Convolutional Neural Networks

A convolution applies the same small learned filter at every image location, producing a feature map whose values say where a pattern is present. The filter's shared weights mean that an edge detector learned in the upper-left can also detect the same edge in the lower-right. Stacking convolutional layers expands each unit's receptive field: early layers see edges and textures, while later layers combine them into larger structures. Stride or pooling reduces spatial resolution, lowering cost and increasing the semantic scale represented by later features.

Weight sharing gives CNNs translation-aware image features with far fewer parameters than a dense layer over pixels. The architecture's locality assumption is useful when nearby pixels are related, but it is not a guarantee of invariance — cropping, augmentation, and data diversity still determine which shifts and distortions the model handles well.

Place the kernel over the top-left corner of the input, multiply each kernel weight by the input value beneath it, sum the products, add the bias — that is one output value. Slide the kernel one position right and repeat; at the end of the row, move down and start again. The output, a feature map, is a grid of those sums, and a large value means the patch beneath that position resembled the kernel's pattern. The kernel's weights are ordinary learned parameters, updated by backpropagation like any others; nothing about them is hand-designed in a trained network.

It is the operation the rest of the module is arithmetic about. Once one output value is understood as 'a dot product between a small learned vector and a small patch', stride, padding, receptive field and channel stacking are all bookkeeping about *which* patch, and the shape formulas stop needing memorisation.

A dense layer connects every input to every output, so its parameter count is n_in × n_out and each weight is tied to one specific input position. A convolution has k × k × C_in × C_out weights regardless of the image's height and width, because the same kernel is reused at every position. Two properties follow. The count is independent of resolution: the same layer works on a 64×64 or a 1024×1024 image. And a feature learned anywhere is available everywhere — shift the input and the feature map shifts with it, rather than landing on weights that have never seen that pattern.

This is the entire efficiency argument, and it is a difference of four or five orders of magnitude rather than a tuning gain. It is also a statistical argument, not only a memory one: every position in every training image contributes a gradient to the same kernel, so a convolution effectively has vastly more training examples per parameter than a dense layer does.

Three settings decide a convolution's output size. The kernel size k sets how much of the input each output cell covers; stride s sets how far the kernel moves between positions, so s=2 samples half as many positions per axis and halves the resolution; padding p adds p rows and columns of zeros on each side so that edge pixels can sit at the centre of a patch. The formula is out = floor((in + 2p − k) / s) + 1. The important special case is k=3, s=1, p=1, which gives out = in exactly — 'same' padding — and is why most architectures can stack many 3×3 layers without tracking sizes at all.

Every shape error in a CNN comes from this formula, and every fix comes from it too. It is also how a network's downsampling schedule is designed: strided convolutions and pooling are the only places resolution changes, so the spatial size at the final flatten is fully determined by the sequence of s values, which is the number the first dense layer's in_features must match.

Pooling downsamples a feature map with a fixed, parameter-free rule. MaxPool2d(2) takes the largest value in each non-overlapping 2×2 window, halving both spatial axes and keeping the strongest response; average pooling takes the mean, producing a smoother, less selective summary. Max pooling buys a small amount of *local* translation invariance — a feature that moves one pixel within its window gives the identical output — and it discards precise position in exchange. Modern architectures often replace intermediate pooling with strided convolutions, which downsample with learned weights instead, but keep one global average pool at the end to reduce each channel's whole map to a single number before the classifier.

Pooling is how a network's compute drops as its channel count rises: halving the resolution quarters the positions, which pays for doubling the channels. The invariance it provides is genuine but small and local, so it is not a substitute for augmentation — and the position information it throws away is exactly what segmentation and detection need, which is why those architectures either avoid it or add paths to recover it.

A convolution over a C_in-channel input does not use a 2-D kernel: each filter is C_in × k × k and covers the full depth of the input at once, so one filter produces one output channel by summing across all input channels as well as the spatial patch. Conv2d(3, 64, 3) therefore holds 64 separate 3×3×3 filters and emits 64 feature maps. C_out is a free architectural choice — how many distinct patterns this layer may look for — and the standard schedule raises it as the resolution falls: 3 → 64 → 128 → 256 → 512, roughly doubling each time the grid halves, which keeps the per-layer compute in the same range.

Channels are where a CNN's capacity lives, and confusing the two dimensions of a weight tensor is the most common source of parameter-count surprise. The k·k·C_in·C_out term means a 3×3 layer from 256 to 512 channels holds 1.18 million weights — vastly more than the early layers — which is why almost all of a CNN's parameters sit near its deep, low-resolution end while almost all of its compute sits near the shallow, high-resolution start.

A unit's receptive field is the region of the *original input* that can influence it. One 3×3 layer has a receptive field of 3. Stack a second and each of its inputs already summarises 3 pixels, so the field grows to 5; a third makes it 7. In general, n stacked 3×3 layers at stride 1 reach 2n + 1. Two 3×3 layers therefore see as much as one 5×5 layer, and three see as much as one 7×7 — but with fewer parameters (2·9 = 18 versus 25, and 3·9 = 27 versus 49 per channel pair) and with a nonlinearity between each pair, which a single large kernel cannot have. Strided layers and pooling grow the field much faster, multiplying rather than adding.

It explains a design choice that otherwise looks arbitrary — why modern CNNs are deep stacks of 3×3 kernels rather than shallow stacks of large ones — and it is the number to check when a model cannot see a large object: if the final feature's receptive field is 35 pixels and the object spans 150, no amount of training will fix it. Depth is added for reach as much as for capacity.

The canonical shape is a feature extractor followed by a head. The extractor repeats conv → norm → activation once or twice, then downsamples, raising the channel count as the resolution falls; the head flattens or global-average-pools the final map and applies one or two dense layers to produce logits. Nothing in it is new: the convolutions are module 13, the BatchNorm is module 11, the ReLU is module 3, and the dense head is module 12. Global average pooling is the modern choice over Flatten for the transition, because it makes the head's input size depend only on the channel count and not on the input resolution.

Being able to write this from memory, with correct shapes, is the practical skill the module builds toward. It also makes the downsampling schedule visible as a design decision — where resolution drops, where channels rise, and what the final feature map's size is — which is the single fact the head depends on and the single fact that breaks when the input size changes.

The diagnosis is always the same: apply out = floor((in + 2p − k)/s) + 1 at every layer and compare against what the tensors actually are. The recurring causes are a specific short list. Padding left at its default of 0, so each 3×3 layer quietly shrinks the map by 2. A stride that does not divide evenly, where the floor discards the last row and column. A Flatten into a Linear whose in_features was computed for a different input resolution. And a (B, C, H, W) versus (B, H, W, C) mix-up after loading data with a library that uses channels-last, which produces a plausible-looking but meaningless result rather than an error.

Shape errors in a CNN surface far from their cause — usually at the final dense layer, several stages downstream of the layer that was wrong. Printing shapes forward through the network finds the first divergence in seconds; reasoning backward from the error message usually blames the head, which is almost never the layer at fault.