Object Detection for Dummies Part 2: CNN, DPM and Overfeat

Lilian Weng's second installment in her object detection series steps back from region proposals and gradients to explain the actual engines doing the recognition work: convolutional neural networks. It's a tour through the architectures that made deep learning the default choice for vision tasks, and it's a useful reminder of how fast the field moved between 2012 and 2015.

The throughline is depth. AlexNet, from Krizhevsky and colleagues in 2012, ran just five convolutional layers plus a couple of fully connected ones and leaned on data augmentation tricks like image flips and patch cropping to squeeze more out of a limited dataset. Two years later, Simonyan and Zisserman's VGG pushed to 19 layers by doing something almost stubbornly simple: stack tiny 3x3 filters instead of bothering with bigger kernels. Stacking small filters approximates a larger receptive field with far fewer parameters, which is the kind of unglamorous efficiency trick that ends up mattering a lot in practice.

Then came ResNet in 2015, and 152 layers, which sounds almost absurd until you remember why it worked. Plain deep networks hit a wall — training loss stops improving and can even get worse as you add layers, thanks to vanishing and exploding gradients. He and colleagues' fix was the residual block, letting information from an earlier layer skip ahead and rejoin the network two layers later. Weng draws a nice parallel here to neuroscience: the human visual cortex has a similar shortcut, where region V4 gets input directly from V1. Biology got there first, as usual.

The piece then pivots to two older but foundational detection approaches. The Deformable Parts Model, from Felzenszwalb's 2010 paper, treats an object as a coarse root filter plus several higher-resolution part filters, scored together while penalizing parts that drift from where they should sit relative to the whole. It's an elegant, almost hand-crafted way of encoding the idea that an object is a whole made of movable pieces. Girshick later showed DPM's inference procedure can actually be unrolled into an equivalent CNN, which is a satisfying bit of theoretical bridge-building between the symbolic and deep-learning eras.

Overfeat, from Sermanet, LeCun and others in 2013, closes the loop by fusing classification, localization, and detection into a single convolutional pipeline. Train a classifier resembling AlexNet, then swap the top layers for a regressor that predicts bounding box coordinates directly, trained to minimize the distance between predicted and true boxes. At inference, the network classifies image patches at multiple locations and scales, generates boxes for each, and merges overlapping ones that agree on both position and class confidence. It's a scrappy but effective architecture, and you can see it as a direct ancestor of the R-CNN family that Weng promises to cover next.