WelchLabsVideo

Residual Connections: Deep Learning's Most Underrated Simple Hack

ResNet is the most cited paper of the 21st century, but its real contribution isn't making networks deeper — it's inventing the residual stream, a kind of working memory that lets a model store and edit information, and that fundamentally changed how we understand neural networks.