Viet-Anh on Software Logo

What is: Residual Network?

SourceDeep Residual Learning for Image Recognition
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Residual Networks, or ResNets, learn residual functions with reference to the layer inputs, instead of learning unreferenced functions. Instead of hoping each few stacked layers directly fit a desired underlying mapping, residual nets let these layers fit a residual mapping. They stack residual blocks ontop of each other to form network: e.g. a ResNet-50 has fifty layers using these blocks.

Formally, denoting the desired underlying mapping as H(x)\mathcal{H}(x), we let the stacked nonlinear layers fit another mapping of F(x):=H(x)x\mathcal{F}(x):=\mathcal{H}(x)-x. The original mapping is recast into F(x)+x\mathcal{F}(x)+x.

There is empirical evidence that these types of network are easier to optimize, and can gain accuracy from considerably increased depth.