What is: Reversible Residual Block?

Reversible Residual Blocks are skip-connection blocks that learn reversible residual functions with reference to the layer inputs. It is proposed as part of the RevNet CNN architecture. Units in each layer are partitioned into two groups, denoted $x\_{1}$ and $x\_{2}$; the authors find what works best is partitioning the channels. Each reversible block takes inputs $\left(x\_{1}, x\_{2}\right)$ and produces outputs $\left(y\_{1}, y\_{2}\right)$ according to the following additive coupling rules – inspired by the transformation in NICE (nonlinear independent components estimation) – and residual functions $F$ and $G$ analogous to those in standard ResNets:

$y\_{1} = x\_{1} + F\left(x\_{2}\right)$ $y\_{2} = x\_{2} + G\left(y\_{1}\right)$

Each layer’s activations can be reconstructed from the next layer’s activations as follows:

$x\_{2} = y\_{2} − G\left(y\_{1}\right)$ $x\_{1} = y\_{1} − F\left(x\_{2}\right)$