**Recurrent Dropout** is a regularization method for [recurrent neural networks](https://paperswithcode.com/methods/category/recurrent-neural-networks). [Dropout](https://paperswithcode.com/method/dropout) is applied to the updates to [LSTM](https://paperswithcode.com/method/lstm) memory cells (or [GRU](https://paperswithcode.com/method/gru) states), i.e. it drops out the input/update gate in LSTM/GRU.

**Implicit Subspace Prior Learning**, or **ISPL**, is a framework to approach dual-blind face restoration, with two major distinctions from previous restoration methods: 1) Instead of assuming an explicit degradation function between LQ and HQ domain, it establishes an implicit correspondence between both domains via a mutual embedding space, thus avoid solving the pathological inverse problem directly. 2) A subspace prior decomposition and fusion mechanism to dynamically handle inputs at varying degradation levels with consistent high-quality restoration results.

ISPL

Implicit Subspace Prior Learning for Dual-Blind Face Restoration

Recurrent Dropout

Recurrent Dropout without Memory Loss

**ShuffleNet V2 Block** is an image model block used in the [ShuffleNet V2](https://paperswithcode.com/method/shufflenet-v2) architecture, where speed is the metric optimized for (instead of indirect ones like FLOPs). It utilizes a simple operator called channel split. At the beginning of each unit, the input of $c$ feature channels are split into two branches with $c - c'$ and $c'$ channels, respectively. Following **G3**, one branch remains as identity. The other branch consists of three convolutions with the same input and output channels to satisfy **G1**. The two $1\times1$ convolutions are no longer group-wise, unlike the original [ShuffleNet](https://paperswithcode.com/method/shufflenet). This is partially to follow **G2**, and partially because the split operation already produces two groups. After [convolution](https://paperswithcode.com/method/convolution), the two branches are concatenated. So, the number of channels keeps the same (G1). The same “[channel shuffle](https://paperswithcode.com/method/channel-shuffle)” operation as in ShuffleNet is then used to enable information communication between the two branches.

The motivation behind channel split is that alternative architectures, where pointwise group convolutions and bottleneck structures are used, lead to increased memory access cost. Additionally more network fragmentation with group convolutions reduces parallelism (less friendly for GPU), and the element-wise addition operation, while they have low FLOPs, have high memory access cost. Channel split is an alternative where we can maintain a large number of equally wide channels (equally wide minimizes memory access cost) without having dense convolutions or too many groups.

Source	Recurrent Dropout without Memory Loss
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com