**MobileNetV2** is a convolutional neural network architecture that seeks to perform well on mobile devices. It is based on an inverted residual structure where the residual connections are between the bottleneck layers.  The intermediate expansion layer uses lightweight depthwise convolutions to filter features as a source of non-linearity. As a whole, the architecture of MobileNetV2 contains the initial fully [convolution](https://paperswithcode.com/method/convolution) layer with 32 filters, followed by 19 residual bottleneck layers.

**CycleGAN**, or **Cycle-Consistent GAN**, is a type of generative adversarial network for unpaired image-to-image translation. For two domains $X$ and $Y$, CycleGAN learns a mapping $G : X \rightarrow Y$ and $F: Y \rightarrow X$. The novelty lies in trying to enforce the intuition that these mappings should be reverses of each other and that both mappings should be bijections. This is achieved through a [cycle consistency loss](https://paperswithcode.com/method/cycle-consistency-loss) that encourages $F\left(G\left(x\right)\right) \approx x$ and $G\left(F\left(y\right)\right) \approx y$. Combining this loss with the adversarial losses on $X$ and $Y$ yields the full objective for unpaired image-to-image translation.

For the mapping $G : X \rightarrow Y$ and its discriminator $D\_{Y}$ we have the objective:

$$ \mathcal{L}\_{GAN}\left(G, D\_{Y}, X, Y\right) =\mathbb{E}\_{y \sim p\_{data}\left(y\right)}\left[\log D\_{Y}\left(y\right)\right] + \mathbb{E}\_{x \sim p\_{data}\left(x\right)}\left[log(1 − D\_{Y}\left(G\left(x\right)\right)\right] $$

where $G$ tries to generate images $G\left(x\right)$ that look similar to images from domain $Y$, while $D\_{Y}$ tries to discriminate between translated samples $G\left(x\right)$ and real samples $y$. A similar loss is postulated for the mapping $F: Y \rightarrow X$ and its discriminator $D\_{X}$.

The Cycle Consistency Loss reduces the space of possible mapping functions by enforcing forward and backwards consistency:

$$ \mathcal{L}\_{cyc}\left(G, F\right) = \mathbb{E}\_{x \sim p\_{data}\left(x\right)}\left[||F\left(G\left(x\right)\right) - x||\_{1}\right] + \mathbb{E}\_{y \sim p\_{data}\left(y\right)}\left[||G\left(F\left(y\right)\right) - y||\_{1}\right] $$

The full objective is:

$$ \mathcal{L}\_{GAN}\left(G, F, D\_{X}, D\_{Y}\right) = \mathcal{L}\_{GAN}\left(G, D\_{Y}, X, Y\right) + \mathcal{L}\_{GAN}\left(F, D\_{X}, X, Y\right) + \lambda\mathcal{L}\_{cyc}\left(G, F\right) $$

Where we aim to solve:

$$ G^{\*}, F^{\*} = \arg \min\_{G, F} \max\_{D\_{X}, D\_{Y}} \mathcal{L}\_{GAN}\left(G, F, D\_{X}, D\_{Y}\right) $$

For the original architecture the authors use:

-  two stride-2 convolutions, several residual blocks, and two fractionally strided convolutions with stride $\frac{1}{2}$.
- [instance normalization](https://paperswithcode.com/method/instance-normalization)
- PatchGANs for the discriminator
- Least Square Loss for the [GAN](https://paperswithcode.com/method/gan) objectives.

CycleGAN

Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks

MobileNetV2

MobileNetV2: Inverted Residuals and Linear Bottlenecks

A **Dual Path Network (DPN)** is a convolutional neural network which presents a new topology of connection paths internally. The intuition is that [ResNets](https://paperswithcode.com/method/resnet) enables feature re-usage while [DenseNet](https://paperswithcode.com/method/densenet) enables new feature exploration, and both are important for learning good representations. To enjoy the benefits from both path topologies, Dual Path Networks share common features while maintaining the flexibility to explore new features through dual path architectures. 

We formulate such a dual path architecture as follows:

$$x^{k} = \sum\limits\_{t=1}^{k-1} f\_t^{k}(h^t) \text{,}  $$

$$
y^{k} = \sum\limits\_{t=1}^{k-1} v\_t(h^t) = y^{k-1} + \phi^{k-1}(y^{k-1}) \text{,} \\\\
$$

$$
r^{k} = x^{k} + y^{k} \text{,} \\\\
$$

$$
h^k = g^k \left( r^{k} \right) \text{,}
$$

where $x^{k}$ and $y^{k}$ denote the extracted information at $k$-th step from individual path, $v_t(\cdot)$ is a feature learning function as $f_t^k(\cdot)$. The first equation refers to the densely connected path that enables exploring new features. The second equation refers to the residual path that enables common features re-usage. The third equation defines the dual path that integrates them and feeds them to the last transformation function in the last equation.

Source	MobileNetV2: Inverted Residuals and Linear Bottlenecks
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com