**Auxiliary Classifiers** are type of architectural component that seek to improve the convergence of very deep networks. They are classifier heads we attach to layers before the end of the network. The motivation is to push useful gradients to the lower layers to make them immediately useful and improve the convergence during training by combatting the vanishing gradient problem. They are notably used in the Inception family of convolutional neural networks.

**Pix2Pix** is a conditional image-to-image translation architecture that uses a conditional [GAN](https://paperswithcode.com/method/gan) objective combined with a reconstruction loss. The conditional GAN objective for observed images $x$, output images $y$ and the random noise vector $z$ is:

$$ \mathcal{L}\_{cGAN}\left(G, D\right) =\mathbb{E}\_{x,y}\left[\log D\left(x, y\right)\right]+
\mathbb{E}\_{x,z}\left[log(1 − D\left(x, G\left(x, z\right)\right)\right] $$

We augment this with a reconstruction term:

$$ \mathcal{L}\_{L1}\left(G\right) = \mathbb{E}\_{x,y,z}\left[||y - G\left(x, z\right)||\_{1}\right] $$

and we get the final objective as:

$$ G^{*} = \arg\min\_{G}\max\_{D}\mathcal{L}\_{cGAN}\left(G, D\right) + \lambda\mathcal{L}\_{L1}\left(G\right) $$

The architectures employed for the generator and discriminator closely follow [DCGAN](https://paperswithcode.com/method/dcgan), with a few modifications:

- Concatenated skip connections are used to "shuttle" low-level information between the input and output, similar to a [U-Net](https://paperswithcode.com/method/u-net).
- The use of a [PatchGAN](https://paperswithcode.com/method/patchgan) discriminator that only penalizes structure at the scale of patches.

Pix2Pix

Image-to-Image Translation with Conditional Adversarial Networks

Auxiliary Classifier

**Neural Architecture Search (NAS)** learns a modular architecture which can be transferred from a small dataset to a large dataset. The method does this by reducing the problem of learning best convolutional architectures to the problem of learning a small convolutional cell. The cell can then be stacked in series to handle larger images and more complex datasets.

Note that this refers to the original method referred to as NAS - there is also a broader category of methods called "neural architecture search".

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com