Viet-Anh on Software Logo

What is: InfoGAN?

SourceInfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

InfoGAN is a type of generative adversarial network that modifies the GAN objective to encourage it to learn interpretable and meaningful representations. This is done by maximizing the mutual information between a fixed small subset of the GAN’s noise variables and the observations.

Formally, InfoGAN is defined as a minimax game with a variational regularization of mutual information and the hyperparameter λ\lambda:

min_G,Qmax_DV_INFOGAN(D,G,Q)=V(D,G)λL_I(G,Q)\min\_{G, Q}\max\_{D}V\_{INFOGAN}\left(D, G, Q\right) = V\left(D, G\right) - \lambda{L}\_{I}\left(G, Q\right)

Where QQ is an auxiliary distribution that approximates the posterior P(cx)P\left(c\mid{x}\right) - the probability of the latent code cc given the data xx - and L_IL\_{I} is the variational lower bound of the mutual information between the latent code and the observations.

In the practical implementation, there is another fully-connected layer to output parameters for the conditional distribution QQ (negligible computation ontop of regular GAN structures). Q is represented with a softmax non-linearity for a categorical latent code. For a continuous latent code, the authors assume a factored Gaussian.