Viet-Anh on Software Logo

What is: Projection Discriminator?

SourcecGANs with Projection Discriminator
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

A Projection Discriminator is a type of discriminator for generative adversarial networks. It is motivated by a probabilistic model in which the distribution of the conditional variable y\textbf{y} given x\textbf{x} is discrete or uni-modal continuous distributions.

If we look at the original solution for the loss function L_D\mathcal{L}\_{D} in the vanilla GANs, we can decompose it into the sum of two log-likelihood ratios:

f(x,y)=logq(xy)q(y)p(xy)p(y)=logq(yx)p(yx)+logq(x)p(x)=r(yx)+r(x)f^{*}\left(\mathbf{x}, \mathbf{y}\right) = \log\frac{q\left(\mathbf{x}\mid{\mathbf{y}}\right)q\left(\mathbf{y}\right)}{p\left(\mathbf{x}\mid{\mathbf{y}}\right)p\left(\mathbf{y}\right)} = \log\frac{q\left(\mathbf{y}\mid{\mathbf{x}}\right)}{p\left(\mathbf{y}\mid{\mathbf{x}}\right)} + \log\frac{q\left(\mathbf{x}\right)}{p\left(\mathbf{x}\right)} = r\left(\mathbf{y\mid{x}}\right) + r\left(\mathbf{x}\right)

We can model the log likelihood ratio r(yx)r\left(\mathbf{y\mid{x}}\right) and r(x)r\left(\mathbf{x}\right) by some parametric functions f_1f\_{1} and f_2f\_{2} respectively. If we make a standing assumption that p(yx)p\left(y\mid{x}\right) and q(yx)q\left(y\mid{x}\right) are simple distributions like those that are Gaussian or discrete log linear on the feature space, then the parametrization of the following form becomes natural:

f(x,y;θ)=f_1(x,y;θ)+f_2(x;θ)=yTVϕ(x;θ_ϕ)+ψ(ϕ(x;θ_ϕ);θ_ψ)f\left(\mathbf{x}, \mathbf{y}; \theta\right) = f\_{1}\left(\mathbf{x}, \mathbf{y}; \theta\right) + f\_{2}\left(\mathbf{x}; \theta\right) = \mathbf{y}^{T}V\phi\left(\mathbf{x}; \theta\_{\phi}\right) + \psi\left(\phi(\mathbf{x}; \theta\_{\phi}); \theta\_{\psi}\right)

where VV is the embedding matrix of yy, ϕ(,θ_ϕ)\phi\left(·, \theta\_{\phi}\right) is a vector output function of xx, and ψ(,θ_ψ)\psi\left(·, \theta\_{\psi}\right) is a scalar function of the same ϕ(x;θ_ϕ)\phi\left(\mathbf{x}; \theta\_{\phi}\right) that appears in f_1f\_{1}. The learned parameters θ=\theta = {V,θ_ϕ,θ_ψV, \theta\_{\phi}, \theta\_{\psi}} are trained to optimize the adversarial loss. This model of the discriminator is the projection.