What is: Singular Value Clipping?
Source | Temporal Generative Adversarial Nets with Singular Value Clipping |
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
Singular Value Clipping (SVC) is an adversarial training technique used by TGAN to enforce the 1-Lipschitz constraint of the WGAN objective. It is a constraint to all linear layers in the discriminator that satisfies the spectral norm of weight parameter is equal or less than one. This means that the singular values of weight matrix are all one or less. Therefore singular value decomposition (SVD) is performed after a parameter update, replacing all the singular values larger than one with one, and the parameters are reconstructed with them. The same operation is applied to convolutional layers by interpreting a higher order tensor in weight parameter as a matrix .