Viet-Anh on Software Logo

What is: Sparse Autoencoder?

Year2000
Data SourceCC BY-SA - https://paperswithcode.com

A Sparse Autoencoder is a type of autoencoder that employs sparsity to achieve an information bottleneck. Specifically the loss function is constructed so that activations are penalized within a layer. The sparsity constraint can be imposed with L1 regularization or a KL divergence between expected average neuron activation to an ideal distribution pp.

Image: Jeff Jordan. Read his blog post (click) for a detailed summary of autoencoders.