Viet-Anh on Software Logo

What is: k-Sparse Autoencoder?

Sourcek-Sparse Autoencoders
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

k-Sparse Autoencoders are autoencoders with linear activation function, where in hidden layers only the kk highest activities are kept. This achieves exact sparsity in the hidden representation. Backpropagation only goes through the the top kk activated units. This can be achieved with a ReLU layer with an adjustable threshold.