Viet-Anh on Software Logo

What is: L1 Regularization?

Year1986
Data SourceCC BY-SA - https://paperswithcode.com

L1L_{1} Regularization is a regularization technique applied to the weights of a neural network. We minimize a loss function compromising both the primary loss function and a penalty on the L_1L\_{1} Norm of the weights:

L_new(w)=L_original(w)+λw_1L\_{new}\left(w\right) = L\_{original}\left(w\right) + \lambda{||w||}\_{1}

where λ\lambda is a value determining the strength of the penalty. In contrast to weight decay, L1L_{1} regularization promotes sparsity; i.e. some parameters have an optimal value of zero.

Image Source: Wikipedia