Viet-Anh on Software Logo

What is: Fishr?

SourceFishr: Invariant Gradient Variances for Out-of-Distribution Generalization
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Fishr is a learning scheme to enforce domain invariance in the space of the gradients of the loss function: specifically, it introduces a regularization term that matches the domain-level variances of gradients across training domains. Critically, the strategy exhibits close relations with the Fisher Information and the Hessian of the loss. Forcing domain-level gradient covariances to be similar during the learning procedure eventually aligns the domain-level loss landscapes locally around the final weights.