What is: Fishr?
Source | Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization |
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
Fishr is a learning scheme to enforce domain invariance in the space of the gradients of the loss function: specifically, it introduces a regularization term that matches the domain-level variances of gradients across training domains. Critically, the strategy exhibits close relations with the Fisher Information and the Hessian of the loss. Forcing domain-level gradient covariances to be similar during the learning procedure eventually aligns the domain-level loss landscapes locally around the final weights.