Viet-Anh on Software Logo

What is: Accordion?

SourceAccordion: Adaptive Gradient Communication via Critical Learning Regime Identification
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Accordion is a gradient communication scheduling algorithm that is generic across models while imposing low computational overheads. Accordion inspects the change in the gradient norms to detect critical regimes and adjusts the communication schedule dynamically. Accordion works for both adjusting the gradient compression rate or the batch size without additional parameter tuning.