What is: Linear Warmup With Linear Decay?
Year | 2000 |
Data Source | CC BY-SA - https://paperswithcode.com |
Linear Warmup With Linear Decay is a learning rate schedule in which we increase the learning rate linearly for updates and then linearly decay afterwards.