Viet-Anh on Software Logo

What is: Child-Tuning?

SourceRaise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning
Year2000
Data SourceCC BY-SA - https://paperswithcode.com

Child-Tuning is a fine-tuning technique that updates a subset of parameters (called child network) of large pretrained models via strategically masking out the gradients of the non-child network during the backward process. It decreases the hypothesis space of the model via a task-specific mask applied to the full gradients, helping to effectively adapt the large-scale pretrained model to various tasks and meanwhile aiming to maintain its original generalization ability.