​​How many layers to fine-tune? Model fine-tuning allows you to improve the quality of the pre-trained models with just a fraction of the resources spent on training the original model. But there is a trade-off between the number of layers you tune and the precision you get. Using fewer layers allows for faster training with a larger batch size, while more layers increase the model's capacity. We've done experiments so you can make more educated choices. Highlights: - Training only the head of a model (5% of weights) gives x2 boost on metrics, while full training gives only x3. - Training only a head layer allows using larger models with bigger batch sizes, compensating for the precision. - If you only have a small dataset, full model tuning will give a more negligible effect