Architecture decision
Test the tuning method on a small representative dataset. Keep model evaluation separate from the training loss.
GLOBALATLAS AI · Fine-tuning
Fine-tuning
Fine-tuning adapts a pretrained model. Full fine-tuning and parameter-efficient methods have different memory requirements.
Choose what will be updated
- 01
Full tuning or a parameter-efficient method
- 02
Sequence length, batch and numerical precision
- 03
Evaluation set and versioned checkpoints
Configuration requirements
Specify method, precision, sequence length and batch size. Keep training data and checkpoints in persistent storage.