Fine-Tuning

Definitions

1. The process of taking a pre-trained model and further training it on a smaller, domain-specific dataset to improve performance on a particular task. Like sending a generalist to a specialized bootcamp.

   In plain English: Taking an AI model that already knows a lot about the world and training it further on specific data — like a general doctor specializing in cardiology.

2. LoRA (Low-Rank Adaptation) revolutionized fine-tuning by freezing the base model and training only small adapter layers, reducing the cost from thousands of GPU hours to hours on a single GPU. QLoRA goes further by quantizing the base model, enabling fine-tuning on consumer hardware.

   Example: 'We fine-tuned Llama on 50K customer support transcripts using LoRA. Total cost: $200 in GPU time. The model now matches our tone and knows our product catalog.'

   Source: LoRA / efficient methods

Etymology

1990s

Transfer learning in neural networks establishes the concept of adapting pre-trained models to specific tasks

2018

ULMFiT and BERT make fine-tuning the standard approach for NLP — pre-train on large data, fine-tune on your specific task

2023

LoRA (Low-Rank Adaptation) makes fine-tuning large language models practical on consumer hardware

Related Terms

• Overfitting
• Reinforcement Learning
• RLHF
• Training Data
• Epoch
• Large Vision Model
• Underfit
• Model Fine-Tuning Cost
• NVIDIA H100