LoRA

Definitions

1. Low-Rank Adaptation — a parameter-efficient fine-tuning technique that freezes the pre-trained model weights and injects small trainable matrices (adapters) into each layer. Reduces trainable parameters by 10,000x while achieving comparable performance to full fine-tuning. Multiple LoRA adapters can be swapped at inference time.

   In plain English: A way to customize a large AI model by training only a tiny add-on instead of retraining the whole thing.

   Example: "Fine-tune with LoRA on a single GPU — it trains 16MB of adapter weights instead of the full 70B model."

Related Terms

• system prompt
• Context Window
• Temperature
• Token
• Agentic Workflow
• Guardrails
• Adapter Layer
• Function Calling
• Gemma
• Large Multimodal Model
• Merging Models
• Batch Inference
• Parameter Efficient