Low-Rank Adaptation

Definitions

1. A parameter-efficient fine-tuning method, commonly known as LoRA, that updates low-rank matrices instead of all model weights. Low-rank adaptation makes it cheaper to specialize large models for new tasks or domains.

   In plain English: A cheaper way to adapt a large model by training a small set of additional parameters.

   Example: "They used low-rank adaptation to tune the model for legal drafting without paying the cost of full fine-tuning."

Related Terms

• Merging Models
• AI Multi-Agent
• Alignment Tax
• Benchmark Saturation
• Capability Overhang
• Chain of Density
• Chain of Verification
• Parameter Efficient
• Soft Prompt