Parameter Efficient

Definitions

1. Describing methods that achieve adaptation or performance gains while changing only a small subset of parameters instead of retraining or updating an entire large model. Parameter-efficient methods are attractive because they reduce cost and storage demands.

   In plain English: Achieving useful adaptation while changing only a small number of parameters.

   Example: "They used a parameter-efficient tuning method so each customer-specific adapter could be stored cheaply."

Related Terms

• LoRA
• Lean AI
• Linearized Attention
• Low-Rank Adaptation
• Mixture of Depths
• Parameter Sharing
• Prompt Compression
• Small Language Model
• Sparse Model
• Token Merging
• Window Attention