Distillation Glossary
Browse 3 distillation terms defined in plain English, from the cultural dictionary of computing.
3 Distillation Terms
- Model Distillation Loss
- The training loss used when a student model learns to match outputs or distributions from a teacher model during distillation.
- Student Model
- A smaller or simpler model trained to imitate or approximate the behavior of a larger teacher model. Student models are often used to reduce cost and latency...
- Teacher-Student Model
- A setup where a larger or stronger teacher model guides a smaller student model during training, often to transfer capability more efficiently.
Related Topics
- Training (2 terms in common)
- Models (1 terms in common)
- Model Compression (1 terms in common)
- Ai (1 terms in common)
- Optimization (1 terms in common)