Teacher-Student Model

Definitions

1. A setup where a larger or stronger teacher model guides a smaller student model during training, often to transfer capability more efficiently.

   In plain English: Training a smaller model by having a stronger model teach it.

   Example: "The teacher-student model approach let us compress the system for mobile use."

Related Terms

• RLHF
• Synthetic Data
• Activation Function
• Active Learning
• Adaptive Learning Rate
• Adversarial Training
• Batch Size
• Model Distillation Loss