Expert Iteration

Definitions

1. A training approach in which a stronger search or expert process generates improved targets or guidance that a model then learns to imitate, repeating this cycle over time. The idea is to bootstrap stronger performance through iterative improvement.

   In plain English: A method where a model repeatedly learns from stronger expert-generated examples.

   Example: "The planning agent improved through expert iteration, using costly search traces to train a faster policy for routine decisions."

Related Terms

• distillation
• Chinchilla Scaling
• Constitutional AI Method
• Context Distillation
• Frozen Embedding
• Goal Conditioned
• Gradient-Free Optimization
• Machine Unlearning
• Mesa-Optimization
• Objective Function
• Process Reward Model
• Replay Buffer
• Reward Engineering
• Score Function
• Surrogate Model
• Training Dynamics