Overfitting

Definitions

1. When a model learns the training data too well — including its noise and random fluctuations — and fails to generalize to new, unseen data. The model has memorized the answers instead of understanding the subject.

   In plain English: When an AI gets perfect scores on its practice tests but fails the real exam — it memorized the specific examples instead of learning the underlying patterns.

Related Terms

• Fine-Tuning
• RLHF
• Training Data
• Epoch
• Underfit
• Model Fine-Tuning Cost
• NVIDIA H100
• Out-of-Distribution