Training Distribution

Definitions

1. The distribution of examples, patterns, and conditions present in the data a model was trained on. A mismatch between training distribution and real-world use often causes poor generalization.

   In plain English: The kinds and frequencies of data the model saw during training.

   Example: "The model struggled in production because customer inputs differed sharply from the clean training distribution used during development."

Related Terms

• Next Token Prediction
• Noise Injection
• Objective Function
• Preference Data
• Refusal Training
• Reward Signal
• Self-Supervised
• Training Curriculum