Preference Data

Definitions

1. Data capturing which outputs humans prefer among alternatives, often used to train ranking models, reward models, or alignment systems. Preference data is useful because it encodes judgments that are hard to express as simple rules.

   In plain English: Data showing which model outputs people prefer.

   Example: "They collected preference data from reviewers who ranked different answer drafts for clarity, caution, and factual grounding."

Related Terms

• Human Preference
• Next Token Prediction
• Noise Injection
• Objective Function
• Preference Model
• Refusal Training
• Reward Signal
• Self-Supervised
• Training Curriculum
• Training Distribution
• Training Recipe