Human Preference

Definitions

1. Human judgments about which outputs, behaviors, or policies are more desirable, useful, or acceptable in a given context. Human preference data is often used in ranking, alignment, and fine-tuning workflows.

   In plain English: What people judge to be the better or more desirable AI behavior.

   Example: "The model was adjusted using human preference signals from reviewers who consistently favored accurate but cautious answers over confident speculation."

Related Terms

• Direct Preference Optimization
• Alignment Tax
• Constitutional AI Method
• Goal Misgeneralization
• Goodhart's Law AI
• Model Alignment
• Preference Data
• Preference Model
• Self-Critique