Reward Engineering

Definitions

1. The design and tuning of reward signals, reward models, or scoring functions so learning systems optimize for the intended behavior. Reward engineering is difficult because bad rewards can produce seemingly good but actually undesirable behavior.

   In plain English: Designing reward signals so the AI system learns the behavior you actually want.

   Example: "Most of the project complexity turned out to be reward engineering, not the policy model itself, because the original reward encouraged shallow shortcuts."

Related Terms

• Expert Iteration
• Gradient-Free Optimization
• Inference Optimization
• Mesa-Optimization
• Neural Network Pruning
• Objective Function
• Reward Signal
• Surrogate Model