Noise Injection

Definitions

1. The deliberate addition of noise to inputs, activations, gradients, or parameters during training or testing to improve robustness, exploration, or regularization. Noise injection can help models avoid brittle overfitting.

   In plain English: Deliberately adding randomness to help a model become more robust.

   Example: "They used noise injection during training so the classifier would become less sensitive to tiny formatting differences in customer forms."

Related Terms

• Elastic Weight Consolidation
• Next Token Prediction
• Objective Function
• Preference Data
• Refusal Training
• Reward Signal
• Self-Supervised
• Training Curriculum
• Training Distribution
• Training Dynamics
• Training Recipe