Self-Supervised

Definitions

1. Describing learning methods that use structure already present in unlabeled data to create training signals, rather than relying on externally labeled examples. Many large modern models are trained with self-supervised objectives.

   In plain English: Learning from unlabeled data by deriving supervision from the data itself.

   Example: "The base language model was trained in a self-supervised way on vast text corpora before later instruction tuning."

Related Terms

• Upskilling
• Few-Shot
• Interactive Learning
• Next Token Prediction
• Noise Injection
• Objective Function
• Passive Learning
• Preference Data
• Reward Signal
• Supervised
• Training Curriculum
• Training Distribution
• Training Recipe