Inference
Noun · AI & Machine Learning
Definitions
The process of running a trained machine learning model on new data to generate predictions or outputs. Unlike training (which learns patterns), inference applies what was learned. Optimizing inference latency and cost is a major concern in production ML.
In plain English: Using a trained AI model to actually make predictions on new data, as opposed to the learning phase.
Example: "Training the model took 3 days on 8 GPUs, but inference runs in under 50ms on a single CPU with the quantized version."
Related Terms
- Inference
- Few-Shot Learning
- Synthetic Data
- Model Card
- Embedding Space
- Guardrails
- Federated Learning
- Embedding Model
- Edge AI
- Flash Attention
- JAX
- KV Cache
- Speech Recognition
- Speech Synthesis
- Speech-to-Text
- Text-to-Speech
- Voice Activity Detection
- Voice Cloning
- Voice Recognition
- Whisper
- Autoregressive Model
- Triton Kernel
- Garbage In Garbage Out