Inference

Definitions

1. The process of using a trained model to make predictions or generate outputs from new inputs. As opposed to training (which teaches the model), inference is the model doing its job. Most of the cost of running an AI service at scale is inference, not training.

   In plain English: When an AI model actually does its job — answering questions, generating images, making predictions — using what it already learned during training.

Etymology

1300s

From Latin 'inferre' (to bring in), inference enters English as a logic term for drawing conclusions from premises

2012

Deep learning adoption makes 'inference' the standard term for running a trained model on new data, distinguished from 'training'

2023

Inference cost becomes the bottleneck for AI deployment; specialized inference chips (Groq, AWS Inferentia) emerge as a hardware category

Related Terms

• Edge Computing
• Token
• Temperature
• GPU
• Edge AI
• Quantization
• tokens per second
• model serving
• Overclocking
• Inference
• AI Inference Cost
• Model Fine-Tuning Cost
• Model Throughput