Latency Glossary

Browse 7 latency terms defined in plain English, from the cultural dictionary of computing.

7 Latency Terms

AI Speed: The responsiveness or throughput of an AI system, usually measured in latency, tokens per second, jobs per minute, or end-to-end completion time. AI speed...
Edge Computing: Edge Computing is a distributed computing paradigm that processes data closer to where it is generated rather than sending everything to a centralized cloud...
Inference Latency: The delay between submitting an inference request and receiving the result. Inference latency is affected by model size, hardware, batching, network overhead,...
Model Latency: The delay introduced by the model portion of a system, from receiving input to producing output. Model latency is a key product metric because users notice...
Real-Time System: A computing system in which correctness depends not only on producing the right result but on producing it within a guaranteed time bound (deadline). Hard...
Region: A geographic area containing multiple availability zones where cloud resources can be deployed. Choosing regions affects latency (closer to users = faster),...
Response Time AI: The time it takes an AI system to return a response, including retrieval, model inference, validation, and any other steps on the critical path. Response time...