Model Latency
Noun · AI & Machine Learning
Definitions
The delay introduced by the model portion of a system, from receiving input to producing output. Model latency is a key product metric because users notice slow responses long before they care about benchmark details.
In plain English: How long the model itself takes to produce a result.
Example: "They accepted a slightly lower quality score in exchange for better model latency during live customer chat."