Latency Budget

Noun · Development

Definitions

  1. The maximum allowable end-to-end response time for a request, broken down and allocated across each component in the processing chain — such as DNS, TLS handshake, API gateway, service logic, and database queries — so that no single stage can silently consume the entire budget.

    In plain English: A time limit for how long a whole operation can take, divided up among each step so you can tell which parts are too slow.

    Example: "Our latency budget for search is 200ms total, and the ML ranking model already eats 80ms, so the database query has to stay under 50ms."

Related Terms