Token Generation

Definitions

1. The step-by-step generation of tokens by a model during inference. Token generation speed and quality shape both latency and the final usefulness of the response.

   In plain English: The process of a model generating output token by token.

   Example: "They improved token generation speed by moving common prompts to a faster serving tier."

Related Terms

• AI Token
• Generative Search
• Input Token
• LLM Completion
• LLM Token
• Max Tokens
• Model Token
• Non-Deterministic Output
• Response Token
• Structured Generation
• Token Prediction
• Token Sampling