Token Per Second

Definitions

1. A throughput measure showing how many tokens a model can generate each second during inference.

   In plain English: How fast an AI model outputs tokens.

   Example: "The model felt usable once we crossed 80 token per second on short replies."

Related Terms

• Core Web Vitals
• First Contentful Paint
• Largest Contentful Paint
• KV Cache
• Batch Inference
• Groq Chip
• Model Throughput
• Parallel Decoding
• Think Token
• Lighthouse Score Obsession
• Inference Speed