Model Token

Definitions

1. A token as understood by a model's tokenizer and counted for context limits, billing, or generation length. Model token behavior matters because different models can split the same text differently.

   In plain English: A unit of text as counted by a model.

   Example: "The migration changed model token counts enough that prompts which previously fit now exceeded the new context window."

Related Terms

• AI Language
• AI Parameter
• AI Token
• Classifier
• Input Token
• LLM Token
• Max Tokens
• Model Benchmark
• Prompt Token
• Response Token
• Token Count
• Token Generation
• Token Prediction
• Token Sequence