Token ID
Noun · AI & Machine Learning
Definitions
The numeric identifier assigned to a token in a model's vocabulary. Token IDs are used internally by tokenizers and models to represent text as discrete values.
In plain English: The numeric ID a model uses to represent a token.
Example: "They inspected token IDs directly while debugging why punctuation-heavy strings were tokenizing so differently across two models."