Token Sequence
Noun · AI & Machine Learning
Definitions
An ordered sequence of tokens representing an input, output, or intermediate model state. Token sequences are the basic units many language models operate over.
In plain English: An ordered list of tokens.
Example: "The parser compared token sequences from two runs to see exactly where the generations started to diverge."