Token Merging

Definitions

1. A tokenization or model-efficiency technique in which tokens or token-like units are merged to reduce computation or represent text more compactly. The term may refer to vocabulary construction or runtime efficiency methods depending on context.

   In plain English: Combining tokens to make representation or computation more efficient.

   Example: "The research prototype used token merging to reduce the cost of long-document processing without changing the underlying model family."

Related Terms

• Image Tokenizer
• Lean AI
• Linearized Attention
• Parameter Efficient
• Parameter Sharing
• Prompt Compression
• Sparse Model
• Sub-Word
• Vocabulary Size
• Window Attention
• Word Piece