Positional Encoding

Definitions

1. A method for injecting token-order information into transformer models so they can distinguish position within a sequence.

   In plain English: A technique that tells an AI model where each token sits in the sequence.

   Example: "Without positional encoding, the model would treat word order too loosely."

Related Terms

• Mixture of Experts
• Differential Transformer
• GPT Architecture
• Graph Transformer
• Decoder-Only Transformer
• Gating Network
• Grouped Query Attention
• RoPE
• SwiGLU