RoPE

Definitions

1. Short for Rotary Positional Embedding, a positional method used in transformer models to encode token order in attention calculations.

   In plain English: A popular way of representing token position inside modern language models.

   Example: "RoPE helped the model handle longer contexts more gracefully than older approaches."

Related Terms

• BERT
• Cross-Attention
• Differential Transformer
• Flash Attention
• GPT Architecture
• Decoder-Only Transformer
• Grouped Query Attention
• Positional Encoding