Decoder-Only Transformer

Definitions

1. A transformer architecture that predicts the next token from prior context without using a separate encoder, which is the common design behind many modern language models.

   In plain English: A common AI model design that generates text based only on what came before.

   Example: "Most chat models are decoder-only transformers trained for next-token prediction."

Related Terms

• Mixture of Experts
• Differential Transformer
• GPT Architecture
• Graph Transformer
• Gating Network
• Grouped Query Attention
• Positional Encoding
• RoPE
• SwiGLU