LLM Proxy

Definitions

1. A proxy service that sits between applications and large language models to add routing, logging, caching, security, or compatibility layers. An LLM proxy can be lighter-weight than a full platform while still centralizing important controls.

   In plain English: A proxy layer placed in front of large language model calls.

   Example: "They deployed an LLM proxy to mask provider differences and capture consistent usage metrics across products."

Related Terms

• LLM Benchmark
• LLM Cache
• LLM Calling
• LLM Completion
• LLM Config
• LLM Memory
• LLM Optimization
• LLM Pipeline