ONNX Runtime

Definitions

1. A runtime for executing models in the ONNX format across different hardware backends with an emphasis on portability and inference performance.

   In plain English: Software for running AI models in a portable standardized format.

   Example: "We exported the model to ONNX Runtime for faster deployment on edge hardware."

Related Terms

• Edge AI
• Quantization
• guardrails
• Floating Point Quantization
• INT4
• INT8
• KV Cache
• LLM Agent