On-Device Inference

Definitions

1. Inference performed locally on a device rather than in the cloud. On-device inference is valuable when privacy, responsiveness, bandwidth, or offline operation matter more than access to larger remote models.

   In plain English: Running model inference locally on the device itself.

   Example: "The translation feature used on-device inference so travelers could still get responses without a network connection."

Related Terms

• Implicit Reasoning
• Inference Budget
• Inference Cost
• Inference Latency
• Inference Server
• Inference Speed
• Inference Time
• On-Device AI