Edge AI

Definitions

1. Running AI models directly on local devices (phones, cameras, IoT) rather than in the cloud, reducing latency and keeping data private. The reason your phone can identify your face without calling Google.

   In plain English: Running artificial intelligence directly on the device in your hand or nearby, instead of sending data to a faraway server, so results come back faster and your data stays more private.

2. The key tradeoff: edge AI models must be small enough to run on constrained hardware (phones, microcontrollers) but accurate enough to be useful. Techniques like quantization, pruning, and knowledge distillation compress large models to run on devices with megabytes instead of gigabytes of memory.

   Example: 'We quantized the model from 32-bit to 8-bit and got it running on a $5 microcontroller. Accuracy dropped 2% but latency went from 500ms to 10ms.'

   Source: compression tradeoffs

Etymology

2016

The proliferation of IoT devices and privacy concerns drive demand for running AI models on-device rather than in the cloud

2017

Google releases TensorFlow Lite, making it practical to deploy machine learning models on mobile phones and embedded devices

2023

Apple's Neural Engine and Qualcomm's AI chips make edge AI a standard feature of consumer devices

Related Terms

• Serverless
• Edge Computing
• Inference
• Quantization
• Lambda Calculus
• Classification
• Federated Learning
• Differential Privacy
• Elastic Weight Consolidation
• ONNX Runtime
• Unlearning
• APK Analysis