Efficiency Glossary

Browse 25 efficiency terms defined in plain English, from the cultural dictionary of computing.

25 Efficiency Terms

Batch Inference
Running model predictions on many inputs together, usually to improve throughput and hardware efficiency compared with one-at-a-time processing.
Burn Multiple
A capital efficiency metric calculated as net burn divided by net new ARR. A burn multiple of 1x means you spend $1 to generate $1 of new ARR (excellent)....
Compute Optimal Training
A training regime chosen to get the best model quality for a fixed compute budget by balancing model size and data volume appropriately.
Energy Efficiency
Using less energy to achieve the same useful outcome, whether in hardware, infrastructure, buildings, or digital systems. In tech culture energy efficiency...
Lean AI
An approach to AI product development that emphasizes simplicity, clear business value, and efficient use of models and infrastructure instead of maximal...
Leverage
The ability to get disproportionately large results from a given amount of effort, capital, product functionality, or distribution access. In startup...
Linearized Attention
An approach to making attention mechanisms more computationally efficient by approximating or restructuring them so cost scales more favorably with sequence...
LoRA
Low-Rank Adaptation — a parameter-efficient fine-tuning technique that freezes the pre-trained model weights and injects small trainable matrices (adapters)...
Low-Touch Sales
A sales motion with limited human involvement, usually relying on simpler onboarding, lighter qualification, and lower-cost reps or automated flows. It sits...
Magic Number Startup
A reference to the SaaS magic number, a quick efficiency metric comparing recurring-revenue growth to prior sales and marketing spend. Investors use the magic...
Mixture of Depths
An architectural idea where different tokens or inputs receive different amounts of computational depth, allowing the model to spend more effort on harder...
Mixture of Experts
A neural network architecture that routes different inputs to different specialized sub-networks (experts), activating only a subset for each input. Allows...
Model Efficiency
How effectively a model uses compute, memory, and time to deliver useful performance for a given task. Model efficiency matters because a slightly less capable...
Parameter Efficient
Describing methods that achieve adaptation or performance gains while changing only a small subset of parameters instead of retraining or updating an entire...
Parameter Sharing
A design approach where the same parameters are reused across different parts of a model or across multiple computations. Parameter sharing can reduce model...
Performance Optimization
The iterative process of identifying bottlenecks through profiling and measurement, then applying targeted changes — such as algorithmic improvements, caching,...
Prompt Compression
The practice of reducing prompt length while preserving the instructions or context needed for good performance. Prompt compression helps with latency, cost,...
Skinny Pipeline
A lean build or delivery pipeline stripped of unnecessary steps, weight, or latency. In engineering slang, a skinny pipeline is praised when it keeps feedback...
Small Language Model
A language model that is smaller in parameter count and operational cost than frontier-scale models, often optimized for speed, specialization, or on-device...
Sparse Model
A model in which only a subset of parameters or pathways are active for a given input, rather than using the full parameter set every time. Sparse models can...
Token Merging
A tokenization or model-efficiency technique in which tokens or token-like units are merged to reduce computation or represent text more compactly. The term...
Training Efficiency
How effectively a training setup converts compute, data, and time into improved model performance. Better training efficiency means reaching useful quality...
Utilization Rate
The percentage of available capacity, billable time, or allocated resources that are actually being used productively. Utilization rate is especially important...
Window Attention
An attention pattern that restricts attention to local windows or neighborhoods instead of letting every token attend to every other token globally. Window...
Work Smarter
A common piece of advice to improve leverage, eliminate waste, or use better tools rather than relying only on effort. In engineering slang, work smarter...

Related Topics