Evaluation Glossary

Browse 90 evaluation terms defined in plain English, from the cultural dictionary of computing.

90 Evaluation Terms

AI Benchmark
An evaluation concept used to measure, inspect, or compare model behavior through ai benchmark. It influences how models are trained, evaluated, or served, and...
AI Evaluation
The process of measuring an AI system's performance, reliability, safety, and usefulness using benchmarks, real tasks, test sets, rubrics, or human review. AI...
AI Fairness
An evaluation concept used to measure, inspect, or compare model behavior through ai fairness. It influences how models are trained, evaluated, or served, and...
AI Hallucination
An evaluation concept used to measure, inspect, or compare model behavior through ai hallucination. It influences how models are trained, evaluated, or served,...
AI Hallucination Rate
A measured estimate of how often an AI system produces false, invented, or unsupported claims in a given task setup.
AI Metric
A measurement used to assess some aspect of AI system behavior, such as accuracy, latency, cost, hallucination rate, acceptance rate, or user satisfaction. AI...
AI Model Card
A documentation artifact describing a model's purpose, data, metrics, limitations, and intended use. It influences how models are trained, evaluated, or...
AI Performance
How well an AI system performs across relevant dimensions such as accuracy, latency, throughput, cost, and user acceptance. AI performance must be judged in...
AI Quality
The overall usefulness and reliability of an AI system's outputs for the intended task, considering factors like correctness, consistency, tone, safety, and...
AI Red Team
A team that deliberately tests an AI system for harmful outputs, misuse paths, jailbreaks, and other safety or security weaknesses.
AI Safety Benchmark
A benchmark designed to measure model behavior on harmful, risky, deceptive, or policy-sensitive tasks.
AI Testing
The broader practice of validating AI system behavior through test sets, scenario runs, regression suites, manual review, and production monitoring. AI testing...
Anomaly Detection ML
An AI task or capability focused on anomaly detection ml and the production of useful predictions or outputs from data. It influences how models are trained,...
Benchmark
A standardized test or dataset used to evaluate and compare AI model performance. MMLU, HellaSwag, HumanEval — the SATs of the AI world. Models are optimized...
benchmark contamination
The phenomenon where a model's training data inadvertently (or deliberately) includes examples from evaluation benchmarks, inflating its apparent performance...
Benchmark Contamination
When benchmark test data leaks into a model's training set, inflating evaluation scores without genuine capability improvement. A pervasive problem in LLM...
Benchmark Saturation
A situation in which benchmark scores become so high that the benchmark no longer meaningfully distinguishes between systems or predicts real-world usefulness....
Binary Classification
An AI task or capability focused on binary classification and the production of useful predictions or outputs from data. It influences how models are trained,...
Classification Threshold
An AI task or capability focused on classification threshold and the production of useful predictions or outputs from data. It influences how models are...
Conditional Generation
An AI task or capability focused on conditional generation and the production of useful predictions or outputs from data. It influences how models are trained,...
Conformal Prediction
A framework for producing prediction sets or intervals with statistical coverage guarantees under stated assumptions. It influences how models are trained,...
confusion matrix
A table that summarizes the performance of a classification model by showing counts of true positives, true negatives, false positives, and false negatives....
Deep Fake Detection
An AI task or capability focused on deep fake detection and the production of useful predictions or outputs from data. It influences how models are trained,...
Elicitation
The process of drawing out a model's capabilities, preferences, or hidden knowledge through careful prompting, testing, or setup. In AI research, elicitation...
Entity Recognition
An AI task or capability focused on entity recognition and the production of useful predictions or outputs from data. It influences how models are trained,...
Eval Suite
A collection of automated tests, tasks, or scoring scripts used to evaluate model quality across multiple behaviors.
Evaluation Harness
A reusable framework or test rig for running evaluation cases against models or AI systems in a consistent way. Evaluation harnesses make it easier to compare...
Evaluation Metric
An evaluation concept used to measure, inspect, or compare model behavior through evaluation metric. It influences how models are trained, evaluated, or...
F1 score
The harmonic mean of precision and recall, providing a single metric that balances both. Ranges from 0 to 1, where 1 is perfect. The F1 score is especially...
Face Detection
An AI task or capability focused on face detection and the production of useful predictions or outputs from data. It influences how models are trained,...
Face Recognition
An AI task or capability focused on face recognition and the production of useful predictions or outputs from data. It influences how models are trained,...
Factuality
An evaluation concept used to measure, inspect, or compare model behavior through factuality. It influences how models are trained, evaluated, or served, and...
Fairness Metric
An evaluation concept used to measure, inspect, or compare model behavior through fairness metric. It influences how models are trained, evaluated, or served,...
Faithfulness
The degree to which an AI system's output accurately reflects its source material, reasoning, or evidence rather than introducing unsupported claims....
Feature Comparison
A comparison of product features across tools, plans, or competitors. In product culture feature comparisons are common in buying decisions, though they often...
Gesture Recognition
An AI task or capability focused on gesture recognition and the production of useful predictions or outputs from data. It influences how models are trained,...
GLUE Benchmark
A benchmark suite of language understanding tasks used to compare NLP model performance. It influences how models are trained, evaluated, or served, and it can...
Hallucination Detection
An AI task or capability focused on hallucination detection and the production of useful predictions or outputs from data. It influences how models are...
Hallucination Mitigation
An evaluation concept used to measure, inspect, or compare model behavior through hallucination mitigation. It influences how models are trained, evaluated, or...
Heuristic Evaluation
A usability inspection method where a small group of evaluators examines an interface against a set of recognized usability principles (heuristics), such as...
Human Baseline
A human performance level used as a reference point when evaluating an AI system. Comparing against a human baseline helps teams judge whether a model is...
Human Evaluation
An evaluation concept used to measure, inspect, or compare model behavior through human evaluation. It influences how models are trained, evaluated, or served,...
Image Captioning
An AI task or capability focused on image captioning and the production of useful predictions or outputs from data. It influences how models are trained,...
Image Classification
An AI task or capability focused on image classification and the production of useful predictions or outputs from data. It influences how models are trained,...
Image Generation
An AI task or capability focused on image generation and the production of useful predictions or outputs from data. It influences how models are trained,...
Image Segmentation
An AI task or capability focused on image segmentation and the production of useful predictions or outputs from data. It influences how models are trained,...
Information Retrieval
An AI task or capability focused on information retrieval and the production of useful predictions or outputs from data. It influences how models are trained,...
Knowledge Retrieval
An AI task or capability focused on knowledge retrieval and the production of useful predictions or outputs from data. It influences how models are trained,...
LLM-as-Judge
An evaluation pattern where one language model grades, compares, or scores the outputs of another model according to a rubric or prompt.
LLM Benchmark
A benchmark used specifically to evaluate large language models on tasks such as reasoning, coding, factuality, tool use, or safety. LLM benchmarks are useful...
LLM Judge
A large language model used to evaluate, rank, or critique the outputs of another model or system according to some rubric. LLM-as-judge approaches can scale...
Loss Function
Loss Function is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for...
Loss Landscape
Loss Landscape is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for...
MMLU Benchmark
A benchmark that measures broad multitask knowledge and reasoning across many academic and professional domains.
Model Eval
An evaluation process or result used to measure how a model performs on chosen tasks, datasets, or business criteria. Teams rely on model evals to compare...
Needle in a Haystack
A type of evaluation that tests whether a model can retrieve or use one small crucial detail hidden inside a very large context. The phrase is often used to...
Needle in a Haystack Test
A test that checks whether a model can find and use a small relevant fact buried inside a long context window.
Neural Scaling Law
Neural Scaling Law is an AI or ML concept used to represent, train, evaluate, or deploy learned systems. It is commonly used for building production models and...
Optimizer
Optimizer is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for neural...
Performance Benchmark
Performance Benchmark is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before...
Perplexity
A metric that measures how well a probability model predicts a sample. Lower perplexity means the model is less 'surprised' by the data. Informally, it...
Perplexity Detail
Perplexity Detail is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before release...
Perplexity Score
A language-modeling metric that measures how surprised a model is by observed text, with lower perplexity generally indicating better predictive fit.
Precision Detail
Precision Detail is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before release or...
precision (ML)
The fraction of positive predictions that are actually correct: true positives divided by (true positives + false positives). High precision means the model...
Prompt Evaluation
The process of measuring how well a prompt performs across relevant tasks, edge cases, and quality criteria. Prompt evaluation is important because prompts...
Proxy Task
A task used as a stand-in for a harder-to-measure real objective, often because the true target is expensive or ambiguous to evaluate directly. Proxy tasks are...
RAG Evaluation
The evaluation of retrieval-augmented generation systems across dimensions such as retrieval relevance, grounding, citation quality, answer correctness, and...
Ranking Loss
Ranking Loss is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for neural...
Reasoning Benchmark
A benchmark designed to test reasoning ability, such as multi-step logic, math, planning, or problem decomposition. Reasoning benchmarks are useful, but they...
Recall Detail
Recall Detail is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before release or...
recall (ML)
The fraction of actual positive cases that the model correctly identified: true positives divided by (true positives + false negatives). High recall means the...
Red Teaming AI
Actively probing an AI system for unsafe behavior, jailbreaks, policy failures, or harmful outputs by simulating adversarial use.
Regularization
Regularization is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for...
Risk Evaluation
The step in risk management where analyzed risks are compared against criteria such as tolerance, thresholds, and business priorities to decide what action is...
ROC Curve
ROC Curve is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before release or model...
Safety Benchmark
A benchmark specifically designed to measure safety-related behavior such as refusal quality, harm avoidance, policy compliance, or robustness to adversarial...
Safety Evaluation
The broader process of assessing whether an AI system behaves safely under expected use and misuse scenarios. Safety evaluation may include red teaming,...
Scaling Law
Scaling Law is an AI or ML concept used to represent, train, evaluate, or deploy learned systems. It is commonly used for building production models and...
Self-Evaluation
An AI system's attempt to evaluate the quality, correctness, or confidence of its own outputs. Self-evaluation can be helpful, but it is not automatically...
Stochastic Gradient Descent
Stochastic Gradient Descent is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning...
Sycophancy AI
The tendency of a model to flatter the user, agree too readily, or mirror stated beliefs instead of providing accurate correction.
Technical Due Diligence
An assessment of a company's technology stack, codebase quality, architecture, security posture, team capabilities, and technical debt performed before an...
Test Set
Test Set is an evaluation concept used to measure model quality, robustness, or efficiency. It is commonly used for comparing systems before release or model...
Triplet Loss
Triplet Loss is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for neural...
Truthfulness
The tendency of an AI system to provide answers that are factually correct and not misleading, rather than merely plausible-sounding. Truthfulness is distinct...
Turing Test Modern
A contemporary framing of the Turing Test that reflects modern chat systems, multimodal interaction, and more realistic expectations about machine imitation.
Vanishing Gradient
Vanishing Gradient is a training-time optimization concept that governs how model parameters are updated. It is commonly used for iterative learning loops for...
Whiteboard Interview
A job interview format where candidates solve or discuss problems on a whiteboard. In engineering slang, whiteboard interviews remain controversial because...
Window Shopping
Looking at tools, ideas, or solutions without serious commitment to adopting one yet. In engineering slang, window shopping can be useful research or...

Related Topics