Safety Glossary

Browse 47 safety terms defined in plain English, from the cultural dictionary of computing.

47 Safety Terms

AI alignment
The research discipline concerned with ensuring that artificial intelligence systems reliably pursue goals and behaviors that are beneficial to humans....
AI Alignment Problem
The challenge of ensuring advanced AI systems reliably pursue human goals, values, and constraints. It influences how models are trained, evaluated, or served,...
AI Red Team
A team that deliberately tests an AI system for harmful outputs, misuse paths, jailbreaks, and other safety or security weaknesses.
AI Safety Benchmark
A benchmark designed to measure model behavior on harmful, risky, deceptive, or policy-sensitive tasks.
AI Sandbox
A restricted environment used to test or execute AI-driven actions with limited permissions, controlled data access, and stronger safety guarantees. AI...
Alignment
Alignment in artificial intelligence refers to the challenge of ensuring that AI systems behave in ways that are consistent with human values, intentions, and...
Canary Deployment
A deployment strategy that gradually rolls out changes to a small percentage of users before deploying to the entire fleet. Named after canaries in coal mines...
Code Hazard
A piece of code or technical situation that creates material risk for correctness, security, or operability. In engineering slang, calling something a code...
constitutional AI
A training methodology developed by Anthropic where an AI system is guided by a set of written principles (a 'constitution') rather than relying solely on...
Guardrail Culture
A culture that relies on clear boundaries and safeguards to keep people and systems safe without requiring perfect judgment every time. Guardrail culture is...
guardrails
Safety mechanisms and filters applied to AI systems to prevent harmful, off-topic, or policy-violating outputs. Guardrails can be implemented at the prompt...
Guardrails
Validation layers applied to AI model inputs and outputs to enforce safety, accuracy, and compliance constraints. Can include content filters, format...
Idempotency Key
A unique identifier sent with an API request that allows the server to recognize and ignore duplicate requests, ensuring operations like payments are only...
Immutable
Data that cannot be changed after creation. Instead of modifying existing data, you create new copies with the changes applied. A cornerstone of functional...
Interpretability
The extent to which humans can understand why an AI system produced a particular output or how it represents information internally. Interpretability matters...
Jailbreak AI
An attempt to bypass an AI system's intended safeguards, restrictions, or instruction boundaries through adversarial prompting or workflow manipulation....
Kill Switch
Kill Switch is a built-in mechanism that disables functionality quickly under defined emergency or abuse conditions. Security teams use it to enforce trust,...
Model Safety
The set of behaviors, controls, and evaluation practices aimed at preventing harmful, unsafe, or policy-violating model outputs and actions. Model safety work...
Moderation
The process of identifying, filtering, or handling unsafe, disallowed, or harmful content in user inputs or AI outputs. Moderation is a core safety function in...
Nullable
A type annotation indicating that a variable may hold either a valid value or null/nil/None, requiring explicit handling before use in null-safe languages.
Null Safety
A type system feature that distinguishes nullable from non-nullable types at compile time, preventing null reference exceptions — the 'billion-dollar mistake'...
Option Type
A type that represents a value that may or may not exist, used to safely handle the absence of data without resorting to null references. Common variants...
Personal Security
The practices and protections an individual uses to secure their accounts, devices, data, and physical safety against digital threats such as phishing, account...
Phantom Type
A type parameter that appears in a type's definition but is not used in any of its fields — it exists only to carry compile-time information. Used to encode...
Prompt Guard
A safeguard or validation layer intended to protect prompts or detect dangerous prompt conditions such as injection, policy violations, or malformed context....
Prompt Safety
The practice of designing prompts and prompt-handling systems so they are resistant to misuse, injection, leakage, or unsafe behavior. Prompt safety overlaps...
Red Team AI
The practice of adversarially testing AI systems to uncover unsafe behavior, security weaknesses, misuse pathways, or policy failures before deployment. Red...
Red Teaming AI
Actively probing an AI system for unsafe behavior, jailbreaks, policy failures, or harmful outputs by simulating adversarial use.
Refusal
A response pattern in which an AI system declines to comply with a request because it is unsafe, disallowed, unsupported, or outside system boundaries. Good...
Refusal Training
Training or tuning methods intended to improve when and how an AI system refuses unsafe, disallowed, or unsupported requests. Refusal training is part of...
Rollback
Rollback is the process of reverting a system, database, or deployment to a previous known-good state after a failed update, bug, or incident. In database...
Safety Alignment
The work of shaping a model so its behavior stays within intended safety, policy, and usefulness boundaries in realistic use.
Safety Benchmark
A benchmark specifically designed to measure safety-related behavior such as refusal quality, harm avoidance, policy compliance, or robustness to adversarial...
Safety Evaluation
The broader process of assessing whether an AI system behaves safely under expected use and misuse scenarios. Safety evaluation may include red teaming,...
Safety Policy
A policy that defines what an AI system should and should not do from a safety and acceptable-use perspective. Safety policies guide training, prompting,...
Safety Testing
Testing focused on unsafe behavior, policy violations, prompt injection, abuse scenarios, and other safety-relevant failure modes. Safety testing is often done...
Safety Training
Training or tuning intended to improve safe behavior, refusal quality, and compliance with policies or human preferences. Safety training is one layer among...
Sandbox Culture
A team environment that encourages trying ideas in isolated, low-risk environments before exposing them to production or other critical systems. It reflects a...
Sleeper Agent
A hypothetical or experimental agent that appears benign under ordinary conditions but behaves differently when specific triggers are met. The concept is...
Static Typing
A type system where variable types are determined and checked at compile time rather than runtime, catching type errors before the program executes.
Strong Typing
A type system characteristic where implicit type conversions are restricted, requiring explicit casts and preventing operations between incompatible types,...
Type System
The set of rules a programming language uses to assign and check types, preventing certain classes of errors. Ranges from dynamic (Python, JS) to static (Rust,...
Uncensored Model
A model advertised as having fewer safety restrictions, moderation layers, or refusal behaviors than mainstream assistants.
Union Type
A type that represents a value which can be one of several specified types. Written as A | B in TypeScript, the compiler ensures you handle all possible...
Unlearning
Techniques for removing or suppressing specific information, behaviors, or data influence from a trained model without fully retraining from scratch.
Watermarking AI
Embedding detectable signals in model outputs or generated media so AI-produced content can later be identified.
Zig
A systems programming language positioned as a pragmatic alternative to C, with no hidden control flow, no hidden allocators, and no undefined behavior. Zig...

Related Topics