Prompt Attack

Definitions

1. An adversarial attempt to manipulate an AI system through crafted input so it ignores instructions, reveals restricted information, or behaves in unintended ways. Prompt attacks are especially important in systems that mix user input with hidden instructions or tool access.

   In plain English: A malicious prompt designed to make an AI system behave incorrectly or unsafely.

   Example: "The red team found a prompt attack that convinced the assistant to treat user text as system-level policy guidance."

Related Terms

• AI Template
• Instruction Hierarchy
• Jailbreak AI
• LLM Security
• Prompt Compression
• Prompt Guard
• Prompt Registry
• Red Team AI