Threat T6
Intent Breaking & Goal Manipulation
Attacks that manipulate the agent's core decision-making to achieve unauthorized goals, breaking control flow, abusing shared context, or interfering with isolated data.
- 8/10 risk (Critical)
- 6 attack vectors
- 6 mitigating controls
- 7 framework refs
Overview
- Impact level: High
- Risk score: 8/10 (Critical)
- CIA impact: Confidentiality, Integrity, Availability
- Affected components: Foundation Model (KC1.1-KC1.3); Agent Orchestration (KC2.1-KC2.5); Reasoning Engine (KC3.1-KC3.4); Memory Modules (KC4.1-KC4.6); Tool Integration Layer (KC5.1-KC5.4)
- Tags: intent, goals, manipulation, control-flow, semantic
- Source: OWASP Agentic AI T&M v1.1 (2025-12)
Attack vectors
How attackers carry out intent breaking & goal manipulation against an agentic system.
| Attack vector | Severity | Example |
|---|---|---|
| Goal Substitution | High | Replacing agent's primary objectives with malicious goals |
| Intent Parser Overflow | Medium | Overwhelming intent extraction with complex nested requests |
| Semantic Ambiguity Exploitation | High | Crafting instructions with multiple interpretations |
| Priority Inversion | Medium | Manipulating task priorities to execute unauthorized actions first |
| Control Flow Hijacking | High | Redirecting agent execution to unintended code paths |
| Context Window Poisoning | Medium | Injecting malicious context to influence decision-making |
Affected components
KC1 Language Models (LLMs)
- KC1.1 Large Language Models (LLMs)
- KC1.2 Multimodal LLMs (MLLMs)
- KC1.3 Small-Language Models (SLMs)
- KC1.4 Fine-tuned Models
KC2 Orchestration (Control Flow)
KC3 Reasoning / Planning Paradigm
- KC3.1 Structured Planning / Execution
- KC3.2 ReAct (Reason + Act)
- KC3.3 Chain of Thought (CoT)
- KC3.4 Tree of Thoughts (ToT)
KC4 Memory Modules
- KC4.1 In-agent session memory
- KC4.2 Cross-agent session memory
- KC4.3 In-agent cross-session memory
- KC4.4 Cross-agent cross-session memory
- KC4.5 In-agent cross-user memory
- KC4.6 Cross-agent cross-user memory
KC5 Tool Integration Frameworks
Mitigating controls
Controls from the catalog that reduce the likelihood or impact of T6.
M4 Prompt Hardening & Jailbreak Prevention
Implement multiple layers of prompt security including structural defenses, behavioral constraints, runtime detection, and advanced techniques to prevent prompt injection, model manipulation, and jailbreaking attempts.
M5 Multi-Stage Reasoning Validation
Implement comprehensive validation of agent reasoning processes using multiple checkpoints, external validators, consistency checks, and fact verification to ensure goal alignment, prevent manipulation, and detect hallucinations.
M8 Adaptive Human-in-the-Loop Controls
Implement intelligent human oversight with risk-based approval workflows, clear decision interfaces, anti-fatigue mechanisms, and adaptive trust systems to prevent approval overwhelm, human attacks, and manipulation while maintaining effective oversight.
M11 Content Security & Output Filtering
Implement comprehensive filtering and validation of agent outputs to prevent harmful content generation, data leakage, manipulation attempts, and ensure compliance with security policies and regulatory requirements.
M13 Secure Development Lifecycle Integration
Embed security practices throughout the AI and agent lifecycle with security design reviews, automated testing gates, dependency scanning, and continuous compliance checks.
M16 Behavioral Monitoring & Ethics Framework
Implement comprehensive behavioral analysis, ethics enforcement, and bias detection to identify misaligned behaviors, prevent human manipulation, and ensure ethical agent operations while maintaining transparency and accountability.
Framework mappings
OWASP Top 10 for Agentic Applications
Cisco AI Security Taxonomy
OWASP AIVSS core risks
- AIVSS #8 Agent Memory and Context Manipulation AIVSS 8.9 (High)
- AIVSS #10 Agent Goal and Instruction Manipulation AIVSS 7.1 (High)
- AIVSS #5 Agent Cascading Failures AIVSS 9.4 (Critical)
- AIVSS #7 Insecure Agent Critical Systems Interaction AIVSS 9.2 (Critical)
OWASP AISVS
Chapters cited by the mitigating controls: C1 Training Data Integrity & Traceability, C2 Input Validation, C3 Model Lifecycle Management & Change Control, C4 Infrastructure, Configuration & Deployment Security, C5 Access Control & Identity for AI Components & Users, C6 Supply Chain Security for Models, C7 Model Behavior, Output Control & Safety Assurance, C9 Orchestration & Agentic Security, C11 Adversarial Robustness, C12 Monitoring, Logging & Anomaly Detection.
Requirements: C2.1.1, C2.1.2, C2.1.3, C2.1.6, C2.1.7, C2.1.8, C11.1.3, C12.2.1, C12.2.3, C7.1.3, C7.2.1, C7.2.2, C7.2.3, C7.4.3, C9.2.6, C9.2.7, C9.2.1, C9.2.2, C9.2.3, C9.2.4, C9.2.8, C9.2.10, C9.6.2, C12.4.2, C7.1.1, C7.1.2, C7.3.1, C7.3.2, C7.3.3, C7.3.4, C2.2.1, C5.2.4, C11.2.1, C3.1.1, C3.3.1, C3.4.1, C3.4.2, C4.1.1, C6.1.4, C1.3.3, C11.1.1, C11.1.5, C12.2.2, C12.3.4.
MITRE ATLAS
- AML.T0015 Evade AI Model
- AML.T0043 Craft Adversarial Data
- AML.T0050 Command and Scripting Interpreter
- AML.T0051 LLM Prompt Injection
- AML.T0051.001 LLM Prompt Injection: Indirect
- AML.T0054 LLM Jailbreak
- AML.T0067 LLM Trusted Output Components Manipulation
- AML.T0070 RAG Poisoning
- AML.T0093 Prompt Infiltration via Public-Facing Application
Attack walkthroughs
- EchoLeak: Zero-Click Indirect Prompt Injection: An attacker emails a crafted message that silently triggers Microsoft 365 Copilot to execute hidden instructions, causing the AI to exfiltrate confidential emails, files, and chat logs without any user interaction.
- Operator Prompt Injection via Web Content: An attacker plants malicious content on a web page that the Operator agent processes, tricking it into following unauthorized instructions and exposing users' private data.
- Goal-lock drift via scheduled prompts: A malicious calendar invite injects a recurring 'quiet mode' instruction that subtly reweights objectives each morning, steering the planner toward low-friction approvals.
- Inception attack on ChatGPT users: A malicious Google Doc injects instructions for ChatGPT to exfiltrate user data and convinces the user to make an ill-advised business decision.
- Travel Booking Memory Poisoning: An attacker keeps reinforcing a fake flight price; the assistant stores it as truth, then approves bookings at that price and bypasses payment checks.
- Context Window Exploitation: The attacker splits attempts across sessions so earlier rejections drop out of context, and the AI eventually grants escalating permissions up to admin access.
- Shared Memory Poisoning: The attacker inserts bogus refund policies into shared memory; other agents reuse them, causing bad decisions and financial losses.
- Assistant Memory Poisoning: An attacker implants a user assistant's memory via Indirect Prompt Injection, compromising current and future sessions.
- T6 (Intent Breaking) & T15 (Human Manipulation) in Sequential: The primary attack vector is manipulating the agent's core KC1 (LLM) through prompt injection.
- T6 (Intent Breaking) & Control-Flow Hijacking in Hierarchical: A key threat is compromising the KC2.2 orchestrator.
- T6 (Intent Breaking) & T7 (Misaligned Behaviors) in Reactive: The real-time decision-making process can be manipulated through carefully crafted environmental inputs or events.
References
Source: OWASP Agentic AI – Threats and Mitigations 1.1 (Dec 2025) · retrieved 2026-11