Misaligned & Deceptive Behaviors

Overview

• Impact level: Medium
• Risk score: 6/10 (High)
• CIA impact: Integrity
• Affected components: Foundation Model (KC1.1-KC1.3); Reasoning Engine (KC3.1-KC3.4); Tool Integration Layer (KC5.1-KC5.4)
• Tags: alignment, ethics, behavior, values, oversight
• Source: OWASP Agentic AI T&M v1.1 (2025-12)

Attack vectors

How attackers carry out misaligned & deceptive behaviors against an agentic system.

Affected components

KC1 Language Models (LLMs)

• KC1.1 Large Language Models (LLMs)
• KC1.2 Multimodal LLMs (MLLMs)
• KC1.3 Small-Language Models (SLMs)
• KC1.4 Fine-tuned Models

KC3 Reasoning / Planning Paradigm

• KC3.1 Structured Planning / Execution
• KC3.2 ReAct (Reason + Act)
• KC3.3 Chain of Thought (CoT)
• KC3.4 Tree of Thoughts (ToT)

KC5 Tool Integration Frameworks

• KC5.1 Flexible Libraries / SDK Features
• KC5.2 Managed Platforms / Services
• KC5.3 Managed APIs

Mitigating controls

Controls from the catalog that reduce the likelihood or impact of T7.

• M4 Prompt Hardening & Jailbreak Prevention

  Implement multiple layers of prompt security including structural defenses, behavioral constraints, runtime detection, and advanced techniques to prevent prompt injection, model manipulation, and jailbreaking attempts.

• M5 Multi-Stage Reasoning Validation

  Implement comprehensive validation of agent reasoning processes using multiple checkpoints, external validators, consistency checks, and fact verification to ensure goal alignment, prevent manipulation, and detect hallucinations.

• M11 Content Security & Output Filtering

  Implement comprehensive filtering and validation of agent outputs to prevent harmful content generation, data leakage, manipulation attempts, and ensure compliance with security policies and regulatory requirements.

• M12 Secure Model Training & Validation

  Comprehensive security practices for model training pipelines including data integrity verification, training environment hardening, model validation against adversarial inputs, and ensuring reproducibility of training processes.

• M13 Secure Development Lifecycle Integration

  Embed security practices throughout the AI and agent lifecycle with security design reviews, automated testing gates, dependency scanning, and continuous compliance checks.

• M16 Behavioral Monitoring & Ethics Framework

  Implement comprehensive behavioral analysis, ethics enforcement, and bias detection to identify misaligned behaviors, prevent human manipulation, and ensure ethical agent operations while maintaining transparency and accountability.

Framework mappings

OWASP Top 10 for Agentic Applications

• ASI01 Agent Goal Hijack
• ASI09 Human-Agent Trust Exploitation

Cisco AI Security Taxonomy

• OB-002 Agentic Autonomy Failures (Misuse, Alignment, Drift)
• OB-015 Harmful / Misleading / Inaccurate Content

OWASP AIVSS core risks

• AIVSS #6 Agent Identity Impersonation AIVSS 9.3 (Critical)
• AIVSS #10 Agent Goal and Instruction Manipulation AIVSS 7.1 (High)
• AIVSS #3 Agent Access Control Violation AIVSS 9.7 (Critical)
• AIVSS #7 Insecure Agent Critical Systems Interaction AIVSS 9.2 (Critical)
• AIVSS #8 Agent Memory and Context Manipulation AIVSS 8.9 (High)

OWASP AISVS

Chapters cited by the mitigating controls: C1 Training Data Integrity & Traceability, C2 Input Validation, C3 Model Lifecycle Management & Change Control, C4 Infrastructure, Configuration & Deployment Security, C5 Access Control & Identity for AI Components & Users, C6 Supply Chain Security for Models, C7 Model Behavior, Output Control & Safety Assurance, C9 Orchestration & Agentic Security, C11 Adversarial Robustness, C12 Monitoring, Logging & Anomaly Detection.

Requirements: C2.1.1, C2.1.2, C2.1.3, C2.1.6, C2.1.7, C2.1.8, C11.1.3, C12.2.1, C12.2.3, C7.1.3, C7.2.1, C7.2.2, C7.2.3, C7.4.3, C9.2.6, C9.2.7, C7.1.1, C7.1.2, C7.3.1, C7.3.2, C7.3.3, C7.3.4, C2.2.1, C5.2.4, C11.2.1, C1.3.1, C1.3.5, C3.2.1, C3.2.2, C3.5.1, C3.5.2, C11.1.1, C11.1.2, C11.2.4, C3.1.1, C3.3.1, C3.4.1, C3.4.2, C4.1.1, C6.1.4, C1.3.3, C11.1.5, C12.2.2, C12.3.4.

Attack walkthroughs

• EchoLeak: Zero-Click Indirect Prompt Injection: An attacker emails a crafted message that silently triggers Microsoft 365 Copilot to execute hidden instructions, causing the AI to exfiltrate confidential emails, files, and chat logs without any user interaction.
• Operator Prompt Injection via Web Content: An attacker plants malicious content on a web page that the Operator agent processes, tricking it into following unauthorized instructions and exposing users' private data.
• Goal-lock drift via scheduled prompts: A malicious calendar invite injects a recurring 'quiet mode' instruction that subtly reweights objectives each morning, steering the planner toward low-friction approvals.
• Inception attack on ChatGPT users: A malicious Google Doc injects instructions for ChatGPT to exfiltrate user data and convinces the user to make an ill-advised business decision.
• Helpful Assistant Trojan: A compromised coding assistant suggests a slick one-line fix; the pasted command runs a malicious script that exfiltrates code.
• Credential harvesting via contextual deception: A prompt-injected IT support agent targets a new hire, cites real tickets to appear legit, requests credentials, then exfiltrates them.
• Invoice Copilot Fraud: A poisoned vendor invoice is ingested by the finance copilot.
• Weaponized Explainability leading to Production Outage: A hijacked agent fabricates a convincing rationale to trick an analyst into approving deletion of a live production database.
• T6 (Intent Breaking) & T7 (Misaligned Behaviors) in Reactive: The real-time decision-making process can be manipulated through carefully crafted environmental inputs or events.
• T5 (Cascading Hallucination) & T7 (Misaligned Behaviors) in Knowledge-Intensive: When the agent reasons over retrieved information, it may blend retrieved facts with hallucinated content, creating believable but inaccurate outputs.

References

• AI Alignment Challenges
• Misaligned AI Behavior Detection