T6 (Intent Breaking) & Control-Flow Hijacking in Hierarchical

How the attack unfolds

1. 1. The setup

   The target is a Hierarchical Agent Architecture. This architecture introduces a KC2.2 (Hierarchical planning) model where a central orchestrator agent decomposes complex tasks and delegates them to specialized sub-agents. These sub-agents may use specialized KC1.1 or KC1.4 (Fine-tuned) models.

2. 2. The attack

   A key threat is compromising the KC2.2 orchestrator. An attacker could manipulate the orchestrator to alter the main goal, misroute tasks to inappropriate agents, or disrupt the entire workflow.

   Threat T6 · Component KC2 · High

3. 3. Technique: Goal Substitution

   Replacing agent's primary objectives with malicious goals. The threat catalog rates this T6 vector high.

   Threat T6 · Component KC1 · High

4. 4. Threat: T6 Intent Breaking & Goal Manipulation

   T6 Intent Breaking & Goal Manipulation: Attacks that manipulate the agent's core decision-making to achieve unauthorized goals, breaking control flow, abusing shared context, or interfering with isolated data.

   Threat T6 · Component KC2 · High

5. 5. In the wild: EchoLeak: Zero-Click Indirect Prompt Injection

   ASI01 Agent Goal Hijack in the OWASP Top 10 for Agentic Applications describes the same pattern: An attacker emails a crafted message that silently triggers Microsoft 365 Copilot to execute hidden instructions, causing the AI to exfiltrate confidential emails, files, and chat logs without any user interaction.

   Threat T6 · Component KC1 · High

Threats involved

• T6 Intent Breaking & Goal Manipulation: Attacks that manipulate the agent's core decision-making to achieve unauthorized goals, breaking control flow, abusing shared context, or interfering with isolated data.

Controls that stop it

• M4 Prompt Hardening & Jailbreak Prevention

  Implement multiple layers of prompt security including structural defenses, behavioral constraints, runtime detection, and advanced techniques to prevent prompt injection, model manipulation, and jailbreaking attempts.

• M5 Multi-Stage Reasoning Validation

  Implement comprehensive validation of agent reasoning processes using multiple checkpoints, external validators, consistency checks, and fact verification to ensure goal alignment, prevent manipulation, and detect hallucinations.

• M8 Adaptive Human-in-the-Loop Controls

  Implement intelligent human oversight with risk-based approval workflows, clear decision interfaces, anti-fatigue mechanisms, and adaptive trust systems to prevent approval overwhelm, human attacks, and manipulation while maintaining effective oversight.

• M11 Content Security & Output Filtering

  Implement comprehensive filtering and validation of agent outputs to prevent harmful content generation, data leakage, manipulation attempts, and ensure compliance with security policies and regulatory requirements.

• M13 Secure Development Lifecycle Integration

  Embed security practices throughout the AI and agent lifecycle with security design reviews, automated testing gates, dependency scanning, and continuous compliance checks.

• M16 Behavioral Monitoring & Ethics Framework

  Implement comprehensive behavioral analysis, ethics enforcement, and bias detection to identify misaligned behaviors, prevent human manipulation, and ensure ethical agent operations while maintaining transparency and accountability.