Replit 'Vibe Coding' Runaway Execution

How the attack unfolds

1. 1. The setup

   The target is a Reactive Agent Architecture. Agents designed for responsive workflows, often event-driven, where actions are taken based on immediate inputs or changes in the environment. Utilizes ReAct paradigm and managed APIs for quick interactions.

2. 2. The attack

   During automated self-repair tasks, an agent generates and executes unreviewed install or shell commands, deleting or overwriting production data.

   Threat T11 · Component KC5 · High

3. 3. Technique: Code Injection via Tools

   Injecting malicious code through tool parameters. The threat catalog rates this T11 vector high.

   Threat T11 · Component KC5 · High

4. 4. Threat: T11 Unexpected RCE

   T11 Unexpected RCE and Code Attacks: Tools or environments enabling unexpected code execution, presenting direct risks to code execution environments.

   Threat T11 · Component KC6 · High

5. 5. Why the Reactive pattern is exposed

   The rapid action-taking nature of reactive agents creates risks when tools are invoked quickly without thorough validation. An attacker can exploit the agent's responsiveness to trigger unauthorized tool executions or inject malicious code through event payloads.

   Threat T2 · Component KC5 · High

Threats involved

• T11 Unexpected RCE and Code Attacks: Tools or environments enabling unexpected code execution, presenting direct risks to code execution environments.

Controls that stop it

• M2 Tool Sandboxing & Isolation

  Implement defense-in-depth isolation for tool execution environments using containerization, virtualization, and strict resource controls to prevent tool misuse, contain breaches, and prevent unexpected code execution.

• M7 Zero Trust & Least Privilege Access

  Implement fine-grained access controls with continuous verification, temporary credentials, minimal permission sets, and dynamic privilege management following zero trust principles to prevent tool misuse, privilege compromise, and unauthorized code execution.

• M13 Secure Development Lifecycle Integration

  Embed security practices throughout the AI and agent lifecycle with security design reviews, automated testing gates, dependency scanning, and continuous compliance checks.

• M14 Runtime Security Controls & Sandboxing

  Implement comprehensive runtime security controls including advanced sandboxing, execution monitoring, and dynamic security policy enforcement to prevent unexpected code execution and contain security breaches in real-time.

• M17 Supply Chain Security & Provenance

  Implement comprehensive supply chain security including model provenance verification, dependency management, secure development environments, and third-party component validation to prevent supply chain attacks and ensure component integrity.

• M18 Incident Response & Recovery Framework

  Implement comprehensive incident response capabilities specifically designed for agentic AI systems, including automated containment, forensic analysis, recovery procedures, and lessons learned integration to rapidly respond to security incidents and minimize impact.