[Backlog]: Tool Invocation and Function Calling Exploitation in Agentic Systems

[vishaljindal1990]

Checklist

• Backlog entry requires creating new sandboxes.
• Backlog entry requires creating new exploitation code and/or tutorials.

CVE List

No response

Description

Summary

Exploitation of tool invocation and function calling mechanisms in agentic GenAI systems to induce unintended actions, command execution, or privilege escalation.

Reference: https://owasp.org/www-project-top-10-for-large-language-model-applications/

GenAI Red Teaming Manual Reference

4.3 Agentic Systems & Tool Use (or closest applicable section)

Sandbox

Reuse sandbox sandboxes/agentic_local_n8n_v1.65.0 (#22) and/or other agent-based sandboxes.

Exploitation

Inject crafted prompts that manipulate the model into invoking unintended tools or functions.

Examples include:

• Prompt injection to force execution of restricted tools (e.g., shell commands, API calls)
• Crafting inputs that result in command injection via tool interfaces
• Chaining multi-step agent actions to escalate privileges or bypass safeguards
• Abusing structured outputs (e.g., JSON function calls) to trigger unsafe operations