AI Security
LLM Agents Can Leak Secrets Through Tool Combinations, New Research Shows
Researchers identify a privacy vulnerability where language models combine non-sensitive tool outputs to reveal confidential information, and propose defenses including specialized training methods.
Language model agents face a distinctive privacy threat: they can chain together multiple tool outputs that individually contain no sensitive data, yet collectively expose confidential conclusions. Researchers have now formalized this vulnerability, built a benchmark to measure it, and tested mitigation strategies across major LLM systems.
The Hidden Privacy Risk#
The problem, termed Tools Orchestration Privacy Risk (TOP-R), arises when an AI agent orchestrates calls to multiple APIs or tools. Each individual tool might return innocuous information—a user's age, their location, their search history. But when the agent combines these responses, it can synthesize and reveal something genuinely sensitive: a person's medical diagnosis, financial status, or other protected detail.
The vulnerability hinges on three conditions: the final conclusion must be sensitive; no single tool output should directly reveal it; yet the compositional result of combining tool returns must make inference possible.
Building a Benchmark and Dataset#
To systematically study TOP-R, researchers developed Library-Grounded Reverse-Inference Seed Expansion (LRSE), a four-library reverse-construction pipeline for generating realistic privacy-leaking scenarios. Using this method, they created TOP-Bench, a benchmark containing 1,000 test instances evaluated under a controlled two-stage tool-use protocol.
Testing across six major LLM agents revealed stark findings:
- Task completion rate: 98.0% on average
- Leakage rate: 88.6% across models
- H-score (a metric balancing safety and utility): 20.4
When models used native reasoning capabilities, the leakage worsened significantly—four models averaged 81.4% leakage in final responses and 82.4% leakage in reasoning traces.
Prompt-Only Defenses Fall Short#
The researchers evaluated three prompt-based safeguards, finding they offer only modest protection. These defenses improved the H-score by roughly 3.4 points on average, suggesting that asking models to be careful through instructions alone is insufficient.
