New Framework Detects LLM Security Flaws Hidden in GitHub Automation Workflows

GitHub's continuous integration (CI) workflows have become a natural place to deploy large language models (LLMs) for automating code review, issue triage, content generation, and repository upkeep. Yet this integration creates a significant security blind spot: outside inputs can manipulate LLM prompts and outputs, potentially compromising security decisions, altering repository state, or triggering privileged commands.

The Problem#

While both LLM security and CI security have been studied in isolation, the risks that emerge when they overlap remain largely uncharted. Externally controllable inputs flowing through a CI workflow—like pull request text or user comments—can be weaponized to shape what an LLM processes and outputs. Those outputs may then feed into critical decisions or actions that affect the repository.

Introducing Heimdallr#

Researchers have developed Heimdallr, the first systematic framework to identify and classify such hybrid risks. The approach:

• Normalizes workflows into an LLM-Workflow Property Graph (L-WPG) to map the execution chain
• Combines multiple analysis techniques: triggerability checking (can an external input reach a vulnerable LLM node?), LLM-assisted dataflow analysis, and deterministic propagation
• Synthesizes concrete threat vectors rather than reporting abstract vulnerabilities

Results in the Wild#

Testing on 300 manually labeled workflows, Heimdallr achieved strong performance across the board: 99.4% accuracy in identifying LLM-related nodes, 99.8% in determining triggerability, and a micro-average F1 score of 0.917 for threat-vector detection.

The researchers have responsibly disclosed findings to 759 repositories containing 802 vulnerable workflow instances, with 71 teams acknowledging the issues. This ongoing effort demonstrates both the scale of the problem and the tool's practical utility.