New technique identifies LLMs by how they act as coding agents

As large language models increasingly operate through agentic systems that inspect codebases, execute tools, and edit files, the ability to identify which model is actually running becomes a security concern. A team from arXiv has introduced a new fingerprinting approach that can distinguish between models based purely on their observable actions during agent execution.

The Challenge of Agent-Based Identification#

Traditional LLM fingerprinting techniques rely on examining text outputs or token probability distributions. However, when models operate within a coding-agent framework, these signals become obscured by multiple layers: system prompts, controller logic, the available tools, and execution feedback all reshape the raw model behavior. This mediation makes conventional fingerprinting methods far less reliable.

The security implications run deep. Swapping one model for another behind the same agent harness can alter mission-critical decisions—whether to verify code changes, how to recover from transient failures, or which approach takes precedence when test results conflict with specifications.

How LIDAR Works#

The researchers developed LIDAR (LLM Identification from Decisions and Actions at Runtime), a black-box fingerprinting method that requires no access to model weights, logits, or internal provider systems. Instead, it uses three pairs of targeted coding probes to expose distinct decision-making patterns:

• Post-edit verification behavior: How the model checks its own changes
• Transient-failure recovery: How it handles and bounces back from temporary failures
• Specification–test conflict resolution: How it prioritizes when specifications and test results diverge

Each probe pair includes controlled variants designed to amplify model-specific responses. LIDAR captures both instance-level features (individual decisions) and distribution-level features (patterns across multiple runs), then compares trajectories against clean references using a lightweight probabilistic classifier.

Results and Validation#

Testing across 36 models from seven different model families and two different agent harnesses, LIDAR achieved high accuracy in identifying models by their execution patterns. It outperformed four existing fingerprinting and API-auditing baselines on the same task. Ablation studies confirmed that all components—both feature levels, all probe pairs, and their controlled variants—meaningfully contribute to accurate identification.