New Framework Detects Synthetic User Activity in Cybersecurity Simulations

Cybersecurity training environments like honeypots and cyber ranges rely on simulating realistic user behavior to detect and respond to threats, yet assessing how convincing those simulated personas actually are has remained difficult. A new research paper introduces PHASE (Passive Human Activity Simulation Evaluation), a machine learning framework that fills this gap by analyzing network traffic patterns to distinguish genuine human activity from artificial behavior.

How PHASE Works#

The system operates entirely passively by examining Zeek network connection logs—detailed records of network activity—without requiring any instrumentation on individual computers or other intrusive monitoring. All data flows through a standard network appliance, eliminating artifacts that might compromise the fidelity of a simulation. Rather than manually labeling network traffic, researchers developed an automated approach using local DNS records to classify activity, making the labeling process scalable and systematic.

Detection Accuracy and Behavioral Insights#

The framework achieved over 90% accuracy in distinguishing human from non-human activity. To understand what makes human behavior distinctive, the team applied SHAP (SHapley Additive exPlanations) analysis—a technique for interpreting machine learning models—and uncovered specific temporal and behavioral signatures that characterize real users. These patterns revealed exactly where synthetic personas fell short in mimicking authentic human interaction.

Practical Application#

In a test case, researchers evaluated an existing synthetic user persona and identified clear non-human patterns undermining its realism. Using those findings, they redesigned the persona's behavioral configuration, resulting in activity that scored significantly higher on human-likeness measures. The refined synthetic user became more effective for security training and testing purposes.