AI
New Framework Adapts ML Safety Nets in Real Time Using Human Feedback
ASAT uses human-in-the-loop learning to dynamically refine how machine learning systems detect unusual inputs while keeping false alarms under control.
Machine learning models trained on familiar data can stumble when faced with unexpected inputs during real-world deployment—a particularly acute problem in safety-critical applications. Researchers have long worked on better scoring methods to measure when something seems "off," but existing approaches rely on static thresholds that don't adapt as new anomalies emerge.
The Core Problem#
Typical detection systems set their alerting threshold based solely on known, in-distribution data to hit a target detection rate. This leaves another critical metric—how often the system falsely flags normal inputs as anomalous—completely uncontrolled. The result is often a cascade of false alarms where legitimate data gets misdirected, while the rigid scoring and thresholds can't keep pace with evolving, real-world anomalies.
How ASAT Works#
A new method called ASAT (Adaptive Scoring and Thresholding) tackles these limitations by incorporating human feedback into an ongoing learning loop. Rather than locking in scoring functions and thresholds at deployment time, the system updates both as it encounters actual out-of-distribution inputs in the wild. Human reviewers help guide these updates, allowing the model to refine its decision-making on the fly.
The approach comes with theoretical guarantees: under stable conditions, it simultaneously maximizes the rate at which true anomalies are caught while keeping false positives bounded. When conditions shift, the system adapts gracefully, though it may experience brief periods of elevated false alarms during the transition.
Evidence and Impact#
Testing on established benchmarks shows that ASAT outperforms existing methods, detecting more genuine anomalies while maintaining tighter control over false positives than fixed-threshold systems. This combination of adaptability and measurable safety guarantees could be especially valuable in domains where both missing anomalies and frequent false alarms carry real costs.
