New AI Framework Helps Robots Adapt to Hidden, Unpredictable Delays

The problem with real-world robot control#

Standard reinforcement learning assumes agents can observe system state instantly and execute actions without delay. But cyber-physical systems—robots, autonomous vehicles, networked control systems—often face unpredictable latencies. Current methods sidestep this challenge by assuming a known worst-case delay, which forces conservative decisions even when actual delays are much shorter.

Introducing adaptive delay handling#

Researchers have developed the interaction layer, a framework that lets agents anticipate and handle delays that vary over time and remain hidden from the decision-making process. Rather than fixing a single delay assumption, the agent generates a matrix of candidate actions across a horizon of possible future delays, accounting for both transmission lag and potential packet loss over networks.

The ACDA algorithm#

Building on this framework, the team created Actor-Critic with Delay Adaptation (ACDA), a model-based algorithm that learns to recognize delay patterns and adjusts its behavior accordingly. Instead of defaulting to pessimistic upper bounds, the system observes actual delays and refines its strategy dynamically.

Real-world performance#

Testing across standard locomotion benchmarks using measured real-world delays, ACDA substantially outperforms existing approaches. The method shows promise for deployment in environments where communication latencies are unpredictable but observable after the fact—relevant for robotics, industrial control, and networked autonomous systems.

Full paper on arXiv