Agentic AI
New COM-based approach teaches AI agents to control professional software like CAD tools
A research paper proposes reframing how AI agents interact with complex desktop applications by treating component object models as executable actions rather than relying on fragile visual grounding.
Researchers have identified a fundamental limitation in how AI agents currently attempt to manipulate professional software: screen-based visual control is brittle and error-prone, while API access remains inconsistent across applications. A new paper introduces a different strategy altogether, proposing the Component Object Model (COM) as a unified foundation for deterministic software interaction.
The paradigm shift#
Rather than training agents to visually navigate graphical interfaces or parse heterogeneous API protocols, the work frames software manipulation as program synthesis—generating executable code that directly invokes underlying software components. This "COM-as-Action" approach treats the discrete, programmatic operations available within applications as the primary unit of control, sidestepping the fragility of visual grounding and long-horizon error accumulation that plague visual agents.
Validating the approach with CAD#
To test this paradigm in real-world conditions, the researchers created ComCADBench, the first benchmark measuring agent performance on actual industrial CAD software. Their experiments reveal a dramatic gap: existing frontier AI models achieve near-zero success when operating through GUI simulation, but immediately show substantial gains when using COM-based execution instead.
The team developed ComActor, a self-correcting agent trained through a progressive three-stage framework designed to bridge the gap between syntactic correctness (generating valid code) and geometric accuracy (achieving the intended design outcome). They also built ComForge, an infrastructure platform for large-scale agent training within Windows containers.
Performance and generalization#
ComActor demonstrates state-of-the-art results on ComCADBench and shows strong resilience in extended task sequences where conventional baselines collapse entirely. The approach also generalizes to external CAD benchmarks beyond the original training data.
