New technique shrinks web page data for AI agents without losing capability

Autonomous web browsing agents powered by large language models face a fundamental constraint: how to efficiently feed an AI system information about what's on a web page. A new research paper from arXiv tackles this problem by proposing a smarter way to compress page structure data.

The Core Challenge#

Web agents designed to navigate the internet autonomously must somehow communicate the state of a webpage to an LLM, which then decides what action to take next. Two main approaches exist: screenshot-based systems that send visual images of pages (smaller file sizes but visually grounded), and DOM-based systems that serialize the page's underlying HTML structure (richer semantic information but often too large to fit in an LLM's context window).

The researchers found that traditional DOM snapshots were impractical—42% of them exceeded GPT-4o's 128,000-token context window. This size explosion made DOM-based agents unreliable despite their theoretical advantages.

Introducing D2Snap#

To solve this, researchers developed D2Snap, an algorithm that intelligently downsamples DOM structures while preserving what matters most: the elements a user can actually interact with and the features that distinguish one action from another.

Rather than blindly compressing the page data, D2Snap focuses on maintaining "actionability"—keeping interactive buttons, forms, and text that help the model understand what can be done on the page. The technique dramatically reduces file size while retaining functional information.

Experimental Results#

Testing D2Snap against GPT-4o on web navigation tasks showed promising outcomes:

• All downsampled DOM snapshots fit comfortably within the context window, using just 16.5% of available tokens on average
• The approach achieved 73% task success, outperforming a 67% baseline from screenshot-based agents by 5.8 percentage points
• Surprisingly, visual grounding (the images in screenshot-based systems) contributed little—text-only grounding alone achieved 62% success, suggesting DOM structure may be more valuable than pixel data for autonomous navigation

Why It Matters#

This work suggests that semantic representations of web pages may be more useful for LLM-based agents than visual approaches. By making DOM-based systems practical through intelligent compression, the research opens a path to more capable and context-efficient autonomous browsing agents that can leverage the web's underlying structure rather than just its appearance.