Agentic AI
New AI Framework Tackles Multi-Page Document Questions Through Active Visual Reasoning
Doc-V* uses an agentic approach to navigate lengthy documents intelligently, outperforming existing OCR-free methods on document VQA tasks.
Researchers have introduced Doc-V*, an OCR-free agentic framework designed to handle visual question answering across multi-page documents. The approach addresses a fundamental challenge in document understanding: how to reason over semantics, layouts, and visual content in long, visually dense materials without relying on optical character recognition.
The Core Problem#
Existing methods for document VQA face a difficult trade-off. End-to-end models struggle to scale efficiently as document length increases, while retrieval-based pipelines tend to be passive and prone to errors. These limitations make it hard to build robust systems that can understand and extract information from complex, multi-page documents at scale.
How Doc-V* Works#
Rather than processing entire documents at once, Doc-V* takes a strategic, step-by-step approach. The system begins by examining a thumbnail overview to understand document structure. It then actively navigates through pages using semantic retrieval and targeted page fetching—essentially learning which pages to look at next based on the question being asked. Evidence is accumulated in a structured working memory, which grounds the final reasoning process.
This agentic strategy allows the model to be selective about which information it processes, rather than being overwhelmed by the full document.
Training and Performance#
The framework was trained using imitation learning from expert trajectories and further refined with Group Relative Policy Optimization, balancing answer accuracy against the efficiency of evidence-seeking. Across five benchmarks, Doc-V* outperforms open-source baselines and approaches the performance of proprietary models. In out-of-domain scenarios, the system improved results by up to 47.9% compared to standard retrieval-augmented generation baselines.
Notably, the improvements come from effective evidence aggregation and selective attention rather than simply processing more pages—a key insight for building efficient document understanding systems.
