AI Struggles With Legal Citation Formatting—But Hybrid Approach Offers Hope

Legal AI has long promised to eliminate tedious, repetitive work that consumes lawyers' hours without requiring much judgment. Yet whether state-of-the-art models can actually handle these routine tasks remains unclear. A new paper from arXiv examines this question through the lens of one of the legal profession's most despised chores: formatting citations according to the Bluebook standard.

How Well Do Language Models Handle Citation Formatting?#

The research team created a benchmark of 2,058 Bluebook citation queries and tested frontier language models in a zero-shot setting. The results were sobering: these cutting-edge systems produced fully compliant citations just 42.6% of the time on average. Even more telling, when the researchers tested a leading "reasoning" model against citation standards from five prestigious law reviews, the AI performed markedly worse than human candidates typically do in those journals' competitive editor-selection exams.

Why RAG Alone Falls Short#

The team also investigated whether simply giving models access to the Bluebook rules—a technique called retrieval-augmented generation (RAG)—could solve the problem. The answer: only modestly. This finding challenges a common assumption that providing rules directly to language models is sufficient to enforce compliance, suggesting the limits of pure retrieval-based approaches.

A Hybrid Solution: Neuro-Symbolic Systems#

The researchers then developed a different approach that meaningfully changed the outcome. Rather than asking the model to produce formatted citations directly, they created a neuro-symbolic system that splits the task: a language model parses natural language descriptions into structured citation components, then hands off the actual formatting to a deterministic rule engine. This hybrid architecture achieved an average accuracy boost of 32.4 percentage points, reaching up to 85.5% overall accuracy on the benchmark.

Implications for Legal AI's Future#

The findings suggest the legal tech field should rethink its strategy. The straightforward promise—that AI alone can automate routine legal work—remains out of reach, even for frontier models. However, combining neural networks with symbolic systems offers a more realistic path forward, demonstrating that the most effective automation may require human-designed logic working in tandem with learned intelligence.