New Framework Makes AI Medical Billing Checks Transparent and Auditable

Researchers have developed a proof-of-concept system that pairs large language models with human-readable decision rules to verify medical billing claims in a way that auditors can fully trace and understand.

The Core Innovation#

The approach rests on a "decision-basis contract"—a framework that cleanly separates two kinds of work. Deterministic rules handle the mechanical tasks: confirming which billing catalog applies, checking whether a code exists, verifying quantity limits, and identifying exclusions. Meanwhile, LLM-based semantic assessment tackles the harder problem: reading unstructured clinical notes to decide whether a claimed service is actually supported by the documentation, contradicted by it, or missing evidence entirely.

How It Works#

The system requires that whenever an AI model flags a billing claim as "supported" or "contradicted," it must cite a verbatim span of evidence from the source documentation. If the model cannot point to explicit text backing its judgment—or if the underlying rule context is unavailable—the system defaults to abstention. This fail-closed posture means the system errs on the side of flagging uncertain cases rather than silently approving them.

Researchers tested this approach on four open-weight models using 36 hand-curated test cases. They ran three versions: the full contract-based system, a variant without explicit documentation requirements, and a simpler end-to-end baseline.

What the Results Show#

Performance varied considerably across the four models tested, and none showed a clear edge over the baseline approach. However, requiring explicit documentation spans did improve all four models' ability to spot cases where required information was missing—a crucial safeguard in billing verification. The evidence gate also revealed situations where a model's raw judgment was technically correct but lacked valid supporting text; the system caught these and marked them as incomplete rather than approved.

Path Forward#

The authors emphasize that their proof-of-concept worked on synthetic data and curated examples. Real-world validation will require testing against actual billing catalogs, independently reviewed documentation, and feedback from human reviewers to determine whether the framework delivers practical value in clinical settings.