New Benchmark Tackles Probabilistic Reasoning Over Tabular Data with Bayesian Networks

Most question-answering systems over tables excel at retrieving single facts—but real-world scenarios often require probabilistic thinking and reasoning about hidden relationships. A new paper addresses this gap by introducing tools for handling uncertainty and latent dependencies in tabular data.

The Problem with Factual QA Over Tables#

Existing question-answering methods work well when the answer lives directly in a table cell. However, many practical questions involve probability and conditional reasoning: What is the likelihood that a customer will churn given certain characteristics? or How does one variable influence another? These questions require inferring patterns that aren't explicitly stored, posing a significant challenge to current systems.

Introducing LUCARIO and Auto-BN#

The researchers propose LUCARIO, a comprehensive benchmark dataset spanning multiple domains and question types, designed to evaluate probabilistic QA over real-world tables. Alongside this, they introduce Auto-BN, a fully automatic system that:

• Extracts a Bayesian Network directly from raw tabular data
• Translates natural-language questions into formal probabilistic queries
• Performs exact statistical inference to generate answers

This neuro-symbolic approach combines neural language processing with symbolic probabilistic reasoning, bridging two computational paradigms.

Performance and Impact#

Testing across various large language model backbones showed that Auto-BN consistently outperformed conventional baselines, including NL2SQL systems, retrieval-only methods, and premise-based approaches. Importantly, the method maintained a near-zero error rate, demonstrating both accuracy and reliability in probabilistic scenarios.

The work fills a meaningful void in the QA landscape by providing researchers with a practical, scalable foundation for advancing probabilistic reasoning over structured data. This opens new possibilities for applications requiring nuanced statistical inference from tables.