LLMs show promise as source of probabilistic knowledge for Bayesian Networks

Large language models appear capable of extracting probabilistic knowledge that can help build Bayesian Networks—formal systems for reasoning about uncertainty across domains ranging from healthcare to finance. A new study published on arXiv suggests LLMs can generate credible probability estimates that rival expert judgment when used to initialize these statistical models.

The Research Question#

Bayesian Networks require prior probability estimates to function effectively, especially when training data is limited or unavailable. Researchers at the intersection of AI and probabilistic modeling have long relied on human experts to provide these numbers. The question becomes: can foundation models like GPT offer a scalable alternative? The authors set out to test whether querying LLMs directly for conditional probability estimates—"What is the likelihood that event A occurs given event B?"—yields meaningful results.

How the Experiment Worked#

The team evaluated their approach across eighty publicly available Bayesian Networks spanning healthcare, finance, and other domains. Rather than relying on conventional methods like random assignment or uniform distributions, they constructed queries designed to elicit probabilistic reasoning from LLMs. They compared the resulting estimates against baseline approaches and measured how well these LLM-derived distributions could serve as expert priors to refine distributions learned from real-world data.

Key Findings#

The experiments revealed that LLM-generated probability distributions outperformed simpler baselines while remaining comparable to other knowledge-extraction techniques, including those based on next-token generation probabilities. Perhaps more importantly, the LLM-derived estimates proved useful as starting points for refining models when observational data was scarce—a common challenge in applied statistics.

Implications#

This work establishes an initial benchmark for evaluating LLM performance in probabilistic reasoning tasks. By combining probabilistic knowledge extracted from language models with real-world observations, researchers can more readily automate the construction of Bayesian Networks. The approach may prove especially valuable in domains where expertise is concentrated or data collection remains expensive.