New computational method uses information theory to spot repetitive patterns in ancient texts

Texts throughout history—whether sacred, literary, or archival—bear the fingerprints of their creators and the communities that shaped them. Repetitive, constrained writing (what scholars call formulaic text) typically contains less information variation than more creative compositions, offering clues about authorship, origin, and how texts were transmitted across centuries.

Researchers from arXiv have developed a new computational approach to uncover these hidden stylistic layers. Rather than relying on human judgment or predefined categories, their method uses information theory—specifically patterns in how much "surprise" or structure each phrase carries—to automatically identify sections of text that share similar repetitive characteristics.

The Challenge of Finding Formulaic Patterns#

Distinguishing between formulaic (highly repetitive, tightly structured) and non-formulaic writing poses a significant challenge in natural language processing, especially for ancient documents like the Hebrew Bible. These texts were often composed and edited across centuries by multiple hands, leaving no labels or metadata to guide analysis. Traditional supervised learning approaches fail because no training data exists.

The researchers needed a method that could work without predefined examples—one that would identify clusters based purely on intrinsic textual properties.

How the Algorithm Works#

The team built their approach on self-information, a concept from Shannon information theory that measures how much novelty or constraint appears in a given piece of text. The method extends this classical discrete measure to a continuous version based on differential self-information in multivariate Gaussian distributions.

This extension allows the algorithm to:

• Handle various types of textual data, from traditional word-level representations to modern neural embeddings
• Operate across high-dimensional, data-sparse datasets common in historical linguistics
• Recover structured partitions in text by analyzing weighted self-information distributions

The resulting clusters are then interpreted by examining their information profiles and characteristic recurring phrases.

Applications to Ancient Texts#

When applied to the Hebrew Bible—a document whose authorial divisions scholars have long debated—the method successfully isolated distinct stylistic layers. This provides quantitative evidence for how complex texts evolved through multiple compositions and editorial interventions.