Researchers Automate Creation of Business Semantics from Raw Application Logs

Modern software systems produce vast streams of raw telemetry events, yet turning that noisy data into useful business intelligence typically demands painstaking manual work. Engineers spend countless hours parsing logs, reconciling inconsistencies, and stitching together connections between low-level data and high-level business metrics.

A Two-Stage Abstraction Approach#

Researchers have developed an automated framework that builds what's known as a semantic layer—a structured mapping between raw data and business concepts—without human intervention. The system works in two stages. First, it identifies high-level business features using large language models informed by domain-specific industry context. Then, it generates finer-grained business entities through a multi-step process: cleaning the data, performing hybrid retrieval, filtering at multiple stages, grouping similar concepts semantically, and assigning standardized names.

Real-World Results#

Testing on production-scale telemetry showed striking improvements. The framework raised human-assessed semantic quality from around 50 to over 80 points on a 100-point scale. It also cut maintenance burden by 80% and removed 74% of noise from the datasets. A machine evaluation metric (Cohen's kappa) reached 0.87, demonstrating the system's consistency and reliability.

Key Advantage Over Existing Methods#

The approach stands out because it requires neither labeled training data nor hand-coded rules—both typically essential for traditional data engineering pipelines. This enables continuous, scalable quality assurance as systems and business needs evolve. The full paper appeared on arXiv in September 2026.