Framework Generates Realistic Long-Context Training Data by Simulating Evolving Information States

Language models often struggle when facts and conclusions shift over time during extended interactions. A new framework called EvolveScaler addresses this by treating information evolution—where later events contradict or update earlier claims—as a first-class synthesis problem.

The Challenge of Evolving Information#

In real-world contexts like contract negotiations, medical histories, or chat logs, information rarely remains static. Earlier statements may be retracted, clarified, or overridden by subsequent events. Successfully reasoning through such scenarios requires models to track valid records, apply updates in sequence, and reconstruct what actually remains true at query time. Traditional natural-language training data obscures these transitions, making it hard to verify whether a model is truly following the logic or simply memorizing surface patterns.

Code-First Design for Clarity#

Rather than generating text and hoping the reasoning works, EvolveScaler flips the workflow. Researchers first write formal operational specifications—human-authored rules that define state transitions, record validity, difficulty levels, and the executable logic needed to compute correct answers. A strong language model then synthesizes a working simulator from each spec. The simulator runs through evolving event sequences, generates the natural-language narrative, and deterministically computes reference answers and verification checklists. This approach makes the underlying reasoning transparent and auditable.

Scale and Performance#

The team built 117 task prototypes with 159 different question operators across five difficulty tiers, generating roughly 35,100 training examples and 585 validated test cases. Scenarios range from 7 to 1,200 events, creating genuine long-context challenges. On the hardest tier, the best model achieved just 59.3% accuracy at average@5 rank, while six others fell below 10%—signaling a real capability gap.

Training Transfer#

When researchers fine-tuned an internal model on 6,000 EvolveScaler examples, it improved across all eight independently built out-of-distribution benchmarks by an average of 5.25 percentage points relative to its base checkpoint. This transfer suggests the framework produces supervision that generalizes beyond its own data distribution.