DZ-TiDPO: A Framework for Reducing State Inertia in Long-Context Dialogue

Long-context dialogue systems often struggle with state inertia—a phenomenon where models prioritize historical context over current intents and fail to adapt as conversations evolve. A new study published on arXiv from researchers at <UNKNOWN> addresses this challenge through a novel framework called DZ-TiDPO.

The Problem#

Standard alignment methods, including Direct Preference Optimization (DPO) and recent long-context optimization techniques, fail to resolve state inertia without imposing a substantial "contextual alignment tax"—a significant rise in perplexity that stems from disrupting the model's pre-trained linguistic priors.

The Solution#

DZ-TiDPO combines two key mechanisms:

• Conflict-aware optimization applied during training to identify and resolve contradictions between historical context and current dialogue turns
• Structural temporal attention bias that decouples state updates from general language modeling

This design allows models to refresh their understanding of the conversation state while preserving overall linguistic coherence.

Key Findings#

The researchers evaluated DZ-TiDPO on two benchmarks:

• Multi-Session Chat: A standard dialogue dataset
• Inertia Challenge (IC-Bench): A newly introduced benchmark designed to stress-test state inertia

The framework supports dual inference modes—a fast "static mode" for general robustness and a precision-focused "dynamic mode" for fine-grained semantic conflicts. The analysis also reveals an important capacity-stability trade-off: mid-sized models (around 7B parameters) can efficiently internalize temporal alignment without requiring larger parameter counts.

Code and data are available from the authors.