AI
New technique helps language models remember facts as they learn new information
Researchers propose a mixing strategy that significantly reduces forgetting when fine-tuning language models on successive waves of new knowledge.
Language models trained on fixed datasets risk becoming outdated as new information emerges. Standard fine-tuning approaches—retraining on fresh knowledge—often backfire, causing models to either fail to absorb the new facts or amplify existing hallucination problems.
A team of researchers has tackled this challenge by studying what they call continual memorization: the ability of a model to learn and hold onto factual knowledge across multiple rounds of fine-tuning. Their findings reveal a consistent problem: models struggle to retain earlier facts when trained on subsequent datasets, with particularly severe forgetting occurring in the second training phase.
The Forgetting Problem#
When language models undergo multiple stages of fine-tuning on new datasets, they tend to overwrite or dilute previously memorized information. This pattern mirrors the well-known catastrophic forgetting challenge in machine learning, but the researchers document how acute this problem becomes specifically for factual recall.
A Simple Solution: Strategic Mixing#
Rather than complex replay mechanisms or specialized continual learning algorithms, the team discovered an elegant approach they call REMIX (Random and Generic Data Mixing). The method involves inserting two types of neutral material during training:
- Randomly generated word sequences
- Generic text sampled from the original pretraining corpus
By sprinkling these benign examples throughout subsequent training stages, the model maintains its grip on earlier facts while still absorbing new ones. This modest intervention outperforms more elaborate replay-based methods and other continual learning baselines, even recovering performance from severe forgetting scenarios.
How Robust Memorization Works#
Analysis of how REMIX affects neural processing reveals an interesting mechanism: models trained this way store factoids in earlier network layers than usual, and distribute them across more diverse layers overall. This distributed, earlier-layer storage makes facts more resilient and easier to retrieve—suggesting that the way information is positioned within the network matters as much as its presence.
