AI
New self-training method lets multiple AI models improve each other without external feedback
A collaborative framework called Peer-Predictive Self-Training enables language models to refine themselves by learning from aggregated responses across multiple models.
Researchers have introduced a novel approach to help language models get better at reasoning tasks without needing human labels or external supervision. The technique, called Peer-Predictive Self-Training (PST), works by having multiple language models work together as a group, using their combined wisdom to teach each one how to improve.
How the approach works#
The method operates through a straightforward collaboration process. When given a prompt, several models generate answers in sequence. Rather than treating each response in isolation, the framework aggregates all the answers to produce a final composite response. This collective answer typically proves more reliable than what any single model produces alone, and becomes the learning target for refining each model's individual responses.
To decide how much each model should learn from its own answer, the system measures how informative that response is relative to the group consensus using pointwise mutual information (PMI). Models whose answers already align closely with the aggregate receive smaller training adjustments, while responses that diverge or add less value get larger updates. This weighting scheme ensures that learning focuses on areas where individual models most need improvement.
Performance gains#
Testing on mathematical reasoning benchmarks—including SimulEq, MATH-500-Numeric, and MultiArith—showed tangible improvements. The approach boosted exact-match accuracy by 2.2 to 4.3 percentage points across three smaller models: Gemma-2-2B, LLaMA-3.2-1B, and Qwen2.5-1.5B. Additionally, it narrowed the gap between generation and verification performance (known as the generator–verifier gap) by 26 to 40%, a meaningful reduction in model inconsistency.
Key advantages#
What distinguishes PST is its independence from external infrastructure. The method requires no teacher model, no labeled data, and no fixed hierarchy between models—just cross-model interactions. This makes it a practical self-improvement mechanism that leverages the natural diversity of responses that multiple models naturally produce when solving the same problem.
