Withdrawn Paper Proposes Two-Stage Training to Boost Language Model Robustness and Generalization

A withdrawn arXiv submission introduced a framework aimed at tackling a persistent problem in neural language models: balancing generalization (performing well on unseen data) with robustness (resisting adversarial attacks). Most existing work improves one property or the other, but rarely both simultaneously.

The Proposed Method#

The authors, led by Sun Yudao, presented UEGR—a bi-stage optimization scheme designed to strengthen language models across both dimensions. The approach operates in two phases:

Forward propagation stage: The method uses adaptive dropout to create multiple variations of the model, generating diverse predictions on adversarial samples. It then combines JS divergence (a statistical measure of distribution similarity) and adversarial losses to stabilize the model's outputs and make predictions less vulnerable to manipulation.

Backward propagation stage: Rather than updating all parameters equally, the framework computes parameter saliency scores to identify which weights matter most. Only these critical parameters are updated, reducing unnecessary drift in the model while reinforcing its ability to resist perturbations.

Theoretical Rationale#

According to the authors' analysis, the framework incorporates gradient regularization to reduce how much the model responds to small input changes, while selective updates help flatten the loss landscape—both features known to improve generalization and robustness.

Experimental Claims#

The research reportedly tested UEGR on 13 public language datasets and claimed state-of-the-art results, outperforming existing approaches at the time of submission in March 2025.

Note: The paper was withdrawn in June 2026, and no PDF is currently available.