New Fine-Tuning Method Hardens Code Summarization Against Poorly Written Code

Code summarization—the task of automatically generating human-readable descriptions of code—remains a crucial part of program comprehension, but current language models struggle when code quality deteriorates. A new study reveals that even state-of-the-art systems like GPT-4o and DeepSeek-V3 experience sharp performance drops when dealing with obfuscated or poorly named variables and functions.

The Robustness Problem#

Existing code language models are trained primarily on well-structured codebases where naming conventions are clear and semantic clues are abundant. However, real-world code often lacks these niceties. Researchers tested leading models on deliberately degraded code—where function names, variable names, and identifiers were obscured—and found their effectiveness plummeted significantly. Attempts to improve performance through prompt engineering and reasoning-enhanced variants yielded only modest gains, confirming that the issue runs deeper than instruction design.

RoFTCodeSum: A Curriculum Learning Approach#

To address this vulnerability, the team developed RoFTCodeSum, a fine-tuning method that combines curriculum learning with meta-learning. Rather than training on static datasets, the approach creates progressively harder training sets by systematically obfuscating code elements—first function names, then identifiers—simulating real-world degradation.

The key innovation lies in meta-updating gradients across these challenging datasets in each training step. This dual-focus strategy optimizes models for both accuracy on clean code and robustness against semantic perturbation, a critical gap in current approaches.

Results and Implications#

Experiments show that RoFTCodeSum strengthens model resilience to code readability problems while maintaining or improving performance on well-written code. The findings underscore the importance of training AI systems not just on ideal scenarios but on the messy reality they'll encounter in production, where code quality varies widely and legacy systems abound.