Diffusion Models Struggle to Find Where Code Edits Should Go
Masked diffusion language models can generate code changes well when given exact edit locations, but falter when forced to identify those locations themselves.
Essential cookies keep the site working and are always on. If you allow analytics, we also record sessions of visitors and of members aged 18 or over across the site, including messaging and profile pages: clicks, scrolls and page layout. Everything you write or upload, names, profile photos and images are masked in your browser before anything is sent, so the text you type never leaves it unmasked. Recordings are not used for advertising. Ad measurement is a separate choice. You can withdraw any time in settings. Cookie policy
Masked diffusion language models can generate code changes well when given exact edit locations, but falter when forced to identify those locations themselves.
A new study reveals that techniques for efficiently allocating model capacity that work well during supervised fine-tuning fail when applied to reinforcement learning optimization.
New research reveals that small language models commit to outputs early in processing and rarely correct course—a phenomenon called prolepsis that repeats across diverse tasks.