Rethinking Token Selection in Diffusion Transformers Reveals Better Acceleration Strategy

Diffusion Transformers (DiT) power modern image and video generation tools, but their computational demands remain substantial. To speed them up, researchers have turned to feature caching—a technique that stores computed features from one timestep and reuses them in the next, bypassing redundant calculations.

A common refinement to this approach has been token-wise caching: different parts of the computation (tokens) receive different treatment, with supposedly "important" tokens being computed fresh while less important ones get cached and skipped. Yet a new study challenges the core assumptions behind this strategy.

The Unexpected Findings#

The researchers posed two seemingly straightforward questions: Is it truly necessary to compute important tokens at every step? And are the tokens deemed "important" actually valuable?

The answers proved counterintuitive. The study found that consistently recomputing selected important tokens across all steps is not necessary for good performance. More surprisingly, the method used to identify important tokens often fails to work as intended—and sometimes performs worse than simply choosing tokens at random.

Introducing Dual Feature Caching#

Building on these insights, the team proposes DuCa (Dual Feature Caching), which alternates between aggressive and conservative caching strategies while randomly selecting which tokens to compute rather than relying on predetermined importance metrics.

Early experiments across multiple models—including DiT, PixArt, FLUX, and OpenSora—show that DuCa achieves faster inference than previous token-wise caching methods, suggesting that the simpler, randomized approach may offer a better balance between speed and quality.