New Solvers Enable Reversible Neural Differential Equations for Precise AI Model Inversion

Deep generative models powered by neural differential equations have emerged as leading tools for creating synthetic data, but they face a critical limitation: the numerical solvers that drive these models tend to accumulate small errors when running backward from data to the prior distribution, making perfect reconstruction nearly impossible.

The Core Problem#

Generative models like diffusion and flow-based systems typically integrate forward from a simple starting distribution to match real data. In many practical applications — including image editing, sampling refinement, and precision-critical tasks — the ability to reverse this process and integrate backward is essential. Standard ODE and SDE solvers, however, introduce discretization errors that compound with each step, making exact inversion infeasible. Prior inversion techniques have been plagued by instability, slow convergence rates, and restriction to ODE-only settings, leaving a significant gap in the field.

Introducing Rex#

A new framework called Rex (Reversible Exponential Runge-Kutta Solvers) addresses this challenge by applying Lawson methods to transform any explicit Runge-Kutta scheme — whether for ordinary or stochastic differential equations — into an algebraically reversible solver. This approach works for both deterministic diffusion ODEs and full stochastic SDEs, broadening applicability beyond what existing methods support.

Key Advantages#

The method offers several concrete improvements:

• Near-machine-precision reconstruction: The solvers achieve accuracy approaching the limits of floating-point computation, eliminating the precision loss that has constrained earlier work
• Theoretical rigor: The framework establishes convergence guarantees at arbitrary orders of accuracy and identifies stable operating regions
• Broad applicability: Results demonstrate gains across multiple domains, including improved Boltzmann sampling with flow models and enhanced image generation and editing with diffusion models

This advancement removes a longstanding barrier to deploying generative AI systems where bidirectional integration and reconstruction fidelity are non-negotiable.