New Loss Functions Unlock Better Training for Generative Flow Networks

Generative Flow Networks (GFlowNets) are a class of machine learning models built to sample from complex, unnormalized probability distributions. A new study from researchers on arXiv examines how the choice of loss function—the mathematical objective that guides training—can significantly shape model behavior and performance.

The Problem with Standard Loss Functions#

Training GFlowNets typically involves aligning the forward flow (the model's probability distribution) with the backward flow (rewards assigned to sampled outputs). Most existing approaches rely on minimizing squared error in log-space, a conventional choice that has gone largely unexamined. While this method is intuitive, it may not be optimal for all learning scenarios, particularly when balancing the need to exploit high-reward solutions with exploration of the full solution space.

A Theoretical Framework for Loss Design#

The researchers establish a rigorous connection between regression loss functions and their corresponding divergence measures—mathematical tools that quantify the difference between two probability distributions. This framework enables principled design of new loss functions based on desired training properties, rather than relying on convention.

The team identifies two key properties that matter:

• Zero-forcing: Encourages the model to focus on high-reward outcomes, improving exploitation and reward maximization
• Zero-avoiding: Promotes broader exploration of the solution space, enhancing diversity and preventing premature convergence

Three Novel Loss Functions#

Based on this theoretical foundation, the researchers propose three new regression losses:

• Shifted-Cosh: Offers a balanced trade-off between exploitation and exploration
• Linex(1/2) and Linex(1): Provide tunable control over the exploitation-exploration spectrum

Validation Across Domains#

The proposed losses were tested on three benchmark tasks—hyper-grid optimization, bit-sequence generation, and molecular design—and showed improvements in convergence speed, sample diversity, and robustness compared to squared error baselines. Critically, these losses are compatible with most existing GFlowNet training algorithms, making them practical to integrate into current systems.