New neural network training method replaces backpropagation with a second forward pass

Researchers have introduced Forward Target Propagation (FTP), a novel training approach that sidesteps the backward pass entirely, addressing fundamental limitations inherent in the backpropagation algorithm that has dominated deep learning for decades.

The case for an alternative#

Traditional backpropagation trains neural networks through gradient calculations flowing backward through the network. Despite its effectiveness, this method faces several constraints: it requires symmetric feedback weights that may not reflect biological reality, assigns credit to neurons in ways that lack local interpretability, and freezes neuron activity during the backward phase. These limitations create barriers for both biologically inspired computing and efficient hardware implementations.

How Forward Target Propagation works#

FTP replaces the backward pass with an additional forward pass through the network. Rather than computing gradients, the method estimates target values for each layer using only feedforward calculations. This approach eliminates the need for symmetric feedback weights or learnable inverse functions, enabling learning to operate at each layer independently without global coordination.

Experimental validation#

The team tested FTP across multiple architectures—fully connected networks, convolutional neural networks, and recurrent networks—on standard benchmarks including MNIST, CIFAR-10, and CIFAR-100. The method achieved accuracies competitive with backpropagation while handling sequential tasks that require modeling long-term dependencies. Notably, FTP demonstrated superior performance compared to backpropagation when operating under quantized, low-precision constraints and on emerging hardware platforms.

Implications for efficient computing#

With minimal computational overhead and a forward-only structure, FTP shows promise for energy-efficient on-device learning and neuromorphic computing applications. The method substantially outperformed other biologically inspired alternatives, positioning it as a practical direction for systems where traditional gradient-based training becomes impractical.