New analytic method sidesteps backpropagation for continual learning, cuts energy use by 20×

A team has introduced CIFNet, an alternative approach to class-incremental learning in neural networks that replaces iterative backpropagation with a direct mathematical solution. Rather than repeatedly adjusting weights through gradient descent, the method solves for classifier updates using a closed-form ridge-regression approach operating on frozen pre-trained feature representations.

Why conventional approaches are costly#

Class-incremental learning—teaching a network to recognize new object categories as they arrive—typically demands repeated optimization cycles. This process consumes significant compute, introduces hyperparameter tuning challenges, and often causes catastrophic forgetting, where the model loses performance on previously learned classes when updated for new ones. Energy consumption and training time are practical bottlenecks in production continual-learning systems.

How CIFNet works#

CIFNet builds on Regularised Recursive Least-Squares (RRLS) to update classifier weights deterministically in a stationary embedding space. When new output neurons are added for novel classes, a lightweight calibration buffer and density-aware resampling strategy prevents initialization bias and keeps decision boundaries globally balanced—all without storing raw images or running gradient updates.

Measured results#

Benchmarks across CIFAR-100, ImageNet-100, and CORe50 show CIFNet matches the predictive accuracy of conventional iterative methods while delivering strictly smooth learning trajectories free from intermediate performance collapse. The most striking gain: up to 20× reduction in energy consumption by replacing per-epoch backpropagation with closed-form moment accumulation.

Implications#

These findings suggest that efficient, mathematically grounded continual learning is possible when leveraging frozen representations—opening a path toward low-energy, stable adaptation in deployed neural networks.