New module brings multiplicative operations to deep vision networks without instability

Deep learning models for computer vision have long relied on additive transformations, but a new approach proposes bringing multiplicative interactions into the picture. Researchers have developed PURe (Product-Unit Residual Module), a modular component designed to slot into existing residual neural networks without requiring architectural overhauls.

The Problem with Product Units#

Product units—which multiply features together rather than adding them—offer potential for capturing more complex interactions between inputs. However, incorporating them into deep networks has proven challenging due to numerical instability during training. This limitation has kept them largely unexplored in modern computer vision architectures, even though they represent a fundamentally different way to process visual information.

How PURe Works#

The core innovation lies in a mathematical reformulation. Rather than performing raw multiplication, PURe uses a log-domain approach with real-valued parameters that keeps training stable across many layers. This allows product units to function as drop-in replacements for standard residual blocks—the building blocks of modern deep networks.

Proven Performance Across Benchmarks#

Testing on standard image classification datasets showed consistent gains. On Galaxy10 DECaLS, ImageNet, and CIFAR-10, networks using PURe achieved better accuracy while using fewer parameters than conventional ResNet baselines. Moderately deep models incorporating the module matched or exceeded much deeper standard networks.

The approach also proved effective beyond classification. In volumetric medical imaging, PURe improved slice-based CT segmentation performance on the AMOS benchmark when evaluated using 3D case-level metrics.

Implications for Vision Architecture Design#

The results suggest that explicit multiplicative interactions deserve renewed attention as a design principle for vision networks. Rather than replacing existing methods, PURe's modularity means practitioners can experiment with multiplicative operations within established training pipelines. This opens a path toward exploring whether neural networks have overlooked an effective way to model feature interactions.