New System Cocoon Accelerates Privacy-Preserving Machine Learning Training

Protecting sensitive data during machine learning training remains a critical challenge. While differential privacy techniques add noise to shield individual records from being reconstructed, they come at a steep cost: models trained this way suffer measurable accuracy degradation, slowing mainstream adoption in industry.

Researchers have recently explored a more sophisticated approach—introducing correlated noise carefully timed across training iterations so that noise patterns cancel each other out rather than compound. But implementing these schemes efficiently proves complex, especially when models grow large or use substantial embedding tables that strain available hardware resources.

The Problem with Scale#

Recent work characterizing these noise-cancellation approaches uncovered a hidden overhead: as models and embedding tables expand, the system must manage vast noise histories that exceed typical GPU memory. This forces expensive spillover to system RAM or other slow storage, undermining the efficiency gains the technique promises.

Cocoon's Multi-Layer Strategy#

To solve this, a team developed Cocoon, a framework that distributes noise data management across multiple tiers of the computing stack. The approach:

• Leverages CPU, GPU, and memory extension modules to store noise histories in a hierarchy matched to hardware capabilities
• Introduces optimizations tailored for sparse embedding tables, which waste less space on zeros
• Incorporates emerging near-memory processing (NMP) devices—novel hardware accelerators positioned between main memory and compute cores—to offload noise-intensive operations closer to where data lives

Real-World Gains#

In tests on a prototype system featuring an FPGA-based NMP device, Cocoon delivered 1.23× to 10.82× performance improvements depending on model size and configuration. The results suggest that with proper system-level design, privacy-preserving training needn't sacrifice efficiency or usability.

The work, available on arXiv, points toward a path where organizations can train with strong privacy guarantees without accepting today's accuracy penalties.