Sound on-device: How Arm CPUs power real-time AI audio generation

A software engineer and music producer has developed a real-time sound generation app that runs entirely on-device using Arm-based CPU technology—no cloud infrastructure, no GPU required. The tool pairs Stability AI's Stable Audio Open model with PyTorch and TorchAudio to generate studio-ready audio samples from natural-language prompts in seconds.

Workflow Without Interruption#

The motivation was simple: avoid breaking creative flow. While working in Ableton Live, a producer can now enter a text prompt—"analog bassline," "cinematic riser," "lofi snare"—and receive a generated .wav file within seconds, ready to import, edit, or loop. The approach generates unique outputs each time, avoiding library repetition.

On-Device Performance#

Running on Arm CPU delivers three key benefits:

• No latency — inference happens locally, not via cloud
• Privacy — outputs remain on the user's device
• Efficiency — Arm's performance-per-watt keeps the system responsive even during multi-step diffusion generation

The implementation optimizes for speed: the code uses reduced diffusion steps (7 instead of many more), enables full thread utilization across Arm cores, and clears memory periodically. Device flexibility is built in—the same pipeline can target Apple Silicon (Metal), NVIDIA (CUDA), or standard Arm CPUs via a single parameter.

Implications for Creators#

As a prototype, the project hints at a broader shift toward edge-based generative AI. With efficient on-device inference, artists and developers can keep tools in their creative hands, avoid data transit overhead, and integrate AI into DAWs and specialized hardware without reliance on external services.