AI
Apple Launches Core AI Framework for On-Device Generative AI
Apple's new Core AI framework enables developers to run large language models and generative AI entirely on Apple Silicon devices while preserving user privacy.
InfoQ
At WWDC 26, Apple unveiled Core AI, a new framework designed as the official successor to Core ML that allows developers to deploy generative AI models directly on device without cloud dependencies.
The framework supports both custom PyTorch models and pre-optimized open-source models, scaling from compact vision models with 3 billion parameters up to large reasoning models with 70 billion parameters. It runs natively on iPhone, iPad, Mac, and Apple Vision Pro.
Key Features#
Core AI underpins Apple Intelligence and brings several technical capabilities to developers:
- Unified hardware access: Workloads can seamlessly distribute across CPU, GPU, and Neural Engine through a single API
- Memory-safe Swift API: Enables zero-copy data paths and fine-grained control over inference memory
- Ahead-of-time compilation: Shifts computation work before deployment, enabling near-instant model load times
The framework ensures zero server dependencies, zero per-token cloud costs, and keeps all user data on-device—addressing privacy concerns around cloud-based AI inference.
Model Conversion and Optimization#
Developers can convert PyTorch models to Core AI format using TorchConverter(). The process includes critical optimization steps like quantization and palettization, which compress models for efficient execution on Apple hardware while reducing memory footprint, inference latency, and power consumption.
When a model first loads, it automatically specializes to the current hardware and OS version. This initial load may take longer, but subsequent runs access the cached, optimized version.
The AI Stack Going Forward#
Apple is maintaining three distinct approaches for machine learning on its platforms: Core ML for classical ML tasks like decision trees, Core AI for neural networks and transformers, and MLX Swift for custom model weights. Community feedback suggests Core AI's long-term value depends on ecosystem growth and future platform development.
