WhatsApp Trials On-Device AI to Flag Scams Without Revealing Message Content

WhatsApp is piloting Scam Alert, an optional feature powered by on-device machine learning that identifies suspicious messages arriving from non-contacts and alerts users to potential fraud schemes. The limited beta release pairs the detection capability with a privacy-centric infrastructure designed to evaluate model performance without exposing message data to WhatsApp's servers.

How the Detection Works#

Once activated, Scam Alert downloads a compact ML model onto the user's phone that analyzes incoming messages using conversational patterns and linguistic markers. The model was developed from patterns extracted from scam reports that users had previously submitted to WhatsApp. When the system identifies a likely fraudulent message, the recipient sees a warning label—invisible to the sender—and gains the option to block, flag, or continue the conversation. Users can also mark specific contacts as trusted, preventing future alerts from those senders.

Privacy by Architecture#

The system treats message content and model distribution as separate security concerns. Classification happens entirely on the device, never transmitted to servers. Separately, WhatsApp measures how well the model performs by collecting aggregated, anonymized data about warnings and user actions. This telemetry flows through an Oblivious HTTP relay using anonymous credentials and lands inside confidential virtual machines—isolated, encrypted computing environments. Before any data leaves the device, the client software verifies the privacy protections and enforces minimum group sizes and differential privacy noise to ensure individual-level insights cannot be extracted.

Model distribution itself receives comparable scrutiny. Every version, tagged with its cryptographic hash, is logged to a third-party transparency ledger before rollout. Devices confirm the ledger entry, verify the model's digital signature, and authenticate the hash before loading. Model downloads use anonymous routing, and experiment assignment—which variant a user receives—happens locally on the phone, preventing the server from steering individual users toward different models.

Foundation and Context#

The design extends Meta's PAPAYA federated analytics system, a research framework presented at USENSI NSDI 2025 that uses trusted execution environments for privacy-preserving analytics. WhatsApp's implementation adapts that approach to a consumer messaging context where telemetry itself is classified as sensitive data.

Google Messages offers a comparable feature: real-time protection against spam and phishing, also built on privacy-preserving detection and on-device processing. The two implementations diverge in which operations occur locally versus server-side.