WhatsApp Detects Scam Messages with On-Device AI

Meta's messaging app WhatsApp has launched a limited beta test for a new feature called 'Scam Alert' that automatically detects fraudulent messages. The feature operates by running machine learning on the device itself to assess the likelihood of fraud in messages from contacts not in the user's address book.

In recent years, fraud losses through messaging apps have increased globally, and handling suspicious messages from unknown senders has become a major challenge for platforms. WhatsApp, meanwhile, places end-to-end encryption—a form of encryption where only sender and recipient can read messages—at the core of its service. An approach to fraud prevention that sends message content to servers could directly conflict with its privacy policy. Scam Alert represents a technical answer to this dilemma.

In Meta's adopted design, message content never leaves the user's device. All assessment processing is completed on the device, and the actual message content is not transmitted to servers. Meta explains that it combines multiple privacy-protection technologies to measure feature accuracy and protect model distribution.

Four technologies are specifically being employed. First is 'confidential computing,' which protects the environment in which data is processed at the hardware level. Second is 'Oblivious HTTP,' a method that relays requests so that the sender and receiver of communication cannot identify each other. Third is 'differential privacy,' a technique that adds intentional noise to data so that individuals cannot be identified from statistical aggregation results. Fourth is 'model transparency,' which aims to ensure the trustworthiness of the AI model itself used for detection.

The significance of this effort extends beyond technical novelty. 'Achieving both privacy and safety' has long been a stated goal, but Meta's approach on WhatsApp—a massive platform—is notable for its attempt to implement this balance in practice. The design of completing assessment on-device offers the advantage of structurally reducing the risk of message content being exposed even if a server were to be compromised.

At present, the feature remains in limited beta and is not available to all users. Regarding future full deployment and feature details, we must await further announcements from Meta. Moreover, operational questions such as how accurately the system can actually detect fraudulent messages and how often false positives (incorrectly flagging non-fraudulent messages as fraud) occur will only become clear once the feature is deployed more widely.

The move by major platforms to combine on-device AI with privacy-protection technologies points to one direction for designing safety features in messaging apps. How other messaging services and apps facing similar privacy constraints respond, and whether this approach influences the broader industry trajectory, warrant close attention.