At WWDC on Monday, Apple unveiled Apple Intelligence, a set of features providing generative AI tools like rewriting a draft email, summarizing notifications and creating custom emojis for iPhone, iPad and Mac. Apple spent a significant portion of its keynote explaining how useful the tools will be — and almost as much time assuring customers just how confidential the new AI system that keeps your data safe is.
This privacy is possible thanks to a dual approach to generative AI, which Apple began to explain in its keynote and offered more details in documents and presentations since. They show that Apple Intelligence is built with a device philosophy that can perform common AI tasks that users want quickly, such as transcribing conversations and organizing their schedules. However, Apple Intelligence can also reach cloud servers for more complex AI queries that involve sending personal contextual data — and making sure that both delivers good results while keeping your data private is where Apple is focusing its efforts.
The big news is that Apple is using its own AI models for Apple Intelligence. Apple notes that it does not train its models with personal data or user interactions, which is unique compared to other companies. Instead, Apple uses both licensed material and publicly available online data, which is mined by the company’s Applebot web robot. Publishers must opt out if they don’t want their data swallowed by Apple, which sounds similar to Google and OpenAI’s rules. Apple also says it omits to provide Social Security and credit card numbers that are distributed online, and ignores “profanity and other low-quality content.”
A big selling point for Apple Intelligence is its deep integration into Apple’s operating systems and applications, and how the company has optimized its power efficiency and size models to fit the iPhone. Keeping AI requests local is key to eliminating many privacy issues, but the tradeoff is using smaller and less capable device models.
To make these local models useful, Apple uses fine-tuning that trains the models to make them better at specific tasks like proofreading or summarizing text. Skills are placed in the form of “adapters” that can be placed on top of the base model and swapped out for the current task, similar to applying buff attributes to your character in an RPG. Similarly, Apple’s diffusion model for Image Playground and Genmoji also uses adapters to get different art styles like illustration or animation (making people and pets look like cheap Pixar characters).
Apple says it has optimized its models to speed up the time between sending a prompt and delivering a response, and uses techniques such as “speculative decoding,” “context pruning” and “bulk request attention” to take advantage of Neural Apple’s Silicon Engine. Chip manufacturers have only recently started adding neural cores (NPUs) to the die, which helps relieve CPU and GPU bandwidth when processing machine learning and AI algorithms. This is part of the reason that only Macs and iPads with M-series chips and only the iPhone 15 Pro and Pro Max support Apple Intelligence.
The approach is similar to what we’re seeing in the Windows world: Intel released its 14th generation Meteor Lake architecture featuring an NPU chip, and Qualcomm’s new Snapdragon X chips built for Microsoft’s Copilot Plus PCs also have them. As a result, many Windows AI features are tied to new devices that can perform work locally on those chips.
According to Apple’s research, out of 750 text summarization responses tested, Apple’s on-device AI (with the appropriate adapter) performed more attractively to humans than Microsoft’s Phi-3-mini model. It sounds like a great achievement, but most chatbot services today use much larger models in the cloud to achieve better results, and this is where Apple tries to walk a careful line on privacy. In order for Apple to compete with larger models, it devised a seamless process that sends complex requests to cloud servers while trying to prove to users that their data remains private.
If a user request needs a more capable AI model, Apple sends the request to its Private Cloud Compute (PCC) servers. PCC runs on its own operating system based on “iOS fundamentals” and has its own machine learning stack that powers Apple Intelligence. According to Apple, PCC has its own secure boot and Secure Enclave to store encryption keys that only work with the requesting device, and the Trusted Execution Monitor ensures that only signed and verified code is executed.
Apple says the user’s device establishes an end-to-end encrypted connection to the PCC cluster before sending the request. Apple says it can’t access data in PCC because it lacks server management tools, so it doesn’t have a remote shell. Apple also does not provide PCC with any persistent storage, so the queries and possible personal contextual data extracted from the Apple Intelligence Semantic Index are apparently deleted in the cloud afterwards.
Each PCC build will have a virtual build that the public or researchers can inspect, and only signed builds that are registered as inspected will go into production.
One of the big open questions is exactly what types of requests will go to the cloud. When processing a request, Apple Intelligence has a step called Orchestration where it decides whether to proceed on the device or use PCC. We don’t yet know exactly what constitutes a complex enough query to trigger a cloud process, and we likely won’t know until Apple Intelligence becomes available in the fall.
There’s another way Apple deals with privacy concerns: by making it someone else’s problem. Apple’s updated Siri can send some queries to ChatGPT in the cloud, but only with permission after you ask some really tough questions. This process puts the issue of privacy in the hands of OpenAI, which has its own policies, and the user, who must agree to offload their request. In an interview with Marques Brownlee, Apple CEO Tim Cook said ChatGPT will be used for requests involving “world knowledge” that are “outside the domain of personal context.”
Apple’s on-premises/cloud-division approach to Apple Intelligence isn’t entirely new. Google has a Gemini Nano model that can run locally on Android devices alongside its Pro and Flash models that process in the cloud. Meanwhile, Microsoft’s Copilot Plus PCs can handle AI requests locally, while the company continues to rely on its OpenAI deal and also builds its own in-house MAI-1 model. However, none of Apple’s rivals have emphasized their privacy commitments as thoroughly by comparison.
Of course, this all looks great in step-by-step demos and edited documents. However, the real test will be later this year when we see Apple Intelligence in action. We’ll have to see if Apple can manage to strike that balance between quality AI experiences and privacy — and continue to develop it in the coming years.
