Apple Intelligence: Apple’s Version of GenAI Is Helpful, Integrated, Private, and Jargon-Free

News coverage focused on Apple's partnership with OpenAI, but that is the least interesting and least important part of Apple Intelligence. Context awareness, the security model, and third-party app APIs look to be transformational -- and that's all Apple's technology and implementation, not ChatGPT.

Generative AI has typically been sold as a way to write things for you (sometimes making things up), generate images for you (as long as you don’t need text or human-looking fingers), or make humans obsolete. AI vendors are always planning trips, asking AI to identify objects, or removing backgrounds from video – impressive feats that don’t reliably work or have limited reach beyond niche audiences. Silicon vendors and AI hyperscalers mask whether any of this is broadly useful this with talk of how many billions or trillions of variables were used to train the model, “time to first token,” or number of TOPS (trillions of operations per second) their chips can handle.

Apple is a software and ecosystem company, and it certainly has used machine learning in its camera app and Siri, but in terms of generative AI, Apple has been relatively quiet while hype has grown around OpenAI and Nvidia. At its annual WWDC developer conference, that changed with Apple Intelligence. (Yes, Apple branded generative AI. Of course it did.)

Apple Intelligence

Let’s clear up one misconception right away: Apple Intelligence is based entirely on Apple technologies, not ChatGPT. (More on ChatGPT later.) Apple is vertically integrated across silicon, hardware, software, and services, and it is utilizing its control over the whole stack to enable generative AI experiences that will be difficult to replicate without at least equivalent access to the operating system and application APIs.

Apple Intelligence does include writing assistance, erasing strangers from your pictures, and the ability to generate pictures of cats wearing hats because those are all table stakes – everyone has to offer that on a modern device. However, Apple’s main focus is on using generative AI to make your devices more useful and contextually relevant while maintaining privacy. Apple Intelligence is making Siri more conversational and giving it the ability to understand your context – what is on screen, where you are, what you’ve used your phone to do in the past, who your contacts are, and even what apps and services you use. It can then pull information from different places or do things on your behalf. This is not limited to Apple’s own applications, but App Intent APIs allow Siri to securely interact with third-party apps as well.

Apple Intelligence examples included asking Siri to find pretty much anything on your phone with just a vague description, which sounds at least superficially similar to Microsoft’s upcoming Recall feature in Copilot+ PCs (Apple is creating and storing a semantic index on your phone, iPad, or Mac). However, you can also use Apple Intelligence to ask for contextual information across apps. For example, "what time do I need to pick Mom from the airport?" requires Siri to know who your mother is, find her text with the flight info, look up flight information, use your current location, and check Maps for traffic. Further, by doing as much processing on-device, Apple’s implementation preserves privacy. If it works, the possibilities are endless and broadly useful.

Apple never called Apple Intelligence a LAM (Large Action Model) but that’s exactly what it’s building, by providing APIs to developers to open up their apps and your data in them to Siri. Since Apple is the arbiter, it can query multiple apps without exposing the data from one app to another or having your data end up in some other vendor’s cloud. Apple’s user base is enormous, and I expect every significant app to utilize these APIs at least in part, because to avoid doing so means that iPhone users will use your app or service less.

Apple did not talk about how big the models are or how many are being used or how many billions of data points were used to train the models because nobody cares: consumers just want it to work. Apple is doing as much processing on device as possible, however, when a query requires more horsepower than can be accessed on-device, Apple encrypts everything and sends it to its own private cloud without retaining any of your data. This hybrid model showcases impressive integration across device silicon, software, cloud servers, and security.

Apple is betting that consumers will trust it to implement AI in a way that doesn’t return bad information and doesn’t steal or leak their data. Apple has been building its reputation for privacy over years — and heavily marketing it. This marketing rings true, in part because of Apple policies, and in part because Apple’s business model is to sell you shiny hardware, not to keep your data and sell it or use it to help others sell to you.

About that OpenAI Deal

Apple is also allowing users to access OpenAI’s ChatGPT within iOS, iPadOS, and MacOS, but it really doesn’t seem to want you to. In fact, every time it offers ChatGPT as an option it pops up a note that reminded me partly of Microsoft’s Clippy: “I can use ChatGPT to help with that.” But there’s a sense of warning, too: “Do you want me to share this photo?” with the unwritten subtext ‘are you SURE you want to leave the privacy and accuracy of Apple’s AI and go to …the dark place?’ Apple isn’t charging for basic ChatGPT access, but there definitely appears to be a mechanism for OpenAI to upsell Apple users on subscription-only features and it wouldn’t surprise me if Apple is paying OpenAI something to use ChatGPT but that there’s a kickback when Apple users upgrade to paid features.

Apple’s third-party AI model extension is not limited to OpenAI. At a media session after the keynote, Apple’s Craig Federighi said that Apple, “is not announcing anything today, but is open to adding access to other AI models like Google’s Gemini.”

Is the inclusion of non-Apple AI models an admission that Apple’s own generative AI capabilities are not state of the art? Maybe, but I would argue that in the real world, it doesn’t matter. Apple implementation of AI -- and its ability to explain the benefits and privacy tradeoffs in plain English -- is more important than any technical or LLM deficiencies.

ChatGPT integration with Siri definitely lets Apple at least partly have its cake and eat it, too. Apple does not need to build out the full suite of capabilities that OpenAI’s leading models have, but it can disavow any hallucinations, cultural, or ethical issues that OpenAI gets embroiled in. For example, Apple’s image generating tools will let you put custom images into your group chats, but if you want to generate something photorealistic or controversial Apple limits that in its own tools, but offers you OpenAI for that instead. If the result is nearly identical to copyrighted material or it conjures up people with six fingers, hey, Apple warned you.

Questions

We didn’t get any live demos and Apple Intelligence is not enabled on the current iOS 18 developer beta, so it isn’t possible to know how well this will work in the real world. How fast will it be, especially if a query needs to handed off to Apple’s cloud? At launch, Apple Intelligence will be English-only for the U.S., and some capabilities will be released later than others - as late as next year. The biggest unknown is how long it will take for Apple’s developers to utilize the new APIs so that Siri can access the data stored in them or start a service. How will consumers know which apps have integrated Apple Intelligence so that a query can include them?

Even though Apple never mentioned the specific technical specs required, under the hood, Apple Intelligence clearly requires serious neural processing capabilities, and it will only run on iPhone 15 Pro and Pro Max phones. The current, base model iPhone 15 need not apply, but I do expect the base iPhone 16 to support Apple Intelligence this September, and it will likely be a big selling point and upgrade push. Apple Intelligence will also work on Apple Silicon M1 or better Macs and iPad Pros, but the M2-powered Apple Vision Pro is conspicuously excluded, even though voice-driven generative AI help would be wildly useful in spatial computing.

The Competition

Not all generative AI tools are too niche or too unreliable. Developers genuinely find AI coding tools helpful – Microsoft’s Github Copilot got a huge cheer at its BUILD developer conference, and Apple announced similar generative AI tools for Xcode at WWDC. Samsung has done a good job finding a few useful AI tools for its phones – some developed internally, some by Google, some running locally, some in the cloud. The tentpole feature for Microsoft’s Copilot+ PCs is Recall, which forms an artificial memory that you can query to find anything you ever did or saw on your PC. (See here for Techsponential’s coverage. Note: since the original publication of this report, Microsoft has pushed back launch of Recall until security changes are made and it goes through a Microsoft Insider’s feedback loop).

However, Google really needs to stop showing off science fiction stuff in its labs, and create APIs for developers to allow OS-level cross-app integration. Google is a web-based company and defaults to doing even basic generative AI in photo editing in the cloud; the bias has to shift to more hybrid and on-device AI, especially since Google has all the pieces in place from Tensor chips with NPUs on its Pixel phones and superb small Gemini models. Google claims that it’s generative AI technology is better than anyone’s -- and in some respects, it may be – but Google needs to make it clear how that advantage is meaningful to consumers, not AI researchers. And obviously Google will want to enable Gemini integration in Siri and Search on iPhone – assuming it can figure out a way to monetize it.