Apple arrived late to the AI party, but now it’s ready to face off against Google and Microsoft. On Monday the Cupertino company announced Apple Intelligence, its AI framework supporting a host of new features across all operating systems, as well as the operating systems themselves: iOS 18, iPadOS 18, WatchOS 11 and MacOS 15 (Sequoia). You can already download an iOS 18 developer beta; all the other operating systems enter public beta later this year.
In the months preceding WWDC, some high-profile topics arose that relate to the underlying architecture of Apple’s operating systems and legal hurdles Apple faces.
Watch this: Everything Announced at Apple’s WWDC 2024
AirPods gets more gestures and TVOS teases new feature
Early in the event, Apple whipped through some updates coming to AirPods and TVOS.
AirPods will be able to recognize head nods, and personalized Spatial Audio will be available for gaming (with developer support), starting with Need for Speed.
Watch this: You’ll Soon Be Able to Control AirPods by Nodding
We were treated to the new shows coming to Apple TV Plus: Presumed Innocent, Fly Me To the Moon, a new season of Silo, Lady in the Lake, new season of Shrinking, Wolfs, The Instigators, Bad Monkey and a new season of Severance.
Home and TVOS has new InSight information (like Amazon Prime’s X-Ray feature), there’s better clarity for audio, 21:9 projector aspect ratios and more. Peanuts licensing brings the comic to your screensaver.
Apple Intelligence was the start of WWDC
With all the operating systems, there’s a ton of AI driving the new features, powering all the text summaries, drafting, image personalization — sketch, animation and illustration — and more. Apple has introduced a programmer kit for adding AI and App Intents support so developers can define actions and access AI. And it’s added code completion and other dev tools to its coding software apps.
Watch this: Apple Reveals InSight for tvOS
It can automatically build Actions and throughout it incorporates everything it sees you do and see for context, similar to Microsoft’s Copilot Plus Recall.
The company stresses on-device processing as a way to keep your info private and secure. It uses a hybrid model, cloud plus local, for more complex requests. It’s created its own servers specifically using Apple silicon to support it, in order to manage the security and privacy through the whole pipeline.
As expected, Siri gets a big dose of AI, with conversational and personal context — it understands pronouns and uses your previous actions as context. You can now type to Siri, switch between text and voice, and get help for using your Apple devices. Siri will support in-app actions as well as cross-app actions. It’s very similar to Microsoft’s Recall.
Watch this: OpenAI’s ChatGPT Is Coming to Apple Apps
The company announced its partnership with OpenAI, so it can use ChatGPT-4o to perform tasks that require a lot more power, like photorealistic generative tools, and in-app integrations for image generation and drafts. It’s free for all, and paid subscribers can link to their accounts for extra features. It’s coming later this year, and future models will be coming later.
iPadOS 18 does everything iOS 18 can, but on a bigger screen
It has everything that’s new in iOS 18, plus new Apple Pencil control and apps that take advantage of the larger screen size. Floating, customizable tab bars work in all apps. The core Apple apps have been redesigned: there are updated animations for app switching and so on, and you may see them in third-party apps.
Screen sharing supports annotating and remote control of others’ devices now, while Freeform lets you select subsections of the screen. And… Calculator! With Apple Pencil you can create Math Notes, which perform basic math and solve simple equations based on your handwriting.
There’s also some smarter Pencil work: Smart Script improves your handwriting and editing; Wand uses AI to drive Apple Pencil-related text. Plus, Clean Up removes unwanted objects in photo backgrounds, and you can use natural language to search images and videos, or create a Memory of your media by picking out photos and choosing themes.
Watch this: iOS 18 Brings New Tapback Features and Test Over Satellite
iOS 18 introduces a ton of long-awaited features
You can finally control where icons sit on the home screen, along with other color and theme customizations for the icons and home screen. Control Center can be broken into control groups, and it allows for more controls and resizing them. You can get to them on the lock screen, too, and decide which is on it.
The operating system introduces the ability to hide or lock specific apps and their data as well, so nothing appears when you hand your phone to someone. Developers can provide more pairing info for Bluetooth.
Messages gets a send later feature, text effects, more Tapbacks, satellite messaging (and SMS) on iPhone 14 and later. Mail categories arrive for your organization needs. Maps gains topographic maps. Tap to Cash, Apple Pay online and Event Tickets get more info and a redesign.
Watch this: Apple introduces Personalized Emojis Called Genmojis
Game Mode comes to iPhone from the Mac, with minimal background activity and lower latency for accessories.
Photos has been overhauled into a single view, with a Library photo grid and collections by time and topic. There’s also new Favorites (personal) and Featured (automated) carousels and views.
AI powers Genmoji, which lets you create Memoji-style graphics based on text prompts, which you can add as stickers or inline. Image Playground lets you remix existing graphics to personalize, using your personal context and images, for use within apps.
WatchOS 11 adds new updates to your Health App
New workout features, Training Load provides a progress report of your current workout, plus you can pause your Rings, and more. The Watch Health app provides more Vitals, with health metrics analysis. The Smart Stack brings more automated widgets, live activities, and developers will be able to add activities. Assisted photo selection improves Photos Face. And there’s a lot more.
Watch this: Apple Updates Health Features With New Watch OS 11
VisionOS 2 wants to bring more than just the theater to you
With the update to the Apple Vision Pro’s operating system, new capabilities include turning photos into 3D — spatial photos — new gestures to give you quick access to the home screen, a Mac virtual display and more. Plus, there’s support for larger and curved displays, up to two-4K equivalent for Mac virtual displays now. And the new OS supports side-by-side screen displays.
And video capture gets the Spatial treatment, too. There will be a new lens for the Canon R7 to capture spatial video, and a new Vimeo app can be used for viewing. Apple Immersive Video will be getting support for new workflows from Blackmagic Design.
Vision Pro sales are expanding to new regions, as well, including the UK and Australia this summer.
MacOS Sequoia adds new screen mirroring feature
A lot of the new iOS features arrive. iPhone Mirroring works as long as it’s within connection range — it shows the iPhone screen on your Mac — and you can get the iPhone notifications and reply to messages, or play audio through your Mac, for example. You can drag and drop between them, as well.
Video conferencing has a Presenter preview, background replacements and more.
Watch this: iPhone Mirroring Comes to Macs with MacOS Sequoia
It finally adds a password manager to control all those passwords you’ve been saving, with autofill and cross-device syncing.
Safari adds Highlights to highlight relevant information while you browse, summaries, an autogenerated TOC in Reader, Video gets a lot of Chrome-like updates.
Gaming ups Metal, Apple’s graphics API, makes it easier for developers to unify their game code and shaders, making it easier to port games to the platform. Ubisoft’s Assassin’s Creed Shadows is coming to Mac and iPad on Nov. 15, at the same time as PCs and consoles. And there seem to be a lot of game announcements to follow later in the year.
AI powers called Rewrite, Proofread and Summarize help with wording and style, summaries and fixing errors. Smart Reply lets you choose responses, which offer a selection based on context. It also manages notifications, prioritizing some and hiding some.
