Apple Intelligence, Siri with ChatGPT, iOS 18

Apple announced that Siri can tap into OpenAI’s ChatGPT when needed. The company said it wanted users to be able to tap into external models.

Siri will ask if you want to share your question with ChatGPT and then will return suggestions from the OpenAI chatbot. ChatGPT is also built into systemwide writing tools. So, for example, Apple said you can create a bedtime story for a child and add images created by ChatGPT. It will be free, without creating an account, and your request and information won’t be logged, Apple said.

The integration will come into iOS 18, iPadOS 18 and macOS Sequoia later this year. It said it will also add other third-party artificial intelligence models.

— Todd Haselton

Apple introduced a new form of custom emojis called Genmoji. Users can create their own emojis using Apple’s artificial intelligence for when they can’t find quite the right emoji to express what they’re trying to say. Users will also be able to create Genmojis that look just like their friends.

You can create a Genmoji right in your keyboard by typing in a description. Apple will create a few options for you to choose from.

Apple’s new Writing Tools are available systemwide, including on third-party apps. Improvements include being able to rewrite your existing copy, change the tone of language, summarize a message in “TLDR” format and pen Smart Replies to messages.

— Rohan Goswami

You can now use Siri via text, correct your statements in real time and offer a new look that is more tightly integrated into the operating system.

It will be powered by Apple Intelligence, the company’s artificial intelligence product. Siri now has the ability to answer “thousands of questions” about how to use your Apple device, even if you don’t know exactly what the feature is.

Siri will now be able to take action in and across apps, such as sending an article from Apple News to a group thread in Messages. Third-party apps will also be able to tap into some of Siri’s enhancements.

The same “personal context” that Apple touted broadly will also come to Siri, allowing it to tap into a broader swath of information and data on your device. For example, it can search through your whole device, so you can ask it to “show me things I sent Todd last week.”

— Rohan Goswami

Apple announced its long-awaited artificial intelligence push, Apple Intelligence. CEO Tim Cook emphasized the need for privacy and personalization, “beyond artificial intelligence,” into “personal intelligence.”

Apple’s Craig Federighi said the generative models behind Apple Intelligence would be available across iOS, iPadOS and macOS.

Here are some of its capabilities:

• Context-driven notifications: Apple Intelligence can recognize which notifications are important to your personal context.
• Writing improvements: Apple Intelligence will introduce systemwide proofreading and style improvements across third-party and native apps.
• Image generation: Apple Intelligence can create generative photos based on your photo library, similar to some other platforms. There are three styles: Sketch, Illustration and Animation.
• Cross-application tasking: Apple Intelligence can delve into your apps and execute tasks on your behalf. One of the examples Federighi gave was asking Apple Intelligence to pull up files sent by a contact during a certain time period.
• Focus on personal context: Apple Intelligence can draw upon the full suite of your activity but also on what is on your screen. For example, you can ask it whether a shifted meeting will cause you to be late to a personal commitment, Federighi said.
• Private Cloud Compute: Apple Intelligence will leverage cloud-based models on special servers using Apple Silicon to ensure that user data is private and secure. If a request needs to go to a cloud server, Apple says it will only send a limited selection of data in a “cryptographically” secure way.

Federighi said privacy was a top priority for Apple. He described Apple Intelligence as a collection of “highly capable” large language and “diffusion models,” as well as an “on-device semantic index” that worked across apps to identify data and feed it to models.

Many of those models will run on device. For those models that need to be stored in the cloud, Federighi touted Apple’s ability to let users control the kind of data you store in the cloud and how it can be accessed.

“We want to extend the privacy and security of your iPhone into the cloud,” Federighi said.

— Rohan Goswami, Kif Leswing

Apple announced a Passwords app for iPhone, iPad, Vision Pro, Mac and Windows. It helps you store all of your passwords, sort of like Keychain did, but includes verification codes, app passwords, Wi-Fi passwords, shares passwords, Passkeys and more. You can manage all of your passwords in the app or see how strong it is. It is a lot similar to 1Password.

— Todd Haselton

Here’s what’s new in macOS Sequoia. Developer betas roll out today, public betas roll out next month and the full OS will release in the fall.

• Continued enhancements from iPadOS and iOS: New features, including text animations and Maps improvements, will also be on macOS.
• iPhone mirroring: Users can now see and control their iPhone from their Mac via Continuity. iPhones will remain locked even if in a virtual session. StandBy will also remain visually undisrupted.
• Unified Notification Center: iPhone notifications will now appear on macOS.
• New Passwords app: Apple is breaking out its iCloud-powered Keychain feature into a discrete app, competing directly with 1Password and other password managers. It will also be available on Windows.
• Video enhancements: MacOS will now offer further enhancements to video calls, including background features and screen isolation.
• Safari enhancements: Safari now has artificial intelligence-powered highlights to extract helpful information from a webpage.

— Rohan Goswami

Apple announced iPadOS 18, with a bevy of new features for the iPad. Here’s what’s new:

• Interface improvements: A floating tab bar and automatic sidebars will allow users to use more of the screen at any given time.
• SharePlay enhancements: You can now take control of a SharePlay session.
• Calculator on iPad: Apple is finally bringing a native Calculator app for iPad. It will also support unit conversions, and when used with the Apple Pencil, will unlock a new feature called Math Notes.
• Math Notes for Calculator: Users can write with the Apple Pencil to automatically solve problems that users write on the iPad, with support for scientific calculator functions.
• Notes: The native app has a new feature called Smart Script, which uses “on-device machine learning” to automatically clean up your handwriting. You can also paste typed words into a handwriting session and they will automatically reform to appear as handwriting.

Apple shared new updates to watchOS:

• Training mode: Apple is introducing training mode, which can track how the intensity and duration of users’ workouts affect their bodies over time. Intensity is tracked using calorimetry data such as heart rate, pace and elevation, and Apple said a “powerful new algorithm” will automatically translate power sensor data into an estimate of your effort rating after your workout.
• Customize the summary tab: Users can personalize their summary tab with information they want to see, such as weekly running distance.
• New Vitals app: WatchOS 11 will track vitals such as your heart rate, respiratory rate and risk temperature, and give you insights about them in the new Vitals app. Users can easily check on their vitals and see when multiple metrics are out of range. They can also learn how their vitals respond to other factors such as alcohol, illness and elevation changes.
• Tools for pregnant users: Apple said cycle tracking now shows gestational aids, and the Health app can display your pregnancy across all charts and prompts.
• Adjustments to rings: Users can now adjust their goals by days of the week, and users can pause their rings if they want to take a break for a few days.
• Photos for your watch face: Apple is using artificial intelligence to identify photos that will work well as a background for your watch face.

Apple announced some improvements to its Audio products and Home platform.

Here’s what’s new with AirPods:

• Siri with AirPods: AirPods can now detect head motions to answer or reject phone calls (nodding to answer, shaking a head to reject it).
• Voice Isolation: AirPods Pro will now be able to isolate your voice, even with noisy background audio.

Here’s what’s new on tvOS:

• Insight: A new feature that will give you information about what is playing on your TV, including on your iPhone.
• Audio enhancements: Apple TVs will now be able to enhance and isolate spoken audio, and will have improved subtitle timing that will automatically show up when appropriate.
• Projector support: Apple TVs will now support 21 x 9 projectors.
• New screensavers: Apple is adding support for Portrait photos as Apple TV screensavers, as well as allowing users to select their preferred screensavers.

— Rohan Goswami

Apple announced iOS 18 with lots of personalization features. Here’s what’s new:

• Customizations for home screen: Users can arrange their apps in new ways, change the colors and adjust the darkness of their apps. For instance, people can arrange their apps around the photo they have set as their background so it doesn’t cover it up.
• New look for Control Center: Users can group controls so they are easier to use, and developers can include controls from their apps. Apple is introducing a controls API for developers.

• Privacy updates: Users can lock apps so others won’t be able to see or access information without authentication. Users can control which contacts are available to different apps.
• New ways to use messages: Users can add new effects to their texts and tap react with new emojis.
• Messages via satellite: Users will be able to send iMessage and SMS messages via satellite.
• Filters for the Mail app: Users will be able to filter their emails by categories, and the feature will be available later this year.
• Tap to cash: A quick and private way to pay another person just by holding phones together.
• New look for event tickets: Apple is also introducing an event guide and support features such as venue maps so you can find your seat more easily.
• Gaming: Gaming on iOS 18 will minimize background activity and support improvement with wireless controllers and AirPods.

• Photos: It will be easier for users to search through their photos. The grid will appear at the top, and a theme, such as time, people or trips, is displayed below. A new filter button shows specific types of content so you can, for example, filter out screenshots.

— Ashley Capoot

Apple Vice President Mike Rockwell announced the next iteration of Apple’s OS for its Vision Pro, VisionOS 2, four months after it first launched VisionOS. Rockwell also said that there were 2,000 apps ready for use on the Vision Pro.

Vision Pro will also be available in China, Japan and Singapore as of June 28. It will roll out to Australia, Canada, France, Germany and the U.K. on July 12.

Here’s some of what’s new:

• Photos: VisionOS 2 will use artificial intelligence to transform regular photos into Spatial photos that are compatible with the Vision Pro.
• New gestures: VisionOS 2 will also debut new gesture-driven controls.
• Ultrawide display: Vision Pro will now support an ultrawide display that Apple says is equivalent to two side-by-side 4K monitors.
• Train support for Travel mode: Apple is adding train support to its Travel Mode. It previously only supported planes.

We’re seated. Tim Cook briefly said hello in the prerecorded video, then kicked it off to Craig Federighi, Apple’s senior vice president of Software Engineering. Cook said it will be action-packed and memorable. He also said Apple will announce “profound new intelligence capabilities.” It’s a little strange he didn’t say “artificial intelligence.”

“Speaking of special, we are incredibly excited … this is a big one … no silly gags, no ridiculous props,” Federighi said.

— Kif Leswing

Here’s a tweet from Apple’s senior vice president of Worldwide Marketing Greg Joswiak teasing an artificial intelligence announcement. He says tuning in to the event is the “intelligent thing to do!” No surprise there, as everyone is expecting AI today.

— Todd Haselton

This is a very important event for Apple. The company is facing pressure to show off its artificial intelligence technology and how it folds into its products and software. The company long avoided using the acronym “AI” when talking about it, instead preferring to say machine learning. But companies such as OpenAI, Microsoft, Google, Amazon and Meta have popularized the term and are racing to add AI to core services, mostly to the benefit of Nvidia, which powers it.

Apple CEO Tim Cook has teased “big plans,” a change of approach for a company that does not like to talk about products before they are released. We will learn about those plans today.

— Kif Leswing

Here’s a peep at what Apple stock is doing leading into the event. It remains the third-largest U.S. company behind Nvidia and Microsoft. Nvidia passed Apple to become the second-largest company on June 5, as the stock continues to rip on the back of the artificial intelligence boom.

Also, check out its performance compared to Microsoft and Nvidia over the past year:

No surprise, but it looks like OpenAI CEO Sam Altman is here, per this tweet. Bloomberg had reported that Apple is partnering with OpenAI to provide some of its artificial intelligence capabilities. Altman has been there in the past, too. In 2008, he was on stage showing off his app Loopt.

— Todd Haselton

I just got to my seat. People are mulling around and waiting for the event to start. The audience will watch a prerecorded video of the event, so don’t expect much live commentary from Apple executives. Based on the conference scheduling, it looks like this event is going to last about two hours.

Also, check this out. Apple opens up a huge door at its headquarters so people can sit inside and outside and see the screen:

There are a lot of big announcements expected on Monday. I’m most excited to hear about iOS 18 and what new changes are coming to the iPhone, and specifically, how artificial intelligence will play into iOS 18. I’m curious if Apple will use a partnership with OpenAI to create a chatbot, or if it will specifically focus on using AI for other tasks, such as summarizing notes, offering live translations and recording voice memos.

But aside from iOS 18, I’m also interested in what kind of changes Apple will make to iPadOS, which is still awfully similar to Apple’s iPhone software. The new iPads Pro have Apple’s latest M4 chip and it seems like Apple should use that extra power to make the tablet’s software more like a MacBook. It probably won’t happen, but I can dream.

— Todd Haselton

Apple CEO Tim Cook will almost certainly open and close the keynote, as is standard. And, similar to most recent events, this is prerecorded, so he won’t be live on stage like he used to be. Instead, everyone will watch the same video you can tune into on YouTube above. Cook will typically say a few words, toss to a quick teaser video and then introduce another executive to break into other topics, such as iOS 18, the new version of macOS, updates to the iPad software or something else.

— Todd Haselton

CNBC Tech Correspondent Steve Kovach arrived at the Steve Jobs Theater super early this morning to get ready for CNBC’s live TV coverage from the event. Kovach snapped this picture before droves of developers and press arrived.

— Todd Haselton