Follow Lifehacker's ongoing coverage of WWDC 2024.
Apple's WWDC 2024 keynote spanned nearly two hours, covering everything from new iOS updates to a deep dive into Apple's grand plans for AI. If you missed the event, you don't have to sit through the livestream on YouTube: Here's everything Apple announced at WWDC this year.
Apple TV+
Apple kicked off its presentation with a preview of upcoming Apple TV+ shows and movies, which has quite the lineup on the horizon. The company announced Severance's second season in addition to fresh content like Dark Matter, Presumed Innocent, Fly Me to the Moon, Pachinko, Silo, Slow Horses, Lady in the Lake, The Instigators, Bad Monkey, Shrinking, and Wolfs.
VisionOS 2
Apple next took the time to show off some new features for visionOS 2. While a modest update, Vision Pro users can look forward to a larger and higher-res Mac Virtual Display, mouse support, and new gestures for Control Center as well as options like seeing the current time, battery level, and volume adjustments.
visionOS 2 also lets you create spatial 3D images from 2D images in Photos, and it will support using the headset on trains. Right now, Vision Pro has a plane mode, but in other moving vehicles, the device struggles to orient itself correctly.
iOS 18
We all thought this would be the moment of the Apple event, when the company would roll out its set of brand-new AI features. That... did not happen. Instead, the company highlighted some small but interesting new changes to iOS, and saved the big AI announcements for later.
First up, Apple is opening up customization on iOS. The company now lets you put your app icons wherever you want, change app icon colors to your preference, and adjust the tint of the icons to match your iPhone's dark theme. You can also completely customize Control Center: Apple lets you choose what functions you want where. It honestly doesn't feel 100% Apple. iOS 18 will also allow you to lock your apps behind Face ID, Touch ID, or your PIN, as well as bury any apps in a new Hidden folder.
Messages gets a big upgrade: Tapback icons (thumbs up, heart, "Ha Ha," etc.) now have a more colorful redesign, but you can also choose any emoji from your phone to react with instead. You can now schedule messages, as well, so you don't need to set yourself a reminder to send an important text down the line. We're also getting text effects and formatting: Now, you can add effects to individual words, and choose from rich font features like bold, italicize, underline, and strikethrough. In addition, you'll be able to message other phones via satellite when you don't have a cellular connection, and RCS is on its way.
Mail has a new look, too: You now have quick tabs at the top of the screen (Primary, Transactions, Updates, and Promotions) to sort through your emails. Mail also intelligently sorts through similar emails to deliver relevant information in one view, so you shouldn't have to scroll through multiple emails from your airline to find different flight data. Maps now has a ton of new hiking data, including new topographical maps and the ability to create hikes of your own. Tap to Cash lets you pay people by tapping phones together, while Game Mode reduces background activity to boost game performance and response time.
Photos has a big redesign: Rather than using multiple sections as the Photos app currently operates, the new app is all one view: Your photos grid lives at the top, while the albums, memories, and other sorted material live below. It's a different approach to Photos than Apple has tried before, and it remains to be seen how legacy iOS users will fare with the new UI.
AirPods and TV
AirPods are getting some new features with iOS 18, as well: You can now interact with Siri by nodding your head for "Yes," or shaking your head for "No." Voice Isolation is also coming to AirPods Pro. It's not clear how this is different from the Voice Isolation already available on iOS and macOS, but Apple is claiming the feature can block out city noises entirely while on a call—at least if Apple's demo is to be believed. Personalized Spatial Audio is also coming to gaming, so developers can integrate the feature into their games.
Apple is also rolling out "InSight," which essentially copies Prime Video's X-Ray: When you pause a show or movie, you'll see, well, insights into the actors as well as what music might be playing during a scene. Enhanced Dialogue is rolling out to more devices, and will use machine learning (aka AI) for an improved experience. 21:9 projectors will also be supported in the latest version of tvOS, and Apple is rolling out new screensavers, like a new Snoopy animation.
watchOS 11
Like iOS 18, watchOS 11 is more of a modest update rather than an overhaul. But there are some interesting features to note: The update introduces a new app called Vitals. Here, you can review metrics like heart rate and sleep, and track how they've changed over time. There's also a new feature called Training Load, which takes all your metrics and body measurements to estimate a proper training effort (with a score of 1 through 10) you should be following in your exercises. The feature takes your own experiences into account, too, if you feel you're working too hard or too little.
Speaking of working too hard, you can now pause your Activity Rings (thank god), so your watch doesn't need to shame you for missing one day of goals. You can even set goals based on the day, so Monday can have a lower exercise goal than Tuesday. You can also customize the Fitness app with tiles, so you can quickly check the datasets and features that matter most to you.
Custom workouts support pool swims, and pregnant users can keep up with additional health-related information more easily. Your Apple Watch will use AI to power a new live translation feature, but we don't know that much about it at this time. Smart Stack gets some upgrades as well (including new Translate and Precipitation widgets), and watchOS will get iOS' "Check In" feature. That way, you can automatically notify a friend when you return home from an outdoor run. More apps will be able to take advantage of newer watches' Double Tap feature, Tap to Cash will work with Apple Watches, and there are new versions of the Photos watch face.
For more information about Apple's latest update for Apple Watch, check out our article here.
iPadOS 18
iPadOS 18 is getting a lot of the same features as iOS 18, including message scheduling and Game Mode, but by far the biggest update is the new Calculator app.
No, really. After over a decade of refusing to put the calculator app on iPad, the mad geniuses at Apple have finally done it. iPadOS18 can do math. And it can do it really well. Specifically, the new calculator app for iPad will be able to take notes, taking advantage of machine learning to spice up the calculator experience for tablets.
Upon loading up the Calculator app, you’ll be able to use buttons for basic calculations as usual, but a quick tap will now take you to the app’s new Math Notes section. Here, you can write out formulas and other equations, and the page will automatically update with answers in your handwriting as soon as you draw an equals sign or other equivalent symbol. You’ll even be able to change already solved equations in real-time, and Math Notes will make revisions as appropriate.
There’s also support for graphs, which you can automatically create and adjust based on your already written equations. Your notes will appear in a history, like written notes do in the Notes app, so you’ll also be able to reference old problems days or weeks later. Teachers are certain to pull their hair out over this new “show your work” cheat tool, but Apple is positioning Math Notes as great for budgeting or even scientific prototyping.
You’ll also be able to access Math Notes capabilities in the standard Notes app, which is getting its own update with a new Smart Script feature.
Smart Script is a bit of an odd pitch, as the idea here is to correct your writing to look more like… your writing. Essentially, your iPad will now use machine learning to figure out what your handwriting generally looks like, then make small tweaks to any new writing to clean it up so it looks like its model of your handwriting.
If that sounds confusing, imagine the following: you’re in a lecture and you can’t afford to make perfectly pretty text with your notes, so you just scribble down what you hear and hope you can make sense of it later. Smart Script will try to take those scribbles and make them look like what your handwriting would resemble in a more ideal environment.
“It’s still your own writing, but it looks smoother, straighter, and more legible,” said Apple senior vice president of Software Engineering Craig Federighi.
Notes is also getting some new formatting options, including five new highlighter colors, the ability to collapse sections under heading and subheadings, and automatic content resizing. For instance, if you delete a paragraph, Notes will adjust your remaining content to fill the empty space.
For general navigation, a new floating tab bar will now appear at the top of several apps across the system and give easy access to different parts of that app, sort of like the menu bar in MacOS. It’ll call out a few key sections by default (the Apple TV version of the tab bar has shortcuts to Apple TV+ and MLS content), but you can also expand it into a sidebar for more detail or add new shortcuts of your own.
Finally, there’s accessibility and Shareplay, which should both help with using the iPad. iPadOS 18 will add eye-tracking for navigation, plus vocal shortcuts that will allow users to map custom sounds to specific tasks and actions. Meanwhile, Shareplay will now be able to show when a user taps or draws on their screen, which should help when walking someone through a task directly. In more extreme cases, helpers could (with permission) take direct control of a device.
M-series powered iPads will also get access to Apple Intelligence features like Image Playground, which we’ll touch on later.
macOS 15 (Sequoia)
macOS 15 is here, and it’s got a new, California-based name: macOS Sequoia. Like iPadOS, it’ll get iOS 18 features where appropriate, like Tapbacks and the Passwords app, and it’ll even have a version of Math Notes specific to typed text. But for exclusives, the focus this year is on window layout, device syncing, and gaming.
Perhaps the biggest of these is a new continuity feature: iPhone mirroring. Now, so long as your iPhone is nearby, you can pull up its screen on a window in your Mac, where you’ll be able to interact with apps and notifications. Your iPhone will remain locked while you do this, and you’ll even be able to share files directly from your Mac to your mirrored screen in supported apps like Unfold.
For when you’re just working on your Mac, you’ll also finally be able to pin windows. Third-party apps such as Magnet already offer this, but this is the first time it will be a native feature. Now, when you drag a window (or tile, as Apple calls them) to the edge of your screen, you’ll be able to slot it into a suggested position on your desktop. There’s also keyboard and menu shortcuts for more advanced customization.
For video calls, a new presenter preview will also allow you to double check which of your windows you’re about to share before going live. There are also custom backgrounds to choose from, or you can use a photo.
Finally, there’s gaming. The updates here are more on the developer side, but they show a promising future for gaming on Mac. Basically, Apple is releasing a new game porting toolkit to make games originally built for Windows run on Mac. This should mean a wider library of titles available on Apple’s computers, with confirmed ports already on the way for blockbuster titles like Control and Assassin’s Creed Shadows.
Like on iPad, any Mac running an M-series chip will also get access to Apple Intelligence.
Safari
Safari is getting a few updates specific for macOS Sequoia. These include a new video mode, article summaries, and extra bonuses powered by machine learning.
Now, when Safari detects a video on the page, a new Viewer mode will kick in. You’ll be able to click a button next to your address bar to hide everything on the page but the video, and when you click away from your browser, Safari will go into picture-in-picture mode to play your video in a corner of your screen.
For text-based content, the new Reader mode “instantly removes distractions from articles,” plus gives you a sidebar with what looks to be an AI-generated table of contents as well as a summary (Apple hasn’t been fully clear on how these work yet).
Finally, Highlights uses machine learning to add all sorts of context to your browsing. For instance, if you’re reading an article about a singer, Highlights will pull up a link to one of their songs in Apple Music. Similarly, if you’re looking at a hotel, Highlights will show you its location.
Apple Intelligence
Apple’s big showstopper for WWDC this year was its entry into AI. Titled Apple Intelligence (pun intended), the company’s AI does a little old and a little new.
First up, what we don’t know. We don’t know what training data Apple is using for its AI, we don’t know how detailed its image generation can get, and we don’t know specific release dates (just release windows). But as for everything else, Apple was surprisingly forthcoming, possibly even more than Google was about Gemini during Google I/O.
Maybe the most exciting reveal was a revamped Siri. While Apple was the first to market with a digital assistant, Apple Intelligence is finally giving Siri the update it needs to keep up with competitors like Alexa. With Apple’s AI, Siri can now understand context, answering questions based on what it sees on your screen or intuitively understanding which contact you mean when you say “pull up my texts with Mike.”
This opens up the bot to new possibilities when it comes to search and actions, allowing you to search your photos and videos for things like “my daughter wearing a red shirt,” or ask Siri to add an on-screen address to a specific contact card for you.
Siri will also be able to answer questions about Apple products, coming preloaded with tutorials on things like “how to turn on dark mode.” Answers will now display in a box right on your screen, rather than in a linked help page.
You’ll also be able to give Siri prompts to create custom Memory Collages from your photos, and the bot will naturally decipher references to specific contacts, activities, places, or musical styles to stitch them together.
Finally, Siri will be able to take on a more traditional AI chatbot role by leaning on ChatGPT. When you ask Siri a question it thinks ChatGPT could help with, the bot will ask permission to send the question to ChatGPT for a response. Apple’s promised that any requests to ChatGPT will hide your IP address, and that ChatGPT will not log requests made through Siri. You won’t need an account to ask ChatGPT questions, but usage limits will apply, and ChatGPT subscribers can link their accounts to access paid features.
Outside of the realm of Siri, Apple Intelligence is also giving iPhones a Clean Up mode that works a lot like Google’s Magic Eraser. In the Photos app, just tap the Clean Up button, then circle or tap a specific subject to have your phone intelligently cut them out of a photo.
In another Pixel-like feature, Apple Intelligence also allows the Notes app to record and transcribe audio, even from phone calls. Participants on your call will be notified when you turn recording on, so nobody is surprised.
There are a couple of other organization goodies here, too. Both notifications and the Mail app can use AI to prioritize and summarize what you see, and the Mail app will even allow you to use AI to compose smart replies.
Then there’s the more traditional features. Apple Intelligence will be able to rewrite or proofread text “nearly everywhere” you write, based either on custom prompts or pre-selected tones.
It’ll also be able to use ChatGPT to generate whole new text, using similar rules as Siri.
Image Playground will be your image generator, and will be able to incorporate AI art into your Notes, Messages, and more. You’ll have access to pre-selected subjects and art styles as well as a prompt box, although it’s not clear how much freedom you’ll have with the tool. Apple’s language emphasizes that users will “choose from a range of concepts” rather than type anything they want. What is clear is that Image Playground will have the same contextual approach as Siri, meaning it’ll be able to generate caricatures of people in your Contacts list with just a name.
Finally, there’s Genmoji, which are similar to Image Playground but are specifically custom, AI-generated emojis. Like regular emojis, they can be added inline to messages and shared as a sticker react.
That all sounds a little cool and a little scary, which is why Apple is emphasizing privacy with its AI. Apple wants to have most AI processing happen on-device, but for content that needs to touch the cloud, Apple is promising that data will never be stored and will only be used for requests. It’s also making its servers’ code accessible for third-party review.
The catch to all this? It relies on Apple Silicon neural engines. That means it’ll only come to devices with an A17 Pro chip or an M-series chip. This limits which iPads and Macs you can use, plus makes it so the only iPhones with Apple Intelligence (at least at the start) will be the iPhone 15 Pro and Pro Max.
Apple Intelligence will be available to try out in U.S. English in the summer and will come out in a beta form in the fall.
from LifeHacker https://ift.tt/0EdaUx3
https://ift.tt/XGso3Ux