Featured • iOS, macOS, tvOS, watchOS • 1:28:15
Join the worldwide developer community for an in-depth look at the future of Apple platforms, directly from Apple Park.
Speakers: Andreas Wendker, Sri Santhanam, Matthew Firlik, Jeremy Sandmel, Jon Andrews, Ali Özer, Kristin Forster, Josh Shaffer, Sebastien Marineau-Mes, Darin Adler, Lori Hylan-Cho
Unlisted on Apple Developer site
Downloads from Apple
Transcript
[Andreas Wendker]
Welcome to the State of the Union, where we'll go behind the scenes of the new technologies we just announced earlier in the keynote. We'll give the inside story that developers need to know. We'll give you details on the new APIs you should become familiar with and the motivation for the deep changes we're introducing. And of course, we'll show you lots of demos of all this in action.
As you heard during the keynote, this is a huge year for our software platforms as they take massive leaps forward. One of the most exciting announcements of this year's WWDC is the transition of the Mac product line to our own Apple Silicon, which will propel the Mac into an amazing new future.
The new macOS Big Sur, in addition to supporting these low-level changes in hardware, has an all-new look and feel to go along with them, making it an entirely new operating system inside and out. Similarly, iPadOS 14 has a refined look and feel that makes the iPad experience even more distinct and powerful, with improvements to many apps and support for Scribble with Apple Pencil.
In iOS 14, users can now interact with your apps in exciting new ways by creating widgets that run right on their Home Screen. And they can discover apps directly in the moment of need with lightweight App Clips without having to download the entire app first. And in the new watchOS 7, you can build a whole set of flexible complications to surface up-to-date information to your users.
From the big screen of the Mac to the iPhone you hold in your hand, every product is carefully designed to have amazing, unique experiences that users love. But for developers, we are making our frameworks and APIs and even the design language in our user interfaces more and more consistent, making it easier than ever for you to create a great user experience across all types of Apple devices.
All the system features are supported by a huge number of new APIs for you developers, all with first-class support in the Xcode tools, and many of them based on SwiftUI, which is one of the most important foundations we use across all our platforms. We're also constantly working on making our platforms open to you, allowing you to plug into our OSs in more and more ways.
You might recall that a few years ago we introduced Extensions, a way for your apps to customize our operating system functionality. Since then, we've been adding plug-ins for share sheets, VoIP calling, rich notifications and more. Starting this year, users can make your app the default app used for e-mail in iOS and iPadOS. And users can change the default web browser too.
And developers can bring their existing web browser extensions to Safari for Mac using the WebExtensions API. With a simple command-line tool that is shipping with our Xcode tools, you can now bring extensions over from Chrome, Firefox and Edge as privacy-preserving Safari extensions and ship them in the Mac App Store.
We are also happy to announce that we are launching a new program that will allow third-party accessory makers and device manufacturers access to the Find My network. Find My is a huge end-to-end encrypted and anonymous network of over half a billion Apple devices that can be used to locate a missing Mac, iPhone, iPad or Apple Watch, even if it can't be connected to the Internet, and show the location to the owner right in the Find My app. It works great for your Apple devices, and it's perfectly suited to locate all kinds of other objects as well. To learn more about this program, and to see the draft specification that is available today, go to developer.apple.com.
There is so much to cover this year. We better dive right in-- starting with the Mac's transition to our own Apple Silicon. Using Apple Silicon inside Macs is an amazing new direction for the future. It will open the door for huge improvements to speed, graphics performance, power consumption, security and more. And unifying architectures across our product lines will allow us to bring unmodified iPhone and iPad apps to the Mac for the first time.
We've been working on it for several years, and we are so happy with the result. It's going to be awesome. Let me give you a sneak peek and take you on a quick tour of macOS Big Sur running on a Mac with Apple Silicon. Here is Finder with the Applications folder. Mail and Messages are fully functional, of course. For example, I can drag and drop attachments between them. And Calendar looks beautiful. It's incredibly smooth. Browsing the web is fast in Big Sur's huge update of Safari. So is reading news.
And moving around in Maps is responsive and quick. Photos looks great on Macs with Apple Silicon as well. Same for my music... and podcasts. Productivity apps like iWork's Pages, Numbers and Keynote run great. And this is one of the most demanding Pro Apps, Final Cut Pro, with real-time rendering of several movie streams. And Logic, our music-creation powerhouse, easily keeping up with large projects.
We already worked with a number of our partners to bring their apps to Apple Silicon as well. Here is Adobe Photoshop, already taking advantage of the new architecture and running fast on this new Mac. And Microsoft has brought up their Office apps, such as Word and Excel. Of course, our developer tools also work great on Apple Silicon and can be used to create native apps.
And as you can see here in Activity Monitor, when we say Big Sur and all this software is running great on Apple Silicon, we really mean it. Every single bit of code you just saw is 100% native-- the entire suite of system apps, Apple Productivity and Pro Apps, developer tools and even the first versions of third-party software packages, like the Adobe Creative Cloud and Microsoft Office.
So we are already far along in our move to Apple Silicon. But you might be wondering why we are making this transition. The reason is simple. We want to build the best Macs possible for our users, and we can't build them without our own processors. The hardware innovations that have taken place in the mobile device space and the extremely tight integration of the iOS software and the Apple Silicon have led to amazing user features and behaviors that other platforms cannot offer to this day.
Now, all the energy we put into refining iOS for Apple Silicon for years and years will come back to macOS in the form of shared operating system components and frameworks, and this will take the Mac to the next level. Because just like code written for the Mac originally served as the foundation for iOS more than ten years ago, we can now take everything we learned about optimizing iOS and make the Mac amazing when running on Apple Silicon. Macs will be more secure, they will be faster and more responsive, they'll have longer battery life, and they will simply be much better overall for graphics, making the Mac an amazing platform for games, for example.
One of the hallmarks of macOS on Apple Silicon is, of course, how the hardware and software work so well together. To give us some insight into how this happens in practice, I'm going to have a short chat with my colleague Sri from our Silicon team. Just like you, for the last few months, I've been having a lot more FaceTime calls, so I'm going to call him in the lab.
[line ringing]
[phone chimes]
Hey, Sri. -Hi, Andreas. How are you? -Hey, I'm good. Thank you. And thanks for taking the time to talk with us. During the keynote, Johny gave us the motivation behind the families of chips we've been building over the last decade. Could you quickly give our developers the high points?
[Sri Santhanam]
[Sri] Sure. We've been building many different chips for our products for the past decade. One of the key ideas when we build chips is that we really focus on what's truly important for each product. For example, in the iPhone, we needed to deliver a level of performance that had never been seen in a battery-powered device that small.
For the iPad, it started with driving an incredibly high-resolution display without sacrificing battery life. And for the Apple Watch, we needed a combination of performance, efficiency and features that had never been put together in such a tightly integrated package. So we built a scalable architecture that powered each of these chips and helped make each of these products best in class. And we haven't stopped. We continue to improve the architecture and drive performance and efficiency while building industry-leading features.
We also executed to scale. When you look across all of our products, we've shipped over two billion SoCs over the past decade, and not only that, billions of additional chips to enable our products. You put that all together, and it makes for an exciting transition for the Mac to Apple Silicon.
That's for sure. So what exactly are you building for the Mac? Well, we're building a family of SoCs designed specifically for the Mac. Just like we did with the iPhone, iPad and Apple Watch, we're making sure the chips we build are tailored to the unique needs of the Mac. And practically speaking, that's gonna mean we're gonna bring to the Mac a whole new level of performance. Our scalable architecture that I've already mentioned translates really well to running the types of application we see on the Mac.
We've never really constrained ourselves to only looking at iOS workloads when designing our chips. We've always looked at a wide variety of application and application tracing because we always wanna build the best chips possible. So this really sets the stage well for the Mac. Oh, it sure does. Uh-- Now, you mentioned performance. How do power and energy efficiency play into all of this? Hmm. Uh-- Yeah, that's a good question.
Uh, the iPhone really taught us that energy efficiency and performance are tightly intertwined. In the most thermally and power-constrained devices, you need to improve energy efficiency, and that means to add performance without increasing power. And that's not as easy as it sounds. All of the technologies we've built over the past decade have focused on this one paradigm.
And it turns out that same discipline and focus, that paradigm, translates really quite well to the Mac. All the systems built today are constrained by thermals, power consumption or both. So that's true for the Mac just as it is for the iPhone, though obviously we're talking about a different range of power consumption.
For any given Mac system, improving efficiency means improving performance. And as you know, all our Macs have different enclosures, so our goal is to deliver more performance within each of these enclosures. And that's made possible thanks to that relentless focus on efficiency, or performance per watt, over the years.
Well, that's great to hear. Can you tell us a little more about what else will come along with this transition? Sure. Uh-- The feature story for the Mac is also a great one. Remember, we build highly integrated SoCs, which include a number of features and technologies that are critical in enabling our products.
We build these technologies for the iPhone and iPad Pro, and they're coming along in this transition to Apple Silicon for the Mac. So in addition to our CPU, our display and media-processing engines, our ISP, our Secure Enclave, advanced power management, are all examples of industry-leading technologies that we're bringing to the Mac.
And for the first time, the Mac will have a dedicated inference accelerator for machine-learning applications with our Neural Engine coming over to the Mac. Of course, one of the things I'm most excited about is that we're bringing our high-performance GPU architecture to the Mac. We've already seen our GPU architecture do really well in the iPhone and iPad Pro. And in the Mac, this architecture is gonna be great for pro application and games.
Well, I'm sure a lot of macOS developers are eager to get their hands on these new Macs. What does the transition mean for developers who build for our other platforms? Yeah, I think this point is a very important one for our developers. This transition will give our developers a common architecture behind all of our products. So whether you're building software for the Apple Watch, iPhone, iPad Pro and now the Mac, there's a unified, scalable architecture behind all of them, which should make it easier to write and optimize your application across all of our platforms. It really is a really exciting time.
[Andreas Wendker]
It really sounds great. Thank you, Sri, for taking the time to talk with me. Thanks, Andreas. And I hope you and our developers have a great WWDC. Yeah. See you later.
[phone chimes]
So, as you heard, there are countless ways Apple Silicon creates a better computing platform for the Mac. Going forward, software can take advantage of the amazing Silicon features we built, such as the unified memory architecture, which lets the GPU access memory without delay, the Neural Engine for blazingly fast machine-learning applications, and the Secure Enclave that safely stores users' keys. And all that will provide users with more powerful everyday experiences than ever before. Let me give you some examples of this.
The advanced power efficiency of Apple processors will let us keep cached cloud content fresh and up-to-date for days, even if your Mac goes to sleep. Or we will be able to run a higher-quality hardware-supported 4:4:4 encoder for even better image quality when connecting your Mac to an iPad with Sidecar. Or users will be able to experience the best-in-class platform security pioneered by Apple. Comprehensive runtime integrity and data-protection technologies out of the box, with the flexibility and configurability of macOS.
And performance will be off the charts in many areas. In particular, the Apple Silicon unified memory architecture, powerful GPUs and our highly optimized Metal APIs are going to allow for responsive, complex image-editing workflows such as the one you see here in Affinity Photo, or Cinema 4D, an app optimized for Metal for real-time, interactive 3D editing and modeling. You can see how smoothly it runs on Apple Silicon. Every Mac with Apple Silicon is going to have that powerful GPU and Metal implementation, making Macs a fantastic platform for running games, like DiRT Rally, at high frame rates.
And every Mac with Apple Silicon is also going to have a Neural Engine, making machine-learning applications so much faster. Here you can see a demonstration using advanced machine learning to identify objects in a video clip, evaluating each frame's image content. On Apple Silicon, the video runs super fast on the GPU, and the machine-learning model gets evaluated separately by the Neural Engine.
As you can easily see, the Neural Engine allows for a whole new experience, and a Mac with an Apple SoC finished the task much quicker. And yet, with all the new capabilities and advantages that Apple Silicon offers, there is one thing that certainly stays the same. Macs will stay Macs the way you know and love them.
They will run the same powerful Pro Apps, they will offer the same developer APIs Macs have today, they will let users create multiple volumes on disk with different operating-system versions, and they will let users boot from external drives, they will support drivers for peripherals, and they will be amazing UNIX machines for developers and the scientific community that can run any software they like.
So how are we going to approach this transition? Well, with this WWDC, we are launching an entire Quick Start Program that will enable you to bring your Mac apps to the new architecture and take full advantage of the capabilities Apple Silicon will offer. We'll provide you with all the tools, documentation and even prototype hardware you will need, and several of the videos from this year's WWDC will provide you with a guide for how to get started. Starting today, we'll even make a Developer Transition Kit, or DTK, available to you.
The DTK looks like an existing Mac mini, but it's completely different inside. It runs on an A12Z processor, the same chip that powers our current-generation iPad Pros. The current preview version of macOS Big Sur will be preinstalled on it, as well as a version of our latest Xcode tools with the capability to create apps that run on both Apple Silicon and existing Intel Macs.
The DTK is a prototype, though, and not fully representative of the hardware we will actually ship to customers. There are some software features that will not be supported on it, but it does have everything you need to get started on supporting Apple Silicon. To apply for access to a DTK, simply go to developer.apple.com later today and register for the Quick Start Program. We'll be shipping out DTKs to you as early as this week.
And so the transition actually starts today, and it starts with you, the software developers. We expect to ship our first Macs with Apple Silicon by the end of the year, and that means that you have to get your software ready for it quickly over the next several months. And the best way to do this is to create a Universal app.
[Matthew Firlik]
We've made it really easy to bring your app to any user, on any Mac, by making your app Universal. A Universal app contains code compiled both for Apple Silicon and Intel processors. The two binaries become slices in a single executable, and the operating system chooses the right one for the hardware architecture of the Mac the user is running on. Everything else in the app is the same. All the resources, such as images and machine-learning models, are identical for both hardware architectures. This means you can distribute a single Universal app to your users, and it runs on any Mac, whether they use Apple Silicon or Intel processors.
We have used this approach before. For example, a few years ago, when we asked you to include both a 64-bit and 32-bit version of your app binaries. For most apps, all it takes to create a Universal version is opening your project in the latest version of Xcode and building. Let's take a look.
During development, Xcode builds my app just for the native architecture of my machine to save time. So getting started on a DTK is as simple as clicking "Run" to launch my app natively on Apple Silicon. To create a Universal version, Xcode offers a new "Any Mac" option. This app is using the standard architectures build setting, which automatically includes Apple Silicon and Intel. So I can select this option, and I am ready to build Universal.
For the rare case where I'm not using standard architectures, the menu lets me know. Here, I can see I previously set my framework to build just for Intel. When I build my framework for Any Mac, Xcode will recommend that I update to use the standard architectures build setting. I can just click "Update and Build," and now I have a Universal version.
[Andreas Wendker]
You can turn most Mac apps and frameworks into Universal binaries with very little effort. This is true for low-level libraries and Mac apps that were created with our Mac system frameworks such as AppKit and SwiftUI, but equally so for apps created with Mac Catalyst. While Catalyst uses a different set of frameworks that utilizes UIKit APIs instead of AppKit APIs, they are native apps compiled for the Mac platform just like any other Mac app, and so they, too, will easily turn into Universal apps when building them in Xcode. For many apps, this work will only take a few engineering days. Even for some of the most complex software packages, we've generally seen that it only takes a few weeks to get a full version up and running.
In fact, as you've seen earlier, we've already ported all of Apple's system apps, iWork, enterprise-management tools and Pro Apps, and worked with a number of important partners to get them started and to proof out our tools and the porting process leading up to this WWDC. And the results have been amazing.
Even Pro Apps such as Final Cut Pro or Adobe Photoshop run beautifully on Apple Silicon. These apps represent some of the most advanced and mature software packages running on the Mac platform, supporting the most demanding needs of pro users, and we have found consistently that adding support for Apple Silicon went quickly and problem-free.
We are also passionate about making sure that open-source projects thrive on the Mac, especially if they benefit the larger software-developer community and can help accelerating the port of other software packages. So we have already done the initial work for some of the more widely used open-source projects to help the community get started, and we will be publishing patches to them in the next days. Another important partner we've been working with is Unity.
[Jeremy Sandmel]
Unity is an industry-leading platform for creating interactive, real-time 3D content. It is used by millions of developers to make games and apps that run across Apple's iPhones, iPads, Apple TVs and Macs. Now we've been working with them, and within just a few weeks, they were able to update their powerful 3D-editing and game-development tools to take full advantage of the power, performance and features of the new Macs built with Apple Silicon.
As a developer-artist, you can use the Unity Editor to create, edit and build your games. So let's see it in action. Here we have loaded Unity's Spaceship Demo project into the 3D Editor, and it's all running on a Mac with Apple Silicon. We can edit our content, our geometry, our textures, shading, even our lighting and much more.
The Spaceship Demo app uses Unity's High Definition Render Pipeline, which uses Metal to drive the Apple-designed GPU in these systems and delivers the advanced rendering features required to run modern, high-end games. Now these effects we're seeing here can be created using the VFX Graph, an artist-friendly tool set that makes it really easy to create complex effects that can be rendered right on the GPU.
Now that we're done editing our project, we can make a build of our Spaceship app. The Unity Editor has also been updated to build Universal apps. So now, with just one click, you can build your game to run great on both existing Intel-based Macs and the new Macs with Apple Silicon. Now that we've built our Spaceship Demo app, let's see it running on a Mac with Apple Silicon.
Here we can see how Unity's runtime engine uses Metal's modern graphics and compute pipeline and unified memory architecture to deliver advanced rendering techniques such as volumetric lighting, ambient occlusion and real-time reflections. Because Unity's runtime has already been optimized for Metal and the Apple-designed GPU, your games will automatically run with a full native graphics performance on the Mac with Apple Silicon.
Next month, Unity plans to release a preview version of their software with support for Macs with Apple SoCs, so you can get started right away building your games as Universal apps. And later this year, Unity plans to release a production version of the editor, runtime and tools to enable you to easily create, build and ship your games optimized for both existing Intel-based Macs and those with Apple Silicon. So with powerful tools like these, we can't wait to see all of the amazing new games you're gonna be able to create.
[Jon Andrews]
Games built native for Macs with Apple Silicon are gonna be great. But if your software needs a little more time, or you're an end user who relies on software that's not been ported natively on Apple Silicon, macOS Big Sur will include a new version of our emulation software, Rosetta 2. This provides compatibility for existing Mac software. Now Rosetta 2 is just an amazing technology written from the ground up for Apple SoCs, allowing existing Intel apps to run seamlessly alongside your native apps.
It's able to take Intel executables and translate them into native ARM instructions. Rosetta uses a number of advanced techniques to let binaries run very fast even during emulation. This is especially true for Metal, where Rosetta produces native calls to the GPUs built into Apple SoCs, so even high-end games execute with incredible performance. Another major technology is ahead-of-time compilation so that nearly all applications are translated before you even launch them.
And if you use macOS packages or the App Store to distribute your software, Rosetta will translate the application in the background during the install. All others are translated on first launch. And if you dynamically load code yourself, or for JITs like JavaScript, then the code will be translated on the fly. All code generated is optimized for Apple Silicon and tuned for our high-performance CPUs.
Now we have completely integrated Rosetta into macOS Big Sur, so there's no need for any special setup. And when system calls are executed, they're translated into native calls to the kernel. Rosetta is also secure, with all its caches integrated with system code signing bound to your specific machine and OS version.
And we're including Rosetta in macOS Big Sur as a transitional technology to give you time to complete support for Macs with Apple Silicon at your own pace. Let's dig into the technology with some quick demos, starting with Xcode running on a Mac with Apple Silicon. I've selected to run under Rosetta. And Xcode builds my app the same way it would for an Intel Mac, but then launches it under Rosetta.
You don't need to configure a special debug mode to work in this environment, which is a real enhancement for Rosetta 2. So here we are, using the app, able to interact with it as you'd expect. And I've hit a break point. The familiar tools for development, including debugging, static analysis, testing and more, are available. I can even use the memory graph debugger to browse objects. Let's load the memory graph. It takes just a moment. And it works like you'd expect. So Xcode can build and run my app for both Rosetta and Apple Silicon.
Now let's look at Rosetta running third-party apps, starting with the app Transmit running under Rosetta. I'm gonna connect to OneDrive and download some drone footage I've taken. We have an emulated version of all the macOS frameworks running alongside the emulation of the apps in the same process. So I'm gonna simply download the file. Because Rosetta isn't running in a special container, file system and network access are just like any other app. Okay, let's take a closer look at apps you saw earlier in the State of the Union, starting with DiRT Rally, which really shows off some great features of Rosetta.
[male driver counting down]
[Jon Andrews]
The translation cache was built during the install from the Mac App Store, completely transparently, which means there's no glitches due to a need to translate new blocks of code, as is really important for games with motion and real-time audio requirements.
[driver speaking, indistinct]
[Jon Andrews]
And things just work, even this game controller that I really need to be able to stay on the road here. CPU instructions will be translated very accurately, whereas the graphics commands are sent through to the native GPU. There's simply no frame drops or lags on the controls. It's amazing.
Okay, now let's look at a Pro App, Affinity Photo, which really helps show off that emulation has very little memory overhead. I've got this huge 82-megapixel image loaded here. There's no additional memory overhead for having such a large file open under Rosetta. And because the translation cache is file-backed, just like the original executable, there's little additional overhead here too. And we didn't just focus on the speed of translation. The translation cache is also secure thanks to the code signatures that are checked before execution, just like the original binary.
Saving a file is an all-native operation through our optimized storage stack. The emulation doesn't add any copies or buffering. Coupled with native Metal, I'm able to apply these effects quickly and save them just as fast. Rosetta 2 is wickedly fast, transparent to users and completely integrated into macOS Big Sur. It's a great transitional technology that will give you time to migrate your apps to support Macs with Apple Silicon.
What about code other than apps? For example, app extensions can be built as Universal binaries too. And since extensions run out of process from the host app, Rosetta can even emulate extensions when they're being used with a native host app. So end users can rely on extensions to continue working on Macs with Apple Silicon while you work on porting them to run natively. And other standard plug-in architectures, such as audio units, run out of process under Rosetta 2.
Drivers utilizing the DriverKit API, introduced in macOS Catalina, are in the same situation. If you ship a driver, for example, for a USB peripheral and have not yet adopted DriverKit, now is a great time to make the change. That way you can create a universal driver that will support all Macs.
We will continue to shrink the surface area for kernel extensions, replacing them with safer APIs going forward. And while we will continue to support running kernel extensions on macOS, you can't run them in emulation, and the increased security we offer on Apple SoCs will require all KEXTS to be notarized. DriverKit offers a much better alternative with less friction for the end user, and we highly recommend you adopt it in all situations possible.
One area that we have worked on removing kernel extension usage is for virtualization products. The Mac is the world's best developer platform. Many developer workflows require using tools from different platforms or performing server deployments. To meet these needs, many of you spin up virtual machines running different operating systems or lightweight containers like Docker. We know that this is critical to your work and important we support these needs. That's why we've been working hard with our partners to support virtual machines on Macs with Apple Silicon.
As you saw in the keynote, we started working with Parallels. And I want to walk you through an early port of Parallels Desktop for Mac running on the Apple Hypervisor that's been updated for Apple Silicon. This is running on a prototype Mac with Apple Silicon that supports virtualization.
Here we have a fully featured ARM version of Debian Linux running on Parallels Desktop. Remember, this doesn't require any third-party kernel extensions to be installed now. I've installed Swift for Linux and started a new project with Vapor, a Swift web framework you may be familiar with. Let's go ahead and build our new project.
And then we'll start the server. I can go ahead and load the page from this server in Firefox for Linux. I'd like to be able to test this from Safari in macOS. We have Webmin installed to help manage this machine and Apache configured to proxy port 80 of the VM to the local Vapor server. So let's start it. Now we can go into Safari in macOS and see how it looks there.
It looks great. These environments can interact due to features built into macOS Big Sur, and thanks to the Hypervisor framework, with complete support for networking, storage, input devices and much more. Virtualization technology is used by container solutions such as Docker, and we know it's incredibly important for many of you, as it's very common for server-side development and testing.
So we'll be working with Docker to enable this in the coming months. We've mentioned before that this transition puts Apple platforms on a common architecture. And one of the coolest benefits of this is the ability to run software originally built for iPhone and iPad completely unmodified. Let's take a look at how that works.
[Ali Özer]
As you may have guessed, the technology that enables running iPhone and iPad apps on Macs with Apple Silicon is the same technology that enables Mac Catalyst. In effect, Catalyst is an implementation of the iOS frameworks for macOS, the same APIs made for Mac. And these frameworks, when running on a Mac with Apple Silicon, are even binary compatible. This makes it possible to run many iPhone and iPad apps without any kind of change.
Unmodified iOS apps, however, will not get all the customizations that Catalyst apps can offer. In particular, it's worth pointing out that running unmodified apps will only be possible on Macs with Apple Silicon, not on Intel-based Macs. So you may want to check that Catalyst box in Xcode and start making your app even better for your Mac users. We are planning to make unmodified iPhone and iPad apps available in the Mac App Store once we launch our Macs with Apple Silicon.
In fact, all the apps that users purchased on iPhone and iPad that are eligible to run on the Mac will simply show up as purchases in the Mac App Store as well. You, the developer, will have total control over whether you wish to participate, but we believe this will be a great addition for Mac users for apps that don't already offer a native Mac version. Now, let's look at an iPad game, Monument Valley. This app is the exact same one available in the iOS App Store.
When we launch it, the app appears in the dock, like any other Mac app. I can minimize and reopen the window. Menu bar is an important part of the Mac experience, and iOS apps get a menu bar generated automatically-- for instance, the Edit menu here, with the usual entries, and the app menu as well. You will notice Preferences here, which puts up an in-app preferences panel automatically created from your app's iOS settings bundle. Also Hide, and even Quit, which is a thing on the Mac. Let's hide our other apps and get back to the game.
We have our beautiful graphics. Mouse clicks are mapped to taps. I can zoom in as expected on the trackpad. I can click and drag to rotate and create a path for our character. The game is fully playable. The application on the Mac is in the Applications folder by default. But as a user, I can rename it or move it, out to desktop, say, if I wish.
On iOS, apps are installed and launched from system-managed app containers and are not used to having their paths arbitrarily modified. To achieve our users' expectations while also remaining compatible with the apps, we play a couple of tricks. First, we have a new app bundle format which wraps your application as is. This allows the user-visible app to be freely renamed or moved.
Second, to ensure that apps are launched as compatibly as possible, we use a feature we introduced in macOS Sierra called App Translocation. This allows the iOS app to be efficiently launched from a sanitized path that is not affected by the user's actions. Let's look at a different type of app, Documents by Readdle.
While Monument Valley is an example of a nice, pure gaming experience, Documents is a productivity tool that uses a number of advanced iOS features, and it's a good example of how these come across on the Mac. For instance, this application supports iPad Multitasking, which means the window on the Mac is automatically resizable.
I can even go into full screen. As you see here, we have macOS scrollers, and I can scroll with two fingers on the trackpad. The app automatically adjusts to macOS's appearance-- dark... and back to light. And since this app supports multiple windows on the iPad, it also does so on the Mac.
From within the app, I can access resources on the Mac, such as my Photos library. Accessing other files brings up a native macOS open panel. Here, for example, is a PDF file. iOS apps can also automatically make use of macOS share services, such as the ability to bring up a Mac Mail compose share sheet for e-mailing content.
I want to show you one more thing, that extensions in your iOS apps work where appropriate on the Mac. Let's open Photos, choose a photo and edit it. I have an iOS app installed, and it provides a Photos editing extension. This extension automatically appears in Photos extension list. I can choose it and enhance my photo.
That's amazing. Using an extension provided by an iOS app, as is, in the Mac Photos app. That's a look at iPhone and iPad apps on Macs with Apple SoCs. Users will love the flexibility to run these apps at their desktops. And remember that for you, the developer, it takes just a few extra steps to unlock more customization and refinement of your app for the Mac through Catalyst.
[Andreas Wendker]
Macs with Apple Silicon represent an incredibly powerful computing platform. They take advantage of the most advanced and power-efficient chip technology, offering unprecedented power-per-watt performance and incredible graphics speed. And they run a wealth of software, ranging from native Universal apps fully optimized for Apple Silicon to Rosetta-emulated apps not yet ported from Intel Macs, to apps from other environments running in virtualization, all the way to unmodified iPhone and iPad apps.
We are looking forward to opening the next chapter for the Mac and seeing how you take it to the next level by leveraging the benefits of Apple Silicon in your own apps. And now, let's talk about macOS Big Sur. Beyond support for Apple Silicon, Big Sur is a huge software release to go along with the transformative hardware changes we are announcing. This release is packed with new capabilities, is faster than ever and has a whole new look.
[Kristin Forster]
The new look and feel of macOS Big Sur eliminates visual clutter in the user interface and gives the Mac an air of freshness. And as you will see in a moment, we've brought the best designs from each of our platforms together. We've created a family resemblance between iPadOS and macOS while retaining the power and flexibility of the Mac. This similarity makes it easy for users to transition between our devices, while making Mac Catalyst and iPhone and iPad apps naturally feel at home on the Mac.
Starting with the Dock, we see a main theme of macOS Big Sur. The Dock is rounded and floats above the bottom of the screen slightly. The Dock corners echo a curvature of a new design for app icons, which all conform to a standard shape. We'll provide templates for you to design your icons to match.
Control Center has come to the Mac, providing easy access to system controls. And Notification Center has a whole new look, built completely with SwiftUI. Notifications and widgets are now shown simultaneously rather than being sequestered into separate tabs. Notifications from each application are grouped together. You can provide a content extension to show custom content using Notification API, which we've brought to macOS from iOS and made available to both AppKit and Mac Catalyst apps. The menu bar floats atop your desktop picture, and the selection floats within it. Menus have a refined layout, and menu selection echoes the rounded theme.
In the Finder, we see a new design for document icons. You can customize your own document icons by simply providing an image and text. macOS will automatically position the elements, mask them and apply the page curl. Sheets come up in a new way. They no longer roll out of the toolbar, but instead float above the window in a rounded platter. The parent window is dimmed, reinforcing the modality of the sheet.
The toolbar in this Preview window sits atop the content area, while the sidebar is a visually distinct vertical element. Your windows will automatically adopt this new structure if you are using a toolbar, a full-size content view and a split-view controller with sidebar styling. Mac Catalyst and SwiftUI windows receive a full-size content view by default.
There are a number of major styling changes in the toolbar. It's taller, and the window title is vertically in line with the toolbar controls. The inline title hides its document icon until you roll over it, contributing to the streamlined visuals. Most toolbar items are positioned to the right, except the navigation items, which remain leftmost in the toolbar. You can designate a toolbar item to be navigational.
Toolbar controls have a new large size and show their backing when you roll over them. The search field collapses into a glyph when the toolbar is space-constrained and expands out to a full field when in use. You can get this great new behavior in windows of your app by embedding your search field in a new search toolbar item. There is additional toolbar structure if you have a three-pane layout like Mail.
The Mail toolbar is split into two sections, with controls corresponding to the message list positioned over that pane, and controls corresponding to the conversation view positioned there. We've added toolbar API to AppKit and Mac Catalyst to express this separation. SwiftUI toolbars attached to NavigationView panes get automatic separators.
As you can tell, the toolbar has gained a lot of new functionality. If you aren't already using NSToolbar, now is a great time to start. Images in both toolbar and sidebar are provided via SF symbols. There are thousands of SF symbols now available on the Mac, and we are using them widely. SF symbols can be configured to perfectly match the size and weight of text.
In the sidebar, the symbols seamlessly adapt to provide correctly sized and styled images when I change my sidebar icon size. Controls have a brand-new look as well, shown here in the System Preferences pop-up and radio buttons. Sliders, progress indicators and other controls have also been updated with a fresh new look.
We are really embracing color in this release. We've given your applications a mechanism to define an app-specific accent color by providing a color name in your plist. This affects not just sidebar icons, but also selection color and the color of standard controls. Mail uses a blue accent color.
We've gone even further with the color theme, giving apps a way to assign colors to individual sidebar items to delineate functionality. In the Mail sidebar, the VIP items are coded yellow, smart mailboxes are monochrome and local mailboxes are teal. The combination of color through tinting and shape through SF symbols contributes to great sidebar design. Notes is another example of app-specific accent color.
Here, the yellow branding of Notes comes alive in the list selection and text highlighting. We hope you enjoy choosing an accent color for each of your apps that expresses its unique personality. As you can tell, every little detail of the Mac has been made fresh and modern, and there are many opportunities for your own applications to participate.
[Andreas Wendker]
Big Sur feels really fantastic, and Mac Catalyst apps also look better than ever before. We continue to improve this technology with every release, and we continue to proof out the implementation in our own apps before making it available to you. Catalyst is basically a Mac-specific implementation of the iOS APIs, most notably UIKit, and while we, of course, recommend SwiftUI and AppKit if you want to take full advantage of the Mac's capabilities, this technology makes it very easy to bring iPadOS apps to the Mac with a single shared code base to support all iPhones, iPads and Macs.
This year, we're making a number of improvements that will make the experience with Catalyst apps even better. Some features require a small amount of adoption work on your part, but you get even better results for the Mac. Most importantly, the new Mac idiom allows you to run your apps at the native resolution of your Mac instead of using the default scaled user interface that matches iPad sizes. The Mac idiom puts you in total control of the user interface, and controls will draw to better match the rest of the apps on the Mac even more closely.
We are also bringing more APIs into the set of supported Catalyst APIs that will provide even better compatibility with iOS and will allow you to make very lightweight customizations in your code that adapt your apps to running inside macOS, on all Macs. We are using it ourselves in critical system apps like Messages, which is now written with Catalyst to give Mac Messages parity with its iOS counterpart. Let's take a look. The new version of Messages has everything you want from a Mac app-- rich menus, key equivalents and multiple windows to take advantage of the large Mac display.
Popovers give quick access to details and integrated Find helps navigate and communicate efficiently. With Catalyst, you get a familiar experience on both the Mac and iOS so you can use new additions like pinned conversations, inline replies and mentions, along with existing favorites such as the Memoji and Messages effects.
-[message chimes] -[party popper pops] The next version of Swift Playgrounds, which we are planning to ship later this year, also shine in the new Mac idiom. It fully takes advantage of the new Mac look and feel, with a simplified toolbar and full-height sidebar, adjustable text sizes and streamlined editing with popovers and the Mac system pickers.
Like Messages, Swift Playgrounds will support multiple windows, making it feel right at home on the large screen of the Mac. If you're an iOS developer and have not yet considered creating a Mac Catalyst app, now is a fantastic time to do so. So, Big Sur is starting the transition to Apple Silicon, is packed with new features, includes a much-improved Mac Catalyst, and on top of all of that has a great new look. It's incredible how much has changed in this release. Big Sur is a huge step forward for the Mac. In fact, so much of the general architecture has improved that we are giving macOS a new number.
Big Sur is macOS version 11, because it really is a new Mac operating system all-around. This is such an exciting year for the Mac. There has never been a better time to be a Mac developer, and we hope you're as excited as we are about the deep changes we are making and the future it opens up for the platform. Next, let's talk about the great improvements we are bringing to iPadOS.
[Josh Shaffer]
This has already been a huge year for iPad. Just a few months ago, we added trackpad support alongside the new Magic Keyboard, continuing iPad's multiyear evolution into a powerful platform for the future of computing. And only a few short years ago, iPadOS gained support for Drag and Drop, a familiar interaction completely rethought for a multi-touch experience. Next came enhanced multitasking capabilities, including the ability to open multiple windows from your favorite apps. And this year, we're taking iPad's capabilities even further, with new ways to navigate and interact with the apps themselves, all designed for iPad.
We've brought these new ideas to apps across iPadOS, and, of course, we added new APIs so that you can bring them to your apps as well. Let's look at a few examples, including the brand-new sidebars, new pickers that streamline data entry, and some new opportunities to refine the interactions in your apps. So let's get started with sidebars.
The new sidebars support a standard two-column layout like you have today, but with just a swipe, they can expand to a three-column layout on every size iPad. This three-column view is even available in portrait orientation, making it easier than ever to browse and organize content, even on the smallest screen.
Adding a sidebar to your app is easy, with all the new functionality available in existing components that you already use. A sidebar can also simplify navigation in your app, like in Photos, where it includes rows for Library and Albums, which were previously found in a tab bar along the bottom. But also rows like Favorites and Places, which used to be nested farther down.
By bringing all the key parts of your app to one place, sidebars can make your app's deeper functionality much easier to access. They also help with organization, with support for displaying hierarchical content in an outline, including collapsible items. These new outlines build on API you're already familiar with in both SwiftUI and UIKit, which have been enhanced to support these new appearances and behaviors.
It's really easy to add a sidebar that perfectly matches other apps on the system, and it can help your app take even better advantage of iPad's large screen. You'll also find some great opportunities to improve data entry in your apps. We've added a number of new pickers, like the brand-new date picker that lets you choose a date by just finding it on a calendar.
It supports this inline style that makes great use of iPad's screen real estate, and if you need to fit it in a smaller footprint, it also has a more compact style. There's also a new inline emoji picker that can be invoked with the same keyboard shortcut as on macOS, and all your iPad apps will get it for free.
You'll also find a new color picker API, which brings a standard way to pick colors across all apps. It's really easy to adopt, and it provides your app standard behaviors, like a place to save your favorite colors for later use, and an eyedropper to pick a color right off the screen.
As you use the new iPadOS apps, you'll notice even more refinements that make interacting with them more fluid than ever, but there are some great opportunities to enhance your apps in similar ways. For example, new lightweight context menus are a great way to organize common actions. We've converted most interactions that previously used an action sheet to these new menus, and I think you're going to love them as much as I do.
Everything we've discussed was designed for iPad, but if you choose Mac Catalyst to bring your app to the Mac, these new APIs are also designed to look great on macOS. And as you look deeper, you'll find many refined interactions in individual apps, like the new inline rename in Files.
Each of these changes is small, but they add up to a much more powerful iPad experience, and finding these opportunities in your apps can be easy too. We looked for ways to make deeper functionality more discoverable, to streamline navigation, and to remove modal states, and I'm sure that you can find opportunities to do so in your apps as well.
These are just some of the enhancements that we've made to apps across iPadOS, and hopefully it gives you some ideas for how you can make your apps more powerful too. It's always exciting what you can do with software alone, but let's also explore how you can incorporate iPad's hardware capabilities to further enhance your apps.
[Sebastien Marineau-Mes]
iPad's capabilities are extended even further through its powerful hardware and accessories. And earlier this year, we announced the new iPad Pro that comes equipped with a LiDAR Scanner. Now, the LiDAR Scanner gives iPad Pro an incredibly precise way of measuring distance, and it paves the way for developers to build entirely new experiences.
The LiDAR Scanner uses direct time of flight to project light onto objects up to five meters away. And that light is then reflected back onto our custom-designed sensor, and by measuring how long that process takes we obtain precise distance measurements of your surroundings. Now you can access this information through APIs in ARKit.
The first is Scene Geometry, which provides you with a 3D mesh of the scene. We also use a neural network to identify the planes and surfaces in the scene, like the floors, the walls, the doors and the windows. The new Depth API gives you access to the precise distance information captured by the LiDAR Scanner. And what we see here is a 3D depth map that was created using the output from the Depth API.
Combined, these APIs bring a new level of realism to all AR experiences on iPad Pro. Now all virtual objects are placed instantly, Motion Capture and People Occlusion is dramatically improved, and with measurements captured by the LiDAR Scanner, virtual objects can be accurately placed in front or behind items in the world. It brings a whole new level of realism to all AR experiences. The LiDAR Scanner also unlocks new capabilities in professional apps like Shapr3D to quickly scan a user's room and generate a 3D map of their surroundings.
And it can also be used for entirely new apps that leverage iPad Pro's precise understanding of the physical world, like those built for industrial maintenance, design and manufacturing, making iPad Pro a powerful tool for manufacturers, architects, and creative professionals. Now another great way to extend the capabilities of iPad is with Apple Pencil.
Apple Pencil is an amazing tool that turns iPad into a powerful drawing and note-taking device. This year, we've added great new capabilities to make handwriting on iPad just as easy and powerful as typed text. Many of these improvements will automatically work in your app without having to do any additional work.
So let's take a look at Pencil's awesome new capabilities in action. Here's a sample app that we've built that helps students practice their handwriting skills. And what's great about Apple Pencil is that it lets you work in a free-form way. You can just start writing anywhere. Writing with Pencil works in any standard UITextField. We've added UITextFields here so that students can write their name at the top of the assignment.
And with the new iPadOS, these text fields automatically support Scribble. And that means that any handwritten word is automatically converted into typed text. Now let's get started on the assignment. Notice that as soon as the pencil is picked up, these beautiful drawing tools appear in the drawing canvas. And because our app uses PencilKit, these beautiful tools are provided automatically.
The new PencilKit makes handwriting as powerful as typed text, so you can select handwritten text using the exact same gestures that you would use for typed text. We've also added a new gesture called Drag to Select, which allows you to select by sweeping the pencil over entire paragraphs, a line, or even a single letter. This makes copying handwritten notes a snap, and it's a sneaky way to get through this assignment quickly.
Now under the hood, drawing with Pencil uses the new Canvas API. These new capabilities have been added directly to PencilKit, so if you're already using it in your iPad app, you'll inherit all of these great new features for free. And this includes the color picker, which makes your apps, and this homework, a little more fun.
Now one of your biggest requests for PencilKit is the ability to get stroke data as Pencil is moving across the screen. And I'm very happy to say that this capability is now available in PencilKit. Using the new Stroke API, you get access to the important attributes of the drawing, like angle, pressure, drawing ink and exact location, grouped into strokes and updated as the user draws. This capability is perfect for our handwriting app. So let's practice.
The student improves their handwriting by tracing these letters. And it was easy to write this app. We simply used PencilKit to show how the Pencil is moving across the intended lines. We also added comparison logic, so that our app can choose to let the student know that they're not quite tracing the letters as closely as they should. It helps them improve as they practice.
The app is built for Pencil, and thanks to the new Drawing Policy API, scrolling with your finger is still consistent with the rest of the system. And iPadOS lets you identify separate areas, some for free-form drawing and others for text entry, which specifies where to apply Scribble. Now as you can see, these new updates to iPadOS and PencilKit make handwriting as powerful as typed text.
Scribble offers a seamless writing experience in any of your app's text fields, and the new Stroke API delivers real-time drawing information enabling PencilKit apps to offer new experiences like annotation, markup and recognition. Now these are just some of the amazing new features that continue to push what's possible on iPad. We can't wait to show you more of the exciting new iPad features this week, and we'll see how all of you customize your apps for iPad. And, of course, iPad apps also benefit from the great features coming in iOS 14.
So let's dive in. iOS 14 is an awesome update that is jam-packed with new features for you to transform your apps. Two of the most exciting developer features in iOS 14 are Widgets and App Clips, both of which have been designed to extend your app's experience and delight your users. First, let's talk about Widgets.
Widgets have always been a great way to convey information at a glance, and in iOS 14 we've completely reimagined this experience, creating all-new widgets that take design to the next level and use on-device intelligence to show users the right information at the right time. They're more beautiful, data-rich, and now they come in three sizes.
The new widgets are available in the Today View, and for the first time we're making them available on the Home Screen. Now users can get at-a-glance information from your apps whenever they go Home, a space that users visit many times a day. The new widgets are written in SwiftUI, making it easy to share code across iPhone, iPad and Mac.
New widgets that work seamlessly across all three platforms. Now displaying widgets on the Home Screen means that they're always visible and always running, so performance and memory efficiency really matter. The previous approach of using a live running extension just could not deliver the efficiency that the Home Screen requires. To solve this for our new widgets, we leveraged SwiftUI's ability to describe your widget's appearance separate from when and how it is rendered, and we've added a new method to serialize the SwiftUI view into an archive.
This archive is very lightweight and can be rendered asynchronously on the GPU to ensure great performance of the Home Screen. The new widget API, WidgetKit, is structured around timeline entries, each of which contains one of these new, efficient SwiftUI archives, as well as time and date and a relevance score.
Each timeline entry is rendered on-demand when it's both timely and visible to the user. So when your widget becomes visible, we can efficiently construct and render a full-view hierarchy from this archive without needing to run your app's code. Your app typically provides multiple timeline entries, so the process is repeated throughout the day without requiring your app to be constantly running. Now each widget the user has configured provides its own set of timeline entries, and the system renders each of them at the right time. And to help optimize the use of space on the Home Screen, users can stack widgets.
Stacks come in all sizes. And the user can easily flip through each widget that is part of the stack. Taking this idea even further is the Smart Stack. Using on-device intelligence, iOS surfaces the right widget to the top of the stack at just the right time. This is where the relevance score that you provide in your widget's timeline entries comes into play. It is your signal to the system that something of high interest is happening, such as this news alert and this weather warning.
Thanks to SwiftUI, we've created this efficient and intelligent widget experience that is unique to every user. And now, I'd like to show you a demo. Before we dive into Xcode, let me show you how easy it is to lay out widgets on your Home Screen. I've already customized my Home Screen with some of my favorite widgets. You'll notice Weather, everyone's favorite topic of conversation, and my Emoji Rangers widget, which shows my panda character in an online multiplayer game.
I configure these widgets in the Widget Gallery. Each of these views in the Widget Gallery are entries that you've created in code. The system knows how to display the timeline entry by writing it to disk to just-in-time render this UI. This technology allows us to create this WYSIWYG gallery for users.
This technology also allows us to create a Smart Stack, which your users can flip through. In addition, the system will intelligently analyze all timeline entries from widgets in a stack and know when to surface the most relevant information for each user. For example, the system knows when there's a relevant calendar event coming up, and the stack can intelligently rotate to that widget.
There. Now I have my widgets beautifully organized on my Home Screen. Now this widget was actually created by my team. We built this widget so that we can see our character's status in the game. Now let's dive into Xcode so that I can show you how we created our Emoji Rangers widget.
In our iOS game, I define my widget in my WidgetKit extension. To define a widget, all you need to do is conform to the widget protocol. Conforming to the widget protocol returns a widget configuration to the system which consists of our kind of widget and our timeline provider. Here I've created a single entry, and I can return that using the snapshot function.
Snapshot is used when the system wants to display a single entry. And I've also created a series of entries in the timeline function which tell the system when to display each individual entry based on the date that I've defined. This is really the engine of my widget. All I need to do is define a widget and return a timeline.
To tell the system when to surface my widget, I define a relevance score. If I have multiple timeline entries, all relevance scores I provide will be weighed against all other relevance scores that I've provided, and the system will dynamically display the entry that is most relevant to my users.
This widget is currently showing my own character in the game, but what I really want to see is how other players are doing and how I'm competing. WidgetKit provides a lightweight configuration ability leveraging the Intents framework. Developers simply pass in the parameters to be configured, and the framework provides the configuration UI. My team has already started to build this out, which you can see here.
They are using the IntentTimelineProvider to generate a new timeline based on the Intents values which are really the answer to the question we're asking. This lets me choose which character to see in my widget. Now I can enter edit mode and edit my widget which brings up this configuration UI.
This UI is created for me by the system. I didn't have to lift a finger to create this view. Just by using the power of Intents, the system can dynamically create this UI on my behalf. Now I can even select my teammate's characters to view their status in my widget.
This is how you define a widget that is easily customizable and can be read by the system to intelligently surface the most relevant information from your app. These are just a few of the amazing possibilities of WidgetKit. Next, let's look at some of the other exciting features in iOS 14.
[Darin Adler]
We're also introducing App Clips. An App Clip is a small part of your app that is light and fast and easy to discover so your users can quickly get what they need, right when they need it. Everything about App Clips is designed for speed. They start with this card which quickly pops up. It's auto-generated by Apple from metadata you submitted to the App Store, so users will find it familiar and safe. And as soon as you see this card, behind the scenes, the App Clip is already downloading.
And with just a tap you can launch the App Clip. You don't need to enter in credit card numbers because App Clips can use Apple Pay for payments. And you don't have to manually log in to an account because it can take advantage of Sign in with Apple.
App Clips won't clutter your Home Screen and will only stay around as long as you need them. But you can easily launch recently used App Clips from the new App Library. And discovery is key. App Clips are all about getting to a part of the app at the moment you need it, so it was critical that we made them really easy to discover. You'll be able to open them from apps you use every day and places in the real world. And when a user sees our new Apple-designed App Clip code, they will know there's an App Clip waiting for them.
And the great thing about App Clips for you, the developer, is that they're made from a part of your app. You use the full power of the SDK. Make your clip as small as possible, so users are in the app instantly. Leave out analytics libraries you don't really need. Do that, and you won't be even close to the ten-megabyte limit.
App Clips can present notifications within eight hours of opening one, like reminding you that your parking spot is about to expire. There's even an App Clip-specific API that lets you verify the App Clip launched exactly where you expect it to be. An App Clip is the best way to introduce users to what your full app offers, so we made it really easy for them to upgrade to it. You can prompt the user to get the full version of your app at the time that makes the most sense for your App Clip. And what's great is that we seamlessly migrate any choices they've made over to the full app as part of that upgrade.
Here I have the Fruta app. Fruta lets you see a menu of delicious smoothies, place orders and explore smoothie recipes. Placing orders works by invoking Fruta with the right URL. Let's make an App Clip for this ordering experience. First, in Xcode, I'll choose "New Target" and choose "App Clip" to create a new App Clip target. Next, I'll select the classes and asset catalog that are needed for ordering and add our App Clip to their target membership. Now, let's build and run.
Great! Here's my App Clip. It preserves my app's ordering experience. You'll notice that my App Clip asks permission to send notifications to customers when their order's complete. We could streamline this by using eight-hour notification permission, exclusive to App Clips and designed for light and fast experiences. To do this, I'll add a check.
If my App Clip has that eight-hour permission, I won't request authorization, so the user won't be prompted. I also want to configure a StoreKit overlay to let the user get my app. When the order is ready, it's the perfect time to present it, along with an explanation of what they will get in the full app.
And there you have it. Now my customers have a streamlined experience in my App Clip, and when they're done ordering, they can get my app and take the recipe home. And that's App Clips. Immediately discoverable in a variety of ways. Built from a small part of your app so they launch fast. And with features like Sign in with Apple and Apple Pay, user privacy is built right in.
In fact, privacy is incredibly important to our users. With our newest products, we're making it easier to give users transparency and control with respect to their data. You now declare privacy information about your apps right on the product page in the App Store. To do this, simply go to App Store Connect and answer a few questions about how you collect, share and protect your users' data. The highlights will be presented to potential users right on the App Store across all platforms. These are just some of the new technologies for you to take advantage of in iOS 14.
[Lori Hylan-Cho]
Last year was a very big year for Apple Watch developers. In watchOS 6 we introduced independent apps, the App Store on Apple Watch, and, of course, the first-ever native UI framework for watchOS, SwiftUI. In watchOS 7, we're giving you a new place to use your SwiftUI skills, and your SwiftUI views, with SwiftUI complications. And we're giving you more space on the face by letting you create multiple complications for each family.
You now have incredible flexibility to provide beautiful and timely information when the user raises their wrist. We're also making it easier than ever to get your app and all its information-rich complications into users' hands with Watch Face Sharing, which integrates App Store purchase and download right into the sharing flow. Let me show you some of these cool new features with an app I've been working on to help my friends in Hawaii track the humpback whales that visit in the spring. And me to live vicariously through them from here, since travel is pretty restricted right now.
In my Whale Watch app you can see recent whale sightings from several viewing stations around Maui, and you can also submit a sighting if you happen to spot a pod. Now I'd like to make it easier for me to see what's happening with the whales from here, and for my friends to log sightings themselves, right from the watch face. It's super easy to describe the complications I'm creating and the families they support.
First I'll offer complications that display data about the most recent sightings for the various viewing stations around the island, similar to how World Clock offers a complication for each of the cities you've selected in the app. I can do that by looping through the list of stations and creating a complication descriptor for each one. As you can see, each complication uses the station name as a unique identifier. This makes it possible to have each complication navigate to a different part of my app. For example, I can have each station-specific complication go to the detail screen for that viewing station.
Next I'll create a complication for quickly logging a whale sighting. Again, I can use the unique identifier to navigate directly to the screen for logging a sighting when the user taps on it, and I can also use it to provide a different template for this complication than for the station-specific ones while supporting the same families.
Finally, I'll create a third descriptor for my Season Data complication. With the other complications I'm supporting all families, but this one is more complex, so it only makes sense in the large rectangular space. I'll finish up here by calling the handler with my descriptors array sorted in the order I want the complications to appear when the user edits their watch face.
I mentioned that watchOS 7 now supports SwiftUI complications, and I have some great options here, from the new native APIs for Gauges and Progress Views to repurposing SwiftUI views from my app. I've already got a chart of recent whale sightings across the island in the app, so I'll use that.
As you can see, that's as easy as wrapping the SwiftUI view in one of the new View templates. I'm using a version of my Chart View, with the font size adjusted down a bit, and populating it with data from the past week, which is a nice, digestible amount of data for a watch face.
To see how everything will look, I'll use my favorite new feature of Xcode 12, complication previews. You may have noticed I was handling all the different complication families in that giant switch statement. Now I can see what those complications look like on different watch sizes and in full color or tinted by writing a bit of SwiftUI. And of course, previews update live, so if I don't like how my complications look, I can adjust the code and see what impact that has.
Moving to my watch, you can see how the complications I've built would appear to a user and where I can customize the complication selections to create the canonical Whale Watch face for sharing with friends. I'll share this one with myself to bundle with my app or post on social media.
While I'm here, I think an island friend who's been testing my app for me would appreciate a face that includes her favorite water sports along with my Whale Watch complications, so I'll make a quick face with surfing and paddleboarding complications, sighting data from the Whale Watch station closest to her favorite beach and the Log a Sighting complication in case she spots a pod of whales while out on her board. She'll get a text from me with a preview of the watch face, and when she taps on it she'll be offered the opportunity to get any apps she doesn't already have from the App Store.
When she adds the face to My Faces, it'll automatically be selected as the default face. So that's a sneak preview of multiple complications, SwiftUI complications and Face Sharing. We're so excited about these and other new tools and capabilities in watchOS 7 that will help your customers connect to the information that matters most to them, and for you and your customers to share your awesome faces and apps.
[Matthew Firlik]
You've seen an overview of the most important changes to our operating systems. And the best way to take advantage of the new functionality is to get the new version of our Xcode developer tools. The first thing you'll notice about Xcode 12 is how it really shows off the new look and feel of macOS Big Sur with a cleaner and more expressive UI. The modernized toolbar puts controls right where I need them, and the icons now have a fresh new appearance.
The font size of the navigator now matches the system sidebar setting, and I can adjust the size separately, too, to get the look I like. Documents now show through under the toolbar, and inspectors have been updated with new spacing and layouts. So Xcode 12, it's really gorgeous. I can get a simple and clean layout and work quickly. For example, I can maximize my work space for code, going into full screen, and then use the slide-out sidebar to quickly access files or even perform a search to move throughout my project.
Now I have something I'm really excited to show you-- document tabs. Document tabs appear in the familiar space across the top of the editor and are a great way to organize my work. I can use the dynamic tab, shown in italics, for most of my navigation. Any file I select in the navigator opens in this tab. When I find a file I want to keep open, I can do a double-click on it, or hold down Option when I click, to have it open in a new tab.
And this works great, too, with actions like Jump to Definition, or with Open Quickly. I can also open any kind of content in a tab, such as the commit view from the branch I'm working on, or a build log I might want to reference later. So tabs let me easily manage the set of content I want to work on.
I can also use many tabs, too. For example, adding to the test classes I already have. I'll use the File menu to split my editor and then drag a group of tests into the tab bar. And now I can work through all the files, one by one. And when I am done working on them, I can close them all at once by closing the split.
Document tabs built into Xcode are fast and fun to use. Xcode 12 supports the new features and APIs in our latest SDKs, providing a great way to develop for all Apple platforms. Xcode's new SwiftUI app templates include everything I need to create common code to share across platforms and then tailor for each UI idiom.
These groups and their targets are created automatically in new projects and help me keep good layering as my project grows. Keeping an app working great on all platforms requires testing, and Xcode 12 makes achieving broad coverage even easier. For tests which have been factored into reusable functions, I can now navigate through the nested calls, just as I would when debugging, to find the source of a problem.
My tests can now also evaluate the responsiveness of my UI by working with animations. UI tests can specify the velocity to use for scrolls and presses, and performance tests can capture diagnostic information. I can review details such as the animation duration, frame rate, even the number of UI hitches that occurred.
Our biggest addition to testing is with StoreKit. The new StoreKitTest framework lets me specify in-app purchase and subscription information right in my project. So I can test without needing to stage or deploy my app. And working with StoreKit is now interactive. Xcode has a new StoreKit Transaction Manager that shows details for apps I'm debugging.
Here's my current session running in the simulator. So now I can select an item, buy it and confirm the purchase, and then review the transaction in Xcode. I can even refund it and my app updates live. So now developing apps with StoreKit is almost as much fun as purchasing items with it. Of course, creating an app is way more fun than testing, and we've made working with Swift and SwiftUI even more awesome in the source editor, with previews, and especially using both.
SwiftUI previews have a new toolbar across the top that speed up common actions. Here I can duplicate the preview and then use the inspector to change the appearance to be dark. This makes working with my previews just like designing a UI control. I can now quickly edit my views by double-clicking in the preview, which scrolls the matching code into view and smartly selects the content to change. Adding modifiers is faster, too. The inspector now recommends modifiers based on my selection, showing them here at the top of the list. And I can search for modifiers, too, to find the ones I need.
Code completion with Swift will feel totally new. The results come back up to 12 times faster, and a simplified presentation makes it easy to find what I'm looking for, particularly amongst similar results. And code completion goes way beyond just helping me type. It helps me get what I want. The same placeholders I get from the Library, that compile with no additional input, now appear for code completion results, too. This means I can make quick additions and customize later, keeping my code and previews compiling the whole time.
Along with the tools helping enhance my UI, my UIs can now enhance the tools. Views and modifiers in my project can now appear in the Library, and with their own default content, by just implementing a protocol. And they can do so from Swift Packages, too. Swift Packages now support all the necessary components, including building asset catalogs and resources and localizing content for use with SwiftUI. So now I can create SwiftUI controls in Packages, design them with Previews and add them to the Library all with just a little code.
So I can switch back to my view, open the Library, and find my control in with the others. And then just add it to my UI and double-click to customize it. It is now easier than ever for me to create my own controls, reuse them in all my projects and share them with others.
There is a lot to discover in Xcode 12 when you try it out. A great user experience with document tabs, new testing functionality with StoreKit, tons of workflow additions for designing great apps, and so much more. And these all build on top of some dramatic improvements we have made this year with Swift and SwiftUI.
[Josh Shaffer]
With last year's introduction of SwiftUI, we took a huge step toward a new way of building apps. We started with a strong foundation of core infrastructure and components and deep integration with our existing frameworks. The full surface of mature frameworks like AppKit and UIKit can't be covered all at once, so we're taking our time to get it right.
We're prioritizing source stability while making the API more powerful over time. As a result, this year's SwiftUI API improvements are all completely additive. Code you've already written will continue to compile and run with this year's releases, with no migration required. That's as important to us as it is for you, because Swift and SwiftUI are playing a central role across the Apple ecosystem.
We're evolving them together and creating a set of APIs powerful enough to build everything from the system features, like the new Control Center and Notification Center on macOS, to entire multi-platform features like the new Widgets. We've even gone as far as implementing the new color picker API in UIKit using SwiftUI. This extensive usage requires great performance, and the language and frameworks teams have worked together to achieve some dramatic improvements this year. You'll notice faster launch and layout, smaller code size and memory usage, and more.
One of the most important pieces of SwiftUI's performance is the layout system. Many of you have been asking for a collection view to enable high-performance display of large data sets. We didn't include one last year because we had a bigger vision that we wanted to pursue. We saw an opportunity to support high-performance collections without an entirely different set of APIs. You can already build complex layouts like this list of photos with SwiftUI today, but large collections can be expensive. This sample contains 20,000 photos and consumes over 500 megabytes of memory.
With this year's releases, you can simply add the word "Lazy" to your existing stack view to enable faster load times and dramatically reduced memory usage for large collections. This same 20,000-photo sample still enables fast scrolling to any point in the list, but now it loads instantly and consumes far less memory. And for more complex needs, we've added new lazy grids as well.
This is a great example of the flexibility that comes from SwiftUI's declarative syntax. You can learn a single set of layout primitives and then use them to compose ten views or 10,000. Combined with new support for switch and "if let" statements in the expanded Swift function builder syntax, you can now efficiently display large, complex collections of views.
Our focus with SwiftUI last year was to enable you to add new views to your existing apps while sharing more of that code across all our platforms, all supported by deep integration with our existing UI frameworks. This year we've, of course, enhanced and expanded on that foundation with new API covering things like outline views and paging views, and we even began adding SwiftUI interfaces to other frameworks like MapKit and AVKit. But we have some bigger enhancements as well. Today we've talked about many technologies to help you bring your existing apps to new places. But with Swift and SwiftUI, we know that you'll also want to build brand-new experiences and deploy them everywhere.
So we're also taking the next big step, expanding SwiftUI's learn-once-write-anywhere story with API that lets you deliver fully native experiences across Apple's platforms, starting from your first line of code. SwiftUI now includes an incredibly concise way to describe your app's structure in just a few lines of code. It supports common elements like windows and toolbars, but also platform-specific features like menus and preference windows. It's a common set of app structure APIs for all Apple platforms that also supports the features that are unique to each one.
Matthew showed you the new shared app template, but let me show you what makes it work. To give you a sense of how powerful this will be, let's build a simple document-based text editor. You'll notice the API is designed around Swift's new "@main" feature, which identifies the main entry point of your application.
Apps use the same declarative structure you're familiar with from SwiftUI Views, enabling your text editor's entire interface to be expressed in just four lines of code. To start, we'll add a Document Group, which manages creating and opening files. Of course, a document needs a view to display and edit its content, so we'll add a simple plain-text editor and connect it to our model.
With just these few lines, we can build and run, and we already have a fully functional multi-document app with all the behaviors that you expect. If we start typing, you'll see it automatically marks the document as edited and needing to be saved, and we even get automatic support for Undo.
The app is fully native to each platform, providing standard behaviors like open panels on macOS, and on iPhone and iPad you get a standard file browser. Of course, it's also easy to add platform-specific functionality. For example, we can add a preference window on macOS with just a few more lines of code. Let's add a Settings scene and fill it with a view that I created earlier.
Composing the different parts of an app is just as easy as composing SwiftUI views. If we run it again, we'll find the preferences window we expect, brought up with a preferences menu item that was automatically added in the right place and configured with the appropriate key command. SwiftUI's declarative app structure can provide appropriate platform-native behaviors like these automatically.
Just like with Views, you can learn these new application APIs once and then apply them everywhere. We're following the same development philosophy, too, starting with a strong foundation and deep integration with our other UI frameworks and then expanding support over time. This is the next big step in bringing you a language and API to build fully native experiences across Apple's platforms, without compromising on what makes each platform unique, all using Swift and SwiftUI.
[Andreas Wendker]
And that concludes our overview of the most important announcements for Apple platform developers this year. The new releases are full of new features. There are so many new capabilities and APIs that we didn't even have time to talk about here in the State of the Union, such as the new ways Core ML allows you to encrypt machine-learning models and to deploy them via iCloud.
Or tvOS, which continues to be the best platform for the big screen, now with AirPlay support for 4K HDR streams, multiplayer support for games and many more new and enhanced functionalities. As usual, we're also making developer previews of all our operating system and Xcode releases available to you for download at developer.apple.com. If you're a Mac developer, don't forget to sign up for the Quick Start Program and apply for a DTK with Apple Silicon there as well.
To learn even more about all the new technologies we've unveiled today, please take advantage of the more than 100 sessions available throughout the week. We can't wait to see how you are going to use the new APIs and technologies and enjoy your great apps. Thank you, be well, and I hope you enjoy the week ahead.