Hi, my name is Edwin and I’m on the software engineering team for Apple Vision Pro. In this session, together with my colleagues Sharlene and Enrica, we’ll take you through some of the incredible new features coming to visionOS. Last year, we released Apple Vision Pro to the world. Since then, we have been amazed and inspired with what all of you have built for this platform. From spatial apps that utilize the infinite canvas, to immersive media content streaming at cinema size in your own living room, to novel productivity apps and games.

Your content comes alive with visionOS. This year, we’re introducing even more features to take your apps to the next level with visionOS 26. We'll start with new volumetric features, allowing you to further harness the power of 3D in your apps. Then we’ll share enhancements that make visionOS an even more cohesive spatial experience.

We’ll cover new features and accessory support for creating groundbreaking spatial games, as well as walk through how your apps and games can be experienced together with the people right next to you. You can integrate Apple Immersive Video right into your apps and deliver spatial experiences through Safari directly from the web. Finally, we’ll cover new, unique, and powerful features that enterprises can take advantage of.

Let’s start with the new volumetric features coming to SwiftUI. SwiftUI is the best way to build great apps for Vision Pro. Whether you’re bringing your existing iOS, iPadOS, or macOS apps to visionOS, or creating a brand new experience, your app will feel spatial from the start. And now you can make them even more immersive with a suite of features we’ve added to SwiftUI, focused on making incredible volumetric experiences.

Let's start with laying out your content. With depth alignments added to existing layouts, new tools for layout-aware rotations, and 3D support for many SwiftUI modifiers, you can create rich 3D layouts in the same familiar way as 2D UI. Let's dive in a little. Depth alignments are an easy way to compose common 3D layouts. with 2D stacks and layout types.

You can use depthAlignment.front to align your views for improved legibility, as demonstrated in this example. If you want to rotate a 3D object in your app, with the rotation3DLayout modifier, you can rotate geometry within the layout system, all while communicating those rotations back to your view, so your app can react. As we’ve seen, many familiar SwiftUI layout tools and modifiers now have first-class 3D analogs. To learn more about them, check out our session, “Meet SwiftUI spatial layout”.

visionOS apps are composed of Windows, Volumes, and Spaces. Volumetric experiences are uniquely compelling, especially in the shared space. So this year, we expanded their support, starting with UI presentations. Momentary information like alerts, sheets, menus, and popovers are frequently used in apps and are an essential part of its user experience. In visionOS 26, presenting this transient content inside and on top of volumes becomes possible. In spatial experiences where 3D content can overlap user interfaces, key elements like presentations will break through the content to remain visible.

With these visual treatments, your presentations will look great in any context. To learn more about Presentations in Volumes, see “Set the scene with SwiftUI in visionOS.” Windows and volumes act as containers that constrain your app’s UI and content, providing defined regions for presenting and interacting with content in the shared space. Now by configuring dynamic bounds restrictions, your apps can blend even more seamlessly with the real world by rendering outside your app’s bounds.

You can apply this effect to your apps by adding a simple view modifier, specifying how much you want your content to peek outside its bounds. This is a great way to enhance the immersiveness of your app without having to change its size. Interacting with virtual objects should feel natural, mimicking the real world. Now, objects can rotate intuitively with the user’s hand as if they were holding the real thing.

We’ve even integrated this with Quick Look, so manipulating and repositioning 3D content within your space feels even more seamless. Applying this behavior to objects within your apps couldn’t be simpler. There’s no need to implement a complicated set of gestures. Instead, you can opt into these improvements with the new Object Manipulation API using a single SwiftUI view modifier.

Or in RealityKit, using the new ManipulationComponent. And if you're using the Quick Look API for previewing 3D models in your apps, you will get Object Manipulation support for free. There’s a host of familiar frameworks you can use to build apps in visionOS. SwiftUI provides the foundation and user interface elements of your app. RealityKit is the best way to showcase 3D content. And ARKit delivers rich contextual information about the real world environment. Together, they provide a powerful toolbox from which to build best-in-class spatial experiences.

Now, a new suite of APIs deepen the integration between these frameworks. Let’s explore a few of them in greater depth. The Unified Coordinate Conversion API makes moving views and entities between scenes straightforward, even between views and ARKit accessory anchors. This radically simplifies content positioning when building in SwiftUI, RealityKit, and ARKit coordinate spaces.

RealityKit’s Entity and its animations are observable, allowing them to be used directly in SwiftUI views. RealityKit can now contribute to SwiftUI updates just like any other observable, allowing you to easily react to changes in your 3D content. You can write your gesture handlers in SwiftUI and attach those gestures directly to RealityKit entities without going through the RealityView as the intermediary. Applying SwiftUI gestures directly to entities produces a more consistent and powerful API surface that further unifies gesture handling across platforms and frameworks.

In visionOS 1, we provided a way to add SwiftUI Views into your RealityKit scene by way of RealityView attachments. Now, we’re introducing a ViewAttachmentComponent that lets you declare your UI in-line with the rest of your RealityKit code. You no longer need to declare your attachments upfront and can add them as desired right into your scene as components.

Model 3D is a great way to present 3D content in your app, and we're extending it to do even more, like switch between configurations defined in your models. Now you can play, pause, resume, and stop animations, control animation playback time, load and select variants from a USD file, and load and select configurations from a .reality file.

But what if you want to do more with your 3D content, like make it glow or sparkle with particle effects? For that, you need RealityView. RealityView works seamlessly with the layout system and includes a new view modifier, realityViewSizingBehavior that gives you flexibility in how your RealityView will size itself relative to the 3D content within it.

These are just a few of the improvements coming to SwiftUI and RealityKit. For even more, see these sessions. And to see an example of how these incredible new APIs come together in a single app, see the Canyon Crosser sample on developer.apple.com. Compelling visuals are just part of the story. Now you can create even richer spatial audio experiences, no matter which playback API is right for you.

Previously in visionOS, all sounds that were played using the AudioToolbox or AVFoundation frameworks, spatialized from the app’s first window. Now, the new Spatial Audio Experience API lets you spatialize each sound from its own window or volume. You can even move sounds seamlessly from one scene to another. This year, we’re also adding new features to RealityKit that can enable your apps to blend even more seamlessly with the real world.

Environment Occlusion enables virtual objects to be occluded by static, real-world objects. This behavior is automatically applied to pin Widgets and Quick Look 3D models. And you can bring this behavior to your Immersive RealityKit apps by adding an EnvironmentBlendingComponent to any entity. Another cool feature is the new MeshInstancesComponent that lets you efficiently draw numerous, unique variations of an asset.

This sample demonstrates rendering and animating 1,000 distinct objects derived from just five unique assets. Combined with AV1 Image File Format texture support and easier ways to display images of all types, including spatial images, you can increase the quality and performance of your 3D scenes in the shared space. We are adding many features like scene understanding updates, entity updates, and a brand new type of image with the new ImagePresentationComponent.

We couldn’t be more excited to see the creative and delightful ways you adopt these APIs in your apps. To learn more about these updates, check out the “What’s new in RealityKit” session. Or to see how you can build a game using RealityKit, download the Petite Asteroids sample on the Apple Developer website. Now I’ll hand it over to Sharlene, to take you through System Features.

Thanks, Edwin. Hello, my name is Sharlene, and I’m a Program Manager for visionOS. Let’s talk about System Features. Advanced Machine Learning is an integral part of Apple products. In March, we launched a suite of Apple intelligence features, and we continued to add even more updates to our foundational models and features like Image Playground. Let's go through some of them.

With the new Foundation Models framework, you get direct access to the on-device, large language model that is at the core of many Apple Intelligence features. Guided generation allows you to generate structured output, and tool calling lets the model autonomously perform actions that you define in your code. To learn more about these capabilities, you can check out these sessions.

In visionOS 2.4, we launched Image Playground for or visionOS, enabling you to turn descriptions, concepts, and people into fun original images, as well as integrate image generation capabilities in your apps through the Image Playground framework. In the latest iOS, macOS, and visionOS, Image Playground gives you more ways to express yourself.

When you’re making images inspired by family and friends, you have more control to customize expressions or attributes like hairstyle. In Image Playground, you can now make images using ChatGPT, as well as access all new styles, like oil painting or vector. And for moments when you have a specific idea in mind, use the “Any” Style to describe what you want. These features are all automatically available through existing Image Playground APIs.

In addition to Apple Intelligence features, we are introducing a new speech-to-text API for iOS, macOS and visionOS called SpeechAnalyzer. SpeechAnalyzer leverages the power of Swift, an advanced speech recognition technology to unlock state-of-the-art speech-to-text capability. It is already powering features across many system apps like Audio and Call Transcription in Notes, FaceTime Live Captions, and more.

SpeechAnalyzer makes it easy for you to build exciting features like these for your own apps. Along with the new API, we are releasing a new speech-to-text model, SpeechTranscriber. This model is faster and more flexible than our previous generation model and is ideal for challenging transcription tasks like media captioning. And SpeechAnalyzer runs entirely on device, keeping your data completely private.

Part of the magic of spatial computing its ability to seamlessly plan digital content into your existing environment. Now, your windows, scenes, and even 2D or 3D content in Quick Look can persist, reappearing in the same place even after restart. Additional SwiftUI restoration APIs allow you to specify what scenes can be restored, as well as react to snap surfaces.

For example, access your windows like Inspector Window in your app can now suppress being created on launch and be flagged to disable restoration with two APIs that might look familiar from macOS. These APIs and a new surface snapping environment property allow you to ground your app in a space, dynamically changing content based on surface type and providing the right set of windows the next time your app is visited in a space. We are also bringing this behavior to a new class of content. Widgets.

Widgets are lightweight app extensions that offer helpful, personalized information at a glance, like a weather forecast or a calendar event. Widgets can snap to walls and tables, blending into your environment and remaining right where you place them. In visionOS, widgets written for iOS and iPadOS using WidgetKit and SwiftUI look even better. Your widget view will automatically adopt the new visual treatments, leveraging depth and dimension, no changes required.

We’re also introducing additional APIs for visionOS widgets. Because widgets live as virtual objects in your space, you can interact with them from different distances. A levelOfDetail API allows you to show alternate views based off user proximity, so your widget displays just the right amount of information. And we’ve added the widgetTexture API, enabling you to adjust your widget’s appearance from glass to paper. Next, let’s talk about games, starting with input.

In Vision Pro, hand and eye input is the primary way for people to interact with your content. For immersive space apps and games that require fast hand movements. visionOS 26 now supports hand tracking that is up to three times faster. This can make your games feel even more responsive with no additional code.

We’re also introducing support for two spatial accessories that offer brand new input mechanisms on visionOS, Sony PlayStation VR2 Sense controller, and Logitech Muse. This spatial game controller is great for games. It can be tracked wirelessly in 6 degrees of freedom, supports hand breakthrough and system navigation, as well as system level gestures like tap out of the box.

With these new capabilities, titles like Resolution Games’s Pickle Pro can unlock powerful new player experiences. We’ve also made it easier to find your connected accessories. Now, your PlayStation Dual Sense, Xbox Series X and S, and Xbox One controllers will break through virtual content by default, even when you’re immersed in gameplay.

For spatial experiences that require precision tasks like drawing or sculpting, Logitech Muse unlocks entirely new creative workflows for visionOS. Its four sensors allow for variable input on the tip and side button, as well as with haptic feedback for integration in your apps. These spatial accessories work when you have one or multiple apps open.

They provide haptic feedback to enhance immersion in your app. You can use the familiar GameController framework to discover Bluetooth connections. And for tracking, you can either use RealityKit or ARKit if your app uses a custom renderer with Metal and Compositor Services. To learn more about how to add spatial accessory input to your apps, see or explore spatial accessory input on visionOS session. In addition to accessories, a new suite of software features expand what’s possible when bringing your games to Vision Pro.

This year, with new increased memory limit support in visionOS, you can bring high-end iPad games to Vision Pro by using App Store Connect. By building your existing game against the visionOS SDK, you can take advantage of Vision Pro's high-resolution display and out-of-the-box game controller support with little to no additional code.

When building with native frameworks, you can use the progressive immersion style to bring more dimensionality to your existing iOS apps and games. This allows you to customize your range of immersion and is a compelling way to gradually present an immersive experience while keeping people grounded in the real world. It is great for existing iPhone or iPad games that have been recompiled to work on visionOS.

This year, we have expanded the progressive immersion style to support new landscape and portrait aspect ratio, which is ideal for presenting high motion games. You can leverage both of these new treatments using SwiftUI and RealityKit, And if you are bringing your own metal rendering engine to visionOS, we are extending the progressive immersion style to Compositor Services.

To learn more about using the progressive immersion style in your game or how you can transform your existing iOS or iPadOS game into a unique spatial experience, check out these sessions. Compositor Services allows you to bring your own metal rendering engine to visionOS, enabling you to render to an immersive space in full, mixed, or progressive immersion style.

This year, Compositor Services is adding two new features for you to enhance your app experience. Hover effects and dynamic render quality. Let’s see what they do. Hover effects enhance interaction by adding a highlight effect when users look at the interactive elements within your app. And this year, we are enabling privacy-preserving hover effect support for Metal Immersive Apps. To use this feature, configure the tracking area’s texture format, add tracking areas to your drawable, then configure your hover effect on the tracking area and draw your render value.

Optimizing your app’s render performance can require fine-grained control, so this year we added support for dynamic render quality in Compositor Services, enabling your highly detailed scenes to shine. Increased resolution in your rendered scenes can significantly enhance the user experience. With this new feature, you can control the quality of the frames rendered by your app.

As you boost render quality, the high relevance area in your texture expands, leading to a larger overall texture size. Increasing the quality also means your app use more memory and power. If you are rendering text or user interface elements, setting a higher render quality is beneficial. To ensure your app runs smoothly, finding a balance between higher resolution visuals and managing your app’s power and memory usage is important.

Another powerful feature coming this year is macOS spatial rendering, allowing you to use the power of your Mac to render and stream immersive content directly to Vision Pro. From previewing spatial content by adding immersive experiences to existing Mac apps, to creating a fully immersive game, your app can combine the incredible rendering capabilities of the Mac with Vision Pro's advanced, high-resolution displays.

To build a Mac-supported immersive experience, we'll use the same familiar frameworks that allow you to create native immersive apps. Compositor Services and ARKit are now available on Mac and can be used with the new Remote Immersive Space SwiftUI scene type. For more details about hover effects, dynamic render quality, and macOS spatial rendering, watch the what’s new in Metal Rendering for Immersive Apps session.

In visionOS 26, we’re also making playing games together even more engaging with new functionality in TabletopKit. TabletopKit enables you to focus on bringing your most imaginative shared games to life by handling networking, input, animations, and game layout. To allow for even more flexibility and control, we’re introducing two new APIs. CustomEquipmentState lets you add your own data fields to game pieces, known as Equipment. This custom state is automatically networked for seamless synchronization. And with CustomActions, you can define your own actions to trigger throughout play. From ownership changes to dynamic color updates, your custom actions are networked effortlessly.

You don’t have to worry about the hard bits like networking and input, so you can focus on what matters most, your game. To learn more, check out the sample TabletopKit Mini-Game. Next, I’ll hand it off to Enrica, to talk about Shared Experiences. Hi, my name is Enrica and I’m an engineering director on the visionOS Apps team.

Let's walk through some of the incredible new features coming to visionOS this year.

Many of the best spatial experiences are shared and they come to life through SharePlay and Spatial Personas. SharePlay makes it easy to bring shared experiences to your apps. There’s something magical about being able to experience virtual content with people halfway across the world. We have seen this capability unlock remarkable use cases, ranging from enterprise, productivity, to entertainment, and more. But what if you want to bring someone who’s physically next to you into a spatial experience in Vision Pro?

In visionOS 26, Nearby Window Sharing enables you to build shared spatial experiences for people co-located in the same room. Like Defenderella by Rock Paper Reality, a tower defense game that comes to life in your own living room. You can take advantage of Nearby Window Sharing in your existing SharePlay apps with no additional code.

Additionally, ARKit has added support for shared world anchors, making it possible to precisely anchor shared content to the room. You can move, resize, and snap apps and content to your surroundings during a SharePlay session. And with Quick Look, you can manipulate, scale, and hand off virtual objects to one another, just like physical objects. And Spatial Persona is now out of beta, complete with an expansive suite of improvements to hair, complexion, expressions, representation, and more. To learn more about building Nearby Window Sharing experiences, check out this session.

The amazing new spatial experiences don’t end there. Let's talk about media. Around the world, people love reliving their memories on Vision Pro. And now, they have an incredible new way to experience photos. You can use RealityKit’s new ImagePresentationComponent to take advantage of on-device generative AI to display monoscopic or spatial stereo content in a 3D spatial scene, optimized for real-time rendering from multiple points of view.

This enables users to lean in and look around content as if they are physically there. Vision Pro is the best way to watch 2D and 3D movies, spatial photos and videos. And the breakthrough Apple Immersive Video, which continues to push the boundaries of what’s possible with real-life acuity and fidelity of presence.

Today, media formats span a comprehensive spectrum. They include 2D video, 3D stereoscopic movies, and spatial videos shot on iPhone or Vision Pro, and Apple Immersive Video, which puts you in the center of the action, complete with Apple Spatial Audio. This year, we're going even further by adding support for three new types of media experiences.

We’re bringing system level support for 180º, 360º, and wide field-of-view media captured by popular third party cameras from Canon, GoPro, and Insta360. Altogether, this gives visionOS 26 the most comprehensive lineup of ways to experience immersive media. This industry leading support is made possible by the Apple Projected Media Profile, which provides a flexible way to express additional per-pixel geometrical operations, called a projection, for ideal presentation to the viewer.

The Apple Projected Media Profile, or APMP, is a metadata-based approach that allows QuickTime and MPEG-4 files, to signal that they are 180º, 360º or wide-FoV. This is fundamental to enabling correct playback and editing workflows. We also built into APMP the ability to enable high motion detection, supporting viewer comfort when challenging content like action camera footage is presented immersively. And since there are many existing libraries of similar content taken by popular third-party cameras, visionOS will automatically generate APMP metadata for select third-party cameras.

We mentioned the unique handling of Wide-FoV media from action cameras. So let’s take a closer look to illustrate the core principles of APMP. The capture device on the left creates a standard video frame. Due to the nature of wide-angle fisheye lenses, content is recorded with a distorted image.

The source footage contains barrel distortion introduced by the lens. Borrowing an approach used by Apple Immersive Video, APMP allows our media frameworks to undo the lens distortion by projecting thepixels against a curved surface defined by the camera lens and device capture mode. This brings content previously viewed in 2D into the world of immersive media, providing you a fantastic way to relieve your own action camera experiences.

With APMP, we’ve fully embraced 180º, 360º and Wide FoV media by integrating support into the system. And we can’t wait to see how this opens up new creative possibilities for your apps and camera devices. You can experience all these immersive media formats, including Apple Immersive Video, immediately in Quick Look via the Files app, or inside your app with full support for Progressive Immersion to customize immersion levels.

Playback is supported by familiar AVKit, RealityKit, and Quick Look APIs, as well as WebKit, so no matter what type of experience you build, you can seamlessly integrate stunning media across a variety of formats. Creators that produce Apple Immersive Video content with the new Blackmagic URSA Immersive Camera, and DaVinci Resolve can play it back in their apps or from the web in Safari.

And in the latest updates to macOS and visionOS, the new Immersive Media Support framework enables you to create, process, and share Apple Immersive Video content from your own production pipelines. You can learn more about APMP and Apple Immersive Video, as well as how to support immersive video experiences in your apps in the sessions listed here.

If you’re recording content on Vision Pro, you can now capture three times more content per session with Reality Composer Pro. The simulator now supports 4K resolution in addition to recording for when you don’t have a device on hand. Next, let’s take a look at some of the new features coming to Safari.

Spatial Browsing transforms the web from a flat 2D plane to a canvas that comes to life with spatial experiences. We have a new effortless way to watch and browse videos, keeping video playback in your field of view while referencing additional content on the page. You can transform web pages to unlock inline spatial media that comes alive as you scroll simply by selecting the Spatial Browsing button in the navigation bar. Any page that can traditionally be viewed with browsing button reader mode on iOS and macOS can be transformed when you enter full screen to see the spatial media presented around you, all without leaving Safari.

And for web developers, now you can easily embed 3D models in your web pages using the new HTML model element. Through a simple declarative API, you can showcase your USDZ model stereoscopically. The model becomes part of the page, can be styled with CSS and configured with JavaScript. It can also be dragged out of the page and placed into the real world with Quick Look, giving you the feeling of being able to touch the item you’re looking at.

As mentioned previously, Safari has also added support for a variety of spatial media formats. In addition to displaying spatial images within Safari, you can now add your spatial videos using all the supported formats, including Apple Immersive Video on your web pages with the existing HTML video element, to enjoy spatial content immersively when entering full screen.

We’re also enabling a new developer preview called Web Backdrop, where you can offer custom immersive environments to visitors through HTML markup. We can't wait to see the creativity that your websites will bring to visionOS. You can learn about these features and even more in our “What’s new for the spatial web” session.

We’ve also added a new hands-free way to navigate the web and content using just your eyes, called “Look to Scroll”. This works in building apps like Safari, TV, Music, App Store, Notes, and Mail, as well as Quick Look. And you can adopt this in your apps with APIs in both SwiftUI and UIKit. As spatial computing transforms personal experiences, it is also unlocking powerful opportunities in enterprise. We’ve been inspired at the incredible diversity of use cases, scaling across industries like healthcare, manufacturing, and transportation, to name just a few.

Across these environments, we love what businesses are doing around collaborative design, simulation, guided work, and more. Now, visionOS 26 makes it even easier for enterprises to bring Vision Pro into their organizations. Using approved enterprise entitlements, enterprise applications can extend the main camera’s access to the shared space, enabling users to run camera-enabled experiences concurrently with other spatial apps. And your app can now access an individual main camera or both the left and right main camera feeds simultaneously.

Using ARKit’s new CameraRegionProvider, you can receive an enhanced, stabilized video feed of a select region of interest. Users can leverage this feature to bring critical information from the periphery into their line of sight, or to share a stabilized portion of their view to a remote participant. You can also provide additional parameters to enhance the fixed region, like altering contrast and boosting vibrancy. Beyond the cameras, we added a brand new API, Protected Content, to better protect sensitive data.

Using a single view modifier, you can mark your app’s view as protected, and the system will prohibit content captures of that view. Screenshots, screen recordings, AirPlay, and SharePlay will be deactivated. Lastly, for visionOS apps with licensed entitlements, you can now place Windows in Window Follow Mode, enabling apps to automatically follow your position as you move around in your space. In addition to these new capabilities, we’re making it even easier to deploy Apple Vision Pro in enterprise settings.

Return to Service allows enterprises to more easily and securely share devices between team members of their organization. When enabled, Vision Pro preserves apps downloaded via MDM across multiple people, while guaranteeing all sensitive information is erased between sessions. This reset can either be triggered at a predefined time interval or manually, readying the device for the next person.

To use Return to Service, devices must be configured within Apple Business Manager. Devices are automatically enrolled within Apple Business Manager when purchased through authorized enterprise channels. For devices purchased elsewhere, we have updated Apple Configurator for iOS to make it even easier for IT professionals to manually enroll their Vision Pros in Apple Business Manager to oversee their organization’s devices.

We’ve also made improvements so people can start using their Vision Pros faster than ever before. With QuickStart enhancements, people can now import their saved Vision Pro setup data stored in iCloud or on their iOS device, removing the need for hands and eyes re-enrollment. And for enterprises that do not use FaceTime and SharePlay today, they can now create locally shared experiences leveraging their own infrastructure.

via the SharedCoordinateSpaceProvider API in ARKit using managed entitlements, you can now request access to the underlying data co-locating people within the same physical space, enabling use cases like collaborative design and review. To learn how to take advantage of so many of the features we have talked about today, visit developer.apple.com for samples and documentation. These are some incredible features coming in visionOS 26. We can’t wait to see what you build with them. Thank you and enjoy the rest of WWDC.