2026-11-11 • visionOS • 5:48:25
Join us online to discover how Apple’s immersive and spatial formats are transforming storytelling across entertainment, marketing, enterprise, and live experiences. In this all-day event, you’ll connect with fellow creators, explore the latest tools and platform updates, and get a preview of how to produce live events in Apple Immersive. Whether you’re already producing immersive video or just starting out, you’ll leave inspired to build your next project.
Speakers: Elliot Graves, Jeff Norris, Tim Dashwood, Bryan Aguirre, Alex Miller, Abhaya Parthy, Grant Petty, Laura Triulzi, Anthony Bottini, Jeff Wilson, Jared King, Yuri Emoto, Tim Amick, France Costrel
Transcript
This transcript was generated using Whisper, it may have transcription errors.
Welcome
Well, welcome everybody, whether you are joining us here in Cupertino. It’s so great to have you in the room with us or to everyone watching online. You know, this morning we have immersive media creators from over 74 countries tuned in today. Yes. Maybe we can give our friends from all our friends around the world. Welcome.
All right. Well, my name is Elliot Graves, and on behalf of a lot of the team working behind the scenes and many of the people you’re going to hear from today, we are very excited to have you here in the Developer Center. And this is just one of four Developer Centers where we regularly host people for sessions, labs, and workshops. We also stream content worldwide. And with our fifth developer center opening later this year in Berlin, no matter where you live, there is always something to learn.
Now, today, we have brought together this community, a very special community of immersive media creators. I think we have filmmakers in the audience, we have developers, we’ve got some ad agencies, we’ve got even some enterprise technologists. All here to learn and pioneer creating immersive media experiences for Apple Vision Pro and to share stories that connect with your audiences.
So whether you are placing someone inside the story with Apple Immersive Video or bringing depth and dimension to a moment with spatial photos and video, or defining new ways to present sonic stories with Apple Spatial Audio, we are here to help you explore these possibilities together. Before we get into the fun stuff though, there is just a few quick things for everyone joining us today.
For those of you in the room with us here, please hold off recording or doing your own live streaming whilst you’re in the theater. But everyone is, of course, more than welcome to use your iPhone, iPad, computer, Vision Pro, or any other device throughout the day to take screenshots and notes.
Speaking of, can I get a little show of hands who brought their Apple Vision Pro with them today? Yes, okay, wow, that’s almost half the room. That’s incredible. Anyone bring their URSA Cine Immersive Camera? Yes, OK. Super committed. Well, maybe come find me later. I’ll have a little surprise for you.
If you haven’t already connected, the Apple Wi-Fi network is available and you can jump onto that. You’ll also find some power for those of you in the theater. If you need to recharge your devices, it’s just at the front of your armrests. Okay, so with the housekeeping out of the way, here is what we have planned.
We have a full day ahead of us, and it is all focused on creating immersive media experiences. We’re going to kick off with a little welcome. And then you’re going to hear directly from Apple engineers and content teams as Tim and Alex take us through the latest news in the immersive media ecosystem. With that, you’re going to get a look at some exclusive new and brand new tools being released here today.
Then we’re going to hear from some special guests, giving you a behind the scenes look at some of the best immersive stories of the year as they share what they have learned along the way. We’re then going to take a quick beat for everyone’s favorite part of the day. That’s lunch. And then Jared will join us to lift the lid on how Apple has been producing shows live in Apple Immersive and what it will look like for you to do your own broadcast very soon.
This afternoon, we will then have an awesome panel joining us from around Apple, Vision products teams and others across Apple. It’s a really great panel and they will be here to answer your questions. So if you have a question for any of our presenters throughout the day, please head to this QR code and drop us your questions online. You can do that throughout the day. If you just prefer to listen, don’t forget you can still upvote your favorite questions via this QR code to make sure that they get answered.
After the Q&A, we will sadly be saying goodbye to the online audience, but for everyone with us here in Cupertino, we’re going to shift from the stage into smaller conversations and the demo area. This will give you an opportunity to go deeper on specific topics with the team, spend time with that team, the creators that have joined us today, and explore the tools and the workflows for yourself. Then, closing out the day, we will be hosting a community mixer in the lobby, and that’s just outside.
So, that is the action-packed day that we have ahead. I hope everyone is prepared and ready. But before we get started with our first session, we have someone very special who wants to join us this morning to say hello. So, with that, I am excited to welcome to the stage Jeff Norris, Apple’s Vice President of Vision Products. Jeff.
Hello
Thank you, Elliot. And good morning, everyone. It is awesome to see so many immersive media creators and developers here today and so many more of you joining us from around the world. I just want to take a few moments this morning to welcome you personally and also to say thank you. Because one of the most exciting things about building a new platform like Vision Pro is seeing what happens when we put it in the hands of creative people like you.
You know, when we introduced Apple Vision Pro, we described it as the beginning of a new era for computing. And although it’s been a couple of years, we are just getting started. Because whenever our teams introduce a new platform, we can build the technology, we can create the tools, and we can imagine some of the things that people might do with them. But ultimately, it’s developers and creators just like you who show us what the platform can truly become.
And some of the most compelling experiences that I’ve personally had in Vision Pro are when it transforms your place into a completely different space, especially one you’ve only dreamed of visiting. Now, before joining Apple, I spent years at NASA’s Jet Propulsion Laboratory creating interfaces to control robots and spacecraft. I spent many days working with colleagues at the NASA Johnson Space Center in Texas. But like most people, I had never actually been inside the Neutral Buoyancy Lab, the place where astronauts train for spacewalks on the International Space Station.
That was until a unique training event for Gates’ Immersive Housing made it possible. Because thanks to some skilled camera operators, some of whom I think are in the audience today, I finally got a chance to be there, floating next to an astronaut while sitting in my house. And this is only one of the remarkable experiences designed for Vision Pro that we’ve seen over the past few years.
Apps like the NBA are making it possible for you to watch five concurrent basketball matches and to see plays from any vantage point. 30 automotive companies, including Kia, have integrated Vision Pro into their design processes, making it possible for designers to iterate faster and to collaborate with teams across the world.
Vision Pro is also in daily use in operating rooms, with thousands of surgeries now performed. Surgeons view high resolution feeds of 3D imaging directly in their line of sight, meaning that they don’t have to crane their neck to see some distant monitor. This ergonomic setup makes it possible for them to maintain stamina during very demanding cases. In fact, one surgeon, Dr. Reardon at Houston Methodist Hospital, completed the longest recorded surgery in Vision Pro, spending over 11 hours in device. Eleven hours.
And Cirrus is bringing their customers closer to the ownership story of their aircraft using Apple Vision Pro and Apple Immersive in markets where demo aircraft are not available. There are so many remarkable experiences, but when I look at the immersive media experiences that everyone here is creating, what often makes them so powerful usually boils down to one thing, great storytelling.
And I think one of the most impressive ways you’ve been doing that all around the world is in Apple Immersive. So now, it’s barely been a year since the Apple Immersive tools were first released, and you’ve taken it to places that the team could barely imagine. You’re doing extraordinary things. But don’t let me just tell you. We want your work to speak for itself. So let’s take a look. If you’re dreaming, you can have it. If you believe it, it can happen.
Welcome to the DCC. That comes tonight. Live your perfect life. ♪ ♪ ♪ We got problems ♪ ♪ See them gather on the shore ♪ ♪ Empty promise ♪ ♪ Can’t say nothing anymore ♪ ♪ I’ve been holding on ♪ Incredible, right? That looked awesome on this screen, but obviously there’s no better place to experience that than in Vision Pro, just incredible stuff. What you have created in less than a year with Apple Immersive has blown us away. And thanks to these creators and many, many more, you’re developing an entirely new audio-visual language, and as a result, establishing new methods to share stories no matter your reason to tell them.
You’re asking questions like where should the audience be when they’re present in a moment? Or what should it feel like to stand beside a performer? to be pitch-side at a sporting event, or to experience a moment that might otherwise be impossible to reach. There isn’t one answer. And that’s what makes this such an exciting creative space.
You are leading the way, improving what’s possible in this space. So once again, from the team at Apple, thank you. Now, let’s get this day started. I’m going to hand it over to Tim to kick us off with what’s new in the immersive media ecosystem. Tim. Thank you, Jeff.
What's new in visionOS 27 and immersive media
Morning, everyone. I’m Tim. I’m an engineer on the Immersive Production Engineering team. Today, we’ll explore the expanding immersive media ecosystem and talk about some new tools and technology now available for immersive content creation. We’ve got a lot to cover, so let’s get started with the platform that powers Apple Vision Pro, visionOS.
From iPhone to professional production tools, anyone can create some form of immersive media for Vision Pro. starting with spatial scenes. We introduced this feature last year to convert 2D images to 3D with look around depth. This feature is included in many visionOS apps, such as the Photos app, making it easy for you to convert even old photos to spatial scenes.
This year, as part of visionOS 27, you can also turn your panoramas into spatial scenes, and even add them as a custom environment in your Vision Pro, letting you spend your days in your favorite places from the past. visionOS 27 also brings new updates to viewing immersive content on Vision Pro.
When someone’s watching Apple immersive video, they may want to temporarily reduce immersion and see their surroundings. With visionOS 27, apps can now choose to shift to a wide aspect ratio portal view when someone reduces immersion. The TV app now enables this wide aspect ratio by default, and developers can customize the aspect ratio to any value they please.
We also have new viewing options for 3D content creators. If you are a creator who works with 3D models on your Mac, you may already be familiar with Mac Virtual Display. This feature allows you to view your Mac’s display in Vision Pro, so you can work on a project with a large display, and complete privacy wherever you are. With visionOS 27, we’re taking this capability even further with Spatial Preview, which lets creators preview spatial content from your Mac directly on your Vision Pro and collaborate with others through SharePlay.
You can check this out today with USD files in the preview app, as well as supported Mac apps like SketchUp, and coming soon, Cinema 4D. These apps are helping transform the creative process, unlocking real-time collaborative 3D workflows. Now, recording your visionOS content is one of the things we’ve heard a lot about from creators and I’m happy to share that with visionOS 27, we now support high quality recording on device. Here to tell you more about it is Bryan.
Thank you, Tim, and good morning, everyone. So my name is Bryan Aguirre. I’m a managing content producer with Vision Products. I’m really excited to talk about this next thing because I haven’t had a chance to. So my team has spent the past few years filming with Apple Vision Pro, and since the very beginning, we really wanted to stay authentic to the experience.
How do you do this, right? For us, it was rendering on device. Not a lot of people know that, but a lot of our captures were actually rendered on device. We all know the best way to experience spatial content is by wearing Apple Vision Pro. But how do you preserve this impact when posting to, say, social media, the App Store, or a website? In the past, on-device captures were limited to foveated 1080p recordings. Or there was a more involved process for higher quality mode, which involved a Mac, Xcode, and additional cables.
To help make recording these experiences easier, we released an update with visionOS 27 that allows you to record your view in high quality directly from Control Center. Let me demo this and show you how to do this. So what you’re gonna wanna do is you’re gonna head to Settings, General, Captures and Recordings, and select High Quality. Recordings will capture your view in un-foveated 4K resolution at 30 FPS for up to three minutes. And now all high quality recordings are saved directly to your Photos library, which is super convenient.
Something folks also don’t know is you can now also AirPlay a preview stream as soon as you begin recording. This can be incredibly helpful to crew members, sharing with Video Village. Your field of view in device isn’t always the same as that 16x9 composition you end up getting, right? So now a director, DP, co-worker, they can guide you to see what’s in frame and help center and level your shots.
Since announcing Apple Vision Pro to the world, we learned a lot while on set. Shooting spatial content takes a bit of, honestly, a different approach, but just a few key considerations can really improve the look of your captures. For example, let me share some. Framing your shots to include grounding shadows on tables or the floor helps provide context for the scale or placement of your content. Also, your space, as you can tell, you see it, right? It’s a large portion of your shot. So sometimes UI can run into some physical objects in your room. Just trying to minimize clutter is actually a really positive thing. Here’s a good example with widgets.
Another way to limit distractions is by reducing head motion. This is really difficult to do if any of you have tried to do this in device. Too many quick movements back and forth can really distract the viewer as you’re trying to share your content. This is something in particular we faced when filming, and we know it’s been a challenge for you. So let me show you something that should help with this. We’re happy to announce that today, Beastgrip has announced the Beastcage for Apple Vision Pro.
It’s kind of hard to see, but it’s really on device. This will fit onto both M2 and M5 models. And it’s going to basically help you stabilize your content. So you can attach it to tripods, cranes, any traditional production equipment. And it can also just lock off the device if you need it for testing or anything else.
So for those of you in person, please stop by the demo area. Our team and Beastgrip will be there with a couple rigs so you can get hands-on with them. And we really just want to see what questions you guys have. We want to see how you’re going to use these. So we’re so excited to see how these will help you improve the look of your amazing content experiences. Thank you, and now back to Tim.
Thank you, Bryan. Those were just a few of the features now in visionOS 27. To learn more and dive deeper into what’s new, please check out developer.apple.com. Next, let’s cover updates to our immersive media formats, starting with spatial video. People love shooting spatial video on iPhone, Apple Vision Pro, or Canon’s dual fisheye lens system, and then reliving those memories in stereoscopic 3D.
Spatial video is easy to view in the Photos app on visionOS, and we curate user-generated spatial video in Apple’s own Spatial Gallery app. When it comes to editing your own spatial video, it is supported in popular post tools like Apple Compressor, DaVinci Resolve Studio, and Final Cut Pro for Mac. Coming next year, though, Avid Media Composer will add spatial video import and export to its existing stereoscopic workflow.
And Maxon’s Autograph will add spatial rendering of 3D motion graphics. There’s also now a way to edit spatial video within the Vision Pro itself. Earlier this year, Adobe released a version of Premiere for visionOS. With Adobe Premiere AI Video Editor, spatial video filmmakers can edit their content and add titles or other effects entirely within Apple Vision Pro, while viewing their footage spatially.
And these aren’t the only apps that can edit spatial video. Several other developers have also released spatial editing apps, and some can even convert standard video to spatial video. For creators looking for a wider field of view, there are also some exciting new tools for the Apple Projected Media Profile, or APMP.
This includes 180 degree, 360 degree, and wide field of view action cam style video in 2D or stereoscopic 3D. These formats are already supported by many popular camera systems, like Canon’s dual fisheye EOS VR lenses, Insta360’s X5, X6, and Ace Pro 2, as well as GoPro’s Max 2 and Hero 13.
And capturing APMP has never been more accessible, with Insta360 and GoPro recently adding support for some legacy models, including the GoPro Hero 11, Hero 12, and original Max, and the Insta360 X4 and original Ace Pro. But brand new this year, 360 degree drones, like the Antigravity A1, are now shipping with APMP support. taking APMP to new heights.
But what about the ultimate immersive media format? Apple Immersive Video, or simply AIV? Apple Immersive Video is the highest fidelity immersive video experience available on visionOS. And the ecosystem to create and deliver AIV has grown tremendously this year. But before we get into some of the new tools, we’re first going to look at how content creators around the world are producing AIV. We have so much to talk about, so to kick us off, I’d like to invite Alex up to the stage to talk about content.
everyone and hello online audience my name is Alex and I’m a producer on the Apple Immersive production team and y’all it has been a very big year for creatives in this space with some incredible titles and moments so let’s take a look at some of these amazing experiences the team behind the Elevated series they were busy in the Big Apple releasing the latest episode, New York City. And this time, not only do you soar above this iconic cityscape, but you get to float ground level, getting up close and personal with the people and the magic of the city.
Also, soccer fans got unprecedented access to one of the most storied clubs in football. In the film The Weight of Greatness, you go behind the scenes with Real Madrid and you even get to take a ride on the retractable pitch, which is seriously one of the coolest moments in the film.
And yes, we heard you, horror is in fact perfect for immersive video. In October, just in time for spooky season, Apple’s second scripted film, Remnants, was released, written and directed by horror legend Ti West. This ghostly piece was shot in Ireland, in a real haunted castle mind you, with an iconic cast led by Lucy Boynton. The world that Ti West builds in this film is both eerie and beautifully disturbing, and we captured some first-time viewers who were not prepared for all of that.
You don’t belong here You can’t leave Oh yeah, I saw some of y’all shaking your heads out here, I know. I was there on set when we filmed that entire production and I still scream like that, so it’s just, no, not for me. But you know, check it out if you dare. But jump scares aside, one of the most novel outcomes of this year has been the arrival of so many talented creators now working with immersive media.
And you know, before the Vision Pro was in the hands of customers, the Apple Immersive community started off with just a small group of us. Yes, that’s me. And we had big dreams, you know, of what this medium could be. And like you just saw in Jeff’s video, those dreams, they got way bigger when we found incredible creators and partners who pushed this medium further than we could have imagined. And one of our earliest partners was the team at Red Bull. This year, alongside the Apple team, they released Backcountry Skiing, the first episode of the World of Red Bull Immersive Series.
Following three free skiers as they navigate untouched mountain terrain in British Columbia, this film showcases some of the most fearless athletes I have ever seen and the industry noticed. World of Red Bull won a Sports Emmy. So can we give it up for that team? Yes. That’s it’s a huge, huge, huge milestone for this medium and proof that immersive storytelling is being recognized alongside the best in the industry. So congratulations to everyone who worked on that film. I think some of you are here today. So thank you.
And Red Bull, it was just the start Because like Jeff said, it’s creators who show us what a platform can really become. And it was only at the start of this year that creators like you all around the world got access to the tools to create in Apple Immersive. And it has been really inspiring to watch what you’ve made with them.
Like CNN’s The Vanishing Edge. and Mayday Beyond Us, GucciCore, and Disneynature’s Wolf, which just debuted at Venice Film Festival. And y’all are in for a treat. Director Jeff Wilson from Silverback Films will be here to tell you more about this gorgeous piece a little bit later. And these four titles, you can expect to see them in the Apple TV app and other apps across Vision Pro soon.
So from news to nature, music to fashion, creators everywhere are making films for Apple Vision Pro. But with all of this content coming to life, we kept hearing the same thing from viewers. They want more immersive video, and they want it to be easier to discover. Well, right now, the Apple TV app is one of the best places to find content, but it is not the only place.
And like those we just saw, there are also some fantastic Apple immersive titles releasing on other apps on visionOS. Apps like Fovii are featuring content from the Audi Revolut F1 team, offering a behind-the-scenes look at what it takes to build a Formula One operation from the ground up.
And the immersive media gallery app, Theater, is hosting titles like Kick from Altitude Films, which is a dazzling take on freestyle skiing in the French Alps. Today, we are very, very excited to announce that independent Apple immersive films from Fovii and Theater will now be searchable right in the Apple TV app, which is super exciting.
And to make sure viewers can always find the best content, the Apple TV app will feature standout titles in a dedicated section. Meaning, from today, you can watch awesome music videos like Cure for Paranoia or Fly Through the Skies of Southern California with Air to Air from Rogue Labs with just a tap. We’re really excited about this one and we can’t wait to see more of your work at home in the Apple TV app.
But what if immersive, Apple immersive, didn’t have to just be something you watched after the fact? What if you could be there in the moment as it happened? Well, this year we announced live in Apple immersive, showing what’s possible when we transport viewers in real time to their favorite sporting events, kicking things off with two amazing experiences.
NBA fans were brought courtside as Spectrum SportsNet broadcast a series of games in the LA market, available in the NBA app and the Spectrum Sports app, and Friday Night Baseball offered fans an unobstructed view from the field. And you’ll hear more about how the team made these experiences possible a little bit later.
Now, live might be bringing audiences into the action, but to finish off, I want to talk about how creators just like you are bringing Apple Vision Pro to their audience with location-based activations. We’ve been really blown away as creators and companies and organizations find powerful new ways and spaces to bring people into this medium for the very first time. This included packing rooms at the Paris Opera, where museum goers got behind the scenes access to rehearsals in spaces normally closed to the public. And iconic venues like the Bernabeu, where a stadium tour is being paired with the Real Madrid immersive film, The Weight of Greatness.
And on the road, the Audi Revolut F1 team brought an interactive app and immersive documentary to race day, giving fans an inside look at their return to the sport. And Vision Pro is even transforming exhibitions like the Vivaldi Immersive Experience, which transports up to 15 guests at a time into 18th century Venice surrounded by live musicians performing Vivaldi’s iconic The Four Seasons.
These experiences and all the titles we’ve talked about this morning continue to surprise and delight audiences around the world. But none of them would be possible without the amazing community of AIV developers and the tools that empower them. So to tell you more about what’s new in that space, here’s Tim again. Thank you, Alex.
So like you’ve just heard from Alex, there are creators deploying incredible location-based experiences. And they’re doing it with tools such as the Apple Immersive Video Utility, Spatial Companion, Immersa, and OrangeCue. OrangeCue set a new record this summer for the most Vision Pros simultaneously playing a fully immersive video with over 40 attendees per screening. Multiple screenings were held at the historic Auditorium Santa Margherita in Venice of Lena Herzog’s animated film, Trinity, which had a runtime of over an hour long. It’s events like these that really unite art and technology in an amazing way.
And with so many of you creating incredible experiences for Vision Pro, it’s important the tools be up to the job. So let’s talk about some existing and new on-set production tools to help you capture your immersive stories. Now, notably, Blackmagic Design has been in step with Apple on this journey to support AIV production, developing the completely new Blackmagic URSA Cine Immersive Camera and adding numerous new features to their suite of post-production tools to support the complete AIV workflow. And that workflow is now mature enough to enable anyone to shoot, edit, and deliver incredible Apple immersive video experiences.
Working with the URSA Cine Immersive Camera is mostly like-- it’s mostly the same as shooting with any normal camera, except you have a fixed lens that can see everything in front of the camera. And because it captures stereoscopic images, it can sometimes be difficult to tell whether the camera height or distance from the subject is actually optimal.
So to help you preview your setups the way you would, and the way you would see the final image, Pomfort is releasing updates to its LiveGrade software that support real-time preview, direct from the Blackmagic URSA Cine Immersive Camera on both a connected Mac and Apple Vision Pro. The app also has a unique warning system to help prevent common compositional mistakes, like positioning objects too close to the lens.
And so far today, we’ve seen a lot of underwater footage. And that was made possible by Gates Underwater Products, which has developed a new underwater housing specifically for the Blackmagic URSA Cine Immersive. Going underwater poses its own challenges, and Gates takes care of them for their customers. The ILPD lens file of each system is separately calibrated while underwater for perfect projection in Vision Pro.
We’ve also seen an explosion of small utility apps that help with AIV production workflows, from iPad slates like Eagle Slate that talk to immersive cameras and integrate with NLEs, to apps like EdgeBlend Studio that help you preview custom EdgeBlend masks for later import into DaVinci Resolve Studio. So next, let’s look at editing and finishing, helping you shape your immersive stories in post.
The AIV workflow is unique because it starts with camera calibration at the factory, which produces a calibration file known as an ILPD. This calibration is recorded into every video file generated by the camera. and it’s carried throughout the post-production workflow, along with presentation metadata that’s added during post, resulting in perfect projection when the content is played back.
Blackmagic Design’s DaVinci Resolve Studio, including Fusion and Fairlight, has supported this workflow since version 20.1. And recent updates in version 21 add new features to foveate content, choose a custom orientation, edge blend mask interpolation, and ILPD retargeting to enable visual effects compositing for AIV. The Blackmagic URSA Cine Immersive Camera records two tracks of video, one for each eye, into a single Blackmagic RAW or BRAW container. That’s a format that macOS doesn’t intrinsically understand. So it can be difficult to organize these clips in the Finder when you can’t even see a thumbnail image.
To make this part of the process easier, BRAW Toolbox now has a macOS media extension that supports these dual track Blackmagic RAW files. Once it’s installed, you can see thumbnails, open files in QuickTime Player, or simply select a clip and press the space bar to view it in the Finder with Quick Look.
It even supports switching between the left and right eyes and looking around the immersive image by clicking and dragging. And yes, this means AIV BRAW files can now be used in Final Cut Pro on the Mac. BRAW Toolbox has also added workflow capabilities that let you export AIV formatted FCP XML to DaVinci Resolve Studio for finishing, along with other export options, like Compressor, and direct transcoding to AIVU files. And on top of all that, BRAW Toolbox will provide live preview support of AIV from Final Cut Pro direct to Vision Pro, so you can preview immersively while editing.
Now, let’s look at visual effects and CG rendering for AIV, helping you build immersive worlds from scratch. DaVinci Fusion fully supports rendering of 3D CGI to AIV via the Spherical Camera node and PanoMap node. Compositing of 3D elements into existing AIV background plates is also possible with the Immersive Patcher node. If you’re interested in taking your 3D rendering to a ray-traced level of realism, Maxon is developing rendering support for AIV in Cinema 4D, which will, of course, leverage the Redshift renderer.
For content creators interested in leveraging game assets for AIV, Pixella Labs has developed a new plugin for Unreal Engine that renders frame sequences as multi-part EXR with embedded ILPD calibration metadata. Both will be available next year. So next, we’re going to move on to encoding and review, helping you deliver at the highest quality and getting feedback faster.
Apple’s Compressor now supports encoding AIVU files and HLS segments from AIV mezzanine bundles generated by NLEs like Resolve Studio. Recently, it added metadata display for validation and stereoscopic preview modes. Compressor uses the hardware multi-view HEVC encoder built into Apple Silicon to encode AIVU and HLS segmented Apple immersive files.
Hardware encoding of MV HEVC is ideal for fast turnaround time, but the highest visual quality at reasonable bit rates requires specially tuned software encoders. So the codec company MainConcept has developed a new MV HEVC software codec tuned specifically for high quality encodes of AIV. DaVinci Resolve Studio and Colorfront’s On-Set Dailies and Transkoder have already implemented the new codec.
Colorfront has also released the Colorfront Immersive Utility on the Mac App Store. It lets content creators produce very high-quality delivery encodes with the MainConcept encoder, as well as validate immersive deliverables, all with a very fast and easy-to-use interface. And coming soon, the app will support real-time preview to Apple Vision Pro. problem.
Also available on the Mac App Store, the Apple Immersive Video Utility has had quite a few updates over the past year, including rendering of USDZ backdrops, subtitle preview, metadata preview, and most recently, head pose visualization. The Apple Immersive Video Utility can also play AIVU files directly from a Mac to one or more Vision Pro devices when they’re on the same network.
This is a great way to present to multiple people at once, but it doesn’t help when someone in a separate location, maybe halfway around the world, needs to review your latest cut. There are several cloud-based solutions available to share your AIVU files, but they all require time to upload fairly large files before anyone can view them.
Wouldn’t it be easier if we could just share directly from a Mac to any Vision Pro anywhere in the world? Well, we’re very excited to announce today that Strada, who specialize in peer-to-peer video file sharing, have developed a brand new visionOS app that can securely stream AIVU files directly from any Apple Silicon Mac to any Vision Pro over the Internet. Mm-hmm.
There are no HLS playlists or cloud uploading to deal with. Simply invite a reviewer to a file or folder of files, and they can then watch directly in the app, fully immersed, and even add timestamped comments. And that’s not all. Strada can also initiate speedy generation of AIVU files from folders of camera-original BRAW files. This workflow enhancement means that even a remote crew can generate dailies that anyone anywhere in the world can review quickly in Vision Pro. And this is going to be a real game changer for any AIV post and review process.
This and many of the other new AIV tools I just mentioned will be on display this afternoon in our demo area. So for those of you here in person, please head up there after the afternoon sessions to see everything for yourself. Most of the tools I’ve just shown you were built on the Immersive Media Support framework.
IMS is the framework that supports the unique metadata driven workflows of AIV. It’s supported on macOS and visionOS. And new this year, we’ve expanded support for much of the framework to iOS 27. This opens up the potential for new AIV applications that can run on iPhone and iPad, like the brand new Immersa app for iOS.
IMS has also received new camera presentation override commands. In complex production scenarios, you may need to introduce new camera parameters that override the default camera configuration. For example, if an immersive camera has to be rigged upside down, the immersive shot flip command can be added to the presentation stream metadata. The video is still encoded upside down, but the shot flip command embedded in the metadata tells IMS to automatically flip and swap the images. And it does this during playback, so you have correct stereoscopic perception when viewed.
New for macOS and visionOS 27, the Immersive Preview Renderer class enables real-time previewing of Apple immersive video during editorial or live production workflows. It enables developers to build tools that generate an accurate representation of how content will look at final delivery. Last year, we introduced the concept of static foveation for AIV to maintain high-acuity, immersive video at a manageable frame size.
Instead of simply downscaling the whole image, the pixels of primary importance can be redistributed within the smaller frame size to maintain a higher density in those areas. Earlier this year, we published a new sample code project for a static foveation command line sample app that demonstrates one method of using the technique with IMS.
Developers can now download the new sample code and the sample files from developer.apple.com. So that’s what’s new in IMS. Now, I’d like to preview something completely new this year that you’ve already heard a bit about this morning from Jeff and Alex, live in Apple Immersive. It all starts with the camera.
The Blackmagic URSA Cine Immersive was the first AIV-compliant camera to ship. And as you’ve seen today, it’s the workhorse of all the video-on-demand productions currently on visionOS. However, there’s a brand new version of the camera coming soon designed for live broadcast, the Blackmagic URSA Cine Immersive 100G. From this angle, they look exactly the same.
But if you look at the side, you will see that the media module can be replaced with an onboard live processor module that will encode using ProRes. And importantly, the internal system has been upgraded to handle 100 gigabit networking, hence the 100G moniker. 100 gigabit networks are the backbone of modern live broadcast production using SMPTE ST2110. If that’s new terminology to you, have no worries. Jared will explain in more detail in his live Apple immersive session this afternoon.
We’ve been using the new live cameras in our live in Apple Immersive productions, most recently for NBA and Major League Baseball games. And while we stream polished shows to Vision Pro users, our team has been using these productions as testbeds to help develop the prototype hardware, develop new specs for AIV over 2110, and support third-party tools development.
Supporting professional live production pipelines takes much more than just a camera. It requires a full suite of tools to handle everything in the pipeline, from video switching, graphics, spatial audio mixing and networking, to transcode, segmentation, delivery networks, and then eventually streaming to viewers. This takes a commitment from all corners of the broadcast industry. So numerous companies that make broadcast hardware and software have stepped up to support Apple immersive video over 2110.
like Blackmagic Design with the camera, the new Solaris switchers and 100G cards, Macnica also with 100G cards, Bridge Technologies with a quality control scope, Lawo with broadcast control software, and Ateme with encoders that can support a high-quality static foveation pattern tuned for live and AIV. On the software side, Vizrt and wTVision are developing updates that support high resolution on-air graphics. and Softron has developed support for immersive replays in M-Replay.
Blackmagic Design has also developed a new live version of Fairlight called Fairlight Live for live immersive audio mixing. as well as some really impressive audio panels to control Fairlight Live. This full live ecosystem is all coming to market soon. And many of these companies will be joining us this afternoon to answer your questions and talk about their updates.
Live in Apple Immersive is the beginning of something special. And I can’t wait to see what shows are being produced by this time next year. But there’s one area of the immersive media experience that we haven’t really talked about yet. And that’s the Apple Spatial Audio Format, or ASAF.
As mentioned earlier, Fairlight has been adding additional features to the ASAF mixing capabilities, which are only available in Resolve Studio. This includes head tracking and support for immersive reference video. This year, we released important new updates to the Apple Spatial Audio Format Production Suite for Avid Pro Tools.
Its AAX plugins can now position objects relative to 360 video, and there are improvements to ambisonics, including a new scene compressor plugin, improved heat map drawing, and a spatial filtering algorithm. It’s available for download on developer.apple.com. But one of the most important aspects of spatial audio, and one often overlooked, is capturing great spatial audio natively to begin with.
Capture spatial audio with the Blackmagic Ambisonic microphone
And to tell you more about that, it’s my pleasure to welcome to the stage Abhaya from our spatial audio team. Thanks, Tim. Thank you, Tim. Hi, my name is Abhaya. I’ve been at Apple for 10 years, and I’ve been working on spatial audio research for more than 20 years. And today’s a very special day for me. I’ve been waiting for this day for very long. And I’m very excited. And I’m very excited to share it with all of you. I think you’re all going to be excited too. Woo!
Today is all about Apple immersive video and spatial audio is an important part of that. Maybe even the most important part, but I’m biased. Let’s talk about where great spatial audio begins, the moment you press record. And I want to start with a simple question. Why do we record anything at all?
We want the feeling of being in another place, like hearing the roar of the crowd at a live sporting event or concert that may be happening on the other side of the world. or the feeling of being at another time, like hearing the voices of your kids when they were young.
And I think the best recordings do both. Capture a moment so well that you feel like you’re in the place where the recording was made. That’s the goal. So if that’s the goal, how do we achieve that? Well, you start with the best microphone you’ve ever listened to. You all brought two of them with you today. Any guesses?
Your own ears, of course. A great mono or stereo microphone already hears tone and loudness about as well as you do. What it can’t do is hear precisely where the sound is coming from. And that’s so important in what we’re talking about today. Your ears can tell you where every sound is coming from without any effort.
They tell you when a door opens behind you, whether a sound is above you or below you, how near or far it is, even the characteristics of the room. In a crowded room, your ears help you focus on the one voice you care about. You do this so subconsciously and naturally, you don’t even notice that.
And that’s only the start. So how does your auditory system actually do this? Let’s say you have a sound source on your left. That sound is going to reach your left ear before it reaches your right ear. Also, the sound is going to be louder in your left ear than in your right ear, because you have your head in the middle.
That tells us horizontal position, but what about elevation? The shape of our head and the folds in our ears act as a spatial filter that modifies the frequency of the original sound. Over time, your brain has learned that a change in frequency is actually a change in position. Put these together and your brain knows exactly where sounds are coming from.
And for everyone in this room, those spatial audio cues are so important. In an immersive film, the sound scene has to match the visual scene. When you get that right, you’re completely immersed in that environment. Just like in real life, the sounds tell you where to look. I could hear Siri over there somewhere, so I think my ears are working correctly. - Oh, shit. Just like in real life, the sounds tell you where to look. This is such a powerful tool in immersive filmmaking.
Apple and sound go back a long way, from when the first Macintosh said hello on stage to iPod that put a thousand songs in your pocket. and it’s really personal for me. Think about your first concert. Mine was Prince. And even before I went to his concert, I spent years listening to him on my iPod, just dreaming of what it would sound to be there.
But until today, you would need a sphere of loudspeakers in an audio lab with no echoes. A multi-million dollar room for one listener at a time. Today you can sit anywhere and listen to personalized head-tracked spatial audio on your Apple Vision Pro and on your iPhone, iPad and Mac wearing AirPods. Apple put that multi-million dollar listening room in your pocket. Playback is ready.
But what about capture? The whole of today is about capturing the most incredible immersive experiences. However, spatial audio capture in high resolution is really complex. In Apple immersive video, it’s critical to have high fidelity spatial audio. Great immersive video needs equally great spatial audio. A breathtaking image with spatial audio that doesn’t match the visual acuity pulls you out of the experience. Blackmagic Design’s URSA Cine Immersive Camera, captures Apple immersive video at close to the acuity of human vision. But what about sound? For sound, we want a microphone that can capture spatial audio at the acuity of human hearing.
Now, if you look at the microphones out there today, there’s a trade-off. At the bottom, a typical tetrahedral microphone arrays with four capsules. They’ve been around since the 1970s. They’re easy and affordable and cost from a few hundred dollars to a couple of thousand dollars. But they’re spatially quite blurry. In the middle we have spherical microphone arrays. They typically have 20 to 30 capsules, are a bit harder to use, and have a bit more spatial resolution, and cost a few thousand dollars.
And up the top are the research arrays. I’ve built many of them over the last 20 years. Research arrays are amazing instruments, but they’re built for universities and labs. They cost tens of thousands to hundreds of thousands of dollars. The equipment powering them often took an entire room.
Even now, you usually need a breakout box, encoding software on a laptop or more likely a workstation. You need to calibrate every single unit. And really, you need a specialist to run it all. Well, I’m excited to tell you that we don’t have to do it that way anymore. Please welcome the CEO of Blackmagic Design, Grant Petty. Thank you.
Thanks Abhaya and thanks to the Apple team for inviting us here today. So over the past year, we’ve released the URSA Cine Immersive range of cameras and hardware. They’re built for Apple Immersive Video and Live Immersive workflows. And it’s been exciting to see what everyone’s been creating. We’ve also released a full suite of post-production tools for Apple Immersive Video. That includes adding support into DaVinci Resolve for editing, color correction, audio and visual effects. So you can edit, color grade and mix immersive all in the one tool. We’ve also introduced Fairlight Live, which is a live audio mixer designed for true 3D audio.
So we have the cameras and the post-production tools, but there is one part missing, and that’s the microphone. So we built one. It’s called the Blackmagic Ambisonic Microphone. So it’s a 7th order mic and it’s designed for Apple Spatial Audio. So let me bring one out and show it to you. Thanks. - You’re so smart! - There it is.
So this all started a while back when we were collaborating with Apple on the immersive camera. And Abhaya gave us a demo of high resolution spatial audio capture. And it wasn’t like listening to normal audio. It was like an acoustic reality. And I’d never heard anything like it before. I was totally blown away. So we’ve been working on this together. Apple did research on how many microphones to use, where they’re positioned, and the filters that convert it to spatial audio. And then we did the engineering, the industrial design, the electronics, and the processing it needs.
So let’s look a bit closer at the design. Inside there’s 108 digital MEMS microphones to capture the entire sound field. They’re all mounted flush in a solid faceted body, so there’s no exposed mesh to damage. That makes it a really strong design for professional use. Plus it’s small enough to mount on top of the camera.
However, the exciting part is the internal processor runs Apple’s encoding filters. So the audio from the 108 microphones is converted live to 7th order ambisonics, direct from the microphone. That means you don’t need a computer to do the processing. It’s all connected and powered using a single USB-C cable.
It’s class compliant. There’s no drivers and there’s no plug-ins to install. You just plug it in and it works. which is great because you can mount it on top of the URSA Cine Immersive camera and it connects to the camera’s USB port. Then the spatial audio records straight into the camera’s immersive files in perfect sync.
or you can connect it to a Mac and then record and mix in real time with Fairlight Live. We’ll also have a SMPTE 2110 converter for the mic. Now, it’s already been road tested, literally, on the side of the road during a drift car rally. No, you don’t need a breakout box for it. You don’t need encoding software for it. And you don’t need a PhD. Well, actually, we did need a PhD, and that’s Abhaya. But he’s already done the work, and it’s built right into the microphone. So let’s welcome back Abhaya.
Thanks Grant. So I’m sure everyone here has captured audio with a microphone before. Most microphones capture a point, how loud, not where. Stereo gives you a line and you can capture that with two microphones. But in reality, you hear all around you. So that’s why we need an ambisonic microphone. It captures the whole sound field from every direction at once, reflections and all. And because you’ve captured the whole field, nothing is locked in. This gives you a lot of flexibility because you can capture the scene and decide how to use it later.
How sharp that capture is depends on the ambisonic order. Just like pixels on an image, order is spatial resolution. First order is four channels. Third is 16. Seventh is 64. Seventh order, like the Blackmagic mic, has 16 times the directivity of first order and four times that of third.
So how much spatial resolution do you actually need? To answer that, you want to look at the spatial resolution of your own hearing across frequency. At low frequencies, we have sounds like the rumble of a truck. In the mid frequencies, we have human voice. At the high frequencies, we have musical instruments like a violin.
The higher the heat map, the more spatial detail your ears use. Low notes wrap around your head and need very little. High notes bounce off every fold of your outer ear. And that’s how you tell height and front from back. Here’s a typical tetrahedral microphone array. It gets the broad strokes, but look at how much of what your ears need it never reaches.
Now typical spherical microphone arrays. More of the field, sounds pull apart, directions get specific. It does well through the middle, then runs out just where your hearing gets sharpest. And this is the Blackmagic Ambisonic microphone. It captures so much more of what your ears need and gives you incredible spatial acuity. Until now, you’d only get this type of performance from a research array in a lab.
So how do you build this? There are 108 microphones spread evenly over a rigid sphere. Blackmagic’s engineering team did an incredible job. This heat map shows the performance of the Blackmagic ambisonic microphone when capturing a sound source. 108 capsules on a compact sphere means that the main beam is narrow and well-defined. This is high spatial resolution.
It also means that the individual capsules are spaced closely together so that spatial aliasing only occurs well into the higher frequencies. Another benefit of having 108 capsules is that it has excellent signal-to-noise ratio across all frequencies. And at the lower frequencies, its resolution drops with the resolution of your own ears, so it’s not amplifying noise.
On the left is the simulated response of the array we challenged Blackmagic to build. On the right is the actual measured response of the Blackmagic Ambisonic microphone. It’s perfect. The gentle ripples you see are caused by a reflection in the measurement chamber. We compared this to other ambisonic microphones. You can see how much more fidelity you get from the Blackmagic ambisonic microphone. But as much as looking at these graphs is helpful, there’s no better way than to show you. So let’s take a listen to how it sounds.
That was all captured using the Blackmagic Ambisonic microphone. It’s hard to believe, I know. This theater has a 7.1.4 system, and as good as it sounds, it doesn’t render the full resolution of the capture. So for folks in here, in the room, just join us in the demo area later to hear it in its full resolution.
This corner up here, really high resolution and really easy to use, has been empty. That’s where the Blackmagic ambisonic microphone fits in. It’s easier to use than any ambisonic microphone that’s come before it and has higher resolution than most other research arrays. And you get to choose what comes out. There’s 7th, 5th or 3rd order ambisonics.
Or standard 5.1 and 7.1.4 like you just heard, if you need a channel mix. or capture the raw 108 channels if you want to do your own processing. And because it outputs ambisonics natively, you can monitor it and play it back with head-tracked, personalized spatial audio on AirPods and Apple Vision Pro.
So that’s the theory, but let’s see it working in real time. So you’d probably expect me to plug this into the camera, but we want to show you something really special. Grant? It’s just an iPhone, one USB-C cable and that’s it. No interface, no laptop, just an app.
What you’re looking at now is this room all the way around the microphone, unwrapped like a 360 photo. That bright spot is me. Watch it follow me. So I can go down below the mic. I’m talking below the mic down here. I can also go above the mic. I’m at the top of the mic. It’s an equirectangular projection, so that’s why as I go up and down it, it sort of spreads a little bit.
And Grant, you want to-- If I talk over here on 90 degree angle, there’s a second heat. I can also talk at the same time as Grant. I don’t normally talk over people, but I just wanted to demo how this works. How about we get some of you guys involved? Can I get everyone on the left to give us a cheer?
What about everyone on the right? Give us a cheer over there. And everyone in the center is at the back, so you’re gonna show up on either side. So can I get everyone in the center? And all of that is now able to be captured spatially just using an iPhone. So whether it’s on a camera or on the iPhone, recording really is that simple. So the Blackmagic.
So the Blackmagic Ambisonic microphone will be priced at $1695. That excludes duties and tax. And it will ship in December. Another place, another time, or both. Now you can capture all of it. Come and hear it for yourself in the demo area upstairs this afternoon. - Thank you.
Big thank you to Abhaya and an extra special thank you to Grant, coming all the way from Australia to join us today. Yeah, let’s give it up for Grant. And I am sure I’m not the only one in the room who is absolutely stoked to use that mic on location. And just one cable-- unreal.
Now, we have had a lot of information from this morning, and we want to give you folks an opportunity to digest that. So we are going to take a quick break, digest all this information, digest some pastries, have a wee bit of coffee, and we will see you back in 10 minutes. See you in a bit. Oops. Thank you. ♪ Our program will continue in five minutes.
Thank you. ♪♪ ♪♪ Thank you. Thank you. Thank you. ♪♪ ♪♪ Thank you. Thank you. Thank you. Thank you. Thank you. ♪ ♪ Okay, welcome back everybody. I hope we are all feeling nicely caffeinated and I think there’s a few people still trickling in. So maybe as people find their seats, I just wanted to say how awesome it is to see this community together again in person, maybe catching up with some old production friends, making new connections. Who knows, maybe you’re even kick-starting your next project. And it’s really awesome. So awesome to see.
All righty. One more in the front row seat as well. Look at that. Okay. On to our next session of the morning. You are about to hear from three incredible creators sharing what they and their teams have learned on their very first Apple Immersive project. And what makes this session unique is that each of them were trying to solve a completely different problem.
from where the line falls between experience and entertainment to how you help customers purchase something that isn’t always accessible. But to kick us off, I am excited to welcome Laura to the stage from the Audi Revolut F1 team, here to share how Apple Immersive introduced the world to a team before it raced a single lap. Laura.
Insights and learnings: Audi Revolut F1 Team entering the race
Hello, everyone. Wow, it’s truly a pleasure to be here. Before we start, I want to say a big thank you to Elliot, Alex, Oliver and the whole Apple team for having us here today. I’m Laura Triulzi and I work on brand activation for the Audi Revolut Formula One team.
This morning, I want to share with you our journey to produce an experience for Apple Vision Pro. And now, hopefully, we were able to produce a moment like this: an iPhone fan in Miami reaching out to touch our driver, Gabriel Bortoleto, as he was standing right in front of them. When, of course, he wasn’t there.
and how we made that possible with a team that hadn’t raced a single lap in the sport. And I should add, we have heard some fantastic presentation today so far, but I’m not a tech expert. So today, I want to show you how our marketing team has used Apple Vision Pro to its fullest.
But first, a quick show of hands. How many Formula One fans do we have in the room? Whoa, and how many Audi fans? Whoa! Amazing! Well, great. Maybe by the end of the presentation, you’re all going to be Audi fans, hopefully. But racing is in our DNA and also is innovation.
It goes back to the 1930s at the Auto Union Silver Arrows, which helped revolutionize racing with a pioneering mid-engine design. In 1980s, Audi Quattro brought all wheel drive to rally, helping Audi secure World Rally Championship titles in 1982 and 1984. In 1981, Michèle Mouton made history in a quattro as the first woman to win a world championship rally. Woo, yeah!
At Le Mans, Audi proved the new ideas could win. In 2006, it became the first manufacturer to win with a diesel-powered car, when many believed that that could not be done. Then, in 2012, the R18 e-tron Quattro became the first hybrid car to win Le Mans. And in 2024, we won the Dakar Rally with our RS Q e-tron, which is a fully electric race car. Audi had almost a century of racing and innovation, but zero laps in Formula One.
So when we started planning our team launch back in September to officially present the team to the public, we faced a very specific challenge. How do you introduce a team that hadn’t raced? New people, a new car, and a team still taking shape. Our job in brand activation is to bring Formula 1 to the people. But there was nothing on track to show them yet.
So we decided not to tell the people about our team. Instead, we will put them inside it. We partnered with Targo, an Emmy Award winning production company from France. Some of you will know their film D-Day, The Camera Soldier. And we made an immersive documentary shot in Apple Immersive. Let me show you a quick preview.
The 13-minute film is called “Entering Formula 1 Racing” inside the Audi F1 project. And you can watch it today on Fovii and hear what we learned making it and taking it to the world. So the first lesson, before we could get to the racing, we had to connect fans to a brand new team.
Second, the access we gave to the fans became the experience, something only this medium can offer. Third, we let people stand inside the history of Audi, not just to see it, but to be there and to really feel it. And fourth, how to design a moment and not just a film. So first, let’s meet the team.
Central to our thinking was introducing the people before the car. When I started working in Formula One, I had no idea how many people it takes to put just two cars on the grid. Or, yeah, right? Or how much happens behind the racing itself. So the film shows the scale of the project and the people behind it. It brings together teams across two countries: the Power Unit, which is developed in Germany, while the chassis and then commercial operations are based in Switzerland. The film connected those locations in one experience.
Together with Targo, we chose to start with the people. Because before there was a car on track, there were already hundreds of people shaping what this team would become. The engineers, the leadership, the mechanics, the commercial team, the drivers and many more. They were all building the team long before the first race.
So that’s the first lesson. The team is the story. Second, make access the experience. Another thing I didn’t know was how carefully you have to tell this story. So much of the work on F1 team involves sensitive and security information. Kept behind secure doors, with access being tightly controlled. This information is what championships are made or broken with.
For this experience, we were able to take the audiences there to go inside a simulator. the turbocharger test bench, the wind tunnel, where our engineers audition new parts to claim the next tenth of a second. and even our 60% scale model running in the wind tunnel. The exact replica of our 2026 car, as you can see here.
And this is the part I think it matters the most for anyone here that has a sensitive story to tell. Apple immersive camera let you control this footage under control. With Apple Vision Pro, Lenny will show the footage in a controlled and safe environment. It lives inside the experience and nowhere else. And for us, that was essential to offer access to the fans while controlling who saw the experience prior the season starting.
And I mean, of course, there were also things that we simply couldn’t film. The biggest example for us was the actual power unit. is one of the most closely protected parts of the entire project. It’s so sensitive that we cannot even show it to guests or partners, even when they are with us trackside. So that’s lesson two. Access is the experience.
Third, let people stand inside a history. Remember those Silver Arrows and quattros? Well, we took people inside one of Audi’s most exclusive spaces, where many of them live, the Audi Secret Garage in Ingolstadt, Germany. Here, they could explore iconic race car up-close. so close to see the dirt still on them from the races that made history.
These were more than car on display. They carried the traces of their racing stories. And this is the type of experience a fan dreams of. Now turned into a possibility, no matter where in the world they were watching it from. So that’s lesson three. As spaces usually reserved for a privileged few became heritage that they could stand in.
And fourth, the one that nobody tells you about. Design the moment, not just the film. And that started by partnering with JigSpace to design an app that would embed our Apple Immersive film alongside an interactive model of the R26 race car. We designed the experience that when the film ended, the race car would automatically appear perfectly in place and scaled right in front of them. It was almost like magic.
We first showed it at our team launch in Berlin to media, partners and a few lucky fans. And after just a few hours, the response was incredible. So we took it to Melbourne, our first race of the year, when we run more than 300 sessions. Every session was a journey. We started with a welcome, where we explained the idea behind the film, what people were about to experience, and how Apple Vision Pro worked.
We run two Apple Vision Pro devices for each session, with two additional devices as backup. We had no real blueprint, so with every activation, we learned more about how to make the experience smoother and easier for the people. For us, the most challenging part was making people feel comfortable with the device.
The whole onboarding took about five to 10 minutes, and we quickly learned that the fewer steps people had to go through, the easier it was for them to relax and enjoy the experience. then the film. 15 minutes, two participants at a time. And then came the reactions. When people took the headset off, they were often amazed, not only by the technology and how immersive it felt, but also by the access they’d just been given to the team. To make the experience work smoothly, we run each activation with two people, one dedicated to each Apple Vision Pro. And next came Miami. More than 200 sessions across three days.
For many people, this was also their first time experiencing Apple Vision Pro, which made the activation about more than just the film itself. they were discovering a completely new way of experiencing our team. but their reaction told the story better than the numbers. So let’s take a look at one of those.
The way we all watch Formula One is going to change forever. It’s absolutely going to change forever. And I’ll tell you exactly why. When we were in Australia, I tried the Apple Vision Pros. Now, this is not an ad. Audi had this place set up where you could basically put on this headset and you would step inside the factory.
So more or less, I was sat there on the sofa watching Wheatley do his interview, watching Gabi in the back of the car with the press manager, watching them build the car, watching them fire up the engine for the first time in the factory. You are there. This is how we’ll all be watching sport in the future.
And this is just one reaction from Jake and Fabio from the Pit Stop podcast, but there were many more. People stood in line. Some of them even became emotional. And for us, that was really awesome to see. We also shared the film across the entire factory, so colleagues who worked on only one part could experience the entire project they were helping to bring to life.
and some of our partners who tried it now they want to recreate a similar film themselves So that was the fourth and final takeaways for us for this project. The film is only half of the experience that we need to build. Bringing all of these takeaways together. Remember those fans reaching out to Gabi from the very beginning? Well, that’s what a team that hadn’t raced felt like. Real.
To finish, I want to say a big thank you to Targo, to Apple, and to all my colleagues who led us through every closed door. We gave the people the feeling of being there, and we brought them closer to our team. And one thought that I would like to leave you with, if a team with zero laps can feel real, Imagine what your story could do. Thank you very much.
Thank you to Laura. What a story. I have to admit, though, that as a self-confessed petrolhead, seeing those Group B rally cars in the Audi secret garage made the little boy in me very happy. And don’t forget, you can watch Inside the Audi Revolut F1 project via Apple TV, and you can stream it on Fovii today. Now we are going to close up the pit garage and open the hangar. Here to tell us how Cirrus took Apple Immersive Airborne and put it in the hands of the sales team. Please welcome to the runway, Anthony.
Insights and learnings from Let's Go Fly
Elliot and his metaphors. I’m here for it. Hello, everybody. My name is Anthony Bottini. I am the director of media for Cirrus, meaning I get to serve the entire internal media team and the videos that we make at Cirrus. And I’ve been really looking forward to sharing an amazing project with you that we started about a year and a half ago.
So I want to start with a question. This is the question that we started with. What if someone could experience flying a Cirrus aircraft without ever leaving the ground? That is really the question that kicked off this entire thing, this entire project. And the answer turned into something much bigger than we had originally expected.
It started during a creative discussion with some internal colleagues in early 2025 regarding the viability of how Vision Pro and thinking creatively around Vision Pro might serve our business with a simple question. Could we, in fact, mount the relatively new at the time Blackmagic URSA Cine Immersive Camera on an aerial gimbal and shoot full air-to-air and create an incredible Cirrus flying experience?
And I didn’t do this alone. This was a wonderful partnership between Rogue Labs and our immersive director of photography, J.R. Racine. I think he’s out there somewhere. And he brought the immersive camera and technicals to the project. Brad Herndl and the team at Immortal engineered the gimbal system that we used aerially. And XBrand, the aerial coordinating and photography company that we got to work with, well, they were part of it too. More on them later.
So first, a little bit about Cirrus. I want you to understand who we are, and this is exactly who we are. Cirrus, an airplane company. I think of it as an experience company. Our mission is to deliver an aviation experience to our customers that is the pinnacle of innovation, quality, safety, and service. That is who we are. That’s what we’re trying to achieve. Focus on experience there in that phrase.
This is what we’re promising our customers, our global customers. It’s this thing we call the Cirrus Life. Through our products, we are getting people to this transformational place where because they can now fly for themselves and truly fly for themselves, they can go where they want, when they want, how they want to get there.
And it really is this thing that if they stick with us and they go down this journey, this path of becoming essentially a pilot for themselves, life really does change, and this is what we promise them. The way that we get them to the Cirrus life is through our two incredible families of products. These photos, the Vision Jet, the Cirrus Vision Jet, the Tony Stark-looking kind of jet.
It is the world’s first civilian, single-engine certified jet. Owners and operators can fly themselves around in that jet. But then we have the Cirrus SR series. This is the product that put us on the map 30 years ago. This was the entry to personal aviation and still is today to our brand and to this idea of personal aviation, the Cirrus SR series. It has been the best-selling personal aircraft in the world for the past 20 years, actually over 20 years at this point.
So maybe you’ve heard of us. Maybe you haven’t heard of us. But if you haven’t heard of us, tangentially, maybe you have because safety is the core of everything we do. In fact, that’s really what put us on the map about 30 years ago. as when the SR series came out, we introduced this. I don’t know if anybody’s ever heard of this. This is the Cirrus Airframe Parachute System. We were originally kind of known as the airplane with the parachute, the plane with the parachute.
What this means is if you, in a very rare case, ever have a critical emergency in a Cirrus, you have this ultimate backup option, a red handle on the ceiling, you pull the red handle, and a parachute deploys to bring you down to the ground safely. To date, we’ve had 152 saves over about those 30 years and have returned 305 people safely to their families. And we’re so proud of that. This is one of our points of innovation, how we’re kind of constantly pushing forward. And this is exactly what it looks like.
If you were really trying to picture that or visualize that in your head, is it true that it’s a full parachute for the whole airplane? Yes, it is. And that’s how it works. And if you want even a little more proof, one of our customers, I don’t know how, but had the presence of mind to pull out a camera on the way down, and once he was safely on the ground, he was happy to share the footage with us.
Called him and said, “We need it. We need the footage.” Unbelievable. That’s been around for a long time. That’s maybe where you have heard of us, the plane with the parachute. But the thing that we’re so proud of that we announced last year, in terms of the innovation around safety and the core of safety of who we are, this is what we’re super excited to talk about.
This is Safe Return Emergency Autoland. In this class, we’re the first ones to bring it to this type of airplane, and it’s kind of what it looks like. Again, in the very rare case that your pilot would, for whatever reason, be incapacitated or couldn’t land the airplane, well, now there is another ultimate backup in the airplane, a red button on the ceiling, which means if you are a passenger, you can push that red button, the airplane turns fully autonomous, navigates around bad weather, and goes and finds the safest, most suitable airport to make a landing on the runway all by itself, comes to a complete stop. There you go, another ultimate backup. I share these two things to help you understand how Cirrus in our DNA have always been very, very forward-looking in how we incorporate technology, not only for the comfort and the practicality of flying, but for our safety. And this is who we serve right here.
Globally, we have airplanes everywhere. Airplanes, customers, and an amazing support network of authorized members that service the airplanes and train people how to fly. So the setup is great. But here’s the thing. For most people, the conversation about the magic of personal aviation begins behind a chain link fence. As a marketing team, as an airplane company, we know the magic. We live it every day. But many times, this is how the conversation begins. something you watch from the other side of a fence. I’m getting to a point here.
Our goal is to grow aviation. We got the fence in the way. We don’t see it as a problem. We don’t even see it necessarily as a challenge, but we do see it as an opportunity. This absolutely is an opportunity. And what we’re trying to do here is to belabor the metaphor of the fence remove the fence line so that more people Buy airplanes, get into aviation, and really believe in what we’re trying to do here with personal aviation. Get more people inspired and more airplanes flying. This is our mission. It’s actually a really fun one. And it causes a lot of discussions creatively about, well, how do you get rid of that proverbial, that metaphorical fence line? Well, that’s where?
Vision Pro came in to the discussion. This really was a serious discussion about what was possible. More specifically, this is where Apple Immersive Video came in. Okay, so this is content as we sort of discovered this. Apple Immersive is content that is not like watching video. It puts you in it. It really does create an emotional impact and experience.
So when we began watching all the content that Apple had already had in this space in Immersive, we all agreed this was an experience far beyond what people think of when they’re just consuming video content. This was different. This wasn’t even like normal VR experience. This was something really different. We got very excited. So we knew we needed immersive. But what was immersive? A relatively new coined term.
And I had to do a lot of internal selling at the company. For everybody out there who’s trying to get the ball rolling on immersive and getting people excited, I was right in the middle of that. Trying to explain to people what was immersive. And I would give a Vision Pro to anybody that would listen to me. But ultimately, it was kind of a process of introduction and understanding of what this was.
And this was the best definition I could come up with. It was an amazing recipe of things that caused something special that’s almost hard to explain. And it was stereoscopic video, 3D video. People generally understood that. 180-degree projection so that your periphery is taken up, which is huge.
Lifelike acuity. Viewer-directed. So now the viewer gets to choose where to look. And with the magic of foveation and definitely spatial audio, don’t skimp on the spatial audio, Customers aren’t just watching anymore, they’re experiencing. That was the argument we made internally to say there’s something different here that might really serve us on the sales side of what we’re trying to do.
So, Cirrus is at the forefront of aviation technology, so why can’t we jump into this immersive game? Why can’t we be on the forefront of this to show our customers that culturally we’re trying to lead in the way we communicate about the magic of personal aviation? So, we brought together Apple Vision Pro, our friends at Rogue Labs, who had all the expertise in immersive, our own internal Cirrus media and marketing team, and the Blackmagic URSA Cine Immersive Camera.
So the goal was, let’s try to make an immersive film. Let’s make an immersive film, go for the most realistic demonstration ride in one of our airplanes possible. Could you get there? What would the realism be like exactly? That was in a Cirrus airplane. Realistic experience from your own living room. Is this possible?
Our strategy for success ultimately was simple, and this is where we were coming from. Success to us looked like we create an app with an immersive film and a 3D AR, augmented reality experience, that we would equip our global sales team with that as a tool. If you’re a Cirrus salesperson, you actually are equipped with a real airplane every year that you get to take around and demo. Pretty cool job. You get to take people flying in kind of your own airplane that you’re assigned. But this was going to be another tool for the times that the weather was bad.
Maybe you didn’t want to demo a potential customer. Maybe you just didn’t have the airplane that day. Well, this was now another way to get people an emotional experience about flying in a Cirrus airplane without technically needing the airplane. Kind of cover that gap. We were shooting for that gap.
So our first gimbal test proved that we actually could get the Cine Immersive camera on a gimbal on an aerial platform. We got very excited after that test worked. Worried about vibration, that kind of thing. Well, it worked out really well. So this is the point where I’d like to take you behind the scenes on how we actually made this. If you’ve seen the experience already or if you haven’t, there’s going to be a little bit of experience appreciation after the behind the scenes here. Because there was so much planning that went into this.
We had three different aircraft working in tandem on every single flight, shooting a tour across what we picked was Arizona and some of the most amazing sites that we could think of as we fly all over the country all the time. Some of the most amazing aerial sites that we thought we could pick out for this experience. So you’re thinking of the right stuff. Grand Canyon, Monument Valley, Sedona, all that kind of stuff.
We wanted to take people on a tour of that region because we knew that it represented the magic and freedom that personal flying has to offer. So to pull this up, to pull this off, every single pre-flight had to be really dialed in to coordinate airplanes, schedules, and the time that we had allotted, and by the way, dodge the bad weather when it came in so we always were given a good experience.
So in immersive, planning is everything because you’re not only just framing the shot. We have the internal team that does all the regular flat videos all the time, and we love doing it, but this was different. You’re not just framing a shot the way you’d normally think about it. You’re designing a landscape, and especially when you’re talking aerially, you’re designing a landscape and a potential for where viewers will draw their attention.
So you can kind of see here in these examples, as we were storyboarding and rendering, and actually this one more specifically, this is one of the opening shots. If you haven’t seen the film, and I can’t wait for you to see it, this is one of the opening shots where JR had actually storyboarded and rendered this to think how we could open up over Horseshoe Bend. So basically what’s going on here is the, well actually here let me show you this.
This was the silly sketch that I did as we got to do the fun part of this job, which was we pre-scouted this entire thing. So what you’re seeing here is on my iPad flying over Horseshoe Bend, scouting this out and just sketching the direction that we wanted this entire path to go.
So in this case the Cirrus would cross below the camera ship, and then we sequence through and then chase it as it conducts a very scenic counterclockwise turn over Horseshoe and exits to the northeast to continue the story on to Lake Powell, etc., etc. So this was some of the beginning planning.
And on the planning for Air to Air, we were so lucky to have a crew that really knew how to do it. This is the part that I brag about sometimes. Kevin LaRosa and Michael Fitzmaurice from XBrand, if you haven’t heard of them, they are the team that did all the aerial cinematography work in Top Gun Maverick and some of the other aerials that you’ve seen in Hollywood over, call it the past decade.
Their expertise was core to being able to pull this off with our own internal formation pilots. So we had our own internal formation pilots, highly skilled, flying our airplanes, interacting with XBrand. You can see this iPhone shot going on from one of our Vision Jet cockpits to understand some of the proximity that we were getting with.
There’s JR in that photo as well. He became fast friends with them. It was a good time. The trick here was for aerial immersive shots, you wouldn’t think about this, but everybody knows it’s a very wide screen or it’s a very wide format. So there was extra briefing that had to go in before we would do these flights because to get the action where you wanted it, you really had to tighten airplanes up.
Real close formation to get this done the way that you wanted. You’re getting a sense for it right there in that video. But it was all about getting the action that we needed to really inspire people and get them emotional in the film. So there’s a lot of pre-planning.
Then came the hardware. We were also setting up POV shots in the left seat in the airplane so that when you’re in there, you get the feeling like you’re the pilot. For anybody that’s ever flown an airplane, you know the left seat is where the pilot sits. So we took a lot of time to set that immersive camera up in a way that was perfectly tailored to the average eyeline of the average pilot in a Cirrus to really make you feel like you’re going along for a ride.
Here’s a quick result, that opening shot over Horseshoe Bend. Pretty amazing here, but also pretty boring compared to what it looks like in the headset. But you get a sense for it, and this is how we were doing all the aerial coordination. Everything intentional, every movement, every position dialed in and pre-briefed. Here’s one of my favorite shots. I call this aerial BFFs.
This was our, when you get into the experience, Justin here, he’s actually one of our sales demo pilots. Very experienced, has been flying for Cirrus for a very long time. He’s got a friendly North Carolina accent, and he’d love to take you flying and be your instructor on this ride. And we were in the back coaching him along and just kind of directing him. But he did a phenomenal job.
And really, when you’re in there, you will feel like I’ve got my North Carolina buddy right here teaching me how to fly. And so it was a great, great collaboration to get the in-cockpit stuff done. And everything was rigged just right. Here’s where I got to have some fun. That’s me doing the flyby. Thank you, thank you.
because we didn’t want to just do the aerial stuff. We were in Arizona. We wanted to shoot everything immersive that we could. That was Cory and JR’s idea, hey, let’s do some ground stuff. And so this was, oh, the hiker’s hiking along, and they get to see an airplane go by, because we want to show relative motion of our cool airplanes flying by. So there’s a little bit of everything in there, and we had a ton of fun.
For the air-to-air shots, nothing was off the shelf. A lot of custom rigging, like I said, with the Immortal gimbal here. Everything had to be designed specifically for this camera and this use case. So I think we kind of cut some new ground here. Brad and the team at Immortal built a custom Phoenix gimbal for the Cine Immersive camera. And together with XBrand and Rogue Labs, the team worked hard to make this kind of shoot possible.
And it was really cool to see it come together. And what you’re seeing there is we actually did have a nose and a tail mounted set of Immortal gimbals and Blackmagic cameras to get this done. Here’s a little iPhone shot from a Vision Jet window again as we were in chase.
And you can see those two gimbals going at the same time. And that gave us the flexibility to shoot what we needed to shoot in a relatively short amount of time. We had four flights over three days to get this done with weather kind of making things challenging.
So this also saved a lot of continuity headaches so that you weren’t in one shot where there were clouds and you were in another shot where there weren’t clouds. There was a lot of that coordination that had to go on. So the two gimbals were key in this. JR knows what I’m talking about. It was a big deal. Here’s another fun one. I’m calling this practice makes perfect. This was the second or third attempt to get the timing just right on a takeoff shot with one of our Vision Jets at sunrise to get that perfect shot.
And they nailed it. In camera, that shot looks sweet. I can promise you. Very exciting. So we’re all kind of cheering. Hey, they got the timing down. All right. So once we were in the air, Once we were in the air, the challenge became framing. And this is where it gets a little bit technical, and you can see it in the composition there. It’s not about normal composition, obviously, rules of thirds and everything else that you’d know in flat video.
It’s more about just comfort and orientation. Not pointing the camera in a direction relative to the motion that would make somebody feel weird. We actually noticed that a lot. If you pointed it out to the wing, then you feel like you’re moving sideways and weird things happen. That’s even when you were shooting air-to-air.
It was about comfort and orientation. We thought of the full frame there that you see as the theater and that center quadrant where JR was focusing a lot of his director of photography time and his skills was right in that square. That was the stage. That was where we really wanted to focus most of our energy.
And in this case, the SR may look a little bit small here, but those four center squares are what you would see looking through the Vision Pro. So John’s focus was showing the aircraft in the best possible way while keeping us safe and not pushing the pilots to get too close together. So the SR would really give you as much action and be as close as possible to give you the best seat that you could ask for in this experience. And this is what that shot ended up looking like.
Again, pretty neat, but awesome in the headset. Can’t wait for you to see that one. So… Keep it simple, keep the subject centered, and make the action as exciting as you can. Here’s another example of it all coming together. - Welcome to the flight deck. Your new Cirrus, but-- - There’s Justin talking to you, being your friend.
- Pull on the runway to take-
- And the cool part about this medium is that I’m able to see my flight instructor but then I’m able to see all the way over, and you’ll see this in a second as we’re taking off, all the way over to the wingtip. There’s nothing claustrophobic about this experience.
- You doing over there?
Nothing claustrophobic, and there shouldn’t be, because when you’re flying in an SR series, there’s windows everywhere, and it feels amazing, and you can see everything. And that camera and that setup captured that feeling of, I can look everywhere now when I’m flying. It’s not just a little porthole anymore in an airliner. It’s, I can look everywhere where I’m flying. And that really does drive an emotional impact when people get in and watch this thing. You look where you want, when you want.
The job wasn’t done when the shooting stopped. I was honestly a little skeptical in the beginning of this that we would be able to do any kind of real DIT station and do dailies and previews as we were going. But sure enough, with the DaVinci Resolve Studio workflow, you can see we were in the hangar every day making sure we got what we needed. In fact, I was reviewing Justin’s emotional, inspiring close-down line here at the end, and I slept better that night knowing we got it. I was amazed that we were able to do a preview on the Vision Pro as we were going along.
And then here’s Eric, our in-house Cirrus editor. So we did all the editing in-house at Cirrus after we pulled it back. And we threw him to the wolves because he is an incredibly talented 2D editor. And we said, here’s a new workflow. Go get him, Tiger. And he did. He did an amazing job. Figured it out. Plugged the Vision Pro in.
Obviously, everything was edited on DaVinci Resolve. And he was able to make the smart decisions with Vision Pro to figure out exactly how we wanted to cut this thing up. and he did an amazing job editing this for us, and we were super excited when we started seeing the results.
Here’s a plug for Spatial Audio. I know there’s been a lot of conversation about Spatial Audio. I will just pile on. Apple Spatial Audio, that format is amazing. Don’t skimp on it. We mix everything in DaVinci Resolve Fairlight, thanks to our friends over at 1.618 Digital. Check this out.
The question becomes, what is immersive without Spatial Audio? Well, I can tell you, it’s not nearly what it should be. if you don’t have spatial audio. JR kept pounding this into me and we finally started doing it. Here’s an example. This is the opening shot to the film. This is without spatial audio. - What if you had an airplane?
Okay, so all of a sudden there’s an airplane in the shot. Cool shot. We get it. But what about the engine noise? What about the Doppler effect? What about the, like we were saying before, cueing people in to where they should be looking? So here it is with spatial.
What if you had an airplane? Oh, there’s the airplane. Excellent. And when we started seeing that effect of spatial, I was a believer, and boy, did we get after that edit. So the aircraft sound comes from beneath you, you’re cued into where to look, and the experience becomes that much better. Okay.
Rather than show you more and more of these clips, I want to give you a teaser, just a 90-second teaser of a behind-the-scenes film that we made for this. And if you want to see the rest of it, it’s about five to six minutes. It’s on YouTube. But at least let me give you a sense and a feel for the crew doing their thing behind the scenes. Here you go, 90 seconds.
What’s better than one Blackmagic Cine Immersive? Two. - 102, take one, marker. Good. I mean, that’s the insurance we need. Excellent. Recording a great video today for VR. Show people what it’s like to fly an airplane. Okay, we’re all systems go. I see Jordan there. And Aerostar is going to come out of the chocks.
Well, there’s always a little bit of nerves. I think it’s more nervous excitement. And yeah, let’s go get it. - So great, I’m not gonna-- - Here we go, camera’s rolling. Here comes my nose cross, stand by for the break. Ready, Fitz? - Ready. - That’s a good break. In three, two, one, hard right.
a little more lens on this one, a little steeper look down with Powell, and I’ll give her a right break. That’s so cool looking. It’s so real that when you’re in there and you’re experiencing flight, it gets you to like a 70% emotional experience of what it’s like to actually fly. This is freaking incredible. Oh my gosh, dude. That’s pretty cool. It’s pretty awesome. Beautiful morning light with some clouds and some beautiful terrain. It’s everything you could ask for. It’s awesome.
Thank you, thank you. Like I said, more on YouTube if you want it, where we get into a little bit more of the detail about how the shoot actually came together. So then came the fun part. Once the production was done and it was time for me to look ridiculous on stage, we started rolling this out and explaining it to people, the product that we had actually created and we were so excited about it. So we actually launched this at what we call, this is our global event for Cirrus. It’s called Cirrus CX. It happens at the beginning of every year. In fact, it’s gonna be happening in about a month and a half from now.
It was in Vegas and we rolled it out to our entire global network who converged on Vegas to learn the news about Cirrus and we said, we’ve got this thing that we think is going to move the needle, especially in democratizing the message around personal aviation and sales.
Over three days, we got crazy, and we set up five Vision Pros in a full station and did over 500 flying demos for people and just had a blast. This is what the setup looked like. We went all out, the backdrop, the branding, the Apple TV that was mirroring at least one station the whole time so passersby could really understand what was going on because, again, it’s such a new idea. We were trying to help people understand, like, no, no, no, you’ve got to hop in line and do this. And we had a line out the door the entire week. It was so cool.
And the one thing that I will note is, and I think some other people have said this, I couldn’t believe the emotional reactions when people saw this for the first time. Lifelong lovers of aviation were here. And there were at least two times where people pulled the Vision Pro off.
They had gone quiet for a while. They pulled the Vision Pro off. And there’s tears flowing down their faces. True story. I’ve never seen something like that before. It’s not like it was a sob story video. It was just an awesome airplane video. And people got that emotional. I’ve never seen that before.
Then we went to South by Southwest. More than 300 Apple Vision Pro demos over two days there with the same kind of setup. And this truly was our getting outside the fence line challenge. South by Southwest, by and large, was people that didn’t know anything about personal aviation or the magic that it could afford or the life-changing stuff and everything else. We had so many incredible conversations with people that got out of this headset that had nothing to do with aviation. all of a sudden they’re starting to Google personal aviation things. Like you can just see it kind of happening in real time. Super fun.
It’s not content, it’s an experience. That’s how we’re treating it. We’re treating it like a ride at a theme park because that’s really how people react to it. So we’re doing this everywhere. We’re doing this at people’s homes and in the mall and at a bunch of different shows.
And in fact, we’re doing it at our Discover Cirrus events. Coming to a hangar near you, we do these small Discover Cirrus events all around the country and all over the world where we invite people to come, have some appetizers, hors d’oeuvres, talk about personal aviation, and now this is a staple experience at our Discover Cirrus events all over the place. The part that I’m most excited about, we have achieved that sales road kit. If you are a salesperson at Cirrus and you’re flying your airplane around, well, you can kind of see the image here.
In your back seat, you have this go kit for the Vision Pro with an iPad so that on those days where you don’t have the airplane for whatever reason or the weather’s bad and you don’t want to take people up in turbulence, “Hey, let me give you the entree to the magic of personal aviation with this new experience that we have.” This was when everything really just clicked.
And here’s one more video I want to share with you. My favorite part of this whole process is when I got to show everybody internally at Cirrus, all my colleagues who know and love aviation, the first time we got to show them this thing as it was just rolling out. This is fun. I wanted to share it with you.
Oh my-- wildly immersive. This is blowing my mind. It’s very cool. I’m-- Blown away. Oh, cool. Wow. I’m getting emotional. Yeah. - That’s pretty-- - Yeah, this is insane. - Killed it. - Crazy. - Amazing. - Legit. - Insane. - It’s so cool. - Whoa. - That shot was a Vision Jet. - I like, I don’t understand. - Whoa. - Whoa. - Hello. - Ah! - Dude.
is crazy. We’re going to sell more planes. *laughter* Those are my friends. I’ve never seen them react to anything that I’ve made like that. It was really, really cool. Our team has found a new way for people everywhere to experience our aircraft. And this is just my kind of last part here that I wanted to share. The sales signal is starting to show up. We actually launched this basically back in June.
The sales signal is starting to show up, and you can read through that however you want. But basically, this was a message from a sales member on our team, actually from Europe. He sent this to me about three weeks ago. And basically what it says was, I had a customer that was kind of on the fence. They were thinking about it. We were trying to drive them to the next thing. And so we sent out one of our other sales team members to get with his family to give them the Vision Pro demo.
And you can see that the deposit showed up quickly afterwards. My favorite part of this is further update. Customer’s wife has also decided to learn to fly, so they both have a private pilot in time for the airplane to arrive, which is super cool. That is the inspiration, the emotional impact that we’re going for. I’m entirely convinced the Vision Pro experience caused her to decide to learn right now and get going. So this is the kind of signal we’re looking for, and we’re seeing more and more of these anecdotal stories as we kind of track the progress of this.
So finally, this isn’t only about selling airplanes. We see this as a new way for people to experience a product and to get emotional about a product. And for us, on the flight side of things, on the airplane side of things, we’re even looking at ways and exploring ways that we might be able to ease people into the idea of flight training through immersive experiences so before they even get into the airplane to start their training, they have a good orientation of the whole thing. We’re starting to look at that kind of stuff right now.
So I’m excited to say Let’s Go Fly is available right now on the App Store. Our customers have already downloaded it widely thousands of times from all around the world. I cannot wait for all of you to get a chance to look at it. And basically what you can do, it’s a simple equation.
Watch the film, get inspired, understand what this world looks and feels like, and then drop a full-scale Cirrus SR series or shiny Vision Jet right in your living room or your front yard or wherever you want to and explore it in high 3D detail. That’s the one-two punch that we were really going after.
So on behalf of the Global Cirrus team, all the employees, our big global network, and all of our customers, it’s been a pleasure for me to share this story with you, and we are excited to be on the leading edge of this particular use case and to see what doors it may open for others in this immersive space. Thank you very much.
Huge thank you to Anthony. And honestly, to see epic reactions in Apple Vision Pro never gets old, and awesome to see him arming his sales team across the world with that. Hopefully, you know, some great inspiration for the brand and sales folks watching today. Now, just before lunch, I hear that our final guest this morning is on their way, having taken Apple Immersive to one of the most remote places on Earth. And I’m told that he likes to travel in a group. So a quick heads up, you might want to warm up your vocal cords. I’m not going to spoil the surprise, but if something calls out to you, It is only polite to answer back.
Insights and learnings from Wolf: An immersive experience
and welcome to my pack. If what I’m talking about today chimes with you, you feel free to howl back to me. My name is Jeff Wilson. Thank you. Thank you. You can all be in my next film. My name is Jeff Wilson. I’m a director and executive producer at Silverback Films.
And I’m here to share with you how we brought one of the most unique places on Earth and one of its most special animals into the immersive space. and how, through some very simple storytelling rules, we boiled down the essence of what it is to be in the wild.
But first, I’d like to introduce you to what we do for a living. Silverback Films makes highly ambitious nature documentary series and feature docs for audiences of billions all across the world. Everything from the globally ambitious blue chip nature series like Our Planet to the equally epic macro worlds in Secrets of the Bees.
Our long term collaborations with the world’s favourite naturalists have resulted in important feature docs such as David Attenborough’s A Life on Our Planet. But we’re not afraid to tell little stories with a big heart, like our otter and man love epic, Billy and Molly. And we love giving the animals we love the best stage we can through long-term collaboration with Disneynature in big-hearted character stories like Penguins.
But really, what is at the core of what we do across all of these films is we work hard to be in the right place, at the right time, with the right equipment, And we always go armed with a curious mind that’s alive to the serendipitous magic of the natural world.
When we do our jobs right, we can take our understanding of the complex beauty of our planet and share it with millions of others. And that is awesome. It’s this ethos that’s led me to the piece I’m talking to you about today. Disneynature’s Wolf: an immersive experience. It’s a story that invites our audience to come alongside our crew as they spend six weeks on Ellesmere Island in the far north of the Canadian Arctic.
It’s voiced by Peter, our assistant producer, who was himself experiencing these wolves for the very first time. and it immerses you in a landscape, inviting you to reset your thinking, to take a pause from the world outside of the Apple Vision Pro and reconnect with something more primal. As you follow our crew through their weeks of tracking and building trust with the wolves, you too get time to experience one of Earth’s last great wildernesses. And you get to spend time observing wild wolves in a way that only a handful of humans have ever done.
Our piece is about what it takes to be a wildlife filmmaker. But it’s also about the responsibility of representing humanity in the wilderness and what that process leaves in your soul. It’s a story that took almost two decades in the telling. The story of how our piece came about started 19 years ago, when a barely recognizable version of me was dropped into the most remote place on our planet, the northern tip of Ellesmere Island in the Canadian Arctic.
My task at the time was to track down and film Arctic wolves in a vast area of high Arctic tundra. For weeks, we searched the remote valleys of this island, piecing together signs and tracks, but really seeing nothing at all. Until one day, one very trusting mother appeared from the crest of a hill and walked directly down to sniff the shoelaces on my boots.
It was a fleeting moment of contact, but one that was to change my life and my perspective forever. The wolves here live their lives with little to no contact with humans, and they are, as a result, naive, uninhibited by the fear that most animals rightfully have of us humans. As someone who grew up surrounded by wilderness and worked with wild animals all over the world, I knew that this moment and these wolves represented something very, very special. It was a glimpse into a past wilderness that exists nowhere else on earth.
The following two months of living alongside these wolves changed me profoundly. They made a deep and indelible mark on my understanding of our place on planet Earth. The tools we had at the time, the latest camera technology, state of the art telephoto zoom lenses, alongside our field craft and a bit of a determination, meant that we were very successful at showing the world new and intimate behaviors of wild Arctic wolves. You could say it was mission accomplished, but there was something missing.
No matter how much I tried to share our experience of living alongside the wolves with friends, or actually really honestly anyone who would listen frankly, it was incredibly difficult to put into words. There was an enormous gulf between the real experience of what others could understand of this world Our 2D documentation only ever managed to communicate a passive form of documentary? It was, in many ways, too narrow a field of view. It left me frustrated because there was so much the wolves taught me about myself and our responsibility to nature that I felt was important, but it was being lost.
Fast forward nearly two decades, and we began working with our long-time collaborators and friends at Disney for their iconic Disneynature strand. And alongside our feature documentary, this time, I saw the opportunity to bridge that gulf of understanding with our audience, the one that hadn’t been able to bridge in the past.
You see, the immersive storyteller I’d seen was evolving rapidly. And in that evolution, I recognized the technology, the resolution of the cameras available, The unbelievable possibilities for rich and immersive sound environments. The human-like 180 degree field of view. And the way that Apple brought all of that together with acuity and final experience in Apple Vision and Apple Immersive on Apple Vision Pro They were all finally at a sweet spot for what I wanted to do.
I knew that if we could create the opportunity, we could do something really special. The Disneynature shoot that followed was every bit as extraordinary as the first encounter decades before. In fact, it was a little bit better. Over six weeks, our expert wildlife cinematographers, Howard Bourne, John Shier, Jamie McPherson, and assistant producer Peter Lytle, tracked and lived alongside the wolves.
Again, they had at their disposal the very best 8K cameras and the very best long lens technology for capturing the intimate lives of the wolves. And you’ll see stuff that you’ve never seen before in the main film. But importantly, this time we were also armed with the Blackmagic URSA Cine Immersive Camera and had Apple’s creative and technical team supporting us, albeit remotely.
At that point, the URSA was so new to the market, we weren’t sure it was going to even make it through the first week. You see, this isn’t your typical film shoot. Bouncing around on an ATV in the middle of nowhere tests every single rivet, every connection, every precision-aligned sensor. If the camera had broken, there was absolutely no way that we were going to be able to replace it. It was all or nothing.
Quite honestly, we had bigger problems. The wolves didn’t read the script. Not once did they read the script. But despite that small technical hitch, it began to emerge that this immersive view of the world could really bring something unique. And it’s a lot more simple than you think.
We began to understand that the camera was like a new member of our crew. And when treated as such, it could place the audience within every success and failure that we experienced on the expedition. And as a new member of our crew, we needed to allow the camera to immerse itself into the location and the story slowly.
In hindsight, it was pretty risky to try and pull off an immersive film in the most remote place on the planet with new technology and a cast who doesn’t give a damn what’s in your script. But amongst what might seem like chaos, we were gifted something beautiful. I should also mention at this point, none of us had ever made an immersive film before, nor used the camera. But perhaps it was because of that naivety, and perhaps because we were clear about what we wanted to show the world, that we would be able to adapt as we went along. We actively learned from our mistakes. And this is what we learned along the way.
Firstly, the best experiences are earned. To immerse our audience in our world, the world of a wildlife filmmaker, we knew we first had to take them as far away as possible from the world outside the Apple Vision Pro, wherever they were. a deep immersion in nature right from the start.
And we achieved this in three ways. Firstly, the extraordinary resolution of the Blackmagic URSA with 180 degree field of view and that fantastic color depth of the Apple Vision Pro. It was the closest to real life I have ever seen. Secondly, a pure and real ambisonic sound environment that situated our audience in the real world of sonic world of Ellesmere Island. And third, a static camera. We wanted our audience to take this extraordinary landscape in for themselves. With time and calm, they could situate themselves in it without direction.
The effect is profound. Even in design, we couldn’t have predicted how powerful a storytelling tool, a simple landscape, nothing more than a vista to immerse yourself in could actually be. That powerful immersive effect gave us the confidence to treat our film as we would treat our field craft.
Taking time to experience the world, reconnecting with deep observation, that’s something that’s sadly lacking in most of our busy lives. It became not only a story point, but a narrative driver. So when the wolves did turn up, keeping a safe distance as they worked out who the field team were, that too became our story, part of the earned experience. And this is how real life first contact actually plays out.
Everything I’d been taught as a director about delivering the audience a hook as early on as possible in a piece didn’t apply in this new immersive space. In fact, it was the complete opposite. The slower we took the story, the more immersive it became. The more we earned the experience, the better the experience became.
The next thing we learned, detail is story. As we began to understand the power of this immersive medium, something unexpected also began to emerge. The detail wove itself into the story. And through this, a whole universe of storytelling avenues opened up. Within the high resolution of the Apple Vision Pro, we could use Peter’s narration to guide our audience towards details in the image.
The beautiful bird call you just heard, a shimmer of the light on the water, or a shift in the direction of the wind on the blades of grass. These nuances became discoveries. And if you give the audience time to engage with them, The fact is, when you discover the detail yourself, in your own time, it’s a storytelling superpower. Because when you found it yourself, it becomes something indelible, a memory.
But visual detail alone is not enough. In this world, all senses are equal. It’s tempting in such a rich and detailed visual environment to focus on just great cinematography. But what I loved about working in this format and with these tools is the ability to see immersive as so much more than just picture. As a small crew, all of our members need to do multiple jobs. So it fell upon our team to upskill with ambisonic sound recording as well. And the greatest example of how this worked in our piece was again unscripted. Let’s take a listen.
Trust me. There is nothing more real about being a wildlife filmmaker than being challenged by a swarm of mosquitoes. In fact, it’s a complete miracle the crew even switched the camera on and the microphones. But the ambisonic sound mix puts you inside the swarm, and the audience is challenged to focus beyond it to what the wolves are doing. to train your brain to go past the first layer of immersion into the second.
It’s absolutely horrendous and brilliant at the same time. And very, very, very real. So by now we understood that taking our time, using detail in story, and building a whole sensory suite all brought our audiences into our world. we noticed we were connecting people to wilderness in completely new ways. So the next thing that we learned is that the director has to be a choreographer.
You might think this is obvious for anyone who calls themselves a director. Yet in the world we wildlife filmmakers live in, where most things are outside of the director’s control, you have to have a laser-focused understanding of timing and depth in a shot that’s happening in real time before your eyes.
when an animal enters frame, how it’s interacting with the landscape and the camera. how other animals provide depth and context, where the sound is coming from, if there’s any at all. And as you’re piecing these pieces of the jigsaw together, you also have to be calibrating for your own sense of adrenaline.
As a crew, we were determined to have as few shots as possible and give enough screen time to every single shot, avoiding fast cuts that remove you from the moment. So every shot became a scene, and every scene needed to carry multiple layers of story. It meant counting under our breath the seconds that had passed between points of interest. And it meant letting go of some things as we waited for the scene to finish. It was unbelievably frustrating for sure. But when we got it right, the captured experience was unlike any other form of media.
Which brings me on to my final understanding. Experience and entertainment are not the same. As a traditional filmmaker, the edit presented its challenges. Time and again, we found ourselves trying to add to the story, to make it more complete. We debated freeze frames to give us longer on screen with the wolves.
We debated use of music to heighten the emotion and engagement in certain parts of our story. And we worried endlessly about how long our audiences would spend within our shots before they became bored. And each time that we tried to apply these tried and tested tools of traditional filmmaking, our experience became less immersive.
further from the experience of being there and further from the story we set out to tell. Entertainment wasn’t necessary here, but being in nature was enough. And therein lay the immersion. Thank you. So experiences are earned. Detail is story. All senses are equal. The director is a choreographer. And experience and entertainment are not the same.
At inception, we already knew that this place and these wolves were extraordinary. Our field team took this incredible technology and brought back absolutely groundbreaking images. You as an audience can join them like never before. But what I’m most proud of is that in finally being able to transport our audience to the world that changed me 19 years ago, I could also take the final step. begin a conversation about what our relationship is with the natural world.
You see, by giving us access to their lives, by trusting our cameras and our presence, these wolves have underlined just how phenomenally rare that experience is. And it begs the question, what is it that we have done to cause virtually all other animals we share this planet with to treat us with caution? And what if we could reset that relationship?
Wouldn’t that be the most immersive experience of all? Wolf, an immersive experience, will be available to watch on Earth Day in 2027. I need to say a huge thank you to our incredible field team, our editor, Jacob Parish, and to everyone at Silverback Films and Disneynature. And thank you all for your time today.
Huge thank you to Jeff there. 19 years from one trusting wolf sniffing his shoelaces to bringing us all out onto the tundra and beside the pack. What a journey. And if you caught Wolf at Venice Immersive, you’ll know it’s one of the most beautiful and probably the most reflective experiences out there because until now, only a handful of people ever got a chance to be that close to these beautiful creatures. But now, thanks to this experience, you can be one of them. And I think that’s pretty special.
Now, it’s been an action-packed morning, and judging by the howls earlier, Jeff’s pack has grown a wee bit. So it’s time to eat. And I apologize in advance that we are not serving Arctic hare, but lunch is ready, and we have plenty of delicious bites waiting for you. So we’re going to take a break so everyone can fill up on some food. Afterwards, Jared will be here to lift the lid on the newest immersive experience live in Apple Immersive. So enjoy your lunch and we will see you back here at 1:25. Thank you.
Thank you. ♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ Thank you. ♪♪ ♪♪ Thank you. ♪ Let’s do it. Thank you. Thank you. ♪ ♪♪ ♪♪ Thank you. ♪♪ ♪♪ ♪♪ ♪♪ Thank you. Thank you. Thank you. Thank you. ♪♪ ♪♪ Thank you. Thank you. ♪♪ ♪♪ ♪♪ ♪♪ Thank you. Thank you. ♪ ♪ Our program will continue in five minutes.
Thank you. Thank you. ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪ ♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪ ♪ ♪ ♪ ♪♪ ♪♪ ♪♪ ♪♪ Thank you. ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ ♪♪ so Thank you. Thank you. Thank you. - Okay, well, welcome back everybody. Yes, thank you.
I hope everyone is feeling suitably fed and watered and everyone is sat down. Everybody feeling comfortable? Yes. Amazing. Ow! So-- Also, for you folks watching online, no matter what time it is where you are, I am very glad you are still with us. And a quick reminder, we have a team of folks in the back helping to answer your questions today. And after the next session, we will have a panel up here to discuss the most upvoted ones.
Now, we have folks joining from our internal content team, so think editorial questions about how we might produce our own Apple Immersive titles. We’ve got folks from the product team joining us as well, as well as engineering, spatial audio, visual effects, post-production, whatever it might be. This QR code is where you can submit those questions and upvote.
give you an extra second for folks taking photos. Awesome. Now, on to our next session. And it’s a very special one because earlier this year, we set out to bring Apple Immersive, that same Apple Immersive experience that customers love, to a whole new format. Now, for the seasoned live broadcast pros in the room, I know there’s many of you here today, you will know that that was no easy feat, but the most exciting part is that it laid the foundations for how you can do the same very soon. So to lift the lid on what it takes to broadcast live in Apple Immersive and what’s coming, let’s welcome Jared. Thank you. Thank you, Elliot.
Meet Live in Apple Immersive
Good afternoon, everyone. I hope you enjoyed lunch. My name is Jared King. I work on the immersive media technologies team here at Apple, and I’m the chief lid lifter of the day. Last year, so we introduced a new approach to video with Apple Immersive Video. And like we heard this morning, so many of you have been putting that approach to work this year, creating some amazing content.
We’ve been completely blown away by the originality, the creativity of what you’ve been able to make. So this afternoon, we’re going to build on this momentum and we’ll take us live. So this spring, Live in Apple Immersive launched in front of the most demanding audience there is, sports fans.
For the first time ever in Apple Vision Pro, fans were transported courtside live. Six Los Angeles Lakers games delivered to the LA market through the Spectrum SportsNet and NBA apps. These games are still available on platform as VOD experiences, so we’d love for you to check them out. But in case you missed it, Live immersive cameras brought you to seats no ticket can buy. Under the basket, at the scorer’s table, in the mix for court player warm-ups and celebrations. Closer to the action than anyone in the arena.
Spatial graphics enhance the action happening right in front of you. And spatial audio, mixed for every position, made the audience a part of the roaring crowd. With fully immersive instant replay, fans never missed a moment, experiencing the biggest highlights seconds after they occurred. All these elements were rolled together into a smooth, live produced broadcast with our Apple production team, and viewers to the best seat in the house in a way that was natural and comfortable.
Then, the next big milestone. In September, Apple delivered four Friday night baseball games to customers in the TV app. This time you got to play ball. Our team brought viewers into the dugouts to dodge sunflower seeds, catch high fives, out to the bullpen to warm up, and gave them the best seat in the park right behind home plate.
Our commentary team anchored the experience right in front of you, talking to fans through every play-by-play, and provided the narrative on every head-to-head matchup during the game. In addition to all our NBA production elements, we built something new for MLB: the virtual Jumbotron. This was a custom, larger-than-life screen floating in the scene where viewers saw zoomed close-up of players, ball and strike calls, and slow motion replay.
So all in, that’s 10 live events, 35 hours of immersive live sports on air. And it continues to excite with more than 40 hours of on-demand highlights and full game replay for you to relive again and again. And that much content on Apple Vision Pro is something to be really excited about. and fans and critics loved it. One viewer even proclaimed they had fallen back in love with baseball because of it. others that didn’t even like baseball, but were still captivated for the whole experience.
And for us, it proved something really important, that the media formats, the broadcast tools, and the production workflows were ready under the real pressure of a live broadcast for mainstream. So this brings me here today with the creators, the engineers, the producers, and operators. And the 5 is coming fast to it.
In the next hour, we’ll explore what it means to create in this format and learn about what changes when that content is created in live. We’ll get an introduction to the new live immersive media formats and step inside of a live production workflow. Finally, we’ll finish with what you need to know to start preparing for your own live broadcast and the key early planning steps to ensure that success.
So let’s warm back up with a quick review of the immersive media format you’re familiar with from the and the post-production work. So you already know what makes Apple immersive video different. First, fidelity of presence. Capturing the world as close to 20/20 vision as we can with our own eyes. Extreme field of view. The image blends into your vision and you never feel the edge of frame.
Bespoke projections per shot. There are no one-size-fits-all projection types, so every pixel stays exactly as the camera, the lens, and the sensor captured it. And of course, world scale accuracy. Straight lines are straight. People, objects, and the sharks feel the correct size. And of course, it’s compound media format with a variety of media elements required in addition to the video. lens calibration data for every clip, motion data for every frame of video, and a presentation track that tells Apple Vision Pro how to play that edit.
And of course, the spatial audio format delivering the audio experience. All that travels conveniently together in a single MOV file containing all the data you need with no sidecars. It really makes working in Apple Immersive Video feel as simple as working in 2D. So what actually changes when you go live in Immersive? There are three big shifts in production, asset use, and transport.
First, live content is just produced at a different time scale. During post-production, you choose your shots, you can mix your audio, add graphics, tweak color, and encode, all in the edit bay, with time to get it right and redo it if there’s an error. In Live, every one of these production tools and creative decisions happens on the fly. There’s a literal ticking clock.
There’s no pause, no redo, no I’ll fix it in post. Every minute counts down to a hard start and up to a final whistle. And the mistakes you make and the successes both ship equally to the customers. and the quality bar for the viewers is no more relaxed. Live requires that same tolerance for fidelity, field of view, and viewer comfort as post.
Second, live elements are often used discreetly within the production facility. So in POST, video, spatial audio, and metadata are stored within one neatly packed MOV file. Most tools work on or with that file. But in LIVE, all those same metadata or media elements are required, but they are often broken out into individual assets for discrete use by the production tools. This allows for a specific process requiring a certain element to work with that and only that element. This reduces compute, latency, and bandwidth requirements of those tools. And third, live media is streamed in real time between production tools to create the content.
In post, media is moved between tools as large file transfer that that device can use once it’s received it. In live, all the same media elements are streamed in real time, bit by bit, between all the devices and tools in the facility used to create the content. Tools receive live content in, process it with minimal latency, and send it on to the next step. At the end, individual video, audio, and metadata elements meet back up to be encoded and streamed out live.
So while there are many similarities between the immersive elements you’ll be working with, the main differences stem from time constraints in the production, the use of those media assets, and the transport of those assets between the production tools. So you might be asking, live broadcast already exists, how different can immersive live really be?
Well, just as post-production was reinvented, as you all know, for immersive’s high quality standards, complex multi-asset workflows, traditional live production must evolve as well. And that’s where our new live format comes in. So to understand why it was designed from the ground up, let’s start by looking at some of the numbers.
To match human visual acuity, stereoscopic Apple immersive video resolution is 32 times larger than HD or UHD resolutions typically used in 2D broadcasts. and it’s produced at twice the frame rate. When you are literally placing your audience inside of the content, fidelity, presence are everything. And every step of the real-time workflow has to preserve that.
and audio scales exactly the same way. We can compare a traditional stereo mix or even a 5.1 surround sound mix with immersive audio mixes. So these mixes can carry over 64 channels containing ambisonic beds, audio objects, surrounding your audience with a sound that’s as rich as that picture.
The key here is traditional broadcast formats and signal transports were never designed for numbers like these. So, immersive live format had to be built from the ground up. To achieve this, we’ve introduced a new suite of standards for immersive live production called the Pro-IM Live Format. Pro-IM was architected in such a way that handles these massive resolutions of video, high audio channel counts, and complex metadata elements that compose immersive media in a way that feels familiar and make live workflows easy and intuitive. So we’ll jump in.
First up, video. So live immersive video moves around the facility as streamed real-time ProRes frames. Very important. As opposed to the uncompressed video typically used in 2D broadcasts. And why ProRes? Because ProRes is a high quality, visually lossless compression that preserves all the fidelity from camera at a data rate that a production facility can actually move around on the wire. Something that’s extremely challenging if that data was uncompressed.
Second, audio. Each spatial mix is built from uncompressed PCM streams. Same as usual, but just a lot more channels. These flows contain specific channel layouts, high flow amounts, carefully set gain structures, all of which must be preserved between every device in the workflow. And third, of course, metadata. Per frame streamed JSON metadata that travels in parallel with the video and audio between all of the production tools.
If you’re familiar with the Immersive Lens Processing Data, or the ILPD file from post-production, this is its live counterpart. It contains all the information on lens projection, camera orientation, audio object position, and the creative commands that are directing it how to be displayed on client. Now, each media payload we’ve mentioned there is only useful if we can get to where it’s going. So whether that’s the output of a camera, the input to a replay system, or the output of a switcher as program. I will go back.
Transport Media. We’ve leaned into SMPTE 2110 as the media transport of choice, the industry standard for media over IP workflows and broadcast facilities worldwide. So let’s jump back to video. So immersive ProRes video is transported as 2110-22 IP flows. This is the 2110 standard for compressed video over IP. And very importantly, both eyes are carried together within that single IP bit stream.
This is worth doubling down on because this represents a huge workflow simplification. Because all the video data is in that one signal, it means there can be no eye swapping by misrouting separate left eye, right eye flows, no latency differences between left and right eyes, causing uncomfortable stereo distortion. So finally, you can confidently work with live immersive video as you would a single 2D flow.
Yes, standing ovation, I know, thank you. Please be seated. So moving on. Spatial audio travels as high channel count 2110-30 flows. This is the standard for uncompressed audio over IP. And immersive metadata, travels as 2110-41 flows. This is the standard for user-defined metadata over IP. This allows us to transport immersive payloads in real time between all these devices.
So it’s easy. Three standard signal types for video, audio, and metadata. And every camera, mic, switcher, and encoder speaks that same language. You can freely exchange those media flows between them. And as a bonus, here’s one of my favorite parts of this design. Because the video already exists as streamed ProRes frames live, there is no encode or decode step in recording this media to disk. It’s a frame copy. Or playing it back out again for tools such as replay, ISO record, or editorial. This means that these applications can run on much lighter weight hardware than live workflows that rely on compressing the data before storing to disk.
So that’s the Pro-IM live format. There’s a lot more to explore here, especially for those packet-inclined among us. But if you’d like to dive deeper into how these formats are put together and how they move through a live facility, watch Build Live Production Tools for Apple Immersive Video from WWDC 26. And for folks here in the room, you can join us in a breakout session this afternoon to dive deeper into how these work in practice.
Okay, so this brings us to the next big idea. Because the formats are new, the infrastructure also is unique. So let’s step inside of a live production. To a viewer, a live event looks effortless, and that’s the point. Most people see a camera, they see an Apple Vision Pro, and they assume that’s most of the story, joined in the middle by the Internet and probably a few computers.
In fact, if you take a small look behind the scenes of our NBA broadcast control room, you’ll see that the middle is much more interesting. We have thousands of media signals moving around in our immersive production trucks. Directors and producers calling the shots, replay teams, graphics designers, camera techs operating the equipment, and lots of engineers sweating to make sure everything is working correctly and no one notices if it’s not.
So as you can see, between these two endpoints, there are countless tools working together in real time without mistakes. So let’s open up that middle. At a top level, here’s every major piece it takes to bring a live immersive broadcast to your audience. Starting with production on the left, and delivery to client on the right.
As creative tools, these are many of the same disciplines you know from 2D. Camera switching, audio mixing, instant replay, and then technical disciplines like signal routing, media encode, and media QC. What’s different is that every one of these tools now has to support the live Pro-IM formats and standards we’ve just outlined. So let’s take a quick walk through it from the lens to the viewer.
It starts with immersive cameras, placed around the event at a variety of angles to capture the action. Every camera connects into a centralized 2110 media network. This is that core of the facility. This network distributes the live signals between every device via those three 2110 flow types we talked about. Dash 22 for video, dash 30 for audio, and dash 41 for metadata.
And around this core sit the rest of the live disciplines. So we’ll walk through them one by one. First, video switching. An immersive video switcher allows the director to cut between any number of cameras deployed in the creative positions around the venue. The viewer is always experiencing the best place for action.
This is also where graphics come together. Spatially placed graphics are keyed into the scene, complete with interesting data such as game clock and score, team lineups, player stats, pitch counts, and embedded 2D feeds. Information that makes the moment richer without pulling the viewer out of it. Next, instant replay. Replay systems record multiple camera angles throughout the event, so the production can put the biggest moments back to air within seconds.
replay is also the gateway between live and post, allowing for post-produced content to be played back into the show in creation of archival media for post-production edit later. Also, a great picture falls flat without great sound. Live ambisonic surround and spot microphones are deployed to create a spatial audio mix for every camera position. When the director takes the camera to air, its mix also goes with it, so what you hear always matches what you see. I think that’s the point. On top of that, you can add commentary, music, and sound effects, just like any other broadcast.
not to be overlooked, quality control. QC and measurement tools help engineers confirm that the live immersive media flows are performing as expected and a key tool in finding and fixing issues fast when the show is on air. And finally, delivery. The content your team produces is encoded, transcoded, segmented, hosted, on-prem or in the cloud, before distribution to viewers around the world. And for app developers, live streaming to an app just works. There’s nothing new to build. Immersive feeds play back through AVKit, just like all video on Apple platforms.
So the big question of the day is how do you get there? So let’s talk about preparing for your first live production. We’ll revisit the workflow map to learn where we are today. Immersive formats are new. Traditional broadcast tools don’t simply plug in. They weren’t built for these formats and bandwidths and spatial audio demands or those metadata requirements. So immersive broadcast demands an entirely new class of tools.
So to prove out live immersive, define those Pro-IM formats, refine the workflows, Apple built many of these first production tools in-house. The NBA and MLB broadcasts proved the workflow could deliver end-to-end and showed third-party tooling partners it was really time to build. So by the MLB games, much of the production was already running on partner tools, a major step towards fully independent third-party productions. So today, we are excited to announce that partners will soon be launching Immersive Ready products across the entire workflow. In early 2027, their tools complete the whole chain, so you can build and run live productions entirely on your own.
Blackmagic Design is bringing a full suite of immersive capable products from cameras to video switchers, networking, ISO recording, and live spatial audio mixing with Fairlight. Softron provides immersive instant replay and ISO recording, with 2110 connectivity brought to you by Macnica’s 100G network cards, the first 2110 card at 100G designed specifically for the Mac.
wTVision and Vizrt create rich, data-driven, immersive graphics to augment your broadcasts. Bridge Technologies’ VB440 QC tools are giving engineers detailed visibility into signal quality, format compliance, and creative monitoring. Lawo, Cisco, and Arista are already providing 2110 support for immersive signal routing in the broadcast facility. And finally, Ateme brings the full encode, hosting, and distribution chain to bring your content to customers on Apple Vision Pro. So here it is, full workflow, lens to lens. This full suite of live tools is launching early next year, and many of these vendors are here today. So for those of you here in Cupertino, you’ll be able to meet them in the demo area after the Q&A.
There’s so much to do today. Some of the tools are ready. So are you. So here’s where to start. First, define the type of facility and form factor that fits where your content lives. Do you broadcast from a new location every time? Then in mobile production, outside broadcast trucks or fly packs may be the answer. In a mobile production, the facility is built into shippable racks or drivable trucks, and you build in and tear down for every event. So this provides a ton of location flexibility, but can add time, cost to every production.
Or, maybe you’re a stadium, a music venue, or a theater, and your event is in the same place every night. Then you may consider a fixed venue approach. In a fixed venue, you install the gear into the building one time and it is always ready. You can go to air quickly and regularly because there’s little set up and tear down, and the cost per show stays low, especially when spread out across many events.
So how and where you deploy that infrastructure is just one variable. You and your teams can also begin to think about the size of that production. A good mental model for this is the location times the creative times the deliverables. This starts to approximate the physical size, creative life cycle, and the content time scale of your project start to finish.
Take, for example, a small theater with a couple of cameras delivering one language for a best seat in the house experience. It’s a very different picture from an international mobile production, with dense camera coverage, graphics, replay, syndicated delivery, and a post-show VOD content obligation, with an SLA mandating two months of on-prem hosting and periodic content refresh. - Oh my gosh.
So run this exercise with your team early. It will give you a real sense of what you’re taking on. But the good news is everything you already consider when sizing a 2D production still applies. So here’s what this can look like as three example builds. The lens-to-lens signal chain is the same in all three. What scales is the deployment type, the creative scope, and that effort to get on air.
First, the theater. Few cameras, spatial audio mixes, and title graphics. Commissioned once, always on, in a low-effort, low-cost way to share your live performances. Next, a basketball arena. Six to eight cameras, video switching, replay, graphics, and spatial audio. High energy, up close production. This could be built into the venue once and ready for every game or event that comes in the door.
Finally, a ballpark produced from mobile trucks. 12 or more camera positions, dozens of audio mixes, multiple replay channels, spatial graphics, etc., etc. And the whole facility is set up and torn down for every event. Plus, back at a studio, a full game recap to be edited and hosted on platform for your customers. So, all very different scenarios with very different sizes.
Next thing you can prepare for is procurement. So a facility usually works best if you have the equipment to build it in the first place. Live requires components that are in high demand right now. Network switching, fiber optics, compute, and venue transmission internet installation can have long lead times. So once you know your strategy, start looking into those lead times so you can plan a realistic timeline to go live.
And finally, build your team’s IT skills. Live runs entirely on 2110. This infrastructure gets simpler every year, but it still asks for more networking expertise than a non-IP-based broadcast. So get your teams learning the 2110 suite of standards, how it’s deployed, and those best practices for operating the networks. There are plenty of great resources and free courses out there to bring your staff up to speed on the fundamentals.
So finally, let’s put it all together. We built on what you already know about Apple Immersive Video and explored what changes with Live. Then we met the new Pro-IM live format. We stepped inside of a live production. Finally, we looked at how you can prepare for your own. So get started today. Begin the thought exercise of sizing your own production. Read the Live in Apple Immersive documentation and examples on the Apple Developer website. and watch the WWDC26 session, “Build Live Production Tools for Apple Immersive Video.”
And for those of you here in Cupertino, come meet the partners building these tools in the demo area this afternoon. So I’ll leave you with this, engineer to engineer, producer to producer, and creator to creator. Live is a bold endeavor. It may take weeks of technical prep, careful coordination, the ability to adapt in the moment, and a little luck.
But when it all comes together, there is nothing like it. Transporting your audiences to an event across the world and letting them share in the excitement of live entertainment together is something truly astounding. This future is incredibly exciting, and we wish you the best delivering your ideas in real time to the world. Thank you.
Thank you, Jared. And what a way to round off our sessions. Since the announcement of Live in Apple Immersive, I know the team and I have received hundreds of questions about how you could do the same. So now you can. And it’s great to know that you’ve got a place to start.
Well, everybody, we are almost at the end of our day here in the theater, and I just want to say thank you so much again for spending it with us. Today has been packed. We’ve covered updates to the platform. We’ve introduced some new tools. We’ve dived deep into the creative and even got a first look at Apple Immersive Live.
Q&A
And all the while, you have been sending in some amazing questions. So we have one more thing happening here in the theater today, and that is our Q&A. Now, if you haven’t sent in your questions yet, there is still time. You can scan the code to submit yours. Perhaps you had some for Jared, who will be joining us in a moment.
And a quick note before we start, that we will try and get to as many questions as possible, but if yours is developer related or we run out of time, you can of course submit a question any time to our team via the developer website. All you need to do is head to developer.apple.com/forums. It won’t come with free people, unfortunately. And if you have a bug or a feature request, then Feedback Assistant is the best way to let our teams know directly. And you can reach that at feedbackassistant.apple.com.
Right, we are going to get the show on the road, so I’m excited to invite up our panel. We have Yuri Emoto from our product team. We have Tim Dashwood and Abhaya and Tim, who you met earlier. I’m going to need my seat, though. One more down. Who you met earlier. Then we have France Costrel, who’s the creative director for our internal content team and helping folks like yourselves create content. And then hopefully Jared is joining us back in the hot seats for some questions. Welcome, everybody.
Well, thank you for joining us up here. Hopefully everyone’s feeling comfortable. Jared may be more comfortable than the last time you were up here. Okay, let’s dive into some questions. And maybe a good place to start is a question that we’ve got, and I’m sure everyone in this room has had in the past couple of months.
Yuri, are we committed to this platform? Yes. What an exciting question to get. I think we get that question a lot, but we want to ensure that we’re obviously very committed to the platform. We’re very excited, actually, in a relatively short amount of time that spatial computing has evolved pretty rapidly. As you saw in Jeff’s presentation this morning, there’s a lot of really exciting use cases that’s coming up in surgery and healthcare.
We have education and training that’s utilizing Apple Vision Pro. We have all of these wonderful creators creating for our platform. So it’s been a very exciting couple of years for spatial computing. And I think this event itself also really speaks to it because we have people from all over the world today joining us today to learn more about our new workflow and this immersive media content. We had new tools that launched today.
We even had new accessory announcement, which is super exciting. So I think this is all testament to say that we are pushing forward spatial computing and Apple is committed to this platform Yeah, and I think everyone in the room here is a part to play in that. Thank you Especially to the folks who saw your titles and the celebration video this morning It’s it’s an awesome industry and community all righty, um Why don’t we move into a creative question maybe to start with we had a question that came in saying, I love what Jeff, I think referring to Jeff Wilson who joined us, I love what Jeff said in terms of this idea of director as a choreographer and other lessons that were learned in the field.
Are there any lessons that a 2D filmmaker might want to know before attempting their first immersive project? I don’t know, France, do you want to have a shot at that? Well, Jeff is brilliant, and I love the way he said it. That was great. The director is a choreographer.
I guess we could extend the metaphor and say that then you’re inviting the viewer to the dance to a certain extent. And we often say that the camera is the viewer and that you’re trying to film the world the way you see it, but you’re also giving them agency to decide where they want to look at and how high your camera will be, how close to your subject the camera will be.
If you want to add some movement to it, it will impact your storytelling. Maybe in more traditional 2D filmmaking, you’ll direct your characters a lot. But here, you’re going to think more about trying to direct the gaze of your viewers. In the case of Jeff, what was great is that in the Wolf films, they gave it clear intention.
The camera is becoming this crew member. And so you’re getting to have that experience to go on that shoot. Going back to this idea of choreography, I guess the spatial audio, it’s the music to what you’re trying to do, right, Abhaya? And so every sound cue will be even more crucial for you. Because while you’re directing less your characters and more your viewer, the sound is what will allow you to try and guess where they may be looking and what you want them to look at or have for great organic transitions.
There’s so much we could cover. I’m just so excited. Someone wants to go and try a new medium. Maybe that’s the more important lesson, really, is to embrace risk and a new medium. And if it worked in Arctic conditions, in front of wolves, for a first film, that there is a great chance that it would work. And also, our team, we’re always so happy to help and give any form of creative support. If you have idea, a special vision that you want to bring to life, we’d be more than happy to support.
France will be here upstairs in the mixer. So any folks that have curious questions about how we’ve been making content with the Apple immersive titles that France and team have been making, you’re in the right place. But just before we move on, maybe Jared, I know that the work you’ve been doing with Live is somewhat parallel to a lot of the teams work with on these broadcasts have come from that same live 2D broadcast.
Could you speak maybe briefly to the Delta in their creative lessons, especially around things like camera placements, camera plots, that type of thing? Yeah, totally. I think the key with the immersive and 2D, maybe camera plot paradigm is no zoom, right? So sports, big format thing, you have broadcasting from a baseball stadium or an NFL stadium, can really zoom in and really capture player reacts, coaches interaction, key fans in the stands, right?
And so I think one biggest thing we learned from camera plot and position is that we need to be close and that we need to be able to give that viewer the sense of presence in the stadium, the best seat in the house, as opposed to be able to get them up in the face of a player, right?
And so things like the virtual jumbotron, that was a way for us to bring in those really close player reacts, coaches interactions, slow motion replays, and other ABS calls like for baseball to not only sit the viewer in the best seat in the house and although they may be a little bit far from the action they can enjoy that and they get that that duality of the benefits of the 2D broadcast as well. So I think broadcast in general and 2D has a lot of tools that have been developed over 50 years right and we’re in nine months into doing live, so the tools will always progress.
But I think it’s really about getting the fan as close to the action and really giving them that best seat. Yeah, awesome. And for folks in the room, we have our AIV, well, live in Apple Immersive breakout session after this where you can meet a whole bunch of the folks helping on this and you can ask those questions in more detail.
Maybe we’ll move from the creative stuff into how you can show your content to a group of people, perhaps. Tim, I was wondering if you might share a little bit of information just about some of the apps that you were talking about earlier and how they’re enabling people to do group sessions and control playback and things like that.
Yeah, so they all use slightly different methods to do that. I mentioned we have the Apple Immersive Video Utility app, and it will work either hardwired or over Wi-Fi. What’s interesting is you can have multiple Vision Pros connected to one computer and just play back your content. Some of the apps also will offload or upload, I should say, the content to each headset beforehand and then it’s a triggered play queue and then they try to maintain sync during the presentation.
I have a little bit of personal experience with the OrangeCue app. What’s interesting with how they work is if someone is a late arrival to the screening, they can enter the screening and the way their sync system works, they’ll be in perfect sync from that point on. And it also checks to maintain sync as they’re going. So yeah, it’s really interesting to see this space start to open up that way. Yeah, and I think there’s a few more. There’s Spatial Film, I think, has a kind of kiosk mode. And the Vivaldi experience is using, I think, Spatial Companion.
I guess with those experiences, though, maybe for you, Yuri, a part of that is kind of getting folks into the device. And we’ve had a question here from RichXR just saying that sometimes that can be a little bit clunky. Do we have any thoughts around Guided Access or MDM to try and help folks get in for shorter demos? Yes, that’s a very good question.
And it is a feedback that we’ve heard before. I think maybe to ground and start the conversation here is around we do have a system level capability called guest user, where we have made it fairly easy in a couple of release ago where you can start guest user from your iPhone.
You’re able to easily select the apps that you want to share to others. You can AirPlay on the spot so you can guide them hands-in-hand so they’re not lost in the experience. And we also even enabled for guests to save their eyes and hands data, which was a great breakthrough. But for folks like the enterprise use cases, we do have capability of MDM where we allow profiles to be downloaded.
And we are still working on ways to make those more enterprise, more showcasing kind of demo easier and easier. And I will be around for the mixer as well. So if you do have feedbacks how you might want to use Apple Vision Pro for some of your demonstration, I will be all ears to hear your feedback today. - That’s a pretty special opportunity.
All right, Abhaya, you’ve had a lot of news to share with the world today. We had another question around how we work with spatial audio and in particular, if we’re mastering 7.1.4, Is there a supported ingest into ASAF or APAC for Vision Pro? Things like do objects carry over?
Should the bed be 7.1.4 channel or HOA? Best for head tracking? Do you have any thoughts there? Yes. So 7.1.4, if you use some sort of universal format like ambisonics, you can then use that in ASAF APAC. So if you start with ambisonics, you can output to 7.1.4.
If you want to bring 7.1.4 in, you can convert that to Ambisonics and bring that into any software that supports Ambisonics, including ASAF. Yeah, so it’s really not just for Apple immersive video. In theory, use the tools you have and you can make it do whatever you’d like. Yes. That’s awesome. Okay.
Here’s one for you, Tim. Sergio has written in to say, we have a scene that starts with calm dancing, and then suddenly heavy snowfall fills the frame. And at this point, the video quality drops dramatically, possibly due to the amount of visual information. So what encoding settings or bit rate adjustments would help you maintain that quality?
That’s a very good question. We deal with finding the right settings and the right process all the time to make sure that when you actually finish and you’re watching your final copy that it looks as good as it can in Vision Pro. I think the key that I can say, if we’re looking at the shot, we might be able to diagnose it a little bit more, but just the general things that contribute to that macro blocking that you’re probably seeing.
One of the things is Snow has a lot of independent, detailed things that are seemingly random, so it’s harder for the compression to keep to that highest quality it can. So I think the key that you can do is obviously you want the snow there, so you need to make everything else as easy for the compressor as you can.
So one thing is a denoise pass that keeps as much detail as you can in the scene but removes any of the random camera noise that’s present in any, no matter what camera you shoot with. And also another key thing is making sure you’re using the, if you’re, say, you dip to black from the previous scene into that snow scene, make sure you’re actually using the, not baking in the dip to black, but using the metadata fade because that will be a huge hit on the compression if you’re fading to and from black without the metadata fade.
So those are just a couple of tips, but I would say it’s a good chance to plug after this, there’s gonna be a live lounge session with some of our finishing team from the internal Apple post-production team that does post on most of the first party Apple stuff. And we’ve got our online editor there who will be doing a session on DeNoise specifically and showing a neat video workflow that we use, that you can use lots of different ways to do DeNoise, but that’s one of the most critical things for maintaining good compression.
On top of that, making sure you’re using, once you’re actually getting to the compression part, MainConcept encoder, you know, if you’re within Resolve, is a big improvement once they added that from the default. So looking into your encoding settings as well. Like I said, if you come to the session, we can kind of, and you have the question there, that’s a little more casual to talk through your specific circumstances, but yeah, it’s anything you can do to advantage the compression quality to maintain the ability to nicely compress the detailed snow. Also, that sounds like a scene I really want to watch in immersive, so I’m curious about the project. I was going to push you for specific denoise parameters, but I’ll let you off the hook.
We’ll be able to get into that in the session, because it really is that we have a little walkthrough demo of the exact process that we go through, And it’s shot by shot. We don’t do like a one denoise pass across the entire show. It’s look at every shot.
You might need to be more aggressive on one shot and less aggressive on the other. So that’s why there’s no magic setting. It’s a lot of trial and error and learning. And you can just do it faster if you have done it a few times. So we can help with some tips. I might stay with denoising if that’s all right, because it came up three or four times in the questions that were submitted.
I think in the work I’ve seen you and the team do, you usually make the call to denoise for a project, either right at the beginning, kind of as you’re going into the edit lock, or at the very, very end, and kind of absorbing a pretty cumbersome render. Could you talk to that briefly for folks maybe who are thinking about denoising and have questions about when is the best time to pull the trigger? - Yeah, absolutely. So that, when you do your denoise, is one of the most critical things for efficiency.
So, you know, a lot of people, you think of that as a finishing step, so you do all your color, you do your visual effects, and you do your long render, and your denoise render is very compute intensive, so it will be probably your longest render. And then after you’re done with that, you look at it and you’re like, ah, I wanna change that one thing. Well now you do your entire highly compute intensive render.
Again, I mean for longer projects, these, depending on your system, this could be over a day. A lot of times we set things overnight and render them. So the key is, once you’ve edit locked your show and you know what your shots are, If you do your denoise pass on all the shots that are in your show with some handles upfront before you get into all the other work that happens afterwards, then that really compute heavy task is complete and you can be much more quickly do your review copies with color changes with, and even when you have handles, even if they’re slight edit changes, you can accommodate those if you’re careful and detail oriented and maintain all the information throughout.
So it’s a huge, huge time saving for efficiency that if you’re not thinking of that final step until you get there, you’re really missing out on huge efficiency. I mean, it saves hundreds of hours across many projects of render. So again, I’m stealing a little bit of Chris Hamilton, our online editor’s thunder for the presentation later, but it really is like one of the biggest things when we help people that are just starting out of how they can really save a lot of time and still make sure this stuff looks as good as it can. ’Cause the clean look will make sure that the compression actually stays clean as well. That’s awesome. Thanks, Tim. All right, where can we find more information about the new Strada tool for live preview of immersive footage?
Well, you’re lucky because the Strada team is here today, and they’ll be up in the demo area, which I believe starts at three o’clock, goes three till five or six. Yeah, just off to the Q&A. Yeah, so please come up and visit, and you can ask them all kinds of questions.
All right, I think this is going to be a two-up maybe for Jared and Tim. When adding Jumbotron-style graphics to immersive footage, what is the correct way to apply warping so that they look natural without strange distortions? In other workflows, we’ve handled this differently, so what’s the recommended approach here?
I’ll start. Yeah, you talk about how you’ve done it. Yeah, correct. So we found that, at least for the immersive Jumbotron, for the MLB example, there’s a couple different ways to do it, right? There’s the make it have that slight bevel, that curvature, and that’s nice, but it really doesn’t emulate, I think, what most fans are expecting, which is an actual flat Jumbotron, either canted off in space, like you would see a Jumbotron in a stadium, or a scoreboard at the event.
So one thing that we really found was straight ended up being better in this particular case. Size was also a big thing. It’s easy to say, Jumbotron, awesome, make a Jumbo, dial up that knob and make it this massive floating thing in space, but you really end up losing beautiful things like the Pirates game that we did. Beautiful skyline there at Pirates Park, wherever it’s called, PNC Park.
Beautiful skyline, great sunset, and we had the Jumbotron massive, right? And we ended up dialing it down a little bit just to give that sky some chance to breathe, make sure that people were more integrated in the environment, and there wasn’t this big oppressive thing standing over your head, right?
Also distance. The way we can place graphics in space is not only left, right, up and down, but also in z-space, like anything else. When you do work with things like virtual jumbos, it’s important to not have it, I call it the porch door or porch roof effect, where you have this thing kind of like shrouding your vision that feels like you have a roof over, or like a visor of a hat. We chose actually to push the Jumbotron out in space, so it’s about over the pitcher’s mound from that camera one position that you saw looking out over the batter.
So really, size and depth is a very, very important consideration when placing things like Jumbotron elements, making it part of the picture, helping enhance the broadcast, but not being a oppressive weight in the sky, you know? - Yeah, I can talk about the projection side for people who use Fusion. If you use the spherical 3D camera in Fusion, you can combine that with, It’s either called plane or quad node, and put the content on that.
And the nice thing about the spherical camera in Fusion is you can set it up with an interaxial that you can publish as a template and control later. And then you would use the PanoMap node to convert that into immersive. And so that projects it correctly. And you can composite it on your footage because that PanoMap, or sorry, yeah, PanoMap node, it will take the ILPD from the media in, right? And so that projection goes into that lens calibration for that camera. So a lot of potential there to do some cool stuff. - Almost like we need Fusion up behind you there, Tim.
- Yeah, Fusion seems like a really steep learning curve. It kind of is, but once you wrap your head around nodes, it’s actually really fun to work in and to experiment.
- One more point on that. So that would be a great example of that post-production versus live kind of workflow. A great way to do that in post. Again, live, all these things are done in real time.
So folks, things like our internal processes, which would maybe take 2D graphics and put them out into space and warp them accordingly in real time, which is an ILPD conversion from 2D to immersive. And then that would be done natively in our switcher. And then for like the Blackmagic switcher that’s coming up, that same process Tim’s described is basically being done in real time, whether that’s an immersive graphic in lens space that you’re bringing in, or it’s a 2D graphic, you know, like the lower third or a score bug you’re pulling in from a host broadcast, that gets converted on the fly from 2D into a warp space and then projected accordingly on the program output. So a very good AB of how you do that in VOD versus live.
One thing to add on that though, because I think for both VOD and live, Jared, you touched on like making sure, you know, size and placement is thought through. I think I want to double down on this concept of when you do float any media that’s not filmed with an immersive camera, understanding, I think people tend to think of it more as like, oh, the users, they may want to have that up there at some times and not want to have it at others. That’s why that thoughtfulness ahead of time, whether it’s live or VOD, of the distance, like where do I think they’ll be looking, when will they be wanting this? It’s a little bit more of like a editorial thought process.
First, I mean, you have to get the technical right, but then also really even thinking the, of course you might know when you wanna bring it in, but like where will they be looking in the scene, even with Z depth, when you are bringing that in. If it’s sports and they’re following the ball, well then you got a pretty good idea.
But if there are times we’ve done this in, you know, other types of documentary, it’s archival footage, thinking of how to creatively draw their eye, if you’re going to bring something in that’s virtual and not actually in the scene, bring them to where you’re going to be looking and then reveal it in that way.
Or you could treat it more like a real element of scene if you’re floating it there and you cut into the shot and this big screen is already floating there, even if it doesn’t look like it’s physically, tactically real, it’s going to occupy physical space in the scene.
So I think as much as it’s important to think of the technical, really just the kind of creative flow of where are people looking and considering the depth especially, because I think when we get questions on this all the time, it’s a lot more about how to get the curvature right, but sometimes where in the scene, I think you used a great example floating over the pitcher’s mound for baseball, makes sense, but if you’re in a locker room, is it actually aligned to the screen that the players look at when they’re breaking down the play and that’s where you put archival. So it can be a creative expression as well as just getting the technical fundamentals right.
- Well, maybe on that, I know we already talked about noise. There’s a lot of questions coming in about denoising. But maybe, France, would you mind speaking to-
- I know you have a great relationship with Tim in post, but how you’ve tackled situations where you know there might be noise, or situations where you want to put graphics in, and how you’ve gone about those decisions?
Yeah, you need someone-- you need a friend in post. But not just in post, and also the way you capture it, obviously. I like night scenes. You know that you’re going to have to be mindful the way you capture it. And our camera team is amazing with that. And in many of our partner films, there is a scene with different type of lighting.
But I think they help elevate your storytelling and also that feeling of the time fleeing and the transitions and you being there. Now, in an Elevated New York film, for example, there are quite a few scenes that are shot at night. And so we work with your team to make it-- how did you say earlier? That was a good way of saying-- maintain the details, exactly, in the city of New York. But I think it was the moment we transitioned to the night scenes when you start getting the chills, really.
Obviously, I wouldn’t switch straight into a night moment from filming during the day. When it comes to your question about graphics, It’s always a question of what’s your intent on the graphic, and are you getting people out of the scene and the emotion, right? Because you’re going to go into more of like, I’m giving you information, or it’s a science way of explaining.
So if there is an organic way to do it, so that going back to this idea that you want to draw the attention to something, and it’s kind of anchored and rooted in what you say, I think it helps a lot. And then when it comes to graphic as in more of a title, I think those are the best to be super creative and immersive.
And working with Andrew and our team as well, the elevator in New York might be one of my favorite graphics ever, because it goes through the buildings as you go. And you see the title. And unlike in more traditional 2D films, you can really embed it in the element, play with the z space, and really have it in a more surprising way. That’s great.
Yeah, for me, Elevated New York was the worst. I was almost emotional in the middle of it, especially with this music choices you made. Maybe we’ll change gears a little bit. We had a question in from Noel. As someone building education products, do Apple believe the Apple Vision Pro can be integrated deeply into secondary or post-secondary institutions in the near future? Do you want to take that Yuri? Absolutely. The answer I would absolutely say it’s a yes.
We have had great trajectory with training and education already. For example, we have example of Complete HeartX has been used for medical training purposes. We also have an app called Built, which is to learn how to build a mechanical object, which was very much used for trade schools.
We also have JigSpace, for example, where you can look at 3D objects, learn them using AI, which is another great example of how you can learn using spatial computing and also AI and combine that all together. It is such a groundbreaking new way to learn new things. So I think we already have a lot of great examples, but I’m very excited how this will even more flourish with the new tools and workflows that we’ve been introducing with visionOS 27 and so forth. Yeah, I think like we saw from Jeff’s presentation, being able to take someone somewhere that they can never go otherwise seems like a pretty easy road in for education. Okay, Abhaya.
For documentary forms of immersive spatial audio, how do you approach the sounds that people technically hear but that they don’t listen to, such as ambient buzz of an HVAC, like an air conditioning unit, the distinction between what their mind’s ear hears and what their literal ear hears. What do you think?
Yes. So your auditory system is very powerful. You can have a noisy sort of HVAC in your room, but once you’re in that room, your brain is already tuning that out. Like if you’re in a noisy restaurant or a cafe, and many of us have been in that situation, you have a table, the restaurant is very noisy, there’s crowds everywhere, but you can clearly hear your friends at the table when they’re talking to you. Because your auditory system, just like your eyes, can focus on the sound you’re interested in. And that’s one of the goals of the Blackmagic Ambisonic microphone is to capture all the cues so your auditory system can choose how to focus. So it’s very powerful.
So giving people the cues allows them to decide what they want to listen to. So if there’s some interesting sound, it gives them sort of the agency, like France was saying, it gives them the ability to listen to what part of the sound scene they want to listen to.
Awesome. I would add just really quickly to that, because for documentary, you’re usually wanting to present what was really there, but there are times where what it really sounded like maybe doesn’t get the point across as well as it should, not because of a microphone, but because, again, sometimes you need to draw someone’s attention to look to the left in a format like ours. They’re not seeing the whole image at all times, so you might need to add elements.
I think in addition to, obviously there’s traditional post sound tools that can help clean up buzzes and hums. If you find those end up being distracting, there are ways to go about that. But I do think it’s important to know, even within documentary, you’ve got to use it tastefully.
But we talk about it as an idealized realism is what we go for a lot of times in our experiences. So finding ways to add, whether it’s planning for some Foley budget to make sure that if there’s things you wanted to emphasize, that weren’t coming through in the recording or that just didn’t happen when and where you needed them to happen.
I really emphasize for people, if you just listen to the experiences that we’ve put out from a first-party Apple Immersive, like the richness of the sound field, some of that comes from really high-quality capture on site and some of that comes from very careful work in post to make sure we give you the feeling and never make you feel like anything wasn’t real. If you didn’t notice that it was added, then we did a really good job.
But ultimately, I do think when people are planning for budgets and timing, one of the best bang for your buck you can get is planning for a little Foley budget and time at the end of your process, ’cause it’s amazing how far you can take things and really drawing up the emotion and tension, scripted as well as unscripted. So there’s a ton of potential there on top of the capabilities of the mic that we’re very excited about. - Yeah.
I feel like maybe the audio is actually one of the biggest difference from the first question when you were like what’s the this is like oh there’s so many things we could be talking about right now but um in your transition like you would cut from a close-up into the dock from a close-up to maybe your b-cam your c-cam a wider and then here’s just like well actually i’m pretty much wide on everything and how am i going to motivate the transition and make it feel organic so it’s just not like my head or i’ve been just suddenly plopped into something.
I feel like sound is so amazing with that. To your point, you’re listening without even realizing, but you’re really ready to be going to the next scene. Yeah, there’s a great example of that with the BBC Proms piece, Debut at the Proms, and they were lucky. They had, I think, just over a hundred microphones, sadly not on one ball like we have today, but dotted around the theater, and they mixed those, and cutting between those shots, it’s a really interesting thing to listen to. You see how the director of that took on the challenge of not making the whole orchestra move in between each cut.
So I definitely recommend checking it out. We have a question all the way from Ecuador from VRO Films. So thank you for joining us where you are. They’re asking, do we have collaboration programs at Apple for South American studios, for Apple Immersive Video, for the URSA Cine Immersive?
And I think we’ve had a couple of other questions also about how filmmakers and creatives using this format can continue their relationship with Apple. I think I’ll answer it broadly for everybody that, of course, events like this are a great place to meet with us and to share your ideas. And ultimately, I speak, I’m sure not just for myself, but everyone on the team that we are as excited about your ideas as you are. So for folks in the room, please do grab members of the team, share with us what you’re working on.
With some of the projects you saw earlier, we have helped kickstart folks with access to cameras and some post-production equipment. We’d love to hear what you’re working on and see how we can help you. The Mixer is probably the best place to do that. If you’re watching online, heading to Feedback Assistant, sharing your app with our Developer relations team can also be a great way of finding more people to talk to and get excited about the work that you’re doing.
With that, we’ve had a question about the process of showcasing Apple immersive films on the Apple TV app. And so maybe I will quickly solve that and then we’ll get back to the more interesting panel, which is that we now have two amazing apps, Fovii and Theater, which are both distributing really high quality Apple immersive video.
So your best place to start is probably to distribute via those apps, and that way the editors, as part of the Apple TV app, will get a chance to take a look at it, and we’re going to do our best to feature the best work we can in that new row that you see in the TV app. So feel free. I think both Theater and Fovii are here in the audience, so I’m sure you can bump into them in the mixer and work out the best way of getting your content on their apps.
All righty. I’m just reading through the next questions. Tim, is there a way to digitally stabilize stereoscopic footage with maybe camera shakes or the same style of automatic stabilization? And what’s the recommended workflow there? And I know there’s two Tim’s on the stage, so I’m going to have to try to solve this. Yeah, you go first. Yeah, I mean, I will say the answer is yes, very carefully.
A lot of the traditional tools just aren’t made for two spherical images, correcting for the things. If you think about the normal way of stabilizing, it’s going to resize and move a part of the image so that it keeps it stable for the viewer. But if you do that just to one eye and do the exact same thing to the other, it’s going to break your stereo.
So you have to, whatever tool you use, and I’m purposely not really going to get into an exact tool workflow here. I would say if it’s a good one, depending on your scenario, maybe you come to the live lounge with Andrew Rakestraw as our VFX supervisor on our team, and he can maybe get a little more detail than I would give.
But I think the key is whatever tool you vet and use, do a couple tests very early and put them in Vision Pro and really make sure it’s giving you the intended result because we’ve seen a lot of people try a tool they’re used to and watch the screen and go, yeah, it absolutely worked, and then move forward with their process and find out at the very end, you know, when they’ve cut their whole show, that these shots weren’t as salvageable as they were hoping. So even for us, you know, like I will say, we did our, we partnered with Red Bull on the surfing show that came out, and I mean, you’re out in the waves.
There’s going to be camera movement. Our camera team did a fantastic job. Their camera team did a fantastic job getting us the best footage we could, but we had to look at, okay, we think these shots are fantastic, let’s go do a quick test for stabilization of how much can you get away with before you have image degradation or a type of warping you just really can’t live with um because i i whereas you know i think in in traditional there’s a lot more shots that can be salvaged i’d say you got to be a lot more cautious up front so plan before you cut your show and you depend on this shot uh do a quick test at the beginning with your tool of choice to make sure you think the quality is acceptable of what sort of stabilization you can get out of it.
Because there’s some stabilization that you know you might, depends on what your subject matter is, like we said with with the Red Bull surfing show we did want you to feel like you’re out on the water. So it’s finding the right balance of okay this is motion that I can understand and I can live with and not get sick from versus you know a completely locked off shot.
You know that that’s achievable for some shots but may not be the effect you want. So my My answer really is test, test, test as early as you can. But yeah, like I said, at the live lounge for finishing, we may be able to get into more specifics if you have those questions.
anything to add? Well, the only thing I would add is just to keep in mind that when you’re trying to stabilize any spherical image, you’re actually doing rotation on three axes, right? Tilt, pan, and roll. But in stereo, you want to avoid roll because you’re going to introduce vertical disparity, right?
And if you have the type of vibration that is actually on the other three axes which is raising up or lateral movement like this I mean that’s much harder to deal with and then the last thing you have to remember is the ILPD that’s already in your footage that camera already has a calibrated ILPD and as soon as you move pixels you have now potentially thrown out the calibration so you have to go into a common projection before you start to do those sorts of things. Go into an equidistant projection or something.
I tend to go to equirectangular because there are more tools for reorienting an equirectangular. The old technique I used to use, doesn’t matter what app you’re using, but it’s just basically picking a point on the horizon and tracking that. And then you just translate that into pan and tilt in the the equirectangular. But yeah, it’s a problem we’ve all dealt with for many years.
Yeah, not just in Apple Immersive. - Yeah, so the moniker of don’t fix in post is probably relevant. - I’ll add a third leg to that stool. But in the live case, there aren’t currently any live stabilization coming, of course. So in the live world, mistakes and success both ship equally, right? So preparation is really your guard for that. Many times in live, you think the best camera position maybe on a handrail where a bunch of fans are sitting. Great.
And then something exciting happens, and that camera is flying around, right? They’re shaking, they’re hitting the barrier, all those kinds of things. So really paying close attention to proper grip scenarios, making sure you’re being thoughtful to the first question, right? Being thoughtful about where that camera is going to be.
The best seat in the house may be an extremely volatile one, and so you do want to take precautions with the league you’re working with, with the venue you’re working with. Seat buyouts, what you have to do to make sure that camera is stabilization, protected because when that camera is moving around it’s not usable on air.
A key example we found and protected against early was for MLB that again high behind home plate shot is mounted to a pole that they have the ABS cameras on, they have a couple robos and it’s stable until two things happen. One, they foul off into that netting then your camera starts to do this and it’s a bad experience. So did a couple things stabilizing that pole, being able to have bailout scenarios when it starts to move cut to an an alternative shot, making sure production was prepped to move to that shot.
The Jumbotron switches accordingly so it’s not blocking something else. And then the other thing is when a robo, so they have robo cameras that are controlled by the 2D operators, attach that same pole, game one, a robo just starts spinning in circles like a Star Wars droid, right? Because it was broken and we couldn’t use that shot because of a 2D mistake and error. So you learn from those things and you build them into your show plan as your season or your event progresses.
Speaking of that, one of the most amusing things I often like to find out from the team is how they’re building their cameras into camera hides. I know that Jared’s been doing that for live. I know, France, you were doing that back for the Alicia Keys piece. They’re always fun to look out for.
Abhaya, just a quick question. I’m not sure if this is a duplicate. So we had a question from Rich XR asking, “For the same Blackmagic Immersive Capture ambisonic mix, is there a supported export for AIV ASAF toolchain to open formats, maybe like H265 plus AmbiX for other devices, other workflows?
Yeah, so the Blackmagic ambisonic mic captures ambisonics, which is an open universal format. So any mixing toolset that supports ambisonics, you can export to that as native ambisonics, and that will keep the full resolution if you export a seventh order ambisonics. If some tools only take third-order ambisonics, you can export third-order ambisonics. If you need a channel mix, that’s what we heard earlier was the 7.1.4 mix. So you can also export to 7.1.4.
So ambisonics is sort of like a universal, like the high-resolution format, and you can mix down to whatever you need for your particular workflow. Thanks. Yuri, maybe one for you. Elena was asking, Apple Vision Pro doesn’t seem to give information about user input, such as where a user is looking to an external app. Do we have any future plans there? Are we planning to provide access for that type of feature?
Yes. So currently today, she is correct. We do not provide the eye information to a third-party app or third-party developers. We do take privacy very seriously. So we ensure that your eye information, Some people may call it gaze information, stay on device. So that is the reason why we do not share those information to third parties.
Now, if you are an entitled enterprise user, for example, there are processes and ways to get access to those information to develop an experience. But currently, we don’t speak about future roadmaps. For now, we do not give out eye data information to third-party developers. Awesome. This is going to be a very short question for you, Tim Dashwood. Why not Canon R5C? Thank you.
- Well, for AIV, you mean.
- Well, that was the question. I think that’s a great point, ’cause we have the whole of APMP.
- The whole reason we have Apple Projected Media Profiles to support all of those legacy formats. So when you’re shooting on a camera with a dual fisheye lens that has a single sensor, you have to remember that, yeah, that sensor might be an 8K sensor, but each eye is only getting 4K, right? So you’re already at half the resolution of the Blackmagic Immersive camera, which has two separate sensors, are, you know, between 70, 7200 vertical, right? So, yeah, that’s basically the reason, is resolution.
- And could you talk maybe briefly to the way that you can shoot on Canon R5C and work into APMP? You had some of the tools earlier that were supporting those.
- Yeah, I mean, that’s possible, and Tim, I’m sure you guys have had to do that, you know, If you’re doing something dangerous, you want a crash cam, you don’t necessarily want to put your Blackmagic Immersive into a place. So I’m sure you’ve mixed and matched from time to time.
An important distinction is if you’re building an Apple Immersive video that you’re going to export in that format end to end and you want to integrate a shot almost like if you’re filming for theatrical and you had a different camera you wanted to mix in the middle. You can do that.
There just has to be an understanding of what is the user experience meant to be. If you’re filming with dual fisheye lenses and it’s meant to be 180-degree field of view, one of the things we talk about a lot when we work with new creative partners is, well, when you cut to that camera, it’s going to look very different from a quality perspective. Does the user know why? All of a sudden their world is going to be half the resolution.
If you’ve built into your story… And the frame rate won’t be 90 frames per second. Exactly. So there’s just a level of immersion that, you know, I think for movies and television on a screen, we’re very used to seeing the quality change and you don’t really ask questions.
When it’s the world around you and all of a sudden I’m looking at all of you and you go half resolution, I go to the doctor and get checked out. So I think the key is if you want to provide a user experience where it’s the same 180-degree field of view and all of a sudden your quality drops, I would be personally guiding you to say, like, well, aren’t people gonna think you made a mistake in finishing, aren’t people gonna be worried that their device isn’t working, you know, so it’s not that you can’t do it, there’s ways to translate it into a more universal format, like insert it into the middle of your immersive show.
It’s much more the question, I try to start all of our technical explorations of what is it gonna mean for the viewer, they don’t have to know anything about how we made this and how we got there, or why it was hard to put a camera somewhere, so really, if you can come up with the way that it makes sense in the story, go for it, do it.
And if you can get the shot, but this is another thing we talk about a lot, like what’s the trade-off? If it’s gonna jump down in quality, but what they’re seeing is so incredible and it couldn’t be captured any other way, that they’re not gonna care that it was half resolution, then use it.
It’s really like, if getting in Vision Pro, I really try to talk a lot about the, how much the, just the testing on the front end. Like if you are confident that what you’re gonna shoot and half of that resolution with a different camera is going to deliver the experience you’re trying to give people, then there’s tools to do that.
But I think it is really key, especially when you can shoot with the lower resolution and then project it as a window in the room and now that resolution is not scrutinized the same way because it’s not presenting itself as reality the way that 180 degree stereoscopic is. So we have a much higher scrutiny for what cameras we mix in when people might think that it was from the same camera that was capturing reality moments before in their show.
For anybody that’s watched the catalog of Apple Immersive Experiences, we’ve put out there, we’ve mixed a bunch of different cameras together, and we have lots of different ways. Even when we use the same camera in another show, we rethink for this show, what’s the right way to present this?
Should we blur the edges and stretch them so you don’t see the black edges? Sometimes we go, you know what, that’s actually distracting. Let’s make sure you see the black edges so you know it’s something different. So we try to start with intent of experience and then find the technical solution that best represents that intended experience.
When you’re talking about cameras in different places, it reminded me of that scene from The Hobbit where it’s the most stunning theatrical piece and then they cut to a GoPro shot for the floating down the river in a barrel and then back to theatrical and it was a bit of a wild ride.
All right. Van had a question. For fully immersive CG video with no physical camera, what is the best way to do this? Is it documented anywhere? And I know, Tim, we’ve got some tools coming. Probably not documented. Yeah, I did mention some future development with some tools coming. But as a general rule, render to equidistant as opposed to equirectangular.
So it’s basically a perfect fisheye. We do provide, I think it’s already in Resolve, there’s an ILPD, an equidistant ILPD, which is kind of like the default ILPD. think the way it works, I don’t know for sure, but I think the way it works is if you render something somewhere, it’s equidistant, you bring it in and put it on the timeline, it should assume that it’s equidistant and then it should just get projected correctly.
I think even today, working with tools like Maxon and Blender, if you do render a left-right EXR file, a multi-layer EXR file with an equidistant fisheye, You can, inside Resolve, inject the ILPD, and you’ll end up with a fully CG immersive. I should mention that in Fusion, I did mention in my deck earlier, but they added to the PanoMap node the ability to inject an ILPD from another source.
So even if it’s in the media pool, there’s actually an equidistant. If you download off of developer.apple.com, the actual sample project for foveation, that file that comes with that, there’s a little download link there, it has the equidistant ILPD on it. So if you were to put that in your project and just hook it up to that PanoMap node, it would inherit that ILPD, the equidistant.
All right, we’re coming towards the end. So if you do have any more questions, you know where they are. A couple more though. Jared, maybe for you, in immersive video, You’d never see lens flares if you were there in real life. So how do you handle them and how do you preserve realism?
If we remove them entirely, that could be one option, or you could keep a glow of the lights and make it feel bright. So how do you do this and potentially even clean up when you’re cleaning up, how do you keep that consistent across both eyes? - Sure.
So again, for live, you deal with the situation in real time. So if that lens flare is coming from something like a stadium light, this is where preparation is key, right? You do not want to place or angle that camera in a position that, you know, would have that lens flare in one or both eyes.
So that’s for, let’s call them static sources that will give you that issue. However, just like any live colorist or V1s we call them in the industry, no, the lighting changes from the daytime when the players are warming up through nighttime when, you know, the game finishes. So that’s all about dynamic correction of those things and making sure that your production, which cameras you’re taking, are not the affected cameras, if that makes sense.
You may have a beautiful shot for the game, let’s use, for example, MLB again, in the dugout facing the players right down the barrel of the dugout there, but for 15 minutes a day, the sunlight will come over the stadium roof, and that will be a less than ideal shot.
Now, if something great is happening there, maybe it’s worth it, but we do put things in the production rundown notes, like this camera is typically not the best during this time of day. But again, that comes with, again, preparation for it. Then in terms of removing lens flares and things live, great tools are always coming. - Yeah, from a post perspective, I can just say-- - Clean lenses.
I like the way it was phrased to say, okay, well, we don’t see lens flares in real life, but I do think that the key is when you see a lens flare, I mean, I’m under some bright lights and I see a little bit of lens flare here, But there’s accurate stereo to those things. So if you get a lens flare that’s in one eye and not the other at all, what will happen is your eyes will, your brain is trying to merge those two things and it’s seeing a total disparity.
It’s very uncomfortable and not just uncomfortable, it also actually just subconsciously draws your eye. So even if it’s not in an area that your viewer is looking, it’s gonna pull their attention, it’s gonna be distracting, and over time it can be fatiguing. So our usual approach from a VFX cleanup standpoint is if it’s in both eyes, we have a conversation of, does it really take away, do we leave it? You know, it’s budget, it’s time, there’s all the reasons that you might wanna leave it in. Sometimes it just looks cool if it hits stereo just right.
But if it’s in just one eye, usually the answer is figure out how to paint it out or mask it in some way or choose a different shot if you don’t have-- - Or clone it. - Or time, or yeah, you can match, it’s a bit of a more advanced technique, but if you can get the stereo just right and clone it from one eye to the other and get everything to match.
Again, just make sure you’re viewing in Vision Pro to make sure you, ’cause we’ve seen, it doesn’t matter how many times you’ve done it, you’re looking at your screen and you go, “Yeah, those match perfectly.” And you put it on and you go, “Oh, now it actually looks like it’s a patch that’s floating close to me when it wasn’t there before.” Because it may look fine on a 2D screen, but yeah, previewing in Vision Pro is key if you’re gonna try to paint it. That’s for removal and for cloning.
But yeah, I think the key is if you wanna keep it, it’s got to be creating a stereo effect that creates at least a realism of like, well, that lens flare was actually there because it’s something that was actually there. You’re never seeing with just one eye, not the other. Nice. And then I think we’ll have time for one or two more.
Yuri, maybe one final one for you. Apple Vision Pro, are there any plans to add headphone accommodations such as audiogram support to visionOS? There’s many folks out there who would want it. Yes, I believe Audiogram, I think it is about accessibility, VoiceOver capability. We actually do have that today for Apple Vision Pro.
So we think all of our product with, of course, accessibility in mind. So on Apple Vision Pro, you are able to turn on VoiceOver. So the content that you’re looking at, the VoiceOver capability will look at the content on behalf of you and read out what it said and what’s shown on the device.
It is a very useful feature for folks who needs it. I think that’s what the question is about And I’m very glad to say that we do have it Yeah If you click too many buttons on the Vision Pro soon discover that it exists or you can go to settings to Accessibility and turn it on All right, and maybe as an open question to folks, please dive in on the answer I guess generically this came in from Lindsay. What do you think creators are fundamentally?
about spatial and immersive storytelling right now? And what opportunities are we missing because we are still thinking in terms of traditional screens? Who wants to take a stab? - That’s a good question. It was phrased really well. - Yeah, I don’t know who wrote that, but a great way to phrase that.
Yeah, that’s probably the number one problem is anyone coming from 2D just is set in that traditional, you know, master shot, you know, coverage over the shoulder, those sorts of ideas. And really, you have to approach it as this is a new medium. I need to learn this new medium.
And I think that’s the most important thing. Yeah, for me, I think Jeff Wilson’s talk embodied that as someone who has an incredibly decorated career in producing high quality 2D documentaries. to have him break down firstly just how a lot of the hooks and the devices that he was hoping to use just didn’t work and how open and candid he was about sharing that I thought was great. So I definitely recommend if you’re up against maybe a challenging client or something, having them watch Jeff’s talk and maybe having them relearn some of the devices that they’ve relied on throughout their career would be a good place to start.
Any final thoughts before we close up? - Yeah, I think on that one I come back a lot to this idea that because it’s the sense of presence that gets talked about a lot, what stories lend themselves to being there? There are some stories where being there isn’t the best thing about being told this story.
But I think a lot about, I go to a lot of immersive theater, I go to, and thinking of like, well yeah, these stories are told this way because being there is what’s interesting about it. And so just coming from not just a standpoint of like, well, there’s a story I wanna tell, how do I make it immersive?
But actually, what stories are best told immersive? I think a lot of the third party stuff we’ve seen this year, it’s been fun to see people’s answers to that question, ’cause we come up with our answers, but there’s far more answers out there that haven’t been tapped yet.
And so my last little pitch though on that is, Don’t be afraid to try things small before you go big. So, like, if you have a big idea, especially if you have access to the camera already, try a little previs version that you shoot around your office or your house that, like, tries to encapsulate that idea and just see how you feel about it before you plan. Because, I mean, you know, a lot of folks on my team have been making immersive for a long time.
Our guess about what’s going to work and what’s not going to work, we’re wrong all the time. So the idea is figure out what you can try and fail fast. And then when you go out in the field and you’ve got, you know, wolves coming up to you that you can’t control, you know what you’re going to do because you played around with your dog at home and had the dog, you know, like interact the way you might.
So I just I think the idea of like, don’t be afraid to try small, fail fast and like do what excites you about being present in the moment. Yeah, those are, I think, good guides. Hear, hear to that. All right. Well, that’s all we’ve had time for, sadly. But thank you again to our panel for joining us up on the stage.
And most of all, thank you all in the audience for joining us too. And what a day. We’ve done a lot. We’ve looked at what’s new in visionOS 27, Immersive Media. We’ve had three incredible creators join us up on the stage. And Jared gave us a behind the scenes look at live in Apple Immersive. And honestly, the best part has been seeing all of you here, having conversations, swapping ideas and getting excited about what immersive ideas to create next.
And if you want to keep exploring, you can, of course, search or prompt for resources from our dedicated website. And you can find that at developer.apple.com forward slash Apple Immersive Video. It’s a great resource base to get started and to find resources to share. Now, for everyone here with us in the room, we will have the choice of two breakout sessions plus a demo area.
A little note on that is we will run the demo area all the way into 6:00 p.m. So if you do want to go back to back on both the hands on sessions, don’t be afraid, you’ll still be able to go up and play around, see all the tools and the microphone, of course.
We will also run the social mixer five to six in that space. Now to everyone watching online, though, sadly, this is where our broadcast will come to an end. So it might be getting late where you are. Thank you for giving us your time and for being part of the day here with us in Cupertino. So wherever you might be, good night, or maybe the sun has finally come up, in which case it is good morning.
And on behalf of everybody here at the Developer Center in Cupertino, I want to say a huge thank you to everyone for joining us, for exploring these new mediums, for sharing your excitement of these technologies. It’s been awesome to have you here, and we hope we will see you again very, very soon. See you in a wee bit. Thank you.