Hi, everyone. My name is Neil Desai, and I'm a watchOS Frameworks engineer, and I'm really excited to be here to talk about how to create some great audio apps for watchOS. And honestly, we've been waiting a little over a year to give this talk, so I'm really happy to be here, so let's just jump right in. In watchOS 4, we unveiled a revamped music and radio app. And in watchOS 5, we unveiled a brand-new podcast app. And we're finding that users absolutely love being able to play audio on the go or in the middle of a workout.

And we're also finding that users really like to be able to remote control some playback that's occurring on another device. For example, your iPhone or HomePod. And I especially really like this feature. I love just being able to play some music out of my iPhone and then being able to control the volume, for instance, directly with my watch.

That's a great feature. And today, with watchOS 5, we're going to show you how to build a great experience like you saw on the keynote like Audible was able to make and just like Pandora was able to make. And so they were able to use some of the new tools available in watchOS 5. And today, we're going to talk about how we can do the same sort of thing. And to do that, we're going to talk about four major themes.

We're going to talk about our native controls and how we can now embed our system controls directly within our application. We're going to talk about, how do we get content over to the watch? And then, once we have that content, how do we actually play something back for our user? And lastly, we're going to go over the whole audio experience and how to build a really rich experience for our users that integrates well throughout the entire system. So let's talk about our native controls.

In watchOS 5, we're unveiling two new controls that you can embed directly within your application. And the first is a Now Playing view which you can embed within your watchOS app. And this is really great for content that may not be yours, for instance. And you can also embed a volume control so you can build your own custom UI to allow your user to control the volume.

And you might be wondering for that first view, that Now Playing view, when exactly would I ever want to allow my user to control some audio, but the content's not exactly mine? Well, let me give you an example. We do this ourselves in the workout app. So I'm in the middle of a HIIT workout. I swipe to the left.

I get my power song, "Eye of the Tiger," and I'm a little bit of a dork, so I actually do listen to this song sometimes when I work out. I love it. And this is a great experience. You just swipe over, and you can allow your user to control the playback. And so if you're building a workout app, and you can do this, build the same sort of experience, just embedding the Now Playing view directly within your application.

Let's talk about the characteristics of that Now Playing view. And the first thing I want to note is that the Digital Crown controls the volume. And because of that, you're going to want to place this view in a non-scrolling controller. So for instance, it's, in my app here, I've actually just pushed directly to a controller that has the Now Playing view in it. Or you could reload directly to a page that has this controller or place it in a page interface similar to the workout app.

And this view is going to automatically switch the sources on behalf. So no matter if it's the iPhone or the watch that's providing the source of audio playback, this view's going to take care of it for you. You don't have to worry about that. And lastly, the Now Playing view, it's going to grab your application's tint color so it fits within the look and feel of your application.

Now, to use this, let's see how it looks in Interface Builder. Here we are in Xcode, and I've selected my Object Library, and then I can just drag and drop my Now Playing view into a controller. Once my user navigates to that view, everything will be done for you. The system will take care of it all. And it really is just that simple.

So that's our Now Playing view. Now, let's talk about our volume control. So the volume control can be set up to either control the iPhone's volume or the local watch's volume. And just like our Now Playing view, we're going to grab your application's tint color and apply it appropriately to this particular control.

And in the case of when the user is actually scrolling with the Digital Crown, then, in that case, the volume control will be tinted to the system green color to match the rest of the system. So for the volume control, to actually decide when or not to control the local or iPhone's volume, there, in the Attributes Inspector is a check box for Controls Local Volume. So if you leave it unchecked, then the volume control will control the iPhone's content.

And if you check it off, then the volume control will control the watch's content. And so if your application might have multiple scenarios where it could either remote control some content or play some content locally, then, in that case, you might have different controls with different volume controls within them, and you make those decisions at design time.

All right, so those are really our system controls, so now let's talk about, how do we actually get content over to the device itself? So I've been building a museum guided tour type application. I want to allow my user to just tap in, listen to a guided tour while they're in the museum, and it's a great experience on the watch because I can just put my phone away. I can just have the entire experience on my wrist and enjoy the beautiful art around me.

And let's say I had a, like a buffer that allowed my user to download the content on, when they're on their way to the museum. In that case, I would want to use URLSession and grab that directly from my server, all my content, and download it to the watch itself. So let's jump into some code and see how we could do that.

So here we are. I have an IBAction which is tied to my WK Interface button. And the first thing you'll notice is I set my frontmost timeout extended property to be true. And we're going to talk a little bit more about frontmost later, but the thing I wanted to mention is that the user has made a user interaction, so it's fair to say that the user, next time they raise their wrist, would want to see our application. And because we're extending the frontmost timeout, then we're getting some additional priority by the system for our URLSessions. And again, I'll talk a little bit more about that later.

And the user can, of course, put their wrist down at any point, so I want to make sure to use a backgroundSession. So that way, when my application goes into the background, my download is still occurring. And then, I can just download that task and appropriately resume it.

And when you're building an experience like this, you want to indicate download progress and, of course, handle any errors. So let's look at how to do that. So here we are. I have my class, my DownloadManager, which is my SessionTaskDelegate and my URLSessionDownload Delegate. And in the first function, I can actually grab the total bytes written and divide it by the total bytes expected to write.

So that way, I can get the progress of my particular download whenever URLSession calls me and gives me that information. And when I do this, I can just update progress for my user in my UI. And then, when the download completes, I can, of course, manage the file, and I might want to post a notification alerting my user that there's a playback opportunity.

And, of course, I'm also going to be called into a didCompleteWithError, so I just want to clean up appropriately and handle any errors at that point, if there are any. And that's wonderful, URLSession, for when we can download directly to the watch, but sometimes the content might already exist on our iPhone. In that case, we can just use watch connectivity and transfer the file directly over to our watch.

And to do so, we'll just use the transfer file API. And new in watchOS 5, you can get the progress of that transfer. So let's take a look. Here we are in code, and I see my WCSession FileTransfer. And new, there's a progress property that's available in watchOS 5 and iOS 12.

And then, I can just get my fileTransfer, query for the progress, and update my view on iOS. And if you want to know some more about watch connectivity. There's some great sample code that's available online. Highly recommend you check it out, the simple watch connectivity sample code. And you might be wondering -- there's URLSession, there's watch connectivity -- which one do we use, exactly? And when does it make sense? Well, for URLSession, we want to use that when it's user initiated on the watch itself. And then, when we do that, we want a query the waits for connectivity property to know that we have a network connection at that point.

And I just wanted to mention that if the iPhone is nearby, then requests are proxied through the iPhone when it's in range. And if you want to know some of the nuances of URLSession, there's a great talk from last year that's available online, Advances in Networking, Part 2. I definitely recommend checking it out.

And then, on the flip side, there's watch connectivity, and that's really important when your user flow might be when it's initiated on the iPhone itself or in the case where you've already downloaded some content and there's no need to request it from your server again. And for both and especially for watch connectivity, you're going to want to set expectations for your user. Make sure to instruct them of when the download, or the transfer's most likely to complete. And that's going to be when the watch is on the magnetic charger.

And for example, we do this ourselves. In the Apple Watch app in the Music pane, at the top, we actually show our user how many songs have actually been transferred over to the watch itself, and we even instruct our user, hey, music's going to sync when the Apple Watch is on its charger, so keep that in mind.

All right, so we talked about getting content, and now that we have it, let's talk about actually playing something back. And on watchOS, we have a bunch of different tools we can use. And some of those audio tools are in WatchKit itself. So there's a presentMedia PlayerController with options, completion that you can use. And there's also WKAudioFile QueuePlayer. And so you can give it a list of items, and then the queue player will appropriately play back that content.

So the presentMedia PlayerController option will present the system UI. So the controls are taken care of for you. There's not really much you have to do. And this is perfect for when it's short-form content and you don't really want to have to think too much about it. And WKAudioFIleQueuePlayer is great for when you have more long-form content that you want to play in the background. And this is more of a conveyer belt type approach where you have some items, you just want to hand it over to the system, and the system will play it on your behalf. And that's exactly what WKAudioFileQueuePlayer is for. And there's also AVFoundation at our disposal.

So there's AVAudioPlayer and AVAudioEngine. And prior to watchOS 5, it was tied to the workout background mode, or when your application was foreground and screen on. For example, Nike Run Club app actually did this exact thing, where you're in the middle of a run, you want some mindfulness, you can play some guided meditation with Headspace.

And prior to watchOS 5, background playback was, with AVFoundation, was really only possible for these types of workout apps. But new in watchOS 5, there's a brand-new audio playback background mode, which I'm really thrilled for. So let's talk about what we have now at our disposal. Kind of broadened our tool kit.

And so we have AVFoundation, and it doesn't have to be tied to any other background mode, and we can now use just AVFoundation directly in the background. We also are exposing the ability to allow you to, allow your user to pick a different route. So maybe some different types of Bluetooth headphones.

And new in watchOS 5, we're also exposing the MediaPlayer. framework, so you can provide some Now Playing information and also handle any media remote commands. And just like our own Music, Podcasts, and Radio app, the long-form content is restricted to Bluetooth routes, which really is just a fancy way of saying you can't play it through the speaker.

So let's take it step by step and talk about how we actually get this background long-form content to play. So the first thing, you add audio to your info.plist background modes. And then, you just appropriately set up your session, and you're going to want to tell the system that you want to play this long-form content in the background.

You want to allow your user to select a different route. And then, you're just going to want to call play and appropriately handle the playback when the user's in the middle of that playback session. So let's break that more, break that down more a little bit. And so in Xcode in the Project Settings, in the Capabilities tab, you just flip the switch for Background Mode, you check off Audio, and you're good to go to the next step.

And so to set up your session, you're going to want to set your routeSharingPolicy to longform on your AVAudioSession. And then, you call the new activate withOptions completion method on AVAudioSession. And then, once it completes, you're going to call play. And in between when you first call the activate function and then when the completion comes back, what we call the route picker is going to get displayed on your behalf. And so this is what the route picker looks like. And so this is going to be showing up within your application, which is going to allow your user to pick whichever Bluetooth headphones route they want to have and control their playback.

So let's talk a little bit more about that route picker. Again, all it is, you just call your asynchronous activate function, which is going to call and show your route picker, and then it's going to come back into your completion, and you handle that, and you just call play. And you're going to want to do this every time you want playback to occur. So that way, you can ensure that you have appropriate route to cause some playback to occur.

All right, so let's jump into code and set up our session. And so we have our AVAudioSession, and then we're going to set our category to playback. And then, we're going to set our mode to default. And then, most importantly, we're going to set our route sharing policy to longform. And then, you want to activate your session, which is going to display the route picker, and then you're going to call play appropriately.

And now, for that route picker, I wanted to call out that we really think of routes in two different ways. And the first way is our Apple wireless chip-based routes, which are commonly referred to as our W1 routes. So these are our AirPods or Beats Studio Wireless. And then, we just have our normal Bluetooth headphone routes. And it's important to make that distinction when we talk about the route picker itself.

And so if the user has an active connected route, then the route picker's going to automatically select that route on your behalf. What's great is that the route picker might not even display to your user. You're just going to get called back in your completion immediately. And in the case where the user has an active W1 route but it's being used by the iPhone, then the system's going to grab that route from iPhone and bring it directly to your watch.

And there's some cases where that won't happen. It's really just when your iPhone has more priority. So for example, say I'm in the middle of a phone call on my phone. Then, we're not going to take that route and put it on your watch because phone call's probably pretty important.

And if there are no active routes, then the route picker will just appear on your behalf. All right, so that's our route picker, and now let's talk about, how do we actually get some playback to occur? Well, you just use the same AVAudioPlayer and AVAudioEngine. And there's a variety of different formats that are supported on watchOS.

And what's great about this is if you have iOS code that uses these classes, then you could have a shared framework between your iOS and watchOS playback operations. And in the case of AVAudioEngine, you can actually play some DRM content in conjunction with AVAudioPlayer node. And so you can now play your own DRM content, decrypt it yourself, and play it back for your user.

An especially important thing to note for watchOS is the power of the device itself, so only play audio when absolutely necessary, when the user actually wants the audio to occur. And for AVAudioEngine, the autoShutdownEnabled property is enabled by default, so if there are no active nodes on your AVAudioEngine, then we're going to automatically power down your AVAudioEngine when necessary.

And so that's our playback. And then, we, let's talk about our MediaPlayer. framework. So we want to, of course, provide our Now Playing information to the system. And then, the Now Playing UI will update, and that's the system-wide notion of what's playing, and it's going to be updated based on the information you provide.

So this is going to show up in the Now Playing app, and even inside apps such as Workout, or in the cases of any third-party apps that embed that Now Playing view. This will also show up there. And then, with the MediaPlayer. framework, we just want to handle the events, any type of media remote commands -- for example, a Next Track on AirPods, for instance. All right, let's jump into some code and see how we can provide that Now Playing information.

So here we are. We have our MPNowPlaying InfoCenter. We just want to grab that. And then, we set up our nowPlayingInfo. And then, we have, we want to set our artwork, our MPMedia ItemArtwork. And it's not currently available in seed 1, but it's coming in seed 2. And in this case, the artwork that you provide will display on the Siri watch face.

And then, we just set up, in this case, our title, album, artwork, and then we set it to our nowPlaying InfoCenter. And then, when we do something such as this, then our data will appear in the Now Playing app. And this is the actual Now Playing app. And I provided some, a song that I was currently listening to in my application.

And when we do this, then, also, the Now Playing complication will also get updated. So in this case, the bottom shows the "Lights in the Sky" song that I was listening to. And then, for media remote commands, again, just use the MediaPlayer. framework. And now, in watchOS, you can handle those commands however you wish.

So let's look at some code. So we just grab our MPRemoteCommandCenter. And then, instead of doing a Next Track, let's talk about how we can actually skip forward for our user. So I have this function. It's a enableSkipForwardCommand and, just by default, takes in 15 seconds. And I set my preferred intervals on my skip forward command. And then, I just add a target and selector, and then I just enable the command within my remote command center.

If you want to know more about the media player framework and especially these media remote commands, I highly recommend checking out the MP remote command sample that's available online. And in this case, when I actually set my preferred intervals, then the system's Now Playing app will appropriately get adjusted. That Now Playing UI, wherever it is, will show the minus 15 seconds or plus 15 seconds, and that's all done on your behalf. All right, so that's the audio experience on watch, or that's our playback, so let's talk about our audio experience.

So there's 4 major things I want to talk about first for the audio experience. There's our auto-launching feature where your app will automatically get launched. There's the frontmost app state and how we can take advantage of that in our application. There's our new interactive notifications. And how to integrate with Siri Shortcuts.

And so we've been talking a lot about playback occurring on the watch itself, but I want to take a step back, and I just wanted to talk through that remote control case. So let's say I was playing some Apple Music on my iPhone, and automatically the Now Playing app will get displayed on my watch.

And I love this feature. It feels like magic when I just look at my wrist and I can easily control my playback. I, it's the coolest thing. And in the case where it's our application that's actually causing the playback to occur on iPhone, then our app automatically gets launched on the watch.

And we call this the auto-launch audio apps feature. And so whenever the Now Playing session is occurring on iPhone, then our Apple Watch app will be brought frontmost at that time. And we're going to stay frontmost for the duration of that session. So as long as the user is listening to content on iPhone, then our app will stay frontmost, unless the user decides to navigate away.

And new in watchOS 5 is we're exposing a Now Playing session API where you're going to know when you're launched for this Now Playing session on iPhone, and so you can take your user directly to the appropriate view. And to do this, you're going to use the handleRemoteNow PlayingActivity on your WKExtension Delegate. So let's take a look at some code and see how we could use this.

So here we are. I have my extension delegate, and then I can implement my handleRemoteNow PlayingActivity. And the first thing I want to check is, where is my user at that time? So I can just grab the visible interface controller, I can check where my user is, and if the user already is at my Now Playing UI, that's great. Let's leave them right there. But in the case they're not, we can just reload directly to our view. So that way, we have that same magical experience where I raise my wrists and I see the remote controls that I want to use.

And in some cases, you might want to opt out of this feature if it doesn't make sense for your user. So just do the right thing. Make that decision. And let's say our app has nothing to do with our phone app, and that's an appropriate use of actually opting out of this capability. And so to do so, use the auto, the opt out of auto-launch feature on your WK extension's info.plist. And in the case you do opt out, then the Now Playing app will show instead.

And now, let's talk about that frontmost app state, and let's give a little bit of a refresher of that. So here I have my app. It's foreground running. I'm looking at it. The screen is on, and this app is considered active and frontmost. But then, let's say I put my wrist down. In that case, the application is still considered frontmost because if I raise my wrist again, I'll see that application. And in that case, the application is just in the background.

So with the frontmost app state, we get some enhanced capabilities from the system. Our background transfers from iPhones -- so say we were transferring our content from iPhone to watch itself, those background transfers, when they complete, the resumes will directly just wake up our app if we're in the frontmost app state. And the same thing in, with our URL session resumes. So if the URL session, our background session before was downloading and it completes, then our app will automatically get waked up and deliver the payload.

And if we're frontmost, we're going to get called into, so we can decide whenever, if a notification appears, how we properly want to handle that notification. Maybe we want to keep our user in the app, or maybe we want to show the notification to our user. And in the frontmost app state, we can make that decision. And when you're in the frontmost app state, you can play some haptics.

And if you want to know a little bit more about frontmost app state, there's the talk from last year, The Life of a watchOS App. It's a great talk. You might recognize the speaker. He's okay. And so you're going to stay frontmost for 2 minutes, but, if you recall, I extended my frontmost earlier. And so in that case, it's just going to be extended by default to 8 minutes. And so, again, this is great for when we were downloading that content from before, and so I can just extend my time out.

And when we're actually playing back some audio, our app will be kept frontmost for the duration of that session, just like when we were actually remote controlling the audio when it was occurring on iPhone. And again, if the user ever navigates away, then you're no longer frontmost, but your background playback will still occur appropriately. And in that case, you can, of course, properly handle all of your background events. All right.

So now, we're -- and when we are in that frontmost state, then we also can integrate properly with notification. So we could post, for example, when the download completes, a content available local notification. And we might want to have play as our primary action. And with our new interaction notifications on watchOS, we can allow our user to actually configure some playback right from within the notification and then be able to call play as the primary action, then get into your app and immediately cause some playback to occur.

So for example, say I had a Salvador Dali exhibit. It's ready to go. I can then adjust the playback speed, immediately call play, that'll launch my app, and then I can play that great content at the appropriate rate for my user. If you want to know more about some interactive notifications, check out the What's New in watchOS talk given earlier, which is available online. It was a great talk, and you'll know a lot more about interactive notifications after watching it.

All right, so do you remember this code from before where I actually posted that notification alerting our user of this type of playback opportunity? Well, let's say our app was frontmost at that time. In that case, when I got delivered the notification from frontmost in user notification, I'm going to get called into and will present notification with completion handler.

And in that case, I can actually decide whether or not I want to still post that notification. Or maybe because my app's frontmost, I could just play a haptic and let my user know that, hey, raise your wrist, and there's some new content. I can update my view. And again, if you want to know some more about the frontmost notifications, check out the Life of a watchOS App.

Great, so that's how we used our interactive notifications and the frontmost app state, but we also want to show up on the Siri watch face. And with audio apps, you can provide suggestions of audio to play on the Siri watch face using relevant shortcuts. So for example, that could be the next episode of your user's favorite podcast or the new music that you just synced over to the watch.

For example, Audible was able to build the same experience where they can just get their user directly within their application from the Siri watch face and resume the playback of an audiobook. And whenever a user interaction is occurring, just wanted to highlight, think through your donations. Make sure to donate your INMedia PlaybackIntent to the system so that way the system knows what your user is commonly doing on the watch. And that way, you can become more relevant on the Siri watch face. And also, just think through your shortcut phrases like design appropriate phrases that your user could use within Siri to get your user directly within your application to allow for some playback to occur.

And all in all, on the Siri watch face, design a great experience. You might have some glanceable information like, hey, there's an exhibit coming soon. Or you could have some tappable actions like Audible was able to make where, hey, we just want to play that audio tour and get our user directly in the app.

And so for shortcuts, donate your INMediaPlaybackIntent. Use the relevant shortcut API to appear on the Siri watch face. And think through your shortcut phrases. And there's been some great talks this week about shortcuts, and I definitely wanted to highlight the Shortcuts on the Siri Watch Face talk. I saw that myself. It was a great talk, and I learned a lot.

And so in summary, for the first time, you can embed your native controls directly from within your application. You can allow background playback to occur however you want and handle media remote commands however you want. And there are better ways that transfer audio assets to your watch and convey that type of progress to your users. So if you want some more information, feel free to go online or join us in the lab this afternoon. I'm really excited to see what type of audio apps are built on watchOS, and thanks, everyone. Have a great WWDC. Thanks.