Good morning. How many of you have apps in the App Store or working on apps that play video? That's quite a few of you. Good. Whether for sports, news, entertainment or education. What's the most important part of your video playback? Is it the Play/Pause control? The scrubbing? Maybe the audio and subtitle settings? No. Of course not. It's the video presentation. So it's important when users are using your application and watching your video that they focus entirely on your video presentation. And they aren't distracted by trying to figure out your user interface.

That's why it's important that video playback be easy, consistent and predictable for viewers as they move between apps on their Apple TV. Video playback on Apple TV appears simple, but it's no easy task. There are many different tools for user interaction, including the buttons and touch surface of the Siri remote, Siri voice commands, older Apple TV remotes, the iOS remote app, Bluetooth keyboards, game controllers, and of course, infrared universal remotes.

Wouldn't it be great if you didn't have to deal with this? I'm Dan Wright and today I'll be talking about how you can accomplish all this easily with AVKit. AVKit provides modern playback with a consistent user interface. Let's take a quick look at some of the features AVKit provides.

Here is AVKit playing a video with a transport bar visible showing the elapsed time, the position in the video and the time remaining. Touching the surface of the remote reveals hints. And clicking on the edges will skip forward or back. Or by holding down the touch surface, viewers can fast forward or rewind.

And when paused, swiping side to side on the remote will scrub quickly through the entire video. Swiping down reveals info panel, which includes information about the video and navigation markers as well as access to settings related to audio and subtitles. Finally, AVKit supports Siri voice commands automatically. And as I go back to the beginning, or "What did she say?" to skip back 30 seconds and replay with captions turned on temporarily.

AVKit uses the modern media stack, the same as on iOS and macOS. AVKit provides the user experience. AVFoundation and CoreMedia provide the core playback tools. And UIKit or on macOS AppKit provides interface elements. Today we're going to talk about three things. First, we'll show you how easy it is to get started with AVKit. Second, we'll look at several of the ways you can extend the playback experience with features unique to tvOS. And third, we'll talk about some best practices. All right, let's get started.

Let's talk about AVPlayerViewController. AVPlayerViewController uses an AVPlayer, AVPlayerItem and AVAsset provided by your application. The AVAsset represents the audio visual media. AVPlayerItem represents the presentation state of an asset. The AVPlayer controls playback. And AVPlayerViewController sits on top, providing the UI. Let's look at how you'll provide your media to AVKit.

There are four steps. First, create an asset from the URL. Second, create a playerItem of the asset. Third, create a player with the playerItem. And fourth, associate the player with the playerViewController. Now, you can simplify all that to one line just to create the player directly from the URL and assign it to the player property of the playerViewController. All right. Next, let's see how you can embed an inline player view.

First, you'll set up your playerViewController. Next you're going to set the frame to the inline view. We're setting the frame directly here, but you should use constraints, of course. And third, add the view from the playerViewController to your view. And add the playerViewController itself as a child view controller of your view controller. Now, once you're ready for interactive full-screen presentation, you can just present the view controller and it will automatically animate from the inline playback to the full screen.

All right. Let's talk about extending the playback experience. AVKit for tvOS was introduced in tvOS 9, last fall, with several enhancements. Namely, the ability to add non-interactive overlays. Restrict playback interaction. Provide informational metadata. Navigation markers. And identify interstitial content. With tvOS 10, we will introduce a couple new enhancements: The ability to modify skipping behavior and presents content proposals.

And now let's talk about each of these. Overlays are for logos and other overlaid graphics. The playback overlay view lies above the video, but below the controls. Views may be static or animated. But views will not receive focus or events. Let's move on to restricting playback. The requiresLinearPlayback property of the player view controller limits user interaction when set to Play and Pause.

Things like fast forward, scrubbing, skipping, chapter navigation and so on are all forbidden when this is set. Typically you'll set it to true when you want to enforce viewing of a particular section of video, for example ads or a legal notice. The external metadata property supplements or replaces information embedded in your asset. Things like title, description, genre, media content and rating, PG-13 or R, that sort of thing, or poster artwork.

Let's go back to our screenshot that showed the info panel we saw earlier. And I want to direct your attention to this top section right here. This is where the informational metadata is displayed on tvOS. Now, in addition, it'll also be displayed in the iOS remote app. Here we see the artwork, which in this case looks like a still from the video. We have the title. We have the duration, which comes directly from the asset, a media content rating and a description.

Now, let's look at how we're going to create external metadata items. And we will create a couple of helper functions to make it a little easier. The first one called metadataItem takes an identifier and a value. And it's going to return an AVMetadataItem. And so metadataItem has a whole bunch of properties. You really need to set three, these three: The value, the identifier, and often overlooked, the extended language tag.

Now, you see here I've set it to the string und which is short for undefined and it acts like a wildcard. So if you do not have a more specific language translation available, this is the version that AVKit will display. If you don't have any translations, if everything, for example, is in English or everything's in French, then just use und so that users, regardless of their language, will see something.

Now, artwork is a little bit different. The value of the artwork metadataItem is an image representation. So it's raw data. You can use PNG or JPEG. You also need to set the data type field to identify what kind of data you're providing. And once again, you set the identifier and the extended language tag. Now we're going to bring it all together.

We're going to create several external metadata items. We create an array. Then we're going to use our helper function to add a title, a description and the poster image to that array -- oh, and sorry, also the genre. And then finally we're going to assign that array to the external metadata property of the playerItem.

All right. Back to our info panel. Now we're going to look down here at the bottom part. This is the navigation section. And this is displaying a navigation marker group. Navigation markers most often are used for chapters. But they allow for easy access to different parts of your video. Navigation marker groups are for chapters or events. Examples of events would be in a sports video you might have game highlights, for example.

A navigation marker group is defined by an object of class AVNavigationMarkersGroup. And it contains an array of navigation markers, in this case chapters. Every marker has a title. And in most cases an image as well, usually a thumbnail from the video. But that's optional. An event group itself also has a title, a chapter group does not.

All right. Another little helper function, the create navigation marker. This takes a title, a description and a time range. Now, the time range is the time range of the thing that you're navigating to. However, the duration is optional. If you don't really care to set it, you can just leave it zero. Here again, we're creating metadata items. In this case, for a title, description. And then finally we're going to create the timed metadata group with our metadata and the time range.

All right. Now let's talk about interstitials. Here we see a video with several interstitials as indicated by the dots on the transport bar. The user is watching an interstitial here. And the time above the transport bar is counting down the time remaining. The time below is the elapsed time into the video.

Interstitial content is content that is unrelated to the main video. So for example, advertisements, legal notices, other things like that. An interstitial time range identifies the portion of an asset that contains interstitial content. And interstitial time ranges collapse. The dots on the transfer bar are [inaudible] holes as we saw in the previous slide. And finally, during scrubbing, interstitial content is hidden. So the user just is navigating through the entire main video and they're not getting distracted by ads popping ups as they're scrubbing.

Now here we have a diagram comparing the timelines of your asset on the top and the transfer bar as seen by the user on the bottom. We have a nine minute asset with a couple of interstitials, both of which are going to be collapsed into dots. And you'll notice that the duration on the transfer bar has been reduced by the length, or rather the duration of those interstitials.

Now let's look at creating and declaring interstitial content. First, on your server you should stitch the interstitial content directly into your HLS asset. Then, in your client app, you will declare the interstitial time ranges. And implement a few delegate methods to enforce playback policy. Here's a short snippet showing how to declare the interstitial time ranges. For each interstitial create an AVInterstititalTimeRange. Then set the interstitialTimeRanges property of your player item to this array. In this case we're just creating a single interstitial time range.

Next, the delegate methods. There are three important ones I'm going to talk about. The first is willPresent. willPresent is called when one of your interstitials begins to play. Typically, this is where you're going to change requiresLinearPlayback to restrict navigation. So for example, if this is an advertisement, you probably need, for contractual reasons, to require the user to watch the entire interstitial once they start.

You set it to true. Then they can't skip out. They could still pause, head to the kitchen or something and come back. But they can't navigate, they can't skip over it once they're inside. didPresent is called at the end of your interstitial. And usually you set requiresLinearPlayback back to False so that the users can once again navigate.

And finally, timeToSeekAfterUserNavigatedFrom time to some target time. So this is called when the user navigates or indicates that they want to navigate to a different part of your video. For example, they skipped forward or back. They scrub and hit Play again. Or they use the navigation markers to jump to another part in your video. The oldTime is the time that they were playing at just before they skipped. The targetTime is the time that they want to watch next. The time returned by this delegate is where you want playback to resume.

So by default, if you didn't implement this method, it would resume at the target time. AVKit would seek to the target time and start playback there. If you provide the start time of an interstitial instead, AVKit will automatically redirect to that interstitial. Now, if you do redirect to an interstitial, unlike in our very simple sample here, you're going to want to save the target time. And when your interstitial is complete, seek back to the target time so that the user can once again resume at the time that they wanted to watch. Now let's turn to skipping behavior.

Here we see the skipping indicator. And this is a new skipping indicator in tvOS 10, the skip by item indicator which can replace the older indicator, which is the skip forward or back 10 seconds. In tvOS 10 you can choose between these two skipping behaviors. If your app is displaying something like a movie or a TV show, it's probably most useful for users to be able to navigate skip 10 seconds back and forward at a time to more easily find the scene that they're looking for.

If you have a series of short videos, for example exercise videos, it probably makes more sense to let users easily skip to the next video, the next exercise, or the previous video, previous exercise, then to move around a little bit within an exercise. So, to accommodate that, you can change the skipping behavior. There are two skipping behaviors defined today. The default, which is to skip plus or minus 10 seconds, and skip item. And there are three properties in the PlayerViewController related to this: The skipping behavior and properties to enable and disable the skip forward and skip backward hint indicators.

So let's look at skipping by item instead of skipping a few seconds. First thing we're going to do is we're going to set the three properties. Select the new skipping behavior. In this case we're just going to always enable skip forward and skip back. I'm being a little lazy.

And then we're going to implement two delegate methods. Skip to next item, all we're going to do is replace the current item with the new video. And skip to previous item. Same thing, but going backwards. Okay. Now for a demo of all these features, I'd like to welcome Jonathan Long to the stage. Jonathan.

Good morning. My name is Jonathan Long. I work with Dan on AVKit for tvOS. And today, we are going to be demoing an app called AVKit Player. I have the project open here in Xcode. And this is the main view controller. It is the subclass of UIViewController.

And we have a reference to an AVPlayerViewController. We obtain this reference in the prepare for segue callback. In addition, we also create an AVAsset, an AVPlayerItem and an AVPlayer, which we assign to the PlayerViewController's player property. I'll go ahead and build and run this so we can see what it looks like.

So here's our playerViewController in full screen with all of the standard user interactions. If I swipe down to reveal the info panel, you can see that there's not a whole lot of interesting information here. Most notably we are missing the info tab. So the first thing we're going to do is we are going to add some external metadata and some navigation markers so that info tab shows up. Jumping back into Xcode, the first thing that we're going to do is we are going to add some external metadata. To do this we need to create some AVMetadataItems, add them to an array and assign this array to the playerItemExternalMetadata property.

I'll go ahead and add some helper functions here. These probably look very familiar, as they are the same ones that Dan showed you in slides earlier. The first creates and AVMetadataItem for artwork, setting the data type to be JPEG. The second creates an AVMetadataItem for all other value types, setting the extendedLanguageTag to be undefined.

Now, I have one more helper function here that takes a dictionary as its argument. And it simply maps the key value pairs of this dictionary to create AVMetadataItems and returns an array of AVMetadataItems. So now with these helper functions, we can set the playerItem.externalMetadata property to be our array of AVMetadataItems. Cool. So we created AVMetadataItems, added them to an array, assigned that array to the playerItem.externalMetadata property.

So now let's move on and add some navigation markers. I have some helper functions here for that. The first creates an AVTimedMetadataGroup with a title, a description and a time range. The second creates an AVNavigationMarkersGroup with a name and four time metadata groups that represent different time ranges in our content.

So now with these functions, I can set the navigationMarkerGroups property on the playerItem to be an array of AVNavigationMarkerGroups. We'll name this one Additional Group. Cool. So we created an AVNavigationMarkersGroup with some time metadata groups with a name. We added that to an array. And then we assigned the array to the navigationMarkerGroups property on the playerItem. So, let's build and run and see what happened.

So here is our Player View Controller again. And I can swipe down to see the Info tab. And now we see in the metadata space all the metadata that we've specified as well as the artwork for this title. And if I select down at the bottom, one of the navigation markers that we've added, we can skip to that point in the content. So with just a little bit of code it's easy to add external metadata and navigation markers to provide a rich user experience in your info panel. Now I'll hand it back over to Dan to talk about new APIs in AVKit for tvOS. Dan.

Thank you, Jonathan. All right. Content proposals. Content proposals are about suggesting what to watch next. You may have seen Up Next suggestions in some of your favorite apps such as Netflix, Hulu, HBO. When you reach the end of each episode of, say, "The West Wing" you'll see a suggestion pop up to watch the next episode and so on and so on until it's 3:00 in the morning. Many apps implemented their own version of this for tvOS 9. And then it was kind of tricky. No more. With tvOS 10 AVKit provides a standard way to present your own customized Up Next experience. We call these Content Proposals. Here is one example

Thank you. Now, you have the entire screen to work with. So you're not limited to anything here. This is just an example. In the lower right we have some metadata and some big buttons that you can press. Up here we have the currently playing video. And here is the thumbnail representing the proposed video.

Now, a content proposal is represented by an object of class AVContentProposal. This is basically a model object. Now, there are several properties on here. I'm not going to spend a lot of time on all of them. But I want to highlight a few important ones. The first is contentTimeForTransition. This is the time within the currently playing video that you want your proposal to be presented to the user.

So, if you specify the duration of your video or zero as a shortcut, that means that you want it to appear at the very end of the video. However, if you're presenting a TV show or a movie or something like that, you probably have a bunch of end credits that the user may not want to sit through.

And you may want to present your proposal at the beginning of those end credits, in which case you can set this property accordingly. Next automaticAcceptanceInterval allows you to specify a timeout for your presentation. You can say, for example, 30 seconds. Which means that if the user does absolutely nothing, they take no action for 30 seconds, then the proposal will be automatically accepted and advanced.

If you don't set this, then your proposal will just stay onscreen until the user takes some sort of action. Title and preview image would be the bare minimum bit of information that you should provide to a user. You can, of course, provide as much additional information as you like: Description, maybe a little widget to set a user rating, maybe additional suggestions for alternate videos. And finally, the way you specify your content proposal is to set the next contentProposal property on the current playerItem. So let's look at creating the content proposal.

Well, this is pretty easy. We just create an instance of AVContentProposal. We're going to specify a time of zero for our contentTimeForTransition. This means we want it to show up at the very end. This is a nice shortcut if you don't have the duration yet. For example, if your playerItem hasn't loaded.

And we're going to set the title and a preview image. In this case, our proposal is for Episode 2 of Season 9 of a fictional TV show called "Happy Hijinks." In this case we're going set one of the optional properties of the content proposal, a URL. And we're going to set it to the URL for this episode, for our own use later. Now we assign the proposal to the playerItem, which should show the proposal. Naturally, this is the playerItem for Season 9, Episode 1.

And, once again, we've got three interesting delegate methods. shouldPresent is called immediately before your proposal is to be presented to the user. Now, you could take this opportunity to return False and prevent the proposal from being presented. But you can also use it as a last minute opportunity to set up to prepare for your presentation. And we'll show an example of that in just a minute.

didAccept is called when the user accepts your proposal. That means they want to watch the video that you've suggested. It will also be called if the timeout occurs. And didReject is called if the user has indicated that they want out of your proposal. They want out of the video. They want to get back to your menu.

Now finally, you will implement your presentation by creating a custom subclass of AVContentProposalViewController. There are a few interesting things in the subclass. Let's look at two, preferredPlayerViewFrame is the frame onscreen where you want the video to appear during your presentation. Now, the base class simply returns the frame of the entire screen, in which case the video will continue to fill the screen. And your presentation controls will be overlaid on top. However, you can specify a smaller rectangle, maybe off to the side in a corner or something, to give yourself lots of space, the rest of the screen to work with for your controls.

And dismissContentProposal is how content proposals are dismissed. You can specify one of three actions: Accept, Reject and Defer. Defer means the user simply wants to hide your presentation, get back to watching the end credits so they can, I don't know, maybe they want to spot their name in the end credits if they worked on the video. And then as the name implies, Defer, it will reappear at the end of the video.

Now let's look at responding to those delegate notifications. shouldPresent. In this case, we're going to create an instance of our custom contentProposalViewController subclass and assign it to the contentProposalViewController property of the playerViewController and return true. This lets us set that up at the very last minute when we know we're going to actually need it and not allocate a bunch of memory that the user may never get to.

didAccept. Here all we're going to do is we're going to replace the current playerItem with a proposed content playerItem. Pretty straightforward. We've seen this code a few times already. You'll notice here I'm making use of the URL property of the proposal that I set when I created the contentProposal.

Now, if this is literally all you're going to do here, you don't need to implement this. Because AVKit, if you provide a URL on your proposal, AVKit will automatically do this if you don't implement the delegate method. All right, and now for a demo with content proposals. Jonathan.

Hello, again. All right. So, let's continue with AVKitPlayer and add an Up Next experience by subclassing AVContentProposalViewController. So the first thing we need to do is we need to create a custom subclass of AVContentProposalViewController. I have one right here that I will go ahead and add to our project.

So, this is our UpNextContentViewController. It is a subclass of AVContentProposalViewController. And we have some UI properties here such as a UIImageView, a UILabel and others. We override the preferredPlayerViewFrame to return the CGRect that we want our playerViewController's view to animate to when the contentProposal is presented. Now, the rest of the viewController simply handles the layout of its view and subviews. So we'll jump back into our main viewController. So there's two more things that we need to do. First, we need to create an instance of AVContentProposal and assign it to the playerItem's next content proposal property.

So, here we are creating an instance of AVContentProposal. We have a transition time, a title and a preview image. We're also setting the automaticAcceptanceInterval to be 15 seconds. In addition, we set the URL and the metadata on this content proposal that corresponds to the next item that will be played if the user chooses to accept this proposal. Finally, we set the contentProposal we just created as the next contentProposal property on our AVPlayerItem.

So the next thing we need to do, and the last thing we need to do, is implement two playerVIewController delegate methods. You can see here that we set the playerViewController delegate to Self. And the two methods that we need to implement are shouldPresentProposal and didAccept. So in shouldPresentProposal we create an instance of our custom subclass of AVContentProposalViewController and we assign that as the playerViewController's contentProposalViewController. In addition we return true to indicate that the contentProposal should be presented.

And finally, in didAccept, we need to handle transitioning our player to the next playerItem. So, to do this, we get the URL from the contentProposal. We create an AVPlayerItem from that URL. And then we simply replace the current item on our playerItem. And we're done. So let's build and run and see what happens. So I'll go ahead and skip closer to the end of the content.

And here is our contentProposal. As you can see, the metadata that we specified is here. The preview image as well as the title. And if I select Play Next, we would transition to our next playerItem. So I think if you have implemented this on tvOS 9 you'll be very happy with this API and you will find that it's much easier to use. I'd like to invite Dan back up to talk about some best practices of the AVKit.

Okay. Best practices. I'm going to talk about a few things. Things to do and things to avoid doing. First, some things that we recommend that you do. Once again, present or present view controler will handle zooming automatically from an inline view. So there's no reason to implement your own animation.

Second, remember Playback is only interactive when full screen. Third, use the new content proposal API. Some of you wrote it the hard way before. And we encourage you to switch over to doing it the new way. I think you'll find your code is a lot simpler. It's easy to understand the flow. And you may avoid some bugs and other things that you may have had trying to get it to work just perfectly before.

And observe the error property of the player in playerItem. This is how you find out about errors that occur during playback. And I'm going to talk about one particular error in a minute that you should handle. But you should watch for any errors that occur and respond appropriately to the user.

All right. Now a few things to avoid. Avoid toggling showsPlaybackControls. This property is not for temporarily showing or hiding the controls. This is for indicating your permanent intent. So when you set this to false, you're saying we don't want the playback controls at all. So if you toggle this back and forth, what you're doing is you're telling AVKit to destroy all the controls and recreate them. Destroy them. Recreate them. And it's not very efficient, as you can imagine.

Second, some people hate this, but avoid adding supplemental gestures to playback because it's going to confuse users. People will not discover them. And by the same token, do not overload the Select button or touch service gestures. Not only will that confuse users, but you'll likely break in the future.

Now, some more general tips. Replace your asset if you see this error: AVErrorMediaServicesWereReset. If you don't respond to this and the media services are reset during playback, video playback will fail. And the user won't be able to do anything except, hopefully, get out. So what they'll have to do is get out, go back to your menu and go back in. If you catch this error and automatically replace your asset, your player item and your player, basically your AV foundation objects, then you can more seamlessly handle this case. And the user will barely even know that anything happened.

Now, some other sessions with best practices for playback include this year, Advances in AVFoundation Playback. It was earlier this week. You can catch the video in the WWDC app. And two years ago we had an excellent session, Mastering Modern Media Playback, which includes tips for both AVKit and AVFoundation and getting them to work well together.

So in summary, AVKit provides standard playback controls and behaviors, support for remotes, game controllers and Siri, full access to the media stack, powerful new APIs and it's easy to get started. Be sure to check out the sample code for more in-depth examples. And we're excited to see your app.

So, for more information, go to the URL on the screen. We'll have links to the sample code and other resources. And check out some of the related sessions. We also have on here the HTTP Live Streaming session also earlier this week. And a pair of sessions on using TVMLKit, which is an alternative approach to using AVKit directly. So, thank you.