Good morning.

Welcome to Focus Interaction in tvOS 11. My name is Matt, I'm a software engineer working on UIKit and I'll be joined later today by my colleague Jon. We have a lot to cover today, so let's get started. First, we're going to showcase some of the new features and enhancements that we've made for Focus Interaction support in tvOS 11. These cover a number of requests that we've received from you over the past year.

Next, Focus Interaction support is now in SceneKit and so we're going to cover everything that you need to know in order to get started. And finally, we've built some great new focus debugging tools in tvOS 11 for helping you diagnose focus related issues in your app. We think these new tools are going to save you a ton of time.

But before we start it's important that we review some of the basics of how the Focus API works into tvOS 11. This information is going to be important throughout our session today. From the perspective of the focus engine your app is app is made up of focus items and focus environments. Focus items are the user interface elements that are capable of becoming focused. They conform to the UIFocusItem item protocol. Examples of focus items include UIViews in UIKit and nodes in SpriteKit.

Focus environments conform to the UIFocusEnvironment protocol. Environments are allowed to influence the focus behavior in your application and they also get notified about focus updates. All focus items are also focus environments, but these also cover other objects like view controllers. Focus items and focus environments are arranged in a hierarchy. In UIKit this hierarchy loosely follows the view and view controller hierarchies and in SpriteKit it follows the node hierarchy.

This hierarchy is important when it comes to getting notified about focus updates. A focus update can happen in one of two ways. First, a user can move focus geometrically, such as by swiping on the Siri remote or your app can trigger a programmatic focus update, such as by calling APIs like setNeedsFocusUpdate.

When a focus update happens, we notify every focus environment that contains both the previously focused item and the next focused item. And this includes all common ancestor environments as well. We notify environments in ascending order starting with the two items that are directly involved in the update. Notifications are sent by calling the didUpdateFocus method, which is defined on the UIFocusEnvironment protocol.

And this leads us straight into our first topic of today, which is that in tvOS 11 we're providing a few helpful new ways to get notified about focus updates. Let's jump back to that diagram. Like I was just saying, we notify your app using this didUpdateFocus method defined on UIFocusEnvironment. But sometimes there might be another object in your app not a focus environment that you don't want to be loosely coupled to the rest of your user interface code.

So, for this case we're introducing a new foundation notification type called UIFocusDidUpdate. And you can observe this from anywhere in your app and it will get sent every time focus updates. This notification includes all the same information that you're used to getting for a normal focus update, including a context that describes the update, as well as an animation coordinator for controlling visual feedback. You can access both of these objects using two new user info keys that we're providing. The UIFocusUpdateContext key and the UIFocusUpdate AnimiationCoordinator key.

We're also introducing another key notification type related to focus updates and that's called UIFocusMovementDidFail and this notification is sent whenever the user tries, but fails to move focus in a certain direction. This will only be sent when the user is trying to move focus not when your app fails to perform some kind of programmatic focus update.

And this is really useful for cases where you might want to provide some kind of feedback to the user when they try to move focus in a direction that they're not allowed to. Now a good example of this is on Apple TV where VoiceOver will provide a sound when a user tries to move focus in a direction that they're not allowed to.

And this is a really helpful way to provide some extra auditory feedback to visually impaired users letting them know that their action was appropriately received, but that it didn't have any effect. So that's it for focus update notifications. We're also providing a few new helpful protocol extensions on the UIFocusEnvironment and UIFocusItem protocols to help you write more succinct and safer code.

The first of these is a pretty easy one we're adding an isFocused property to the UIFocusItem protocol and this is just a simple way to check if any item is currently focused. We've exposed this property on UIView since tvOS 9, but now that we're exposing it on the protocol UIFocusItem item directly you can use this with any type of focus item and that includes SpriteKit nodes.

We've also added a new contains method to the UIFocusEnvironment protocol. This method allows you to easily check if one focus environment contains another focus environment, but without having to know the underlying types involved in the check. Let's look at an example of this. Suppose we want to check if an item that just got focused is contained by our current view controller. We can accomplish this by simply calling the contains method on the view controller itself and passing in the next focus item directly.

This will work if that item is a view, it'll work if it's a SpriteKit node, it doesn't matter what the types are involved, you can just call contains and it'll just work. And so, this will hopefully cut down on the amount of typecasting that you have to perform in your code. So that's it for new protocol extensions. For our next enhancements, I'd like to invite up my colleague Jon to talk to you about some new APIs for focus animations.

Good morning, last year we showed you the coordination API that you can use to keep your animations in sync with the system animations during a focus update. And this is done by using the UIFocusAnimationCoordinator that is provided whenever a focus update occurs. Today, I'd like to talk to you about some enhancements that we've made to this API in tvOS 11. But before I get started let's take a more in-depth look at how animations work today.

Focus animations are a really important part of the tvOS user experience. Not only do they show your users where focus has moved they help them feel more connected with your content. But as with most things, timing is essential. If the animations are too fast it creates a jarring effect for your users. And if they're too slow well they're getting in the way. So, UIKit has a lot of work put into it to manage these animations and their timing.

We do a lot of manipulations, such as duration changes, to keep the animations feeling fluid. Quicker movements through multiple focus items will result in quicker animations. And when we move slowly from one item to the next we slow down the animations to match. When a user moves focus to an item offscreen we make sure to add an appropriate delay until that item is fully visible before we perform the focusing animations.

And you might have noticed that focusing animations and unfocusing animations have a different set of timing. This is because focusing animations should be prominent and they should grab the user's attention, whereas unfocusing animations should be subtler and more in the background. So, you can see here as focus moves between these items, the item becoming focused pops up very quickly and the item becoming unfocused slowly moves into the background. In fact, it leaves a trail behind it as focus has moved from it to the next item.

So, when you call addCoordinatedAnimations to synchronize your animations we automatically determine whether these are focusing or unfocusing animations based on the focus environment hierarchy. So, here's the same diagram Matt showed earlier for the didUpdateFocus notifications. Whenever you call addCoordinatedAnimations from either the previously focused item or any of its direct ancestors up to but not including the common parent environment we consider these unfocusing animations and we coordinate them as such.

Conversely, if your animations are added from either the next focused item or any of its direct ancestors we consider these focusing animations and so they get that more prominent timing. And this works really well. But in tvOS 11 we're now allowing you to specifically target either the focusing or unfocusing animations. And we're doing this by adding two new methods to the UIFocus AnimationCoordinator, addCoordinated FocusingAnimations and addCoordinated UnfocusingAnimations.

So, addCoordinated FocusingAnimations will function exactly the same as if you call addCoordinatedAnimations from within the next focused item or any of its direct ancestors. And addCoordinated UnfocusingAnimations will function the same as addCoordinatedAnimations from within the previous focused item or its ancestors. And so, the best way to show you how this works is with a demo. So, let's dive right in.

All right, so I'm working a very simple timeline application that shows some entries from past dubdub events and their keynotes. And each entry is represented by this cell that has a number in it and as we move focus the corresponding image above moves down to give a little bit of emphasis.

As I move through these you might notice that it feels a little bit choppy as we move quickly. Well let's hop over and take a look at the code. And you'll see that everything is done within the addCoordinatedAnimations block. And this is a great use for our new API, what we have here is the focusing view and I've gotten a reference to this by using the nextFocusedIndexPath to grab the supplementary view it corresponds with and I'm simply doing a transform of 100 points down. And then similarly for the unfocusing view the image that's going back up I get it by using the previouslyFocusedIndexPath and then transforming to the identity.

So instead of using the single addCoordinatedAnimations call which in fact makes everything coordinate alongside the focusing animations, we can now split this up into two explicit calls, one for the focusing animations for the focusing view and then one for the unfocusing animations for the unfocusing view. So, now let's rerun this and take a look.

Now as we move from one item to the next you can see that the same trail as focus moves in the cells itself exists in the image views themselves and this creates a much nicer user experience for your application. All right, so now let's hope back over to the slides. And there's one other thing I'd like to talk about regarding animations.

In addition to the two new methods we're also providing an instance of the UIFocusAnimationContext and this is a new object in tvOS 11 that gives you some additional timing information about the animations occurring which should make advance animations even easier. So, let's say we'd like to run an animation during the second half of the system focusing animations.

Well this is very easy to do, all we need to do is grab the duration off of the animation context, divide it by two, and then pass these values into the standard UIView animate method and we're done. One important point to mention is the overrideInheritedDuration option that needs to get passed in because without this the nested view based animations are going to ignore the duration that you gave and instead use the outer duration.

But this doesn't just make you UIVIewAnimation easier it also provides better non-UIKit support. So, if you're using SpriteKit which is fully supported by the focus system it allows you to create a more consistent feel for your animations using SK actions for example. This is an example of using SK actions to fade the opacity to 50% using the exact same timing that the system UIKit focusing animations would.

All right, so the existing API works really well. And if you don't have a need to explicitly target either focusing or unfocusing continue using the existing API because it does the right thing for you. However, if you need to target your animations to focusing or unfocusing regardless of their association with the focus environment hierarchy use the new APIs and you'll be able to do that to great effect.

And lastly, the focus animation context is provided for some additional timing information which should make advanced animations even easier. So, next I'd like to talk about focus sounds. Where animations give a visual representation of the focus update focus sounds provide an auditory queue to the user that focus has moved.

And just like animations there's some subtlety to sound playback to create the best user experience possible. UIKit performs a number of volume and panning modifications to every sound that's played. So, faster focus movements result in a quieter sound and slower focus movements result in a louder sound, which nicely complements the same duration modifications we make to animations.

We also use the location of the focusing item onscreen to pan the volume either stereo left or stereo right. So, if you swipe right across the screen you'll notice sound starts to pan to the right and as you swipe back left you'll notice sound pans to the left bouncing out when focus is centered horizontally. Again, this helps create a much more immersive user experience.

But a lot of requested that we allow you to provide custom focus sounds or to opt out of the UIKit sounds and started in tvOS 11 I'm happy to announce that we're allowing you to do both. So, to play custom sounds you simply need to register your sound with an identifier and then provide that identifier during a focus update that will play a focus sound. This is only a user initiated focus movement. And then once we have that identifier we'll play the sound it's associated with. To register sounds you'll use a class method register soundFileURL for sound identifier which is exposed on the new UIFocusSystem, a new object in tvOS 11.

The sound registration is a global action, meaning that once you do it that custom sound is available throughout your entire application. Because of this, you should only register a sound for a single identifier. In fact, reregistering with the same sound identifier is an error. But it's perfectly acceptable and reasonable to have multiple sounds in your application, just give each one of them a unique identifier.

Because it's global you also want to register early. There's a nontrivial performance cost to setting up your custom sounds for playback and we want to make sure there's as much time between registration and user focus movement so that sounds can play seamlessly. The registration method accepts all standard iOS file formats that are locally stored on your device.

So, when you actually need to play the sound you'll use the new optional method on the UIFocusEnvironment protocol. So, we call this method soundIdentifier ForFocusUpdate. In the same ascending order that the didUpdateFocus notifications are delivered. And we start by calling this method on the next focused item. If the next focused item gives us a no focus sound identifier we use that to control sound playback for this focus update and then we're done. However, if we don't get a sound identifier back from the next focus item we look to its parent and we try again.

Now if the parent doesn't return anything we repeat this process and we go up to its parent and we repeat this over and over until we get to the root focus environment which if none of your focus environments have chosen to provide a custom sound then we simply use the default that UIKit would use on its own. So, there are a number of options for what values you might return from sound identifier for focus update.

Of course, you're able to return any of the sounds that you had previously registered and doing so will play the sound that that identifier was registered with. UIKit also provides two sound identifiers, default and none. By providing the default sound identifier it explicitly signals to UIKit that you want to force the default sound that UIKit would play on its own to be played for this focus update.

Now keep in mind if none of your environments wish to control sound for this update then this is value we'll use anyway. So, it's probably rare you'll need to use, this provide this for the cases that you do. And last, the non-identifier can be used to entirely opt out of focus sounds for this focus update.

You're able to very easily conditionally modify which sound is played and using the same contains protocol extension that Matt showed earlier we're able to determine when focus moves through a particular environment and when it does modify the sound, otherwise deferred to pair it. So, I here I have a particular environment, I'll call it sound enabled environment where when focus moves through this environment I want to play a sound that I registered with my custom sound identifier. Again, it's very easy to do, take the next focused item and then just check whether or not that's contained within the sound enabled environment.

So, a couple quick points on opting out of focus sounds. First, users expect a sound to be played when focus movement occurs. Like I mentioned before, this is an auditory queue to them that focus has moved. So, you should only be opting out of sounds if the user is explicitly expected to. A great example of this might be an in-game menu where the user can turn off sounds for your entire application.

Also, if you were think about opting out of focus sounds and then playing your own when a focus update occurs, I highly encourage you to use our API instead because not only do you get the same volume and panning adjustments that we perform for our internal sounds which are probably very hard to replicate.

We also respect the user's device settings for playing these sounds. So, Apple TV has a setting that allows you to turn off navigation sounds for the entire system and you don't want your application to be the only one playing sounds if the user says they don't want them.

All right, so to recap. You should use good judgment when determining when and where and which sounds to play for focus update because your sound should enhance the user experience they should not detract from it. A great example of this might be differentiating between elements onscreen. We do this in UIKit based on the size of particular items.

So, very large focus items have a larger, deeper sound and smaller items have a smaller, higher pitched sound. Again, this is all to enhance the user experience. All right and with that I'd like to turn things back over to Matt who will talk about support for SceneKit. Matt.

Thanks Jon. All right, I'm excited to announce that in tvOS 11 we're now extending focus system support into the SceneKit framework. So, how does this work? Well if you recall back to tvOS 9 it used to be that UIView was the only type that was able to be focused. In tvOS 10 we introduced this new UI focused item protocol and we extended focus interaction support into SpriteKit by adopting this protocol on both UIView and SKNode. Well now in tvOS 11 we're adding SCNNode to the UIFocusItem family.

So, how does this work? Just like SpriteKit you need to opt in in order to get focus support in your SceneKit apps. This means that nodes are not focusable by default. In order to make them focusable it couldn't be easier, all you have to do is set the new focus behavior property to focusable and that's it, it's that easy.

FocusBehavior is a new property on SCNNode and we've also made this property available on SKNode so now it's even easier to make your SpriteKit nodes focusable as well. But I think it would be a lot more fun to just show you this in action with a quick demo.

All right, I've been building a tic-tac-toe app for tvOS and I really wanted my app to just have an extra bit of pop and so I decided to implement it with SceneKit. And so, you can see my three-dimensional tic-tac-toe board with full perspective and I've already implement focus interaction support in my tic-tac-toe game. So, I can move focus around using any type of input device to each of the tiles on my board. And if I press Select I can drop an X or an O anywhere on the tile, on any tile.

So, implementing this couldn't have been easier, in addition to just setting up my SceneKit scene all I had to do was mark each of the nodes that represent the tiles on the board as focusable using that one line code I showed earlier. And then provide custom animations using those same animation APIs that Jon was just talking about a moment ago.

And that's really all that we had to do. Now we can fully interact with our SceneKit scene. And the focus engine automatically takes care of everything else for us. In fact, I added two UIKit buttons, UI buttons to the bottom of our SceneKit scene by just placing them on top of our SK view and the focus engine will automatically handle moving focus between our nodes and these buttons. And it'll do the correct thing no matter whether I'm on the left or right side of the board hitting the correct button based on what's geometrically closer to our nodes. And so, that's just a quick demo of SceneKit support for focus interaction.

There are just a few more things to keep in mind when implementing focus interaction support in your SceneKit apps and games. First, SceneKit obviously operates in three-dimensional space, but focus interaction operates in two-dimensional space. So how does this work? Well when a user tries to move focus in a certain direction we're going to look for the next focusable SceneKit node based on its area as it's drawn onto the screen.

Second, now that we support focus interaction in SceneKit that means we're now also able to support your focusable SpriteKit scenes that are embedded within SceneKit scenes. And in fact, we automatically support moving focus between UIKit, SpriteKit and SceneKit objects with no extra work required on your part. So, you can build your user interface using whatever tools make the most sense for different areas of your application and focus interaction support will just work automatically. And then finally, all the focus APIs that we define in UIKit using the UIFocusItem and UIFocusEnvironment protocols also work for SpriteKit and SceneKit.

And so, those custom sound and animation APIs like Jon was explaining earlier, those work for SceneKit as well and really helped to create a more immersive user experience. To learn more about working with both SceneKit and SpriteKit we encourage you to check out some of these related sessions that we're putting on this week.

Now for our grand finale today I'd like to talk about something that definitely affects all of us and that's debugging. I'm happy to say that in tvOS 11 we're introducing some great new focus debugging tools that we've built right into UIKit. And first up is live focus update logging.

By including the UIFocusLoggingEnabled option in your Xcode schemes launch arguments you can turn on live focus update logging. Now what's really helpful about this is that for each programmatic focus update that gets logged we're going to include a step-by-step outline of the preferred focus environment's search logic that the focus engine used to determine the next focus item. And so, this is incredibly helpful for diagnosing issues where focus is not programmatically updating to the item that we would expect it to.

And because these are logs this can help you diagnose issues like focus not updating to the correct item on app launch and during transitions. However, this tool does come with one warning which is that please only use this for debugging. There is a nontrivial performance cost that comes with adding logging to your application so you never want this enabled for shipping applications.

So that's focus update logging. The next tool that we're unveiling today is UIFocusDebugger which is a new class that we're exposing in tvOS 11. UIFocusDebugger is a tool for diagnosing focus related issues at runtime in your app and this is really cool. It's structured like a miniature command line tool that's built right into UIKit available for you to use. To take advantage of it all you need to do is call any of its methods from LLDB while your app is paused.

Let's take a look at some of its capabilities. The simplest one is that you can simply call status to quickly print out the currently focused item at any time. And this is often step one in diagnosing some focus related issue, especially if you can't see where the currently focused item is.

Next, you can use the simulate focus update request from environment method to diagnose issues related to your app's preferred focus logic. Calling this method is similar to calling the setNeedsFocusUpdate API and having that focus update execute on the environment that you pass in immediately. But what's great about this is it outputs that same step-by-step preferred focus environment search logic that we also include in the live focus update logging. But with this simulation method you can simulate any focus update anywhere in your application without having to actually trigger that focus update manually in your UI. And so, this helps with quickly diagnosing different preferred focus environments issues in different places of your app.

Personally, my favorite tool in UIFocusDebugger is this next method, check focus ability for item. And this is used to print out a list of potential issues that may cause an item to not be focusable. We've all had that kind of issue where there is some item on the screen that we expect to be focusable, but for some reason we just can't focus on it and we don't know why. And in fact, there are many different reasons why an item might not be focusable and they could be issues not just with that item itself, but with ancestors of that item that affect their children.

And so, this is where this new focusability for item method really shines. It will automatically detect and diagnose all of these potential issues for you and print out a list of the issues that it detects and this allows you to save a ton of time. Obviously, investigating all of these issues on your own would just be a huge waste of time.

And then finally, like any good command line tool you can always just ask UIFocusDebugger for help and this is going to print out a detailed list of instructions for each of the commands that are available and so don't worry about remembering all of these methods right now because the help command has you covered. But the best way to show you all these new tools is with a quick demo.

All right, so I'm in the process of building a new tvOS application, but I've already run into some pretty serious focus related issues so let's see if we can figure out what's going on. I'm going to build and run our application. All right, so this is just a simple app that showcases some my favorite movies and the first thing you might notice is that nothing appears focused onscreen. And in fact, if I swipe on the remote or if I try to move focus around the screen using the arrow keys nothing appears focused. So, I'm at a loss here, don't really know what's going, let's see if UIFocusDebugger can help.

We're going to go back to our application, pause it, which will be LLDB and like I was saying earlier, a good thing to check is the status of the current focus system. So, I'm going to type in UIFocusDebugger status and these calls work with both Objective-C and Swift as well.

And what status is telling me is that there is an item currently focused, it's one of those poster cells. But obviously, if we take a look it doesn't appear focused and so this points to some kind of potential issue with the visual appearance of our cells. Let's go to our storyboard to see if we can figure out what's going on.

All right, I've got my storyboard, let's open up our collection view and we have a template for our cell and the cell contains an image view. And what we want is for that image view when the cell is focused to float up above our UI. And now I'm taking a look at this I noticed that we forgot to switch on this adjust on ancestor focus option and this is what creates that floating effect and so that's probably our issue. Let's rerun our application and see if that fixes it.

Awesome, now we can move between these cells, it clearly indicates which cell is currently focused with that floating effect so we're looking in better shape. There is another issue with this allocation though, which is that I can't seem to focus on this more info button that's in this bottom shelf of our app and I don't really know what's going on there, there's nothing I can do from here. So, let's go back into our application and see if we can figure out what's going on.

We're going to go to our mainViewController, set up breakpoints, let's trigger that breakpoints. All right, so this is a great opportunity to use that new check focus ability method. I'm going to type in P-O UIFocusDebugger. What's great is that we get full autocomplete support and we're allowed to invoke this, we need a Swift code. Let's check the focusability of our more info button which I have an outlet referenced to.

All right, so it detected an issue and what this is telling us is that the item is being visually occluded, that's visually blocked by some other item on the screen. And it really helpfully even printed out a reference to the item that it thinks is causing the problem. Let's go back to our storyboard to see if we can figure out what's going on here.

All right, I'm going to go to our mainViewController. All right, so here's our more info button. So, it said that we were being visually occluded by some other item, but I can obviously see this more info button here so what's going on. Well now I'm looking at the storyboard I notice this other sibling view to our more info button and this is a semitransparent white view that we use as the background of the bottom shelf in our application. And I'm noticing that this view was accidentally placed on top of our more info button, but that was really hard to see because it's semitransparent. So, let's try dragging this just below our more info button and relaunch our application and see if that fixes the issue.

All right, let's try that again. All right, make that a little bit bigger. Okay now I can go and focus on that more info button. As our application is looking in better shape now we can start to move on to the more important work of building out our new features. Let's [inaudible] back to the presentation.

So that's just a quick example of how UIFocusDebugger helped us to identify those kinds of simple mistakes that are easy to fix, but that can nonetheless waste a ton of time during application development as we try to track them down. So that's focused debugging in tvOS 11. To recap our session for today.

Please take advantage of the new focus update notifications and the protocol extension methods that we're adding in tvOS 11 in order to accomplish common tasks with less code. You can use the new animation APIs and the custom sound APIs that Jon was talking about earlier in this session in order to provide a more immersive experience for your apps and games.

Definitely check out the new focus interaction support in SceneKit. This is a great way to add interactivity to your SceneKit games. And then finally, use the new focus debugging tools to save yourself a lot of time. For more information on today's session, you can go to developer.apple.com. If you're interested in learning more about developing for tvOS you can check out some of these related sessions that we're putting on this week. Thank you very much.