Hi, everyone. My name is Neil Desai, and I'm a WatchKit Frameworks Engineer, and I'm really excited to talk about the life of a watchOS app today. So, watch apps have a phenomenal capacity to be simple yet incredibly powerful. Well-designed watch apps have the potential to be the ultimate companion on your user's wrist.

Now, let's imagine your user for a second. What are they asking for when they want to use your watch app? Maybe they want to quickly glance at some information, or complete a simple task. We think about great watch apps that are designed to help your user accomplish their goals in a simple, quick way so that the technology disappears and the user can go about their day and have an enhanced experience, all because of your app.

So, today we're going to design an app that I've been working on in watchOS 3. We're not ready to ship it just yet, but we'll get to take advantage of some of the new capabilities and enhancements in watchOS 4 to take our app to the next level. Today, we're going to talk through four different additions in watchOS 4 that can help design our app.

We're going to talk about some new things, such as our new Unified Process Runtime. We're going to get some great performance improvements in our app, and I'm going to show you how to do so in watchOS 4. We're going to talk through our new Frontmost App State. We're going to take advantage of this state and build an amazing experience for our user.

Also, we're going to talk about some new improvements we've made to Background App Refresh, and how we can take advantage of them. And lastly, we're going to talk about some new background modes that are available in watchOS 4, and take advantage of one of them in our app.

So, let's get started and dive right into our new Unified Process Runtime. So, we've designed a single-process architecture that will help your apps become even faster and more responsive to your user. So, let's now just revisit some of the history of where we came from and where we're now at, today.

So in watchOS 3, your apps UI lived in the app process, which we managed for you, and your code lived in the watch extension, so we had two separate processes, in this case, that lived on the system. So, we changed things internally and unified our process runtime, so now the UI and your extension code all get executed from the same process. And now, you might be wondering, why exactly did we do this? So ultimately, we wanted to improve the performance of all WatchKit apps on watchOS 4, so users could have an even better experience with your apps. And, I'm really happy to say, we've accomplished our goals.

We can see touch latency improvements of almost up to twice as fast in some cases. So, from the moment a user actually touches the screen, to when your code gets called into, we've seen a dramatic improvement. Also, for example, we've seen dramatic frames-per-second improvements when tied to general pan gestures, and we expect launch performance to also improve.

And now, since we just have one process to manage your UI and your extension code, we now have upped the memory limits to account for this. And, you might be wondering, there's a lot of architecture that we did internally, so what do you all have to do to change your apps to adopt this new mechanism? And, the great thing is that there are actually no changes required at all. So, you'll be happy to hear that all existing native WatchKit extensions in the App Store today will get this behavior as soon as your user upgrades to watchOS 4.

This new Unified Process Runtime is completely backwards compatible. As long as your app is built with watchOS 2 or later, you don't have to anything else. So, this means the app we're going to improve today doesn't need to change in Xcode, or on the App Store. It just works.

So now, let's talk about our new Frontmost App State. On the watch, a user many times uses an app in a matter of seconds, and then they put their wrist down and expect the app to finish up whatever task needs to be finished. Many times, a user wants to raise their wrist again and see your app has finished doing whatever it needs to, and they want it to feel like magic.

And, users shouldn't have to keep their wrist up, for your app to have to finish doing whatever task it needs to finish, and this is a really common pain point for our users. So, we thought a lot about all the different ways we could give you all more runtime to help finish your tasks for your user, and today, we're introducing a collection of new capabilities we call the Frontmost App State to help you design experiences for this case.

So, without further ado, let's talk about our app. So, the app is called Apple Pie Me, and let's see how it looks. So, it's a food delivery service where you can get apple pies delivered wherever you are, so an apple pie a day keeps the doctor away, right? I think so, at least, I really hope so, because I eat an apple pie every single day, and they're delicious. And, I'm pretty busy, and I always just want to quickly order my apple pie, and get it delivered wherever I am.

And, most importantly, I want to tap the order button and then simply put my wrist down and go about my day. In watchOS 3, it was a bit difficult to build a perfect experience, but with watchOS 4, I'm going to show you how to surprise and delight your users using our new Frontmost App State. So, to help explain the Frontmost App State, we're going to talk through what exactly application state means on the watch, what does the term frontmost mean, the enhanced capabilities that you're going to receive, and lastly, we're going to walk through architecting our Apple Pie Me app.

So, let's say our user decides to launch our app. So here, the app is foreground, the screen is on, and the application state, in this case, is active. So now, let's say our user decided to go back to the watch face. So, in this case, the app is actually in the background and, for instance, your application might be running for a complication update, for example.

And now, let's say our user decides to go back to our app. So, we're active in foreground again, and now the user's using the app, but then they decide they want to put their wrist down. So, the screen turns off, and your app is no longer foreground because of this. However, since your app was the last thing used, it's the frontmost app, so when the user raises their wrist again, your app will be what they see, instead of the watch face.

During this time, we think of your app as the frontmost app. So now, when your app is frontmost in this way, the application state will be background, and you might be wondering how exactly, or how long does the app stay frontmost for? So, it's going to be two minutes for all apps.

However, if there is an expectation by your user to see your app next when they raise their wrist, then you can use a new API to extend this timeout to be eight minutes. So, for example, if you're a ride-sharing app in the middle of a ride, then you might want additional time in the frontmost state, because there's a reasonable expectation that the user wants to see your app the next time they raise their wrist.

However, in many cases, if you forget to turn this off when it's not needed, you can unwittingly annoy your user and leave them with a subpar experience. So, always just make sure to turn it off, to reset the time back to two minutes when you no longer need this experience.

So, let's jump into code to see how we might enable the extended timeout. So, it's really simple. It's a Boolean property on our WKExtension, and when we want eight minutes, we can set it to be true, and when we want to go back to the default of two minutes, we just set it to be false.

So, there's one other point I wanted to mention. If the user raises their wrist and views your app, then the timeout is reset, regardless if, if you use the frontmost extended API or not. Essentially, if the user is still interested in using your app, your app will stay frontmost.

So, now we know what frontmost means, but what does this all really mean? Like, what exactly does this new state give us? Well, we're going to get some enhanced capabilities. So, we're going to change the notion of how we get our WatchConnectivity resumes, and our NSURLSession resumes. So, our transfer and connectivity is now improved. We'll also change a little bit about our task completions.

And then, we're also going to allow haptics to be played in this frontmost state. And lastly, we're going to get a frontmost notification, so we can handle this via the user notifications framework, and choose the right experience. So, let's dive right in. So, for WatchConnectivity, we've changed the way our background transfers work, so when you use on the phone, update application context, transfer user info, or transfer file, the request, when you send it from the phone to the watch, it'll just go right through, wake up your app on the watch, and then just deliver your payload.

And, there's one point I wanted to mention in relation to WatchConnectivity, is that send message won't work when you're frontmost, but the screen's off, because your application state is background. So, just always make sure that call is reachable first, and then that can allow you to send the message.

And, along the same vein as WatchConnectivity, we're also going to change the notion of our resumes for NSURLSession. So, when we have a background download, for instance, and the system's downloading it for us, and then when the system is finished, it's going to automatically wake up our app, if we're in the frontmost state, and deliver the data right to our application.

And, of course, with NSURLSession, there is actually two major parts. There is the resume, when the system has actually completed downloading whatever it is you need, but then there's also the initiation of a download in the background. So, in most cases, this will occur immediately, also, and in very rare cases, it could get delayed up to 10 minutes, but it's a very rare situation.

If you've ever used NSURLSession, which probably almost everyone has here, for a background transfer, this is a really big deal on the frontmost state. To reliably depend on the initiation and the resume of our connectivity is really going to change the design landscape for the watch, and also, in addition, with our WatchConnectivity and getting requests right from our phone.

So, another thing is, a lot of times, a user might put their wrist down, and in that moment, your application goes into the background, but you might need to finish up a task. So, say you need to close a database, or do something else. So, you can just use the NSProcessInfo class to perform expiring activity API to get up to 30 seconds when foreground initiated, and up to 10 seconds when background initiated. So, for example, for background initiated, maybe you're running in a complication for an update, and you just need a little extra time to get done. So, you can just use the NSProcessInfo class to get some additional time.

And now, when you're frontmost, and you use that API, then you'll get increased priority by the system. So, because the system believes that your app is more important to the user, because it's still in the frontmost state, the priority by the system will get increased, so you're more likely to get the runtime that you need to finish up whatever task you need to do for your user. And, another benefit of being frontmost is, you can now play haptics, so when you have runtime for another reason, you can easily just play a haptic. Thank you.

So, sometimes haptics can get a little confusing here or there, so let me just dive in and talk to you about, so if haptics, if you play a haptic, and let's say some audio is playing out a speaker, then only the audio from a haptic will play, and it ducks what's currently playing. If audio is playing out of Bluetooth headphones and the screen is on, then the haptics will play, and if the screen is off, then just the audio will play, because that makes the most sense for your user.

And so lastly, the other benefit of being frontmost is, when you receive a remote or local notification via the user notifications framework, you'll then get called into, and then you can decide the right experience for your user. So, maybe you want to just update your UI, and play the notification like normally, you can have that choice, now. And so, you'll get called into, when you just override the function userNotification willPresent notification withCompletionHandler.

And, the key of all these benefits, and this can really change the design landscape, but the best part about all this is it's free, so if your app gets a WatchConnectivity or NSURLSession resume, your app will now get additional runtime. If your app implements the willPresent notification callback, you'll now get that for free in the frontmost state. Also, if, for example, you want to use a NSProcessInfo performExpiringActivity API to complete a task on applicationWillEnterBackground, you'll get some additional priority by the system.

You can change nothing in watchOS 4, and your current, native application will automatically be more glanceable and actionable by default. And, with the Unified Process Runtime we talked about earlier, your application will be even more responsive. But, we want you all to take it a step further. Your apps can now leap forward in functionality if you take these new behaviors into account. You need to architect your apps for this experience. So now, let's talk through the Apple Pie Me app, and get started architecting our app using these new capabilities. So, let's revisit some of the design goals for the app from the perspective of our user.

So, here we are at a timeline of our user, and really, the user just wants to order apple pie, and then they want to eat apple pie, and sometimes in between, they just want to view the status of their order. So, it's a pretty simple app from the user's perspective. But now, let's break down the problem from our app's perspective.

So, when that Order button is pressed, then we're going to send that order to our server, and eventually, when the restaurant has received the request properly, and we know that the pie is cooking, we're going to send a push notification. And then, we also want to tell our user when a courier is on the way, and when a courier is outside. So, let's break this down some more, and take it step by step.

So, we're at the Receive Order, and how exactly do we do this? Once the user taps the Order button, what exactly should we show? We're going to be sending the order to our server here, and can't guarantee it'll come back immediately, no matter what. And, do we show something like this, maybe a Loading indicator? So, sometimes Loading indicators do make sense to your user, but in this case, it's actually pretty confusing. I might see this, and I might think, "Oh, I need to hold my wrist up.

How long do I have to hold my wrist up?" This isn't really a great experience on the watch. But instead, we could actually show something like this. So, hey, we'll tap you when the pie's ready. So, the user knows they can just drop their wrist and go about their day.

So, now we know at the Receive Order screen, we know we just want to show this new UI immediately, so now let's break this down into its separate components. So, we want to use NSURLSession and post the order to our server, and we also want to extend the timeout, and then just update our UI.

And lastly, there are times when our server might not be responsive, or something could have gone wrong, and we need to handle that case. So, we need a mechanism that allows us to schedule something around, let's say, five minutes from now. And, we also need something that doesn't depend on the network, or something that, regardless of frontmost or not, should alert our user.

We also need something that we can cancel later on, and so, if you're thinking about a local notification, then you'd be absolutely right. So, we could just show a local notification that says, "Oh no! Something went wrong." And then, the user can easily just tap on the Reorder button and get another apple pie, or they could just tap on the App icon and go into the app and see what went wrong. So, let's just add that last step, and now, let's just jump into some code and see how this can be done.

So, here we are at our IBAction for when the Order button is actually pressed, and then we just want to grab our background session, and then we're going to want to post the order to our server. And then, because there's a reasonable expectation that the user wants to see our app the next time, because they just pressed the Order button, then we're just going to extend our frontmost timeout. And then, we're going to trigger that just-in-case scenario, our fallback local notification, and then we're just going to want to reload some new UI.

Great. So now, we've received the order, and eventually we're going to send a push notification for when the pie is cooking. So now, let's say our user puts their wrist down sometime after we've received the order. So, right before we send that notification, we expect to be in the Frontmost App State, but really, we, the developer, we don't really need to know too much about when the user puts their wrist down, or raises it up.

The next event that's really just important is our push notification, to just alert our user that their pie is cooking. If the app is frontmost, we'll get our willPresent notification callback we talked about earlier. Otherwise, if we're not frontmost, the user just gets a notification like normally. And so, let's jump back into some code and see what we can do.

So, here we're just going to override our user notification center, willPresent notification withCompletionHandler. And then, the first thing we want to do is, we just want to make sure that we cancel our local notification. And then, we just want to reload our new UI, and then, because we did that, we want to just tap our user, so they know to raise their wrist. And lastly, we want to call the completionHandler and finish up. And, I wanted to point out for the completionHandler, it actually takes a UNNotification PresentationOptions, and so that's the options of alert, badge, or sound.

And, in this particular case, we actually don't want a notification to appear on top of our app, because we've already tapped the user, and we've updated our UI, so instead, we're just calling completionHandler with none of the options, so essentially we're just going to be consuming the notification, and then, because the tap comes in, the user would raise their wrist.

So, we just tap the user, and they might see something such as the following. So, great, their pie is coming. It's about 12 minutes away. It's cooking in the oven. So, here's where we were, and now let's go back to our timeline. So, now we know our pie's cooking at the restaurant.

And now, let's say we want the user to be able to raise their wrist and see as up to date of an ETA as possible. So, as soon as the user raises their wrist, we will, of course, ask for the most up-to-date ETA, and this means that the next time the user raises their wrist again, we'd expect them to see that up-to-date ETA.

Great. So, we handled our push notification. We now have a mechanism for updating our ETAs, and now we just want to send a push notification again for when the courier is on the way. And, if we receive this push in willPresent, then we're just going to tap our user again, and then maybe we might want to add a map and an updated ETA showing either where we are, or where the courier is.

And, let's say, after we sent a push notification for our courier is on the way, the user decides to maybe raise their wrist. So now, the user would see the map and where their courier is, and the app would provide a great, glanceable experience. And next, we can just send another push for when our courier is outside.

And then, if we're still in the frontmost state, we can easily just play another haptic like before, and update our UI to show that the courier is right outside. And, at this point, let's just make sure to disable our frontmost timeout extended API, since we expect our user to be eating apple pie now, instead of actually just using our app. So, let's take a look at our overall timeline.

So, here's our whole flow, and it all looks great, but let's try to challenge our assumptions as much as possible. Now, what happens if the user decides to go back to the watch face? So, we're no longer frontmost in that case, but does that really change anything? Not really, right? Let's take it through our app code again.

So, we use the background NSURLSession to post the order to our server, so if, right afterwards, the user decided to go back to the watch face, we know that that will continue, and then properly post the order to our server. And then, we just send a push notification for all the different stages. So, even if we're not frontmost, the user just gets a regular notification.

And so, here we are at our overall timeline. So, we've handled the cases when our server wasn't working properly with the local notification, and we've worked through how to use some of the new Frontmost App State to provide a great experience to our user. And, the best part about all of this is that our app works great with or without the Frontmost App State.

And so, we've now gone through the Frontmost App State, and our Unified Process Runtime, and how it affects our app. So, let's just now take some time to revisit Background App Refresh, which was introduced last year, and see what improvements we've added, and how we can apply it to Apple Pie Me. So, let's just go through a brief overview. So, in watchOS, the system wakes your application by handing it a task, and the system has a limited number of these to hand out, so just make sure to make the best use of each one you get.

And, when the system wants to wake your app, it gives you one or more of these tasks, and then you can do any work you'd like in the background, and you just make sure to hold this task for as long as you're doing work. And, the way in which the system will deliver these tasks to you is when you implement your handle background tasks callback on your WKExtension delegate.

And so, when you finish your background work, you just return the task to the system by calling setTaskCompleted. So, we've changed a little bit of the API, so let's walk through that now, now that we have a good, conceptual overview. So, before, we would implement our handle background tasks, and then we would switch case on all of the different tasks. So, in this example, we have a WKApplicationRefresh BackgroundTask, and then, let's say we perform whatever functions we need to, to get our app up to date, and then we just want to schedule a snapshot.

And so, we use the scheduleSnapshotRefresh, and we use a preferred date. And then, we just call setTaskCompleted. And, the important thing here that was a little tricky was, it was really important to call setTaskCompleted after you actually scheduled a snapshot refresh, because if you called it before, then, like I mentioned, you're actually, by calling setTaskCompleted, you're returning the task to the system, and no snapshot would then be scheduled. So, this was a little bit cumbersome, and not the best experience.

So now, all you have to do is just call a new function that says setTaskCompletedWithSnapshot, and you set it to be true when you want that snapshot, and set it to be false when you don't. And so, for all tasks, the default is just to take a snapshot immediately. However, for snapshot, just make sure to remember that there's a special call to setTaskCompleted with a snapshot. So, let's take a look at that.

So, here it is, again, we're implementing our handle background task, and this time, we just have a WKSnapshotRefreshBackgroundTask, so we're just going to call setTaskCompleted with a restored default state, and then our estimatedSnapshotExpiration. So, this is really great, and it's very simple to use, but I actually want to point out one cool trick.

So now, you can actually use setTaskCompletedWithSnapshot on your snapshotTask. So, what that does is it will, of course, take a snapshot immediately, and then it'll actually schedule a snapshot an hour from now, if you set it to be true. So, essentially what this means, if you just keep calling setTaskCompletedWithSnapshot as true, you're going to set up a one-hour cadence, and it's really easy to constantly make sure that your app is up to date every hour.

And, if you want more custom data behavior, for example, then just make sure to call setTaskCompletedWithSnapshot false after scheduling your snapshot refresh, because if you call it with true in this case, then we're actually going to replace your previously scheduled snapshot, and just replace it with one an hour from now. So, if you do want that custom behavior, just make sure to set it to be false.

So, another important thing for our handle background tasks is how we actually update our complications. So, in watchOS 3, we actually soft deprecated on our CLKComplicationDataSource, our getNextRequestedUpdateDate, our requestedUpdateDidBegin, and our requestedUpdateBudgetExhausted. So now, in watchOS 3, I should say, if you implemented handle background tasks, these just wouldn't get called into, and now they're just fully deprecated, so they won't get called into at all.

So, if you want to update your complications, again, just make sure to handle background tasks, and then just call scheduleBackgroundRefresh with whatever date you want to update your complication. And so, last year, a lot of times complications can be tied to some sort of networking task, and so last year, we guided you all to essentially, basically just schedule a background refresh, and then that way you would schedule it maybe for, let's say, 15 minutes from now, if you wanted a complication update by 7:45. So, essentially you would schedule that background refresh, and then you would get woken up around 7:30 to get that NSURLSession, and then you could trigger that, and then by 7:45, you would have enough time to update your complication.

But now, with some new NSURLSession APIs, we can actually just reduce a step here, and the reason we had these steps was really because we wanted to ensure we gave enough background running time to our NSURLSession. But now, with some new API in URLSessionTask, we can actually just set an earliest begin date.

So, when we actually call the download task, we can actually just set on that task our variable, earliestBeginDate, so that way, we're telling the system, "Hey, we don't need this right now, we just need this at whenever date we've specified." And so now, we can actually just reduce that step.

So, here in our schedule background refresh, we probably also just want to kick off our URLSession task, and set it up with the earliest begin date, and so that way the system, on our behalf, will actually just trigger that NSURLSession around 7:30 p.m., and then when that resume comes back, we can update our complication at 7:45 p.m.

So now, let's talk about how we can actually incorporate some of the Background App Refresh things we just talked about into our app, Apple Pie Me. So, here's that timeline we looked at earlier. So now, let's just focus in on the Pie's Cooking notification. So, if you remember, the code, before, it looked a lot like this. We implemented userNotificationCenter, willPresent notification with CompletionHandler, and then we just cancelled those fallback local notifications. We updated our UI. We then played a haptic, and called our completionHandler.

And now, the great thing is, once you call that completionHandler, your app will get suspended, and you will get called into via handle background tasks with a snapshot task, and then your reason for that snapshot would be app backgrounded. So now, when you have an ETA you can share, you can just make sure your snapshot looks as up to date as possible in the doc.

And then, another thing I wanted to point out from before was, if you remember, we actually used, when the user raised their wrist, we then kicked off an NSURLSession to then grab the most up-to-date ETA. And so now, with that new API we just talked about on our URLSessionTask, we can actually just even schedule an ETA for some time from now. So, let's say, for example, maybe our pie's coming in 12 minutes, and we just want to schedule an ETA, like a refresh, around, let's say, like, eight minutes from now, for example.

So, let's think about our overall timeline once again. This was a really simple app, actually. It was just, we had an Order button, and then we were able to eat pie. And then, but, it was actually really powerful under the surface in our app code, and so we were able to take advantage of our NSURLSession resumes, our haptics in conjunction with our frontmost notifications.

We're also taking advantage of our Background App Refresh, as well as we were able to schedule NSURLSession ETA requests. So, creating a great watch app that delivered you apple pies was certainly possible before, but now with watchOS 4, you can take a huge leap forward and wow your users.

So now, let's dive into some of our new background modes. So, in watchOS 3, we introduced our workout session as a background mode, and now we actually have audio recording sessions, and then we also have navigation sessions. So, let's dive right into our audio recording session. So, in watchOS 3, you could record audio, but it would, essentially the API would present a modal presentation that would show our UI, and then the user could then stop the recording, and then that file would get delivered to you all, the developer. However, now you can actually just use your own UI, so you can build the exact recording experience you want in watchOS4. And, the important thing is, now it can be background running but, of course, it's foreground initiated and background running, just like our workouts.

And then, your app will stay frontmost while it's recording, and what's great about that is, you actually have the ability to play haptics. So, let's say, for example, you're an app that wants to detect what sort of song is playing around you. You could easily just press the Record button. The user would put their wrist down. You could detect what's playing, and then you could easily tap your user, so they know that, hey, you detected what song it was, and then you could show that to your user.

And then, what's also great is that, just like workout sessions, it's a background running mode, but the heart rate sensor is not going to be on, so there's actually less battery life impact because of this. And, another thing I wanted to mention is, if for some reason the user decides to go back to the watch face, for instance, we're going to show that Microphone icon at the very top, so that way the user knows that something's recording, and then they can also tap on that icon and get launched right back to the app.

So, for playback, in watchOS3.1, we introduced AVAudioPlayer, and so now, for recording, all you need to use is AVAudioInputNode via AVAudioEngine, or AVAudioRecorder, and then our AVAudio recording for permissions. And, there's a whole bunch of formats that are now supported. And so, let's talk about some of our locations background mode. So, just like audio recording and our workout sessions, it's foreground initiated, and then just background running. And, of course, you're going to be frontmost while in session.

So, one thing to note is, you must call startUpdatingLocation while your app is in the foreground, and you should also just make sure to set allowsBackgroundLocationUpdates to true only when the navigation session has actually begun, and then set it to be false when it's not. So, if you recall, for Apple Pie Me app, we actually had two users.

We had myself, the person that was about to eat that awesome apple pie, and then we actually had a courier that was going to bring it to us. So, in this case, we might actually just want to build a companion driver app for that driver, so that way they can easily be navigated on their wrist to our destination.

And, the great thing is, you can even play haptics while in session, so now if, for example, for a right turn or a left turn, you could play a different haptic, or maybe when a user has missed a turn, for example. And, I'm really excited about this one, because, like, for example, transit apps that, you could imagine, you want to, just right before the stop you're about to get off at, the user could be easily tapped, and then they could know, oh, I need to go to the doors, and so I can get off at the next stop. So, we're really excited about our new Unified Process Runtime, our new enhancements to Background App Refresh, and our new background modes, and I especially am really excited about the new Frontmost App State. And, we think the design landscape will change in a really unique way on watchOS.

So, I challenge you all to design apps for the new Frontmost App State. Think through the enhanced capabilities, and design experiences where the interactions are just two seconds. And so, we went through how to use NSURLSession, haptics, and frontmost notifications, but that only really scratches the surface. With the enhancements to watch connectivity and task completions, you could design even more powerful interactions that are incredibly simple to use. And so, I'm really excited to see how you all take advantage of these new capabilities.

So, if you want more information about this session or any other session, feel free to go to developer.apple.com, and then we have some great related sessions. There's a great one tomorrow for planning a great Apple Watch experience. I highly recommend checking it out. I'm super excited for that one. And, I hope you all have a great WWDC. Thank you.