Good afternoon, and welcome everybody to our session, Best Practices and What's New in User Notifications. I'm Kritharth Jain and a junior on the iOS notifications team and I'll be joined by my colleague, Teja, for this session today. So, today we're going to start by doing an overview of notifications and all the different APIs that you as app developers can use for doing notifications for your applications.

Then, we're going to jump into four different use cases for notifications. We're going to start out by introducing a new concept for iOS notifications about hidden notification content which we think you're going to be really excited about. Then we're going to jump into our second use case about modifying push notification content before delivery to the user. Then we'll talk about customizing rich notifications and how you can completely own the UI and feel of these notifications. And lastly we'll talk about user input customization associated with these rich notifications.

So, let's start with the notifications overview. Now, I'm sure you all know that there are two types of notifications. There are local notifications. These are the ones that are scheduled by your application on the device itself and then these get presented to the user. And when the user handles these notifications, your app has the opportunity to handle its response.

The other kind of notifications are remote notifications that are associated with your remote server which then sends a notification payload to the Apple push notification service. Now, if your payload has content, then this notification gets delivered to the user's device and presented to the user. And again, if the user handles this notification, you can handle this response in your application. The other kind of remote notifications are silent notifications. Now, these don't get presented to your application users but these allow your application to be launched in the background and do any extra work that your app wants to do associated with your remote site application.

Now, it's important for remote notifications to use the latest HTTPS2, authentication method, wherever you're doing remote notifications. Now, notification content exists, consists of different properties that you can set on the notification. These include the title, the subtitle, the body as well as media attachments that you can set on your notifications.

Notification triggers determine when your notification gets delivered to your app users. Now, all remote notifications are associated with the push trigger and for your local notifications, you have the option of using time interval, calendar or location-based triggers. Now, these notifications get presented to your user in different ways.

In iOS 11, we've combined the notification list into a single list. So, if the user wakes their device or pulls down from the top, they get to go to the same notification list. And when the user is in that state, the notification gets added to the list right there and then. And if you want to see your older notifications, then you can just slide up on this list to see your notification history.

If the user's device is unlocked if they're in the home screen or in any other app, then the notification rolls down as a banner from the top. And if your application is in the foreground, you do have the option of presenting the notification banner right there and then. And along with these different presentation models, you'll also have the option of having a sound alert as well as badging your application icon.

Now, notification actions are what make your notifications really interactable. And your user gets to them by 3D touching on the notification where they see the list of all these actions as well as the dismiss action. And then when the user takes any of these actions, then again you get the response in your application to handle this action as you see fit.

Now, there are different mechanisms that you can use with these notifications. Notification update allows you to modify notifications that you might have already scheduled or published to the user. In this example, you can see that we're aggregating the number of likes that your social media posts might have gotten.

And after a certain time, say this count changes, then you can just go ahead and update this notification in place. So, your notification list for the user is not cluttered with redundant information. In the same vein, you also have the option of removing notifications. In this example, there's a message about the video that was sent to you. And once you've seen that video, you can just go ahead and remove this notification locally by your applications.

Now, with iOS 10, we introduced extensions to your notifications and there are two types of extensions. Namely, the first one is service extensions. Now, what a service extension allows your application to do is to intercept a notification before it gets delivered to the user. So, you can modify the notification content or do other background stuff that might be associated with this notification right in the service extension. We will take a look at service extensions in depth in one of our use cases today.

The other extension points are content extensions which allow you to customize which notifications. And again, is one of our use cases of the talk today. Some examples of content extensions from first-party applications include the messages app which gives you the full thread right there in the notifications so you can reply and get responses right from the notification without having to go to the application.

The calendar app shows the full schedule as well as gives you the option of accepting or declining a request. And the photos application sends you the photo within the notification itself and the ability to write or comment on it. And again, we'll look at how you can use all these different tools for creating these notification styles for all your applications in this session.

Now, watchOS notifications work in sync with iOS notifications. All notifications that are sent to the user's phone also get delivered to the user's watch if the phone is locked at that time. Now, if the user's watch is not in range of the phone, then you also have the ability of scheduling notifications right to your watchOS apps as well.

So, how do access all these different features for notifications? For this we introduced two new frameworks last year, namely User Notifications and User Notifications UI. We went in depth covering all those different APIs at last year's WWDC and we highly encourage you to go check out these two sessions from last year which contains much more information about everything we've covered in the overview. All right, let's move on and jump into our use cases, starting with the first one, about hidden notification content. And to tell you all about this, let me invite Teja up here.

Hi, everyone. My name is Teja Kondapalli and I'm also an engineer on the notifications team. You just heard a great recap of the API that we announced last year and now I'm here to talk to you about the API we want to announce this year. This year we decided to focus on a feature that we call hidden notification content which is a way for your users to hide the private information in their notifications when it doesn't need to be seen.

Why do we care about this? It's because your users care about their privacy. We all know notifications are very visible and the content in them is richer than ever, which means we have to be careful about protecting the sensitive information that may be in them. And actually, you all had a taste of this in iOS 10. A message notification on the lock screen could look like this. If you had gone into your settings under notifications, set for messages that you only want previews to be shown when unlocked.

So, now you can see that the message, the content of the message is obscured, and that's because the user is not authenticated. Once they've authenticated, the content of the message is revealed. And I'm really excited to announce to you that this year we've extended the support of this to all apps, including third-party apps.

We've also given your users a global setting. They can simply go into settings notifications and right at the top they can select whether they want their previews to be shown always when unlocked or never. And this should apply to all their notifications for all their apps. We've also given them a way to customize these settings per app. So again, under settings they can simply go to a specific application and choose the privacy setting for the previews for that application. And they might want to fine tune these because some applications may show more sensitive information in their notifications than others.

And now, because it's up to your users to determine whether a notification's content is hidden or shown at a certain point, we have given you as app developers an API to customize what a hidden notification might look like. So, let's take a look at this application I've written that sends me some notifications. And you can see here, this notification has some rich information in it. If my user has previews turned off, this is what my notification would look like.

And as you can see, all of the content is hidden, the title, the subtitle and the body. And the body is replaced with a placeholder. Our default placeholder says notification. But if I had a bunch of notifications associated with this app, then you can see here clearly that I have two different threads going on. One about work and the other about a Thai vacation. If my user has previews turned off, this is what my notification list could look like. But in order to prevent that, we went one step further and we have automatically coalesced these for you by thread identifier.

So, if you're already using thread identifiers in your notifications, this will work automatically for you. Let's take a look at one of my threads. You can see here it says two notifications of the default placeholder. And this thread is actually about my Thai vacation. And these are comments. So, I want to customize this further and we can do that.

When I'm setting up my notification category that I'm going to associate with these notifications, I can pass it a new, a value for a new property. So, here I'm setting up a comment category and I am also setting the hidden previews body placeholder. I'm just passing in a regular string here and it says comment. And now my notification looks like this. Which is still not quite right because I know that there's two notifications that have been coalesced and this just says comment. So, instead I decide to pass in a format string.

Which is great. Now my notification looks as intended. It says 2 comments. But this is not going to work for the singular case. It would've said 1 comments if I had just one notification. Luckily there's already a way to account for the singular-plural problem that's already supported by foundation, and that's a strings dictionary.

A strings dictionary is a p-list file that sits in your project. And you can, and it looks somewhat like this. You can see here that the first thing I have is a line that says comment key. And this indicates to a system where to find this pluralization rule.

And below that you can see that I have a key value mapping for the singular and plural rules. It says 1 with a format string that says comment and other, with a format string that says comments. And it's important to use a strings dictionary because not all languages have a distinction just between singular and plural. For example, a strings dictionary in Arabic looks like this.

And you can see there are a lot more options than just singular and plural. So, to account for pluralization as well as localization, it's important to use strings dictionaries. So, now that I've set up my strings dictionary, when I'm setting up my comment category, instead of passing in just a format string, I'm passing in an NS localized string. And the first parameter that I pass to it is comment key which is what I associated my pluralization rule with, the key that I associated with. And the second parameter that I'm passing is just a note about what this string is about.

If you want to know more about localization, we had a chat earlier during WWDC, so you can catch that on video. And we also have great resources in our docs. And now, my notification correctly says 2 comments and I have confidence that it's going to say 1 comment if there had only been 1 comment in here.

But if we look further, actually Jane sent me an image. She didn't send me a comment, and I want to indicate that. So, I can do that as well. When I'm setting up my categories, after I set up my comment category, I also set up an image category the same way and I associate Jane's notification with the image category and John's notification with a comment category. Now I have two notifications with two different categories but they're being sent to the same thread. So, the coalescing will automatically work for me and it correctly says 1 image and 1 comment.

But if you recall further, this is all one thread and maybe I want to distinguish this thread from another thread that is also for the notifications in this application. And I can do that. So if you see here, the title says Thai vacation and that's indicative of what thread this is.

So, maybe I want to show that. And again, when I'm setting up my categories, I can pass in one or both of two new options that we have. Hidden previews show title and hidden previews shows subtitle. So, here I'm just trying to pass in, I'm just passing in the title. Because showing the subtitle is revealing too much information. I just want to show the title.

And now my notification correctly says Thai vacation, which indicates the thread, 1 image and 1 comment. Now that we know how to customize hidden notification content, it's important to remember that this is a user-defined setting. It's up to your users at any point to determine whether a notification content is going to be hidden or shown.

We've given you a way to find the user's setting and you can retrieve it by using the show preview setting property in view and notification settings. It's also important to use thread identifiers to take advantage of the automatic coalescing that we already have supported for you. And it's important to use strings dictionaries to account for pluralization as well as localization.

So, we've looked at our first use case of hidden notification content. And the second use case is something I know a lot of you will be doing, which is sending push notifications to your applications. And we can do some interesting things with push notifications if only we're able to modify them, such as attachments or encryption.

And what I mean by that is imagine I want to send some media with my remote notification. But that media doesn't fit in the push payload. Well, I can instead send some metadata about this media such as the URL that it can be found at and then on the device I can download that media and then present the notification.

Another great example is if your server side wants to encrypt a content in the notification, you would want to decrypt it on the device before displaying it to the user. And another great application is a pending context-specific information to your notification such as location or time or health data. This information may only be present on the user's device. So, before it's presented to the user, you can append these to your notification. And to do all this, we would a service extension, which is something we introduced last year.

So, let's deep dive and look at an example. I want to send a video with my remote notification. So, my remote server would send a push payload to APNS. And in the payload, I packaged some metadata about my video. The notification will then get delivered to my device and the service extension has the opportunity to intercept this notification. At that point, I can talk to the remote server and download a video preview before attaching it to my notification and displaying it to the user. Let's see how we do this in code.

The remote payload would look something like this. It's important to set the mutable content flag. This is what indicates to the device that it needs to launch the service extension before presenting the notification. And I also have some additional metadata such as the type of this media, the video URL, the location that it was shot at and also the user that uploaded this.

Now, when setting up the service extension, I simply go into Xcode, create a new target and select the notification service extension template. And I would get a template that looks something like this. So, in the service extension, when a notification gets delivered, you'll receive a call to the did receive request with content handler function. And the first thing I do in there is store both the content handler and the remote content that was sent. Just in case I'm not able to complete my operation, this is my backup. So, I've stored those.

And then I can handle my request the way that I want. So, the first thing I do here is I grab the video URL from the remote payload. And because this was some custom metadata, I use the user info dictionary to access this value. The next thing I do is download from that URL using whatever mechanism I want and I pass a completion handler to my download method that I want to execute after the download is complete.

After the download completes, I take that file and I create an attachment out of it. I attach it to my notification content and I send my notification on its way. And now my notification looks great. It has an attachment, a video attachment attached to it. But what if I'm not able to complete the downloading of the video in the set time? There is a limit on the amount of time that I have to use in my service extension. So, if I'm not able to do that, there is a fallback, and this a function that we call also in the service extension called service extension time will expire. In here, I can make use of both the content handler and the content that I had stored earlier.

And the first thing I do is I grab the type of media this is that I had also packaged with my metadata and I just set the title to say incoming media, or incoming video. And I send my notification on its way. Now that we know how to use service extension, let's remember few things. Which is that there is a short execution time.

There's a limit on execution time and there's also a memory limit. But make use of the fallback in order to do the best possible thing for that notification if you're unable to complete your operation. It's also important to note the notification that can be handled by the service extension are UI notification and they will be presented.

And these are different from silent notifications. What I mean by that is when APNS delivers the notification to your device, the service extension has the opportunity to intercept it, as we just saw. Any and all work done in the service extension should pertain to this incoming notification. There should be no additional background work for your application that should be done here. All work should be either about modifying or enhancing this notification. The service extension also doesn't have the power to drop this notification or prevent it from being displayed. This notification will get delivered to the device.

If instead you want to launch your application in the background and run some additional processing, you should send a silent notification. You can also send a silent notification and launch your app in the background and your app can determine whether or not to schedule a local notification if you want to present a conditional notification.

So, now I have my great notification with an attachment and let's say I 3D touch to go into the rich notification. Without doing anything, this is my default look. Which looks good, but I might want to customize it. And in order to show you guys how to customize these views, I'm going to invite Kritharth back up on stage.

Thank you, Teja. So, so far we've looked at our first two use cases. The first about hidden notification content and how we can customize them. And secondly about modifying push notification content. Let's move to our third use case about customizing rich notifications. Now, as we just saw, this is the default look for a media attachment rich notification.

We overlay the video with the play button so that the user can play the media right there and then as well as default content shown right below it. And if there was no media attached to it, then all that the rich notification would show would be the default content.

But what if you want to customize this rich notification and give it a custom look and feel that's specific to your application? What if you want to completely custom9ize the display of this notification content as you want to do. And in the process you make these notifications very interactive and dynamic for your users so that they can complete these operations right in the notification without having the need to go to the application. Well, we can achieve all of this and for that we use our second extension point of content extensions. So, let's take a look how we can set content extensions up.

So, so far we've seen that the service extension was where we downloaded the media and then the notification was presented on the user's device. Now, if you have a content extension set up having the same category identifier as the notification, then the system will launch this content extension. And set up all the different views and view controllers associated with this content extension before opening the rich notification. And then uses these view controllers to present that custom rich notification user interface.

What are the different things that you can customize as part of these rich notifications? You can change the title that's right above the content. You can completely customize the content and the way it's shown. And you also have the option of just removing the default content if you've already included that information in your custom section.

So, to set up a content extension, Xcode gives you a default template to get you started. In this customization process, the first place where we look is the info.plist. Now, of course the first thing we do is set the correct category for all the notifications that we want to associate with this content extension.

Now, some of the easy ones we can get here are by setting the default content hidden flag to Yes. And this, we'll just go ahead and remove the default content before, below your custom UI. And also by overriding the default title, you can do that by simply setting this flag here as well.

Now, when we look at our code, you can see here that our class notification view controller implements the UN notification content extension protocol. Now, the entry method into your content extension is the did receive notification method. The UN notification object here contains all the information that's pertinent to this notification.

So, the first thing that we do is go ahead and extract the content from this notification. And then we can just go ahead and get the attachment that we had downloaded in our service extension here. And then we go and set it up in our custom media player that we set up in our custom UI.

There are more things we can do in this customization. One of the things is setting the title that we had overridden at the top of the content extension. Now, the user info object is a key value pairing that you can associate with your notification payload which allows you to send a lot more information.

So that you're not restricted to just the title, the subtitle as well as the body of the notification. So, in our example here, we package different information associated with this notification such as the user who posted this video. The location [inaudible] was taken as well as a description that might be associated with it. And we can just extract that simply from the user info object.

And we use this to set up our custom labels which, again, we've tied to the storyboard associated with our content extension. So, now that we've set this up in Code, let's take a look at how our custom UI looks. So, the user goes ahead and 3D touches in this notification. And then you can see that we have a completely different UI here which might be something similar to what our application looks like.

So, we've gone from something that looked like this to something that's completely customized. You have your own custom title at the top. You have your complete custom UI where the video is surrounded by all the relevant information that you want to show as part of this notification. And we've hidden the default content because we've already included that information in our custom content.

All right, so let's take a look at the transition again and pay attention to the initial sizing of this content extension as it's being loaded here. Now, you might have noticed that it starts at a larger size and then grows smaller to the exact size of the content extension. Now, this might not be a good experience for your user if they're going into your content extension after 3D touching. Well, we do have the option of fixing that as well. And for that, we go back to our info.plist.

And there we have a property that you can set which is the content size ratio. Now, this ratio is a best guess that you want to tell the system for the ratio between the width and the height of your content extension. So, the system knows what's the starting size it wants to load that content extension at. So, in our example here we just set the value of 0.8 and once we've done this, now when we do the transition, it loads right perfectly to the size of your content. So, it's a much better experience for your users.

So, before we move on, let's take a look at some of the best practices for customizing these rich notifications. Now, as I've emphasized, you can have custom UI elements and you can display all the relevant information using these elements as you see fit. You can also simply reuse your app view controllers in these notification content extensions. So, you don't have to duplicate any work. These are simply view controllers.

It's also important to have the correct sizing and that the starting loading size is also what makes sense for your content, for your custom content. And lastly, it's important to have fast loading and layout for your content. Because we do want the transition from the notification into the rich notification to be super smooth for your users.

All right, so this is where we ended up with our custom UI. Now, we did attach a video here. However, there's no way to play the media. Also, this post looks like it's coming from a social media stream but there's no way to interact with it. Well, we can add all these different mechanisms to this notification and that leads us to our last use case about user input customization.

Now, some of you might be familiar that the content extension body does not allow any touches. However, fear not. We do give you some multiple input mechanisms that you can use here for making these notifications interactable. And these are namely media buttons, notification actions, and custom user inputs. And we'll take a look at all these three different interaction models in the section.

Now, wouldn't it be nice that we could just overlay our video here with a play button on top of it? Well, you can do that. Content extensions allow you to overlay your content with media buttons. Let's take a look how we can set this up in code. So, here we are back at our, in our class which implements the content extension protocol. And for getting these media buttons, you have to implement a few methods. The first one is the media play pause button frame which allows you to tell the system exactly where you want this media button to be overlayed over your custom content.

You can also customize the look and feel of this media button by specifying the color as well as the type. In this example, we just use default. Now, to know when the user actually pressed the play or the pause button, you also get method callbacks, namely media play and media pause. And in our example, it's important that call to our media player in our custom content.

All right. Now that we've set up our media button, let's take a look how it works in action. So, the user goes into our custom content and just by simply pressing the play button, the video starts playing. It's as simple as that. So, let's move to adding more interaction to this notification. And like I said, this was a media post and we want the user to give us feedback to this video post in the form of, say, maybe a like and a comment action. So, let's see how we can set these up.

Now, normally action handling works that when the user takes an action, you get the response in your main application. However, if you do have a content extension set up, you do have the option of intercepting this action right in the content extension. So, you can act on this action as well as update your UI the way you want to do it. Also, you have the option of just forwarding this action to the application if you want to centralize all your action handling.

Let's see how we can set this all up in code. So, here we create our two actions, one for like which is a simple UN notification action. And for comment action, we create a text input action. And then we set up our category with these two actions that we've created, giving it the same identifier that's associated with the content extension.

Now, when we come back to the content extension code, the main method that you implement for handling these actions is the did receive response with completion handler method. Now, here the UN notification response object contains all the information about the action that was taken as well as the notification request on which this action was taken.

So, the first thing in our example that we do here is post this response to our server because we just want to handle our action in the content extension without sending it to our app. Then we look at the action identifier and determine it's the like action. And then just go ahead and update our custom UI with the like emoji.

And lastly, we call the completion handler with the option do not dismiss. Because we want the user to still interact with the content, because the video might still be playing after they've completed this action. So, once we've set this action up, let's take a look at how this looks.

So, the user starts playing the video and just by simply taking the like action, we've updated the UI and it gives a good feedback to the user that they're engaging with this content right there and then. And the video's still playing and the content hasn't dismissed. Let's continue and look at the next action that we've added which was the comment action. And here again we look at the action identifier and determine that it is the comment action.

Now, since we know that we'd set it up as a text input action, we first check if the response is associated with a text response. And we extract the user text that the user might have entered as part of this action. And then set it up on our custom label that's again associated with our custom UI. And again, we call the completion handler without dismissing this notification.

So, once we've set this up again, now the user starts playing the video, presses the comment button and just adds their comment there. And as soon as you post it, we just update our content right there and then. So, it's, so the user gets real-time feedback of what they're doing and they're super engaged with the content that's associated with the notification. And they don't have to go to your application to do all these things.

All right, so we've set up these two actions but sometimes a like is not enough and a comment might be too much. For social media posts maybe, we want to have the user to give their reactions to this post. So, one way we could do that is add all these different reactions as just actions associated with this notification. However, this doesn't look really nice and it's a lot of options that we're duplicating here. What if you could just replace this with a custom user input view? That shows up right where the user's keyboard would've gone.

Perhaps something like this? Well, you do have the option of doing this. And for that, we use an existing UI kit API for UI responder. And let's take a look how we can set this up and code them. So, now we create our new action for reactions which is a simple UN notification action. And then update our category with this action that we've created.

Now, in the class that's implementing the content extension protocol, what you need to do is override a few methods that tell the system that you're providing your own custom input feed. The first method that you have to override is the can become first responder and you return the value true.

And then you override the input view method where you're returning your custom input field. Now, you have complete control of how you're setting this input view up. And all the actions and the different interactions that you might have associated with this. You're not going to go through the default notification action handling for this because this is your custom input view.

So, once we've set this up, we come back to the did receive action response method and here determining that it's the action associated with the reaction [inaudible], all we have to do is call self.become first responder. Just simply this call will let the system know to display your custom input view where the keyword would've gone.

And in our private method for handling the bottom press for these, we update our custom label and also we just call resign first responder which then dismisses your custom input view. Now, I know that's a lot of code. Let's take a look how this actually works. So, you can see here that we have our new action for reactions.

And then when the user presses this, our custom input view just shows up at the button and by pressing either of these emojis, we can just update the UI there. So, it's a very customized way of how you want your users to interact with your content. You're not limited by just simple action button presses. So, we've gone from something that looked like this.

To something that's much more concise and a much better user experience. So, what are some of the best practices for this user input customization? Now, let me repeat that media buttons are the only mechanism that you have for overlay controls. So, use them where it makes sense. You can associate them with video as well as some playback.

Make sure your actions are specific to the context of the notification so that the user knows exactly what they're doing with these actions. It's up to you to determine whether it makes sense to dismiss the content extension or not depending upon the kind of engagement that you're presenting as part of your notification. And then the custom user input view gives you complete control of your input mechanisms for these notifications.

Now, before we go today, we did want to showcase a few apps that we feel have really used these notifications APIs. Most of them that we announced at last year's WWDC and show how they're using these different APIs in different ways. The [inaudible] application provides a very rich interface with the media as well as the news title and a blurb about this content right there and then. And gives you the option of just interacting with this content with actions.

The Castro Podcast app shows you your next upcoming podcast as well as gives you the option of just modifying your podcast queue list right there and then, again with notification actions. And then the Shazam content extension gives you a play button so that you can play your last recorded song right from the notification as well as the option to buy and share this media.

All right, so let's summarize everything that we've looked in our session today. We started out by doing an overview of notifications and all the different APIs that you can use for doing notifications for your applications. Then we introduced a new concept about hidden notification content and how you can totally customize how that looks to your application users. Then we talked about modifying notification push content using service extension.

And we talked about customizing rich notification interfaces using content extensions. And tied to that, we talked about customizing these user input that's associated with these rich notifications. So, we're really excited to see how you go ahead and use all these different tools for making the notification experience much richer for your application users. For more information about this session, do check out this website. That's all for the session today. Thank you so much for coming and have a great WWDC.