Hello, everyone, and welcome to Building Voice with Siri Shortcuts. My name is Amit, and I'm here today with my colleague Ayaka Nonaka. And we are so excited to be here and talk to you about Shortcuts. And you all know, shortcuts allow you to accelerate your users to the things they value the most in your apps. What we are really excited about is that your users can add these shortcuts to Siri and perform them with their voice wherever they are.

So, it's paramount that we build a great voice experience for all of our users accessing them through voice. And that's what we'll focus on today. So, let's get started. First, we'll talk about how your users can add shortcuts to Siri and how simple it is to write -- use it right away.

Then, we'll talk about how you can provide downloads that can be spoken by Siri, through custom responses. Then Ayaka will go over some of the best practices that you should adopt to create great voice and UI experiences for your shortcuts. And finally, we'll go over how you can bring the experience of setting shortcuts to your apps.

So, before we dive into all the exciting details of adopting shortcuts and custom responses, let's take a look at the experience of setting a shortcut with Siri. My teammates and I have been working on this app called Soup Chef. It lets you order soup. Now Soup Chef adopts all our new shortcut APIs, and every time a user places an order, Soup Chef lets the system know by donating a custom intent to Siri. So, yesterday, I ordered some tomato soup. And it was really great. So much so, that I would like to order it again. And wouldn't it be great if I can just do that from Siri? Well, now we can.

So, when we go to Siri's Settings UI, you see that Siri's already suggesting that donated intent to be created into a shortcut. So, let's tap on it. And we are presented with this screen, where we can set up a phrase that you can say to Siri and perform your shortcut. There are also a lot of details on this screen from Soup Chef that convey to me as a user what this shortcut would exactly do.

How to get these right, we'll talk about that in a little bit. But for now, I would like to associate a phrase with this shortcut. Something that is short. Something that's memorable and I can say it again and again. It's not always easy to come up with that right in the moment, so I figure that Soup Chef is suggesting I call it, Soup Time. I like that. I'll take it.

Now, when I record the phrase, I have a shortcut to order soup with Siri. Let's try it out. Now, when I say to Siri, "Soup Time," I'm asked to confirm whether I would like to run the intent, and when I say, "Yes," the order is placed. And it's that easy.

Your users will set up shortcuts for things they do most often. And they can use these shortcuts wherever they are, from their iOS devices, from their wrist using the Apple Watch, in their homes using the HomePod, as well as on the go with CarPlay. So, we should pay a lot of attention to the voice experience that they encounter.

Towards that goal, let's talk about custom responses. Shortcuts allow you to provide dialogues that can be spoken by Siri, and this gives you the ability to provide relevant information, important information, right with voice. Fundamentally, shortcuts are -- custom responses are template strings whose structure you define in the Intent Definition File.

And based on the type of information you provide in your responses, shortcuts can be one of four types. The first is a confirmation custom response. This gives you the opportunity to provide critical information before your users commit to running the shortcut. Next, is a success response. And it gives you a chance to provide an auxiliary relevant information, after the shortcut has run, and the structure of a success response consists of Siri letting the users know that their order was successful, or their shortcut ran successfully, followed by any information that you provide attributed to your app.

The third type of custom response is an informational response. And we are really excited about this. Informational response allows you to provide information like transit schedules or ski forecast right within Siri, with voice. This opens the door for a whole new category of apps which was not possible before with Siri.

And finally, when things do not work as expected, custom error responses give you the opportunity to explain what went wrong instead of just saying, "Something didn't work." The structure consists of Siri communicating to the users that their intent did not successfully execute, followed by information from your app, and the interaction punctuated by Siri, advising users to continue in your apps.

Now, that we understand what custom responses are, let's take a look at how you would adopt them in your apps. Custom responses work hand in hand with custom intents. So, the first step is to define an intent that models the use case you are trying to create a shortcut for. For Soup Chef, we created an intent called Order Soup, and it lets you, you guessed it, order soup.

Alongside the intent, you have to pick a category for your intent. And the category you pick influences how Siri speaks about it or what you see across the OS like the order buttons. So, pick something that feels natural, that works well with your use case. Once we have defined our custom intent, let's go over to the Responses Tab. And in the Responses Tab, you see that we already have a success and failure response codes.

But these are just a generic response codes which don't convey much more information besides the status of the operation. What I would like to do, is let our users know how long it will be until their order is ready and they can to the store and pick it up.

So, what I want to do is define a custom success response for that. Towards that goal, the next step is to declare the response properties that you will use in your templates to provide information at run time. So, if you go over to the Properties Section, I'm going to add a Wait Time and a Soup Property.

With that in place, the only thing left to do in our definition of custom responses is to define the template. The template is what is spoken out by Siri. So, when we look at the template section of our Intent Definition File, now I'm going to add my success template.

Not just that. Our store is getting a little popular and we tend to run out of soup these days. So, it will be great if instead of just throwing a random error, I can let the users know that the soup is out of stock. So, I will add another response code called, "Failure Out of Stock," with a template that communicates that the soup is out of stock.

With that, we have defined our intent responses. And when you do that, XCode automatically cogenerates a custom response subclass. This subclass has all the properties that you defined as well as custom constructors for your templates, which take all the important properties that you need to construct that template.

Now the last step that is left to do, is to use these responses in our Intent Handler. Handling a custom intent comprises of two steps: confirm and handle. In confirmation, we are asked whether we are ready to handle this intent at this time. So, we'll first find out whether the soup the user has requested is in our menu.

If it is, we'll let the system know that we are ready to place the order. But this is our chance to also check that the soup is available right now, and that we can place the order of that. So, let's do that. First, I'll grab a reference to my menu item. Then check if it's available. And if it isn't, I'm going to call the Completion Handler with our new custom Failure Out of Stock Response Code.

If everything looks great, we'll continue as before. This is also our chance to provide custom response -- custom confirmation response. Support for those will be coming in upcoming [inaudible]. Alright, that was confirmation. Now, it's time to handle the intent. For us, that means to place the order for the soup. So, I'm going to build an order from all the information that is present on my intent.

And then place the order with our Order Manager. And if it's successful, we'll let the system know, by calling the Completion Handler with the Success Response Code. However, since we defined the Custom Success Response Template, this is our chance to provide the wait time. So, let's do just that.

And with that, we have added support for two really useful custom responses to our custom intent. Let's see how it looks. So before, all we could let the users know was, "Your order was placed." But now, Siri can speak out that it will be about 10 minutes before they should come down to the store. And that's really helpful.

The improvement to the usability is even more stark if you look at the error case. Before, "Sorry, there was a problem with the app." But now, we can let the users know that the soup is out of stock and then they can decide what they would like to do. Either, they can go into our app and order something else or try again later. So, you saw that Siri and shortcuts allowed you to provide custom dialogues, but that's not all. As soup engineers, we work really hard to build superb experiences for our users.

And wouldn't it be great if we can bring those experiences from our apps to our shortcuts wherever they are run? With Intents UI extensions, you can do just that. Intents UI extensions allow you to provide custom UI, familiar UI from your apps, to shortcuts. And it is displayed all across the OS, on Lock Screen, in Search , and of course, within Siri.

To learn more about how you can build great, Intents UI extensions, please take a look at the WWDC 2017 session, "What's New in SiriKit?" So, we built a custom Intents UI extension for Soup Chef app. And let's see how that looks. Again, this is where we started out with. But with a custom Intents UI Extension, now I'm able to show the users the entire invoice and convince that we got all the details right, that it will be 2.95, before they place the order, which is incredibly helpful.

Once they place the order, instead of just saying, "It's done," we can now show them the receipt and reaffirm that we got everything right and it will be ten minutes before they should come down there. Alright, so you saw how easy it is to add support for Custom UI and Custom Responses. Now, let's take a look at how we did that in Soup Chef.

Alright, what you see here is our app Soup Chef. If we go to the Intents Definition File, you can see that we have a custom order soup intent defined. This intent is off-category order. Now, let's look at the response section that is associated with my intent. Here, I have properties and template sections.

What I would like to do is add the custom success response that we just talked about. So, let's start by adding the wait time property. I'll click the Plus button, and name my property Wait Time. I'll leave it of the Type String, so I can format the date however I see fit before providing it into the dialogue.

To reaffirm which soup the user ordered, I'm also going to add the soup property. Now, soup is a custom object in our app. So, I'm going to select the type "Custom." Now, let's take a look at the response codes. We have a failure and a success code. Here, I'm going to define the template for our Wait Time Response Template. So, let's do that.

"Your Soup Will Be Ready in Wait Time." I would also like to add our Out of Stock Error Code. For that, I'll click the Plus Button and create another response code. I'll call it "Failure, Out of Stock." Since this is an error code, I'll uncheck the Success checkbox. And now, finally, let's define the template for it. "Sorry, Soup is Out of Stock." And with that, we have defined our custom responses. Now, let's head over to the Intent Handler file.

Alright, here we are. As you see here, we have a Confirm and a Handle Method. Let's look at Confirm first. We find out which soup the user has requested, and then check with our menu manager whether that soup exists on our menu. If it does, we return the response code "Ready." What I would like to do here, is to check whether that soup item is available currently to order. So first, let's grab reference to the menu item from our menu manager. Then, I'm going to check if it's available. If it isn't, I'm going to call the Completion Handler with our new Failure, Out of Stock Error.

And provided the soup object that the user requested. Alright, so we have provided the custom error code here in completion. Let's take a look at Handle. Handling this intent means ordering our soup. So first, we build the Order Object from Intent. Then we place the order, using our order manager.

And finally, let the system know that the order was successful. But this is our chance to provide users with more helpful information like wait time. So, let's call the Completion Handler with our New Custom Response Code. This response template takes the soup and wait time strengths. So, I'll provide that from my intent.

And we can grab the wait time from our order. And that's it. So, you saw how easy it is to add support for custom responses to our custom intent. You start by defining a custom intent that works for your use case. You follow it up by defining custom responses that you want Siri to speak out. And finally, you provide them in your Intent Handler.

Now, I placed that order for that soup earlier, so I better head back and get it. So, please welcome Ayaka Nonaka, who will talk to you about some of the best practices that you should adopt in order to create great voice and UI experiences in your apps. Thank you and welcome Ayaka.

Thanks so much.

Hi, everyone. So, as Amit just showed you, it's super easy to get started with Siri Shortcuts. Once a shortcut is donated, it is available for the system to suggest as a shortcut to use with Siri straight out of the box. I'm going to be showing you some more new shortcuts APIs, and the best practices to help bring the greatest experience to our users. And I know you care about this because your users are going to notice these small details and the large amount of care that you put into your apps.

If you saw the first session, "Introduction to Siri Shortcuts," you learned that there are two ways to donate shortcuts. The first way is through NS User Activity. User Activities are great if you want to do a basic integration, where you just want to open the app and show your user a particular piece of content. And the second way is through Intents.

Intents are great if you want to do a deeper integration where you can run things in the background and inline within the Siri experience, so you don't have to context with your users out of that to get things done. And for both NS User Activities and Intents, there are several parameters that we can configure. The first one is the title.

So, in this case, Order Clam Chowder, with the subtitle underneath it and image next to it, and finally, a suggested invocation phrase which you'll get to later. For each of these, we're going to go over how best to configure them. So, let's get started with the title and subtitle.

So, when you're thinking about titles and subtitles, there are two high-level things that you should keep in mind. First, the title should represent what happens when the user runs the shortcut. Your user is going to want to know exactly what happens before they run the shortcut or add that shortcut to use with Siri. And secondly, the subtitles will provide more information, but only if you need it. So, let's dive into a little bit more of the details.

The first easy thing to do is see a sentence case. This is to make sure that we provide a consistent experience across all different apps and their shortcuts. Next, it's important to keep the title concise. This is because we're dealing with potentially a limited amount of [inaudible]. And you can do this by dropping articles like "a" and "an" if your language allows for it.

Next, it's super important to include critical information. For example, if you have a payments app that lets you send money to your friends and you have a shortcut for that, it's important to put the dollar amount or your currency amount in that. Probably in the title in this case.

Next, if we're working with Intents, it's important to include a verb. And if you're working with English, we should put that at the beginning and prepend the verb to your phrase. And last but not least, to make sure that we provide the shortcuts experience to everyone around the world, it's very important to localize.

So, not let's take a look at some things that we should avoid. So, the first thing that we don't need to include, is the name of your app. This is because our RUI already attributes the name of your app, and by not including this, you can save even more space. And second, we don't need to put duplicate information in the title and subtitle for similar reasons to save space and leave room for critical information.

And lastly, we don't want to use quotation marks unless you're actually quoting something that's going to be used verbatim in the shortcut. For example, if you have a shortcut to send a message, you can put the content of the message in the quotation marks. So, let's take a look at an example.

This example says, "A clam chowder from Soup Chef. The best way to get soup, Soup Chef." And just based on what we learned about, I think we can all agree that this is not a good example. So, let's try to make it better. So, the first easy thing that we can do is to drop the "From Soup Chef," because as you see, Soup Chef is already attributed in this UI.

So, now we have "A Clam Chowder." But when I'm looking at this, I'm not sure what's going to happen when I run the shortcut. Am I ordering the soup or just looking at it? Who knows? So, let's fix that by adding a verb. And while I'm doing that, I'm also going to drop the article "a" to save some space.

So now I have "Order Clam Chowder." So, when I'm looking at this, I'm thinking, "I want to order a Clam Chowder, not a Clam Chowder, which doesn't sound very delicious and we're trying to sell soup here, so that's really not ideal. But the good thing is, we can fix this really easily by simply dropping the quotation marks.

And now we have, "Order Clam Chowder." And I think at this point, our title's looking pretty good. But let's take a look at the subtitle. The subtitle says, "The best way to get soup." But that's basically Soup Chef's tag line. I already know that Soup Chef is the best way to get my soup. And it's not adding any extra information that is specific to the shortcut.

And, so that's already not good, but a more critically bad thing is that Soup Chef recently started offering delivery service, and I don't know where I'm getting my soup delivered to. Am I getting it to my home, my office, someone else's office, no idea. So, let's fix that by adding a delivery location. So, now I have "Order Clam Chowder to 1 Apple Park Way." And now I know exactly what happens when I run the shortcut. So, that is great.

And in the Settings UI, it might look something like this. And this is looking pretty good. But I think we can do a little bit better. So, as you can see, the Soup Chef app icon is used for both the Clam Chowder and Tomato Soup cases, but wouldn't it be great if we can show an image of a clam chowder for the clam chowder case and an image for the tomato soup for the tomato soup case? Sort of like this. And for this, we provide two APIs. One for User Activities and another for Intents. So, let's take a look.

So, to set some context, the user has just viewed one of their past soup orders and we want to donate it, so we can associate it -- we can associate that with a shortcut. So, the first thing that we want to do is create a CS Searchable Item Attribute Set which by the way is the same thing that you want to use if you want to provide a subtitle to your user activity, but in this case, I want to set an image [inaudible]. First, make an image with chowder.

Get the PNG data out of it and set it as the thumbnail data property on my Attributes. And next, I'm going to take those attributes, set it as the Content Attribute Set on the User Activity, and finally I'm going to donate the User Activity by setting it as the User Activity property on my view controller.

So, let's take a look at how it works during Intents. For Intents, I'm actually able to set multiple images. One for each parameter that is defined as an intent. So, I'm going to set an image of a chowder for the parameter named "Soup," and an image of an office for the parameter named "Delivery Location." And once I have that, I'm going to create an IN Interaction out of it and donate it. So, at this point, you might be wondering, "If I have a shortcut that is based on a shortcut type that contains multiple parameters that have images, how do I know which image gets used?" For that, we're going to go to our Intent Definition File.

And in particular, the Parameters Section. So, in the Parameters Section, we want to make sure that these parameters, at least the ones with images are ordered from least specific to most specific. So, in my case, I care more about the soup image than the delivery location. So, I'm going to make sure the soup comes below delivery location.

And once I do that, my list of shortcuts might look like this. So, in the first two cases, where the shortcut type contains both a soup and a delivery location, I see an image of a soup, but in the last case, where the shortcut type just contains a delivery location, I just see the image of a delivery location.

Now, these shortcuts are looking so great, that I just want to tap on one and add it to Siri, so I can use it. And our users are super excited about this new feature too, and they're all so eager to start setting up new shortcuts to use with Siri. But as more users starting using this feature, we noticed something. We noticed a problem.

When they got to the recording screen, they had no idea what to say. They could use some creative inspiration in choosing a great suggested shortcut phrase. And as Soup Chef app developers and general soup enthusiasts, we think we have some pretty great ideas on what would make a great soup ordering phrase.

You know, something like "Tomato, Tomato" or "Chowder Time." So, would it be great if we can show a suggestion like this? "Hey user, you can say something like, 'Chowder Time.'" And of course, we have an API for this, so let's take a look. So, building on a previous example, all we have to do is add one line. So, User Activity has a property called Suggested Indication Phrase. And I'm going to set that to Chowder Time.

And for Intents, it's the exact same thing. Intents also have a property called Suggested Indication Phrase which I'll also set to "Chowder Time." So, the API for this as you can see, is super simple. But what actually makes a good suggested invocation phrase? So, the most important thing to keep in mind is that the suggested invocation phrase is both short and memorable. You have to remember that your users are going to have to say this phrase exactly every, single time they run their shortcut. So, ideally it would be somewhere between two to three words.

Next, it's important to not include the phrase, "Hey Siri," in your shortcut phrase, because if your user accidentally sets their shortcut phrase as "Hey, Siri, Chowder Time," when they actually run the shortcut, they might have to say, "Hey Siri, Hey Siri, Chowder Time," and no one wants to do that.

So, let's take a look at some examples. This example says, "Hey, Siri, please place an order, thank you." Although this example is polite, it's not a good example. First, it contains, "Hey, Siri," which is pretty obvious, but it's a little bit too long for me to want to repeat it every, single time I want to run the shortcut. So, that's out.

What about this example? This example says, "Order a clam chowder to my office." Seems pretty reasonable, but this is still not good because it's still too long. Like, it's not short and memorable. And I don't think I can or want to remember to say this phrase every, single time I want clam chowder delivered to my office.

What about this example? This simply says, "Chowder Time." This is both short and memorable and doesn't include the phrase, "Hey, Siri." So, this passes all of our tests and that's awesome. But we can't [inaudible] in order to provide a truly great shortcuts experience to your users, around the world, you have to make sure it's localized.

For example, I speak both English and Japanese, and it would be great if I can suggest a Use Shortcuts in Japanese sort of like this. So, instead of "Chowder Time," I can be like [foreign language spoken] which means, "I'll eat chowder." To do this, we can localize. And when we're thinking about localization, there are a couple of things to keep in mind. So, in addition to localizing your code, we want to make sure to localize the content in the Intent Definition Files, because as you can remember, there are a lot of strings in there.

And second, we have to think about pluralization. So, even if you're not localizing to any other language besides English, if you have to handle things like, "Order one clam chowder," versus, "Order three clam chowders," you have to think about localization. And to learn about all of this stuff, check out the Localization Session and their labs. They'll teach you everything that you need to know, and more. So, cool. For now, let's go back to English.

Now, that we've followed these best practices, when we go to the Settings UI, you are presented with a beautiful list of suggested shortcuts. And right now, these are based on a combination of suggestions based on the user's past routines and the things that the user did recently. But in addition, we provide an API to allow developers to suggest their own shortcuts. And this is useful because this allows you to suggest things for things that the user hasn't necessarily done yet or hasn't done recently.

For example, if you have a music app, you might want to offer a shortcut for playing a particular playlist, even if the user hasn't played it before or hasn't played it recently, because presumably, if a user has created the playlist, they probably ought to play it at some point in the future.

So, in the Soup Chef app, I think it'd be great if we can suggest a shortcut for ordering the soup of the day, even if they haven't ordered it recently, or have never ordered it, because based on past experience, our users seem to love it because it's both delicious and offered at a special price. So, let's take a look at the API on how to do this.

So, for Intents, you can create an IN Shortcut out of Intent, well, and for User Activities, you can also create an IN Shortcut out of it. I'm going to wrap it in a suggestions list and pass it into IN Voice Shortcut Centers Set Shortcut Suggestions Method. And once you do that, you'll see my suggestions alongside all the other system's suggestions. And by the way, the same set of suggestions is shown in the shortcuts app, so you can create custom shortcuts, using your shortcuts.

So, now that we put in all this work to provide a great voice experience for your users, we want to make sure that your users know that they can add shortcuts to use with Siri. And the best way to do that is bringing shortcuts into your app, so that users can create shortcuts right from inside your app. So, for example, I just ordered a soup, and I'm going to need a suggestion to add this to Siri.

And when I tap, "Add to Siri," I'm presented with the same system UI that allows you to configure a new shortcut to use with Siri. So, let's take a look at the API to do this. So again, I'm going to take my User Activity and create an IN Shortcut out of it, and then using that, I'm going to create an IN UI Voice Shortcut View Controller, set the Delegate, and present it.

And it's the same thing for Intents. So, instead of creating the IN Shortcut with a User Activity, I will create it with Intent, create my View Controller, set the Delegate, and present it. And once I do that, your user will see this setup screen, right from inside of your app. In addition, rewrite an API to allow users to delete and edit any existing shortcuts.

We also have an API that lets you get a list of shortcuts that are already added to Siri by our users. So, for example I can use this information to indicate which soups on my menu have the user has already added the shortcut to use with Siri. So, as you can see, for clam chowder, I have "Chowder Time" associated and for tomato soup, I have "Tomato, Tomato." Awesome. So, to summarize, let's go over what we learned about today.

First, we learned that Custom Responses are critical in providing a great voice experience to our users. And Amit showed us just how simple that can be. Next, we saw how Custom Intents UI can help bring the apps experience into Siri and to the other places in the system like Search and Lock Screen.

We also saw that details matter. Even something small like adding an image to your shortcuts, or just dropping the quotation marks from around "clam chowder" to make it a big difference. And bigger details like localization can make a huge difference. Lastly, we learned about the various different APIs that you can use to bring the shortcuts experience into your app, so that your users can create shortcuts to user with Siri at the most relevant moments in your app.

So, the documentation on how to do this is all on developer.apple.com, and in addition, we'll be holding shortcuts labs throughout the week, including one later today. And we also have a Watch Session right after this, that will show you how to bring the shortcuts experience to the Watch and to the Siri Watch face so that -- and they're also going to show you how it works, even if you don't have a Watch app. So, it's a total must watch. Alright.

Alright, well, now you know everything that you need to know to create a great shortcuts experience for your users. We can't wait to see what you build with our new APIs. Thank you for listening, and I hope you have a great rest of WWDC.