Hello. Good morning, everyone. Welcome to Part 1 of iMessage Apps and Stickers. My name is Bhaskar and I'm an engineer on the Messages Team at Apple. Today we are introducing iMessage apps. And what that means is that you will be able to write apps that run within the context of the Messages application. iMessage apps can provide three main types of content.

Number one, they can provide interactive messages. Interactive messages are a type of message that your application generates and when your user taps on this message, it will launch your application. For example, in the screenshot shown here, I'm using the DoorDash app to send a menu to my friends for the party that we are having later tonight.

On their phones they will be able to see that message and if they have the application installed, they will be able to tap on it and tapping will launch DoorDash. At that point they will be able to modify the menu, add stuff to it, and send a reply back to me.

iMessage apps can also provide Stickers. Stickers are images generated by your application and we believe it helps create a very fun and interactive experience between iMessage users. And lastly, iMessage apps can provide every other type of content that Messages has always supported, like photos, videos, text, links, et cetera.

Also in iOS 10 we will have a Messages App Store. What that means is that the apps that you write and create will be shown and displayed in this Messages App Store. Users will be able to get to this App Store via Messages and they will be able to download your applications. Applications that your users have downloaded will be displayed within Messages in the UI which we call the Messages App Drawer.

We also have this great feature that allows your users to discover your applications very easily and we call it inline app attribution. Let's try to understand what that means with an example. In the screenshot shown here I have received a Sticker from my friend but I do not have that application installed from which he sent the Sticker.

In that case, as you can see the zoomed in area, beneath the Sticker we will have the name of the application from which the Sticker was sent. And tapping on the piece of text will launch the Messages App Store and present me with the application from which the Sticker was sent. And then I can download the application and use it like any other iMessage app.

In this example shown here, I used the Sticker as an example to illustrate inline app attribution. However, it applies to every other type of content that your iMessage app can generate. Also creating an iMessage app is very similar to creating any other iOS application that you have already created. You will be able to use the same set of technologies and framework that you already use currently.

For example, some of the things that you can use are: You can do in-app purchase within your application; you will have access to Apple Pay; and your iMessage app will also be able to use the camera. And we believe these technologies and features will allow you to create great apps for your users. Now let's talk about supportive platforms. iMessage apps will only run on iOS 10. However, the content that your application generates will be sent to, and can be viewed on watchOS and macOS devices.

In watchOS 3.0, within the Messages application, we also have a new page called the Recents Page. The Recents Page, amongst other things, will display Stickers that your users had recently sent from their phone. And you users will not only be able to see these recent Stickers, they will also be able to send these Stickers from their watch just like they are able to send it from their phones. So that was about the Messages App Store and the Messages platform. Now let's get into details of creating iMessage apps.

This is what we're going to cover in the rest of the session today. First we are going to talk about the Messages framework. Messages framework is the framework that you guys need to familiarize yourself with to create iMessage apps. Then we're going to talk about creating a Sticker Pack application.

Next we are going to go into details of Sticker file formats and other constraints around Stickers. And lastly we're going to talk about creating a custom Sticker experience using the Messages framework. What we will not cover in today's talk is how to create an application that provides interactive messages.

We will cover such an application in Part 2 of our talk and I do request you guys to please come back for that. That will be here on Thursday at 1:40 p.m. Messages framework, Messages framework is the new framework that we are introducing in iOS 10. Messages framework acts as an interface between your iMessages app and the Messages application.

When your user taps on your app icon to launch your application, it is through this framework that we discover your app, launch it, and provide your application with data and other context your application needs to lay out its UI. Once your application has presented its UI your users are ready to interact with it.

And at the end of that interaction your application will hopefully generate some content that your user wants to share by Messages and your app hands this content back to Messages by this framework. And at that point your users are ready to share that thing via Messages with whomever they're talking to at that point. I would like to mention that the Messages framework and iMessage apps are built on top of existing app extension technology.

And in that sense your iMessage app is an app extension and in this scenario the Messages app is acting as the host application for your iMessage app. For those of you who are not familiar or who have not made extensions before, I do recommend that you please go back and take a look at this WWDC talk from 2014 that introduces extensions in iOS and OS X.

So as I just mentioned, iMessage apps are extensions, however, there's one key difference as compared to other extension points on iOS. Unlike other extension points you do not have to provide a containing application for your iMessage app. You can only ship your iMessage app if you want to. But if you already have a containing application that you created or that you're already shipping to the App Store, you can always include an iMessage app within your containing application because, as I mentioned, these are extensions.

So if you provide a containing application, then your application will show up in the main Home screen of your users' phones and the Messages App Drawer within Messages. It will also show up in the main App Store and in the Messages App Store. But if you do not provide a containing application, then, as you can see, your application will not show up in the main Home screen and the main App Store. It will only show up in the Messages App Drawer and the Messages App Store.

I would like to mention that if you do not provide a containing application, you still have to provide an icon for your containing application. This is the icon that will be used in other parts of the system like the Settings application to show information about your application, for example, the amount of storage it is using.

So now let's talk about Stickers. Stickers are a fun way and a new way to communicate with other iMessage users. Your users can send Stickers inline just like any other iMessage your users already send. Or your users, not your Stickers, or your users can drag Stickers out of their applications and attach them to other Message bubbles. And we believe this lends itself to a very expressive and engaging messaging experience.

But what do Stickers mean to you? Stickers are basically images that your application provides. These images can be animated or they can be static. And we believe Stickers are a great way to share or promote any content that you may have. And we generally think Sticker applications are going to be very, very popular amongst iMessage users.

So now let's talk about creating Sticker apps. There are two ways to create a Sticker app. First, you can create a Sticker Pack application using Xcode. If you go down this route, you do not have to provide any code for your application. You only have to provide the assets for your Stickers and the app icons. But if you want to customize your application, you can do so by providing code. And your code in this case will provide Stickers via the Messages framework.

At this point let me show you a demo of how to create a Sticker Pack application using Xcode. So I have my machine here and I'm going to launch Xcode. And here I'm going to say I'm going to create a new Xcode project. And under the Title of Application I'm going to select Sticker Pack Application. And then I'm going to hit Next. And I'm going to name my project "SFStickers" as the assets that I'm going to use are going to be assets depicting various landmarks in San Francisco. So I click Next here.

And for now just let me create it on my desktop. I'm done. Let me quickly make Xcode bigger. Great. So, as you can see here in the template that you get in a Sticker Pack application, there are no source code files. That is a very deliberate design of this template and so let me reiterate this, if you create a Sticker Pack Application, you cannot provide any code. You only provide the Sticker and the app icon assets. So let's see how to do that.

I go into the Stickers -- that's the Catalog which has two catalogs within it. One is the Sticker Pack and the other is the iMessage App Icon. I click on Sticker Pack. And, as you can see it says I do not have any Stickers and I need to drag and drop the images to add new Stickers. So let me do that. So go to Finder and I go to Desktop. And I have some assets here that I want to use. So I'm going to click on the first one and I'm going to drag and drop it here. So that's Golden Gate Bridge.

I'm going to drag the second one. That's Chinatown. And just to be quick I'm just going to select everything else and drag it here. Great. So I have all my assets. So these assets are a PNG type. Xcode also allows you to create an [inaudible] in your Sticker using the individual frames of the PNG. Let me show you how to do that. So I'm going to click, or rather right-click here and I'm going to go to Add Assets.

And I'm going to say New Sticker Sequence. So when you create a new Sticker sequence Xcode expects you to provide the individual frames of the Sticker that you're about to create. So I'm going to go back to my desktop. I'm going to click here where I have the individual frames.

I'm going to drag the first one in. I'm going to drag the second one in. Third one. And finally the fourth one. Now let's go back to Xcode and see what it does. So as you can see, I have an enlarged version of the Sticker here and click in its center will actually show it to me animating. Looks good. Now let me just throw in the app icons. I go to the App Icon Catalog. I go in here and I select these things, drag it here. Looks like I may have missed one. Let me quickly do that one again.

All right. So I provided the app icons. At this point I'm ready. My app is done. So let me try and see how it looks like. I'm going to go here and test my application against the simulator. I'm going to select iPhone 6s as the device I want to test against. And I'm going to go to Product and I'm going to hit Run.

I'm just going to wait for this thing to show up. Great. So Messages has shown up and since I'm creating an iMessage app, I'm testing it against Messages. And I'm going to run it. And the simulator is launching. Great. So here you can see I have two simulated conversations in Messages in the simulator.

By simulated conversation what I mean is if you go into one of them and send a message, it will show up as an incoming message in the other conversation. And just to clarify, you will not be able to send messages out of the simulator to real devices. So going into the first one and I type in something.

And at this point I'm just talking to myself so. As you can see, this is my message here. Great. Now let me see what my app looks like. So I go in the App Drawer, and there you go. My application is ready here to be launched. I click on it.

There you go. The assets have shown up right there. These are the same assets that I just added and I can scroll down and look how they look like. And, as you can see the animated PNG is also here. And I can click on one of them, peel them, drag them out of the application, and attach to a message bubble right here. Great. And as you can see, I can go back out, go into the other conversation. I see the Sticker here.

So this is how you create a Sticker Pack application. However, there's one more thing that I want to show you here and that is how to customize certain aspects of this application. So going back to asset Stickers, I will show you now how to change the sizes of the cells in which your Stickers are being displayed in this grid here. So to do that, let me go back to Xcode.

And I'm going to click Go To The Top right here. And as you can see, there's a drop-down menu by the name Sticker Size. I click on that. There are three options, Small, Medium, and Large. And I'm going to click Large and see how that changes my application. I run it again.

And Messages is launched. Let me go back to my application. And there you go. As you can see, the Stickers are now presented two-in-a-row and they're presented in larger cells. And there's more space between them. I'm going to go into more details of the implications of changing the Sticker cell size later in this session.

In that demo I showed you how to create a Sticker Pack application using Xcode and in this technique, like I showed you, you do not have to provide any code. You only provide the assets for the Stickers and for your app icons. I also showed you how to use the tool built into Xcode to create an APNG Sticker using the individual frames of the APNG.

And lastly, we also talked about how to change the Sticker cell sizes. Now let's talk about supported file formats for Stickers. We support four formats. These are PNG, APNG, JPEG, and GIFs. And out of these four formats, if you want to provide animated Stickers, we support APNG and GIFs.

We are also making certain optimization under the hood to make sending Stickers faster as compared to other iMessage attachments and also to reduce the on disk storage space for Stickers. And to help us make these optimizations we have to limit the file sizes for Stickers to 500 kilobytes. So as I just mentioned, we support four file formats, however, we do recommend that you use PNG and APNG for most, if not for all of your Stickers.

Let's try to understand why we have that recommendation by taking a look at these screenshots. So here I have two conversations which look nearly identical but there's a subtle difference between them. And so let me zoom in to show what that is. As you can see, the conversation to the left has Sticker placed on the photo and it is in PNG format.

And the conversation to the right has the same photo and the same Sticker but that Sticker is now in a GIF format. If you observe carefully and in the zoomed in areas, the one on the left, that is the PNG, does not display any artifacts around it's edges. But the one on the right, that is the GIF, is displaying certain artifacts around the edges of the Stickers.

Now this is happening because PNGs and APNGs handle transparency very well but GIFs do not handle transparency that well. And if you were to compare the transparency properties of JPEGs, JPEGs in fact do not handle transparency at all. And hence keep in mind your Stickers can be dragged out of your applications and attached to Message bubbles and therefore transparency is a very important factor to consider while designing your Stickers. We also request you to please try out your Stickers on various colored backgrounds like black, white, and even multi-colored backgrounds because, again, your Stickers can be placed on photos.

In the demo I just showed you how to change the sizes of the cells in which your Stickers are being displayed. There are three supported cell sizes and they are small, regular, and large. And for each of these sizes we have a recommended maximum dimensions of the Stickers that it supports. For small it is 100 by 100 points at 3x the scale. For regular it is 136 by 136 points at 3x the scale. For large the maximum supported dimension is 206 by 206 points at 3x the scale.

If you provide Stickers that are larger than the dimensions that are supported by the cell size that you have selected, then your Stickers will be shrunken down and displayed by Messages framework and in that case they may not look like exactly the way you intended them to look like. But if your Stickers are within these bounds, then they will be displayed as is.

I would also like to clarify that we always request you to provide us Stickers at 3x the scale regardless of the resolution of the device in which your application is running. Messages will take care of scaling down your Stickers if needed according to the device in which your Stickers are being currently displayed. So that is all I had to talk about the Messages framework and creating Sticker applications. At this point I would like to hand it over to Lily who is going to talk about creating a custom Sticker experience. Thank you.

Thank you, Bhaskar. My name is Lily and I'm an engineer on the Messages Team. I'm going to talk to you about creating a custom Sticker application. First, why would you want to create a custom Sticker application? The first major benefit is you will be able to customize how you display Stickers in your application. In our previous example we saw Stickers displayed in a nice grid layout. However, if you wanted to position your Stickers in any other format or even change the background color, you would need to create a custom Sticker application.

A custom Sticker application will allow you to dynamically create stickers. In our previous example we bundled assets into our application and in order to update those assets we would have to push an update to the App Store. If you want to pull your assets from your servers, or allow you users to dynamically create them on device, they could do so and then you wouldn't have to update through the App Store.

Messages extensions will also provide camera access, allowing your users to create fun Stickers using the camera. And finally your application can utilize In-app purchases, allowing your users to unlock Stickers. We can create a custom Sticker application using Messages framework. Messages framework provides the building blocks for creating a Sticker application.

In our example of Sticker Pack applications built through Xcode, they use the same classes in this framework so you could ideally create the same application using Messages framework. In the screenshot we have onscreen we have an application that's displaying Stickers. Well, you'll notice that the background color is pink. So I'm going to show you how we can customize and create an application like this. So let's go into a demo.

So I'm going to open up Xcode and I'm going to create a new project. Bhaskar created a Sticker Pack application but this time we're going to create a Messages application. So I'm going to go ahead and choose that. And I'm going to call my project "Nature Stickies" as we're going to use nature assets. And we'll go ahead and create this project and we'll put it on our desktop.

Great. So you'll see here that we have two folders: Nature Stickies and Messages Extension. Nature Stickies does not have source code as this is the containing bundle application. You'll provide an app icon here for your System Settings. However, we're interested in the folder, Messages Extension. So we're going to go ahead and click on MessagesViewController.

And we'll see that MessagesViewController is going to be our principal class. It is a subclass of the MSMessagesAppViewController. The MSMessagesAppViewController is going to be the controller that handles callbacks from the Messages application. We'll be talking more about these callbacks and how your application can handle them in Part 2 of our talk. So we can go ahead and get rid of these functions here.

I'm going to select them and then press Delete. Great. So let's take a look at Messages framework. If we click here we have an MSStickerBrowserViewController. The MSStickerBrowseViewController will allow us to display stickers. It has two properties. It has an MSStickerBrowserView which is the view that our Stickers will be displayed in. And it also has an MSStickerSize which decide the cell sizes of our grid for the MSStickerBrowserView.

We'll also see that the MSStickerBrowserViewController conforms to the MSStickerBrowserView DataSource protocol. This protocol, when implemented, will return the number of Stickers your application will be displaying and the Sticker at a given index for the MSStickerBrowserView. So if we want to display our Stickers, we'll need to create an MSStickerBrowserViewController.

And to implement the data source we'll go ahead and subclass this. So I'm going to go ahead and create a new file. And I'm going to choose Swift and click Next. And I'm going to call this NatureStickerBrowser ViewController. Great. So I'm going to go ahead and import UIKit.

And then I'll go ahead and import Messages. Great. So now let's create our class. I'm going to go ahead and create our NatureStickerBrowser ViewController class as a subclass of the MSStickerBrowserViewController. As I mentioned, the MSStickerBrowserViewController has two data source methods that we'll need to implement. So I'll go ahead and fill those in.

So now we'll need a way to create our Stickers and store them. I'm going to go ahead and create an array to hold our Stickers and we'll see that our Stickers are at MSSticker. And MSSticker is an object that has a file URL that points to an asset on disk. It will also have a localized description which will be used for accessibility. So let's go ahead and create a Sticker.

I'm going to go ahead and add assets to our bundle. I'm going to click Add Files and then go to my desktop and choose these assets here. And I've added them to my application. In this example we're just adding assets to our application but you can imagine how your applications can either pull these assets from your servers or allow your users to create them on device. So now we'll need to create a Sticker.

So we'll write a function to do that. Great. So here we take our bundle. We create a file URL pointing to our asset on disk, and then we initialize our MSSticker with a localized description, and then add it to our array. So now let's go ahead and write a function that will load all of the Stickers. Cool. So now we can go back to our MessagesViewController and create an instance. And we'll create a NatureStickerBrowser ViewController, and we'll instantiate it, and viewDidLoad.

Great. So now we instantiated our browserViewController with a Sticker size. This will allow your application to display your Stickers in the grid layouts that we showed you earlier. So this application will create a Sticker application like we did through Xcode without code. So let's go ahead and customize this. I'm going to write a function that will allow me to change the background color of our MSStickerBrowserView. So we'll write that function here. And we're just changing the background color of the MSStickerBrowserView. And I'm going to go back to the MessagesViewController and call that.

So now we can change the background. And let's go ahead and run our application. So I'm going to choose the iPhone 6s and I'm going to hit Run. Oops. Oh, I forgot something. We forgot to actually implement our data source. So let's go ahead and do that. So I'm going to return the Stickers count and I'm going to return a Sticker out of given index. And now we're ready to run. So I'll choose Messages as the application I want to run in.

Great. And now we can open up the App Drawer and then you'll see that we don't have an icon because we forgot to add that. And we'll see that we have our app named Messages Extension. In order to change what our display name is we would have to go into our project, click on the Messages Extension Target, and then we could change it here. But for now let's just see what our application looks like.

So you'll see we now have an application that has Stickers with a pink background. So this is one small example of how you can customize your Sticker applications. So let's go back to the slides. So to recap what we just went over, we introduced the MSStickerBrowserView Controller. This ViewController will display Stickers in your application using the MSStickerBrowserView. This controller also conforms to the MSStickerBrowserView DataSource protocol.

This protocol will return the number of Stickers your application will be displaying and the given Sticker. We also introduced the MS MessagesAppViewController. This ViewController is what your principal class will subclass. This MSMessagesAppViewController handles callbacks from the Messages application. Again, we'll talk more about this in Part 2 and how your application can handle these callbacks.

So that was a small example of how we could customize a Sticker application by just changing the background color. But what if we want to do more? Your application can add Sticker support using just two classes, the MSStickerView and an MSSticker. So if we wanted to create an application that displays Stickers in a different way than the MSStickerBrowserViewController does, we can do so by using these two classes.

On the example I have behind me we have an application. This application is actually using a UICollectionViewController to display Stickers. And each cell of the UICollectionView has an MSStickerView. The MSStickerView is the class that will be providing the drag and drop functionality of Stickers into the Messages transcript.

So we also see that there's a Plus icon which doesn't have the MSStickerView. Tapping it will expand our application, allow us to choose a Sticker, and add it to our collection. Also note that we were able to transition how our Stickers were displayed. Your application can go from being in a compact mode to an expanded mode.

Your users can also toggle this so by clicking on the chevron in the bottom. So your application will have to be able to handle being in a compact and an expanded mode. So I'm going to show you how you can handle these callbacks and create an application like this. So let's go into another demo. So I'm going to open up an application I already have. And it's called Sticker Picker.

And let's go ahead and make this bigger. Great. So we're back in our MessagesViewController which is our principal class, again, subclassing the MSMessagesAppViewController. We had two views that we displayed, our UICollectionView holding MSStickerViews, and we also had a view that would allow us to add Stickers to our application. These two views are going to be represented in a StickerPickerCollection ViewController and in an AddStickersViewController. Let's take a look at the StickerPickerCollection ViewController.

The StickerPickerCollection ViewController, subclass is a UICollectionViewController. The items that we'll have are a CollectionViewItem. So it can either be the AddSticker item or it could be a Sticker. So let's go ahead and look at the Sticker cell. The Sticker cell has an MSStickerView. The MSStickerView will have a Sticker and, in order to handle animations of your animated Stickers, we would have to call startAnimating and stopAnimating when we are displaying our MSStickerView. So let's go back to the StickerPickerCollection ViewController.

Great. So when we create this ViewController we're going to go ahead and create all of our items. We'll create an AddStickerItem and then we'll load our Stickers the same way we did in our previous example. We'll return the number of items in our collectionView and we'll create Sticker cells or the Add Sticker cell for an item at a given index path. When creating the Sticker cell we're going to want to assign the Sticker value of our stickerView. So let's go ahead and do that.

So we have now put a Sticker on our stickerView. And, as I mentioned, our Stickers can be animated. And we'll need to be calling start and stopAnimating to control these animations. When we're going to display our Sticker we're going to want to call startAnimating and when we stop displaying our Sticker we're going to want to call stopAnimating.

So we can write a function to check if the Sticker is animatable or not by loading the image file URL, and using a CG image source, and counting the frames. If we have more than one frame, we know that we have an animated Sticker. So we'll go ahead and implement this.

So we'll check to see if our Sticker cell Sticker can animate. And if it can, then we'll call startAnimating. And then when we're done displaying our cell we'll go ahead and check if our Sticker is animating, and if it is, we'll call stopAnimating. Great. So let's go back to the MessagesViewController. And we'll go ahead and instantiate the StickerPickerViewController as that's going to be the view we want loaded when we first launch our application.

So we'll do that in viewDidLoad and then we'll present it. When the user taps the Add icon we'll want to create the AddStickersViewController. And then we're going to want to transition into an expanded mode. We can do so by calling requestPresentationStyle. This will tell the Messages application that your application wants to be expanded.

You'll then get a call back from the Messages application once you've done this transition. So in didTransitiontopresentation Style we'll get a call back and we'll first check to see if we have an AddStickersViewController. And if we do and our presentation style is expanded, we'll go ahead and present that view.

Or else we'll present the other view we want to show in compact. After we choose a Sticker we're going to want to go back to compact. So we'll then again call requestPresentationStyle .compact. So if we run our application we'll be able to see all of this in play. So I'm going to go ahead and choose Messages as the application I want to run.

And we're going to launch. I'm going to go ahead and click on the App Drawer. And we're going to click on Sticker Picker and wait for it to load. Great. So now we can go ahead and add a Sticker. And I'm going to choose this cute panda right here. And we'll see it also animating here. And then we can send our Stickers.

So to recap what we just went over, we introduced, we showed you how to create a Sticker application using a UICollectionView and in each of the cells of the UICollectionView we had an MSStickerView. The MSStickerView allowed us to drag and drop Stickers into the transcript and it also had an MSSticker.

We also showed you how you can transition from a compact to an expanded presentation style. And we also showed you how you can handle such callbacks from the Messages application. Remember that your users can toggle between the two by tapping on the chevron icon so please keep that in mind on designing your application.

Bhaskar talked about the Sticker File Formats that we used when creating a Sticker Pack Application. These same file formats apply to creating a custom Sticker application. So to summarize what we went over today, we introduced iMessage apps as a platform to you as a developer. There's a new way to interact with Messages and it's really fun and exciting. We also introduced Messages framework and how you could create Sticker Pack Applications using Messages framework.

We showed you how to create a Sticker Pack Application using Xcode with no code. And we also showed you how to programmatically create a custom Sticker application. In Part 2 of our talk we're going to show you how to create an application that will allow your application to send interactive messages. For more information please visit the website and we'll have a related session back here in the Presidio at 1:40 p.m. on Thursday. Thank you so much. We can't wait to see what you guys create and please come to our labs. Have a great rest of the conference.