Good morning. My name is Bruce Nilo. I'm one of the engineering managers for UIKit. And today we're going to be discussing a new feature you might've heard that we're introducing in iOS 11. This is going to be the first of four dedicated sessions where we're going to explore the breadth and depth of the new drag-and-drop APIs. We're going to show you just how easy it is using these APIs to add drag and drop to your app in ways that your users are going to love.

We're going to start by talking about the goals and concepts behind drag and drop. We're going to go deep into some of the core APIs that you're going to need to add drag and drop to your apps. And then, we're going to follow that up with a demonstration showing just how easy they are to use.

Finally, we're going to wrap up with some suggestions about what you can do to quickly adopt this great, new feature. So what is drag and drop? I think we all know that it's a way to graphically move data from one application to another or even within the same application.

When we set about designing drag and drop and bringing it to iOS, we had some goals in mind. We wanted it to be fast and responsive. In particular, we didn't want to move or copy data that we didn't need to. Therefore, we leveraged the capabilities of the new file system on iOS 11 and we made our APIs, we structured our APIs in a way such that we never move data unless it's requested. And moreover, they're designed to deliver the data to your applications asynchronously so your apps never block the run loop and prevent your users from doing what they want to do.

We also wanted drag and drop to be secure in a way that the pasteboard isn't. In particular, data is only visible to the destination application that your user indicates they want to drop over. But that's not all. We wanted our source applications to have the ability to restrict access to their data to just their app, to other apps developed by the same team, and then, of course, to all apps on the system. And when we GM, we will also be supporting manage configuration for our enterprise customers.

So finally, the most important thing was that it be a great Multi-Touch experience. And to do this, it was clear we were going to have to reimagine drag and drop in many important ways. So let's take a look at some aspect of that experience we had in mind.

And before I start this video, let me give you some context. I recently took some pictures of the Empire State Building in New York, and in this video, you're going to see me use a bunch of different fingers to compose an email to some friends. We're going to start off in Spotlight, where we're going to drag a link, swipe up the doc, hover over the Safari icon to launch it, and drop that link onto the + button to open up a tab in Safari.

We're then going to select some text, long press, and begin to move and realize that wasn't really the text I wanted at all. I can easily cancel the drag, select the text that I care about. And again, the gesture is a long press where the text lifts up, followed by a move, at which point I can interact with the rest of the system. I can open the Split so that I can better see the photos that I care about, and now I can select a number of the Empire State Building photos.

You'll notice that I'm still dragging the text with my other hand and I'm adding photos to my right. I'm going to hand that collection of photos to my left hand so I can better swipe mail into the right split. I'm going to pass it over to my right hand again so that I can drop them into the mail Compose sheet.

So that's what we mean by a great Multi-Touch experience. What did we see? The interface is alive. It's deeply integrated with all of iOS. You can drag out of Spotlight. You can bring up the doc. Indeed, the system UI is implemented on top of drag and drop. There's great visual feedback, fantastic animations. You can hover over controls to navigate. You can add items to an existing drag session. You can transfer drags between your fingers, between your hands, and you can even start multiple drag sessions at once.

So before we get into some of the concepts, let me talk about drag and drop on the iPhone. We initially envisioned drag and drop as an iPad productivity feature. However, all of the APIs are in fact available on the phone. There is one restriction, and that is on the phone, we only allow drags to execute within the same application. So let's turn our attention to some of the key concepts. These are going to provide a great basis for understanding the APIs that follow.

So we're going to talk about the phases of a drag session. We break a drag session into four of them, in which both the source application and the destination application have a part to play. And it all begins with the lift. In the lift, that's when you long press, and the view lifts out of the screen. When, and then, when the user begins to drag, you enter the drag phase. In the drag phase, you can update the previews of what the things look like as you're dragging. You can tap to add. You can hover to navigate.

When the user lifts their finger, one of two things can happen. Either the drag is canceled or you actually want those items dropped where the user lifted their finger. We provide great targeting APIs to make those set-down animations look fantastic. And finally, we enter the data transfer phase, and that's the phase where the destination application requests the data from the source.

So before we go into the building blocks of the drag APIs, let me take a moment to say that it's all built around a concept called an interaction. An interaction is a really simple concept. It's basically takes an input and generates an effect of some sort. The drag-and-drop APIs are built around this for a couple of reasons. One is, is it lets us to provide a consistent look and feel for drag and drop across the system.

But most importantly, it makes it really easy for you to add drag and drop to your app without having to rearrange your code or your view hierarchies. So let me introduce the drag interaction. It's attached to a view, and as you heard in the, perhaps in the previous talk, it's very similar to a gesture recognizer. If you've ever used them, the pattern is the same. You create an object with a delegate, and it is via the delegate that you interact with the system. So for example, the delegate is asked to return the drag items just before the lift animation happens.

This is the one required method for a drag interaction delegate. The delegate is free to return no items whatsoever -- in which case, the gesture will fail and the touch will be processed as normal. OK, so what's a drag item? Well, the drag item is really the whole point of it all. It's the model object that's associated with the view that the interaction is attached to.

Drag item for drag and drop embodies both what the items look like as they're moved around the screen, the preview, as well as a promise by the source application to deliver the data to the destination when it requests. We call these promises item providers, and they're implemented in iOS 11 by some new additions to NSItemProvider.

So how do you enable a drop? It's easy. The first way to do it is to take advantage of a new property that we added to UIResponder called paste configuration. What is a paste configuration? Basically, it's a declarative way of specifying the types that your responder can either accept as a paste or as a drop. To take advantage of paste configurations, you need to implement a new method called paste itemProviders. And if you do, you will be able to support paste and drag and, and drop out of the box.

However, for more sophisticated drop interactions, you're going to want to add a drop interaction to your view. And you do so in a very similar way that you add a drag interaction. And when you do, when you are dragging over a view that has such an interaction, we're going to query that delegate to tell us whether or not it's interested in the items that are actually being dragged over it. And it will reply with and intention or a proposal, as we call it.

Now, on touch up, the delegate might've said, not interested. In which case, the drag will be canceled and all those previews are going to animate back to whence they came and, or off the screen to indicate that it was canceled. However, the more interesting bit is when the delegate says, yeah, I'm interested in those items. In which case, the system is going to say, perform the drop.

And it is within the scope of this callback and it is only within the scope of this callback that the delegate can request the items that it's interested in. Or more precisely, the representations of the items. And once, when it does so, at that point, the source application will fulfill its promise and the system will deliver that data back to the destination app asynchronously.

OK, there's a lot more to discuss. For example, what's the life cycle of a drag session from the perspective of the destination or the source? How do you make your set-down transitions look great? What do you do if the data transfer takes a long time to complete? Before we answer some of these questions, it's going to help to look at this picture because it's going to give you a good bird's-eye view of how our API is structured. On the left in blue, you're going to see those objects and classes that the source application is going to use to realize a drag.

And on the right, what the destination application uses to realize a drop. And in the middle are those objects and classes that both the source and the destination use to fulfill the contract of drag and drop. Now, to discuss these foundational classes in more detail, I'm going to bring Kurt Revis, my colleague, up on the stage.

Thank you, Bruce. So Bruce gave you the big picture of drag and drop. Now, I'm going to tell you more about using the drag-and-drop API. So I'll talk about three things. I'll give you a timeline about what happens during drag and drop, I'll introduce you to these essential API methods that you need to implement to get drag and drop working in your app, and then finally, I'll introduce you to the full API.

So drag and drop looks pretty simple, right? There's an object on the screen. The users touches it. It lifts up. Then, they move it around inside that app, maybe over to a different app. Finally, they release their touch. The touch ends. The object drops down into its new position. Of course, there's a lot more going on underneath, so let's talk about what happens when.

It all starts when a touch comes down on a view that has a drag interaction. UIKit sets up this lift animation, and then as that touch stays down, we run that lift all the way to its completion. Then, when the user moves their finger far enough, we start the drag in earnest. The object lifts up over everything else in the view and over all apps, and the user can drag it around on the screen.

Finally, at some point, the user is going to release their touch, and what we do depends on where that was. The first possibility is that the view underneath is not interested in accepting the drop. In that case, we run a short cancellation animation. Stepping back, maybe the location was on a view that wanted to accept the drop. In that case, there's more to do. We ask the delegate of the drop interaction to perform the drop, to request the data, and then two things happen in parallel.

We run the drop animations to animate those things into their final positions, and we do the data transfer. So those two boxes are the same size, but really, that data transfer is asynchronous and nobody knows how long it's going to take ahead of time. It might be very quick. It might, excuse me, it might be very quick. In fact, it might be faster than those drop animations. Or it could take substantially longer -- for instance, if the data needs to be downloaded.

Now, you've seen this whole timeline. That's everything that can happen. UIKit's responsible for running this timeline, but we need your help at three specific places. The first one is when the drag starts. We need to find the items to be dragged. We do that by calling this required method on the drag interaction delegate, dragInteraction itemsForBeginning session. You return us an array of drag items. I'll give an example here. We're going to drag just a single string, so my object will be the string "Hello World."

I put this inside of an NSItemProvider. This is the data representation, data transfer level of things. Now, note that I'm casting this to an NSString. This is because NSItemProvider only deals with objects. It doesn't deal with Swift struct. Next, I make a drag item. I go up to the drag-and-drop level. So I make a UIDragItem with that item provider. And finally, I just return that single item.

The new API essential is on the drop side. Now, while you're dragging that object around on the screen, UIKit wants to know at every point, what would happen if the user lifted their finger right here and now? You tell us that by implementing this method on the drop interaction delegate, dropInteraction sessionDidUpdate, and you return a drop proposal. So this is called when we enter your view and then also when it moves around inside of your view every time you can return a new proposal.

You make a UIDropProposal and use a drop operation to do that. Now, a drop operation is an enum. There's four possible values. I'll talk about these in order from most likely to least likely. The first one's cancel. This is saying that when the user lifts their finger, I don't want to accept the drag. Just cancel the drag.

Second is copy. This is the opposite. You're saying, I do want that data. I will accept that data and copy it into the view that the user is dropping on. Most of the time, this is what you should propose. In general, on iOS, drag and drop copies from one place to another.

The next one is move. This is a lot like copy, but you're indicating that you want the data to be moved from the source to the destination. This is a bit more complicated, and you should know that UIKit can't make it look like a move for you. We just give you the data the exact same way as with a copy. Your delegates need to cooperate in order to make it look like a move.

So this is only allowed within a single app. Your drag interaction delegate must allow it. There's a delegate method where we'll ask if it does allow it. And the drag interaction should check this property on the drop session allowsMoveOperation. If that's true, then you can propose the move operation.

Next, the forbidden operation. This is just like cancel, except that we show this additional badge on the drag item. This is telling the user that although normally you could drop here, in this specific case, you can't. For instance, if I was dragging an image over a folder, normally that would be allowed, but if that folder was read only, then I would use the forbidden badge to tell the user that it can't happen right now.

So this is a bit forbidding, so don't use it all the time. Use it sparingly. The next API essential, the third one, you proposed an operation earlier. Now, if it's a copy or move, you need to perform that operation. So on the drop delegate, drop interaction delegate, you implement dropInteraction performDrop. And here's your chance to load data from the session. In fact, this is the only place you can do that. So I'll show two examples of loading data from the items in the session.

The first one is a simple one. We'll use a convenience method on the session called session loadObjectsofClass. Here I'm using UIImage. I call this, and I provide a closure to be called back later on when those objects are available. When they come in, I can iterate through that list.

I know they will be UIImages, so it's okay to do this force cast here. And then, I can update my UI directly. So here I'm changing the image inside of an imageView. So this convenience method calls me back on the main queue. I can update my UI right there and then.

Here's a more complex example. I'm going to iterate through the items, the UI drag items inside of my session. I could do different things for different ones if I wanted to. I, now, I'll take, for an item, I will use the NSItemProvider, the lower-level API for loading objects. So I'll load an object of a class. Again, UIImage. You see that my closure gets two things now. It gets an object and it also gets this error object.

The other thing about this is that it's lower level. This calls me back on a background queue. So if I had more processing to do of the data, I could do it right there and then. But also, it's my responsibility to dispatch back to the main queue when it's time to update my UI. So of course, dispatch back to the main queue. Handle that error. So that was the three API essentials: getting the objects to drag, getting a drag proposal, and then actually performing the drop.

Now, we'll cycle back all the way to the beginning of this timeline again and talk about all the other delegate methods because there's a lot more. First, I'm going to talk about it from the point of view of the drag interaction delegate. So immediately after you provided those images, those items to drag, if you don't implement anything else, we will just lift up your entire view. So the drag image, the preview will be a snapshot of your entire view. If that is not what you want, you can customize that preview. So implement drag interaction previewForLifting item and return a targeted drag preview. I'll explain what this means with an example.

Targeted drag preview has really two parts to it. There's the drag preview part and the target part. The drag preview part is just saying, what does the item look like while it's being dragged? And you tell us that by providing us a view. So in this case, I'm making an image view containing an image from my asset catalog. That automatically gets sized based on that image. That's what my thing will look like.

The second part is the target part. Now, you'll note that that's a freestanding view. It's not in the view hierarchy anywhere. So UIKit doesn't know where to show it. You need to tell us with a target. A target is two things: It's a containing view and it's a location within that view. So here from my containing view, I'm going to use the interactions view. This is where the drag is starting. And for the location, I'll use these sessions location in the view. This is where the user's finger is.

I take those two things, I put them together into a drag preview target, and then finally, I make a UITargetedDragPreview with that view, with a default set of parameters, and with that target. That's the lift. The other thing you can do during the lift phase is animate your own UI alongside the lift.

So maybe you've got other UI inside of your view that needs to be dimmed out during a drag. Something like that. You can implement this method -- drag interaction will animate lift with an animator -- and use that animator to add animations alongside. Here I'm just changing the background color of a view.

You can also get the completion of that lift animation. Now, note that, with the lift, there's two ways that it can end. If the user holds down their finger long enough through the whole lift, we'll go all the way to the end. The position will be end. If the user lifts their finger partway through the lift, we'll run it in reverse all the way back to the beginning and we'll tell you the position is start.

As that session begins after the user moves their finger, we'll tell you the session will begin. We will then ask you some questions. For instance, we'll ask you, does this session allow the move operation? And then, as that session moves around, we'll tell you every time it moves. So you can find out the new location there.

Also, during this drag phase, you can add items to the drag-and-drop session. So I've got one touch that's moving that drag around. Another touch can come down and tap on any view that has a drag interaction. It can, it doesn't have to be the original view. It could be a completely different view. When we detect this tap, we will ask the delegate for items to add to the session.

You do that by implementing drag interaction items for adding to session with a touch at a point. So you can hit Test using that point, find out what items, if any, that you want to add to the session. If you've got some, return them. If you don't have anything to add, just return an empty array, and then we will pass that touch along just like we normally would have. And of course, you can customize what the preview of those items looks like.

Finally, when the session ends, the user releases their finger. We'll tell you the session will end with the operation that was chosen. And if it's ending in a cancel, there's a cancellation animation, which you can customize. So implement drag interaction previewForCancelling item with a default and return a targeted drag preview.

You can return nil here, and the item will just disappear in place. You can use our default one and change its target. So if you know where that thing should fly back to, all you have to do is give it a new target. It will fly right back there. You can animate alongside the cancellation animation the same way as before. Finally, when we're done, we'll tell you that the session did end.

If it was a copy or a move, there's no cancellation animation, so we just tell you the session did end and we also tell you when the data transfer is finished. So that was the drag interaction side. Now, let's talk about the drop interaction side. This is a little more interesting.

There's only one drag interaction that's involved in starting that drag. But it can go over lots of different views, and they can all have their own drop interaction with the same delegate or multiple delegates. But in the end, only one of them gets to choose what happens when the user lifts their finger.

So when that drag-and-drop session enters your view, you've got a drop interaction. The first thing we'll do is ask you, can you handle that session? So we implement, you can implement this method, drop interaction canHandle session. And you can't check what data is actually being dragged. We don't give you access to that yet, but you can find the types of data that are being dragged. So let's say you're only interested in images.

You can check session canLoadObjectsofClass UIImage. Or you can do something more specific. If you're interested in specific uniform type identifiers, you can check if any items in that session conform to that specific uniform type identifier. So here I'm using image PNG, if I was only interested in PNG images.

Next, after you've said that you can handle the drag session, we'll tell you when that session enters your view, we'll tell you when it moves around inside of your view -- of course, you return a proposal there -- and we'll tell you when it exits. And these can of course happen multiple times -- enter, exit, enter, exit.

Also, while the session is happening, springloading might happen. So we have built-in support in UIKit for springloading. Several different classes, such as UIButton. All you have to do is say, isSpringLoaded equals true. When that drag-and-drop session hovers over that button, the button's action will automatically run. Or you can set up your own springloaded interaction with its own handler to be called when it's time to activate and add that to any view you like.

Finally, when the session ends on the dropInteraction delegate, we'll tell you the session did end. So every interaction, every drop interaction that has ever seen this session will be told that it's ending. If you were keeping track of it, now you can stop keeping track of it. If there's a drop to perform, of course, there's more to do. We talked about the perform phase. Now, let's talk about the animations and the data transfer.

The animations are exactly the same for drop animations as they were for cancellation animations. Just the API names are different. So now, it's on the drop interaction delegate preview for dropping item, but you can do exactly the same things. You can animate alongside the same way. And finally, when all those drop animations of all the items are done, we'll tell you conclude drop.

For the data transfer, of course, this closure gets called when that data's available. But something I didn't mention earlier: When you start the load of the data, you can get a progress object, and this progress object can tell you how much of the data transfer is done, is it finished yet, and you can even cancel the data transfer from there. Also, you can get those progress objects per item, or there's a progress object on the entire session that wraps up all the progress of all the item loading.

And there you have it. That's the full timeline of drag and drop. That's everything that can happen. You know about these three essential API functions that you should implement to get drag and drop working. And I hope you got the real message here, which is customizing drag and drop is done through these interaction delegates. That's how UIKit and your app work together to make a fantastic drag-and-drop experience. I'll hand you over to Emmanuel, who will show you how to add drag and drop to an app.

Thank you, Kurt.

Using drag-and-drop interactions, we created skeuomorphic pinboard for images. So we can start a drag from photos, for example. We can bring it over the pinboard. And when we drop it, we'll copy the image and show it in the pinboard. Notice the drop animation scales down the image and, when we receive the data in the pinboard, will show the full image. We can also drag it on the pinboard and see that we're going to dim the original image. And we can also drag that back into photos to add it. Hopefully.

Yeah, that's -- this used to work. [laughs] We can also drag it in the same pinboard, and then when we drop it, we will just rearrange it. And see, look again at the animation. We dimmed the original image, and when we drop it, we hide the original image, and then we show it in the new spot. So let's switch back to the code and see how this is implemented.

So we start in our main view controller by adding a drop interaction and a drag interaction to our view. And then, we're going to set the view controller as the delegate of those interactions. Then, we're going to implement a few drop interaction delegate methods. The first one is drop interaction can handle session, and this will return true if the session can load images. It's very simple.

The second one is drop interaction sessionDidUpdate, and here we return a proposal where we specify what kind of operation we want to perform. So we have two cases. The first one is if the drag started in another application, we want to perform a copy operation. And we can check that by looking at the session localDragSession property. So if it's new, it means that the operation, that the drag was started in another application and we're going to do a copy. Otherwise, we are just rearranging the item and we're going to do a move operation.

Third, we're going to implement dropInteraction performDrop session. Again, we only want to copy the data if the drag started in another application. So again, we do this check if the local drag session is new. And then, we're going to perform the drop first by getting the drop point of where the user lifted the finger by calling session location in view. And then, we are going to go over each drag item in the session and call a helper function that will load the image from the item provider, put it in an image view, and then display it in the pinboard using that center point that we got.

We also want to perform a drop animation when we release the finger. So again, we have two cases. In drop interaction previewForDropping item, we want to return a preview that scales down when the drag comes from another application. And otherwise, if we're moving the image from the pinboard, we are not going to apply any effects. We're just going to drop it in place.

So we can check if the item is coming from a local drag or from an external drag by checking this property on drag item, which is called localObject. And if the item is coming from another application, then we return nil. And this will have the effect that we saw before. Just will scale down the preview, and then it will stay in place. The default preview instead will scale down and move to the center of the pinboard, but that's not what we want.

In the case we're moving an image from the pinboard, so we're just rearranging it, we're going to animate it in place. So to get that, we need to retarget the default preview. And as Kurt explained, we need a container, which is the pinboard itself, and the center in that view, which is the center of the preview. And then, we're just going to retarget the default preview and return that.

Next, we're going to implement drop interaction willAnimateDropwith animator. Here we want to animate a local move because, as you've seen before, we had dimmed out the original image view, and when we drop it into a new place, we're going to hide the previous, the image in the previous location and then show it again at the end of the animation into the new spot. So we're going to add an alongside animation and we're going to fade the drag item to alpha equals 0.

And then, at the end of the animation, so by adding a completion block, we're going to set the new center of that image view, and then we're going to show it back by setting its alpha to 1. And here you can see that we're again checking a local object. In this case, I'm passing an index so that I know where in my model object the image is, and we're going to see in a moment in the drag interaction how to set that up.

So I'll go into the drag interaction delegate. I have to implement, first of all, what kind of data I want to pass in the drag. And I installed the interaction on the whole pinboard. So what I want to do is give the image that is under my finger.

So to do that, I first need to get the touch of the drag by calling again session location in view. And then, if there is an image under my finger, I want to get the image, the data for that image, wrap it into an item provider, wrap the item provider in a drag item, and then attach that additional information that will allow me in the drop site to recognize that local drag item.

Next, we need to provide a preview for that item. So we said before, if we don't provide a preview, the drag interaction by default will take a snapshot of the whole view. And we clearly don't want to do that. We just want to provide a preview for the single image that we're dragging.

So we do that by getting the index of the drag item, we get its associated image view, and then we pass that as a targeted preview. Finally, we want to dim out the original image view once we start the drag. So we're going to add a completion block to this drag interaction animator pass in a willAnimateLift delegate. And so we're going to add the completion block, and the lift ended, so if the position is end, then we're going to fade the original item to alpha 0.5.

And that's really it. So you've seen how simple it is to add drag-and-drop interactions. This example also contains, also shows how to implement a [inaudible] configuration that you can use together with your drop interaction, and the sample code will be available online, so make sure to check it out. Back to Bruce.

Thanks, Emman.

Okay, so by now, you guys are just going to run back to your laptops -- I see you already have them -- and add drag and drop to your apps. And so I'm going to just share a couple of words of advice about how to expedite that process. First of all, don't do that right away.

Explore the system. See what we've done. See the types of set-down animations that we have implemented and the types of data that we actually transfer. I think you'll be surprised. And then, go simple. Try to add a drop target, maybe using a pace configuration, and just kind of get a feel for how it works. Experiment with a drag source.

And even try to springload a control or two. And then, you're going to look at it and say, it just doesn't look right. What are we missing? And that's when you're going to have to dive a little bit deeper into these drag-and-drop APIs because there's a lot of them. And we've made this easy for you.

And so I'm going to refer back to this picture again, and I'm just going to start out. There is a talk on mastering the drag-and-drop APIs, which is going to go even deeper into them than what Kurt gave you an overview of. There's going to be a session on what we've done with NSItemProvider, and you're going to find it fascinating. We can drag and drop files, for example, that you can open in place, and you can actually provision the data from your source using a file provider. And we're going to go into that in detail in that talk.

There's a bit more. For example, text view has a delegate, a higher-level delegate for supporting drag and drop. So if you have custom attachments or want some kind of custom text processing, you should explore the APIs on those delegates. And I know many of you have table views and collection views in your apps, and if you want them to support drag and drop, we've created some great higher-level APIs, and there's an entire talk dedicated to that topic which you should really go see.

As usual, there's a lot more information about this talk in particular along with the sample app at this link. Here are some of the related sessions. I added one about the file provider enhancements as well as how you can, one on Spotlight, which'll talk about how you can actually drag and drop items that your, that Spotlight finds for your app. If you have any questions, please drop by. And I hope you guys have a great WWDC.