Good afternoon. I'm Ralaiigh Ledet. I'm a Cocoa Software Engineer and welcome to session 115, scrolling, swiping and dragging, now with more animation. We've got a jam-packed session for you today. So, first off I want to preface that everything we're gonna be talking about is all new stuff in Lion, and of course it's only gonna be appearing on the Mac OS.

So we got a lot to cover for scrolling. We've got a new look to scrollers, and we've got a new elastic scrolling behavior, so we're gonna cover how you can integrate this into your application. We have a new method for doing fluid swiping. You might have already seen this in Safari when you go back and forward in pages, you can track what the user's doing on the trackpad and make it real nice, smooth animation. We also have multi-image dragging, and no longer is dragging a static process.

It's no longer a fire forget where it's just this one single image, dragging is now a static More fluid and has, you can change the image on the fly and it's really nice so we'll show you the API for that as well. So let's get right down to it and start off with scrolling and our new content focused redesign. And to talk about scrollers, I'm going to bring up Troy Stephens.

So as you've all probably have the chance to notice by now, scrollers in Lion have been the recipient of probably the biggest redesign in their look and feel in the history of OS X since OS X 10.0 probably. For many releases of OS X now, we grew accustomed to sort of the trademark Aqua scroller look. The scrollers were this liquid blue capsule on the side of your windows, on the side of your scrollable areas that really was very eye-catching and attention-getting design.

Even if you run in graphite mode and drain them of their color, they're still kind of a heavyweight user interface element relative to some of the other stuff in Aqua, especially now on Lion where everything is really toned down and subdued, everything that surrounds and accessorizes your content. But what are scrollers really? They're kind of this invention that we came up with in the computer industry to enable users who have a mouse to position their content within a smaller visible content area.

So fast forward to Lion, and scrollers are really just that. They're accessories that are just there when you need them. They're much more subtle and subdued, certainly in color. They're more slender in size. They appear only when you're scrolling. And they disappear and get out of the way and put the focus back on your content when you're done scrolling.

So what is there to know about scrollers that will affect your applications in Lion? Well, one of the most important things to know, first off, is that we actually have two new scroller styles in Lion. The first that we hope people will be using in most cases are the ones we call overlay scrollers that I was just talking about.

And these are called overlay scrollers because they're composited atop the user's content area. So within your scroll view, the scrollers actually now go on top of the clip view. This gives the user more potentially viewable content area, you know, instead of having us shave off sides of the potential content area and devoting those entirely to scrollers that you don't need all the time, the scrollers are composited atop the content. Again, they fade out when the user is not scrolling, so they're just there when you need them, and then they're out of the way, enabling you to focus on your content.

Now, the first time we presented... So the first time we presented a scroll view to the user, the first time its window is ordered in or it's added to the window, the scroll view has a behavior of automatically pulsing its scrollers to visible. This is very similar to what's done on iOS. So this gives the user a cue that, oh, there's more to see here in this viewport than I can see right now, and I can potentially mouse in there and touch scroll to scroll that area.

This is done automatically in most of the cases where you'd want it to be done, but you're always free to send a flash scrollers message to your scroll view, and that will trigger the same behavior to happen again. You can send this as many times as you want. And we'll batch those up and just do a nice, smooth flash of the scrollers.

Now, if I put my two fingers down on the trackpad and begin scrolling, the scrollers show, and they stay shown for as long as I'm scrolling, and also as long as I keep that touch held on the trackpad or Magic Mouse. Only when I let up are the scrollers free to hide. Now, if I begin scrolling again, once I start moving the mouse in the content area, I have all the time in the world to get down into the scroller, and I can use all the familiar mouse-based scroller behaviors.

So I can drag the thumb. I can even option drag to find scroll. As long as the mouse stays in the track, both of the scrollers stay pinned to visible. Only when I mouse out are they free to hide again. So on the Mac, you sort of get iOS scrollers now, but it's the best of both worlds because you also have the ability to manipulate the scrollers using the mouse if and when that's more comfortable for you.

Now, in addition to the overlay scrollers, the second style that we have, we call the legacy scroller style. This is really the 10.6 and earlier scrollers. Restyled for Lion, so again, they're also more subtle and subdued looking, no bright blue capsule in there. But other than that, they're metrics compatible with the earlier scrollers and they have all the same behaviors. So these are provided for out-of-the-box compatibility with existing applications, and also to accommodate the user's preferences and any accessibility needs they might have.

Now, users on Lion can always ask for their scrollers to be shown to them always. This can be done in system preferences under the general category, used to be called appearance. If users ask for this, we need to show them legacy scrollers everywhere. It's a preference that they've expressed that we need to obey.

We also -- you'll see these used if you have an application that inserts accessory views into the scroll views margins. This has been a very useful technique for a long time now because if you've always got the scrollers there but you don't need the full length of the space, you can kind of shim in an extra, say, a Zoom pop-up like you might see in text edits wrapped to page mode or other accessory views like that.

If we detect these in there, they interfere with the potential to use overlay scrollers, so we'll fall back to legacy scrollers, but that's a really easy problem to solve, and I'll show you how to do it. Also when we encounter subclassed NSScroller instances that we aren't sure are compatible with Lion's new scroller behaviors, we will fall back to be sure we're compatible to the legacy look and behavior as the basis for those, but it's real easy to show us -- to inform us that your scrollers are Lion-compatible.

So, given that we have these two scroller styles, how do we determine which one -- which style to use in a particular case? Well, as I said, it all goes back to appearance -- to the user's general preferences, rather. The user has a choice in the general settings. They can ask for their scrollers to be shown always. And again, that means we give them legacy scrollers everywhere.

They can ask for scrollers to be shown only when scrolling. So they are explicitly asking for overlay scrollers wherever we can give them to them. We still have to fall back in those two cases that I mentioned to legacy scrollers if we detect accessory views in the way or subclass scrollers that we don't know are Lion compatible. But in all other cases, we'll give them overlay scrollers.

And then the last -- and actually the first option here, automatically based on input device, this is the default for new and existing users. And what it tells us is you decide. It leaves it up to the system We do pointing device detection. We look at all of the trackpads and mice and tablets and whatnot connected to the system. We look at which ones have gesture scroll capability.

And if you have external devices connected, we'll disregard an internal trackpad in favor of looking at the most capable external device to see if we have at least one external device that has gesture scroll capability enabled, then we can use overlay scrollers. The full set of rules and logic is somewhat complicated. If you're interested in it for some reason, you can find it explained in the application kit release notes. But mostly we try to make this automatic and sensible.

So because we have these two scroller styles, because users can change their preferences over time, and because they can connect and disconnect devices over time, your applications need to be prepared to work with both types of scrollers. Both at app launch time and over the lifetime of your process, the preferred scroller style may change. So you need to be able to react to that.

Fortunately, in most cases, you don't have to do anything. AppKit keeps track of all scroll views that are in use. So if you're using scroll views in your apps, which you most likely are, AppKit will automatically find every scroll view instance and send setScrollerStyle to each of your scroll views when the style changes with the new scroller style, either overlay or legacy. The scroll view knows to automatically update itself, update its layout, redraw, and present the new style of scrollers to the user.

But just in case you have any other code that wants to respond to these changes, maybe there are other aspects of your UI layout that you want to change, there's a preferred scroller style method on NSScroller. You can always ask it, what's the system preferred scroller style? Either overlay or legacy. And there's a notification that you can subscribe to to observe for when this changes.

Within the overlay scroller style, just as on iOS with iOS scrollers, we actually have three sub-styles. We have different knob styles for use on different backgrounds. So the default style is really designed to be visible on as wide a variety of backgrounds as our graphic designers could manage. It's sort of a semi-transparent black with a thin white halo around it that tries to optimize for better visibility on darker backgrounds.

But if you have a background that's really pretty dark and you try to use that default knob style, it may be hard for the user to make out on top of that background. So if you have knowledge about the nature of your content and is it light or is it dark, you could in this case opt for the light scroller knob style.

So this is something you would explicitly set. If you set it on the scroll view, the scroll view will tell both of its scrollers, if it has two of them, to update accordingly. So just set this on the scroll view and let the scroll view manage its scrollers.

And there's also a dark style that looks pretty much like the default style. It's a little bit lighter, except without that light halo around it. And you'll see this in play in Safari online. They're actually taking advantage of this. So most web pages tend toward lighter content. They use the default scroller style on those.

But on darker pages, such as this, they will tell us, "Hey, use the light scroller style. We figured out that the background color, it's overall black, so we want to use the light scroller style." And that helps users out a lot when you have the knowledge to be able to do that.

So how do you make your scroller subclasses overlay scroller compatible? Well, that's a great question. Well, it's really pretty easy. You might imagine that there would be a method such as, "Is compatible with overlay scrollers?" that you'd override, and there is. And you could simply override that to return yes, but if you want to be a little more clever, you can use the technique shown here to avoid speaking for any potential subclasses of your scroller subclass in case you're developing a framework.

And the contract that you're agreeing to here is really very simple, and it should be easy. We hope to rearrange any code you have to keep it. The most important thing is, if you do any drawing customization for your scrollers, you need to override the parts-based drawing methods rather than overriding draw-rect wholesale to draw your entire scroller.

And the reason for this is that we need to be able to show and hide the scroll thumb and the scroller track separately. The scroll thumb is visible any time the scrolling is in progress. That's the first part of itself that a scroller shows. The track is only visible when the user mouses into the scroller track area.

So we need to be able to fade those in and out separately. We need to be able to call you to draw them separately. But as far as the crossfade and the behavior, we take care of that automatically for you. You just draw at full opacity. And then similarly for hit testing, override the parts-based methods if you must.

You need to be okay with the fact that scrollers, both overlay and legacy, no longer have arrows on Lion. So if you ask for the rect for part for arrows, we're going to return empty rectangles for those. And you need to be okay with the possibility that overlay scrollers may have different widths and layout metrics than legacy scrollers, which we've painstakingly tried to make have the exact same metrics for compatibility. In practice on Lion, we found that for compatibility, it was best to keep the metrics the same, but in the future, the two might diverge.

Accessory views, the second leading cause of fallback to legacy scrollers besides user preferences and custom classes. So what are the implications for accessory views? Why do we care about these? Well, with overlay scrollers, again, what we're trying to do is give space and attention back to the user's content. So we've got these scrollers now that we composite them atop the content area instead of shaving space off of that clip view.

And we automatically hide them when they're not in use, so they don't interfere with the user's content. But what's going to happen with your accessory views if we don't do anything about them is they'll end up laid out atop an edge of the user's content area. Now, that could be unimportant margin, or it could be important content that the user needs to be able to click on to select or interact with something we don't know.

So we probably need to do something about it when we see them. We can't really leave them obscuring the user's content for the reasons I just gave, but at the same time, if we were to presume to automatically hide them with the scrollers, well, that could be problematic for users too, right? I don't want to have to touch scroll just so that I can get my zoom pop-up to show so I can change the zoom on my document. That's not going to be what I want either.

So therefore, when we see those accessory views just in the right-hand or bottom scroller margins of a scroll view where we think the scroller should be, we'll fall back just for that scroll view instance to the legacy scroller style. So you may see this happen in your applications.

It's really easy to solve by just moving that UI elsewhere. As soon as we don't see accessory views anymore, that scroll view is eligible. It's also possible to use overlay scrollers. A few quick notes about API changes related to this. First of all, if you have ever tried to subclass in a scroller to provide a custom appearance, you may be happy to know that AppKit now consistently consumes our public rec-for-part API, so you can override that to change the layout of elements of your scroller.

The methods and constants for dealing with arrows are of course they're soft deprecated now. You can still call them but they don't really mean anything. And incidentally the control tint we used to have this Aqua versus graphite control tint. That doesn't really affect scrollers anymore for either style. They have the same appearance for blue or graphite.

There are some layout methods, class methods on scroller and scroll view that have been useful if you're creating your scrollers and scroll views programmatically, and you want to pre-flight the layout and figure out how much space am I going to need. For scroller, there's a new replacement method, scroller width for control size, scroller style, that lets you specify regular or small scroller size and also which style, legacy or overlay, because that may make a difference in the future. So the older methods, you can still use them, but they assume regular control size and whatever the system's preferred scroller style is.

Likewise on scroll view, frame size for content size and its counterpart, content size for frame size, et cetera, new replacements that allow for the possibility that scrollers are subclassed, that you may be using regular or small control style for your scrollers, and that the scroller style, again, may be legacy or overlay.

The existing methods, still usable, but they assume no subclassing, regular control size and the system's current preferred scroller style. And that really is all there is to know about scrollers. But there's a whole lot more to know about enhancements to scrolling and swiping and dragging. And for that, I'll return you to my colleague, Raleigh Lede. Thank you. Raleigh Lede: Thank you, Troy.

So continuing with our content-focused redesign of scrolling, we've brought over elastic scrolling, also known as rubber banding from iOS. Since the scrollers aren't visible, we need to provide some more feedback to the user on what is scrollable and what directions that the user can scroll. And you can participate as well in this as a developer.

In a scroll view we have a couple of new methods, horizontal scroll elasticity, vertical scroll elasticity, and their associated set methods. And likewise, in Xcode 4, right in the interface editor, you can select the scroll view and change the settings right there with a couple of pop-up buttons.

So what in a scroll elasticity options do we have? Well, the default is in a scroll elasticity automatic. We have some heuristics in place that try and figure out the look at your document view. How big is it compared to the size of the scroll view? And we err to the side of vertical scrolling. Most things are going to be vertically scrolling oriented. And we automatically figure out which axis we should allow scrolling--scroll rubber banding to happen on.

But this doesn't always work in all cases. Particularly a good example where our heuristics fall down is if your application has a timeline and the timeline scrolls dominantly in the horizontal axis, or possibly only in the horizontal axis. In that case, you would want to set the vertical axis to NS scroll elasticity none. Now when the user attempts to scroll in the vertical axis, you won't get any rubber banding on that axis and the user knows that content will not continue to go down in the vertical orientation, so they're not missing anything.

But likewise you're going to want to set NSScrollElasticityAloud on the horizontal axis. This way, even if your content for the timeline, like when you create a new document, it hasn't created enough content to exceed the bounds of the scroll view on the horizontal axis yet. We will still rubber band on the horizontal axis. And this gives feedback to the user to let them know that in general for this document and this particular view that there will be horizontal content that they will want to scroll to, if not now, definitely in the future.

That's the scroll elasticity settings. We also have another property here, "Users' Predominant Axis Scrolling." And this is an interesting, uh, property. If you look at a traditional scroll wheel mouse, you can only scroll in one direction at a time, typically vertically, or if you know the appropriate modifier key to hold down while you scroll, you can actually scroll in the horizontal axis.

But you can only scroll on one axis at a time. With the introduction of the Mighty Mouse, for that matter, but also, in particular, the Magic Trackpad and the Magic Mouse, we allow the user to scroll on both axes at the same time. And this works really great for certain types of applications, a drawing application, a painting application, or perhaps you're showing a map or some kind of large canvas area, and you get a nice panning effect whenever the user scrolls. It works out really great.

But in other applications, this doesn't work so well. Perhaps some kind of text editing document or just browsing the web, where generally what the user wants to do there is scroll in one direction at a time. And the problem is, the problem with the Magic Mouse and the Magic Trackpad is it's real hard as a user to move your fingers perfectly vertically or perfectly horizontal.

You're gonna have some drift along the other axis. And this is gonna come through a scroll wheel event, and a scroll view will scroll in that direction. You'll see this drift. And if your content-- this isn't the appropriate way to view it, this isn't what your user is going to intend to do, you can set user's predominant axis scrolling to "yes," and we will attempt to treat the devices more like this. So we'll make the user a scrolling only in one axis, and whichever axis they're dominantly scrolling at the time.

Thank you. As I'm sure a lot of you that's installed Line have already seen, when you move your fingers up on the trackpad, your content scrolls up. We want to have the feeling that you're -- the user's actually physically moving their content as opposed to moving the thumb in a scroller. And likewise, whenever you bring your fingers down, the content scrolls in the other direction. This feels really great and we really love it. But for some people, they really, really, really want the traditional way that scrolling worked.

So we have user preference in trackpad and also in mouse that they can set this on content follows finger. And the way we actually perform this is deep down in the system, we look at the user setting and whatever the hardware is doing, because the hardware still does what it did before, we invert the delta X and the delta Y automatically for you and you automatically get the new behavior.

Unfortunately, this doesn't work in some cases where already you were looking at the scrolling delta and figured out what the correct direction is. Perhaps you were doing a volume controller, for example. And scrolling up always increased the volume and scrolling down decreased the volume. And that made perfect sense. And now if the user has content follows finger set to yes, this is completely inverted for you.

In these rare cases, when it's appropriate, you can look on the scroll wheel event for is direction inverted from device and regardless of the user setting, you can now re-invert the scrolling delta and up will always be increase the volume, down will always be decrease the volume. Regardless of what the user setting is. In general, please though, respect the user's preference unless it absolutely positively makes sense for your application and in your specific situation in your application.

We also have some new scrolling delta X and scrolling delta Y properties on NSEvent for scroll wheel events. These supersede the traditional delta X and delta Y because we hopefully--they will have more precise scrolling information in them. And what do I mean there? Traditionally, every time you have a tick of the scroll wheel happens, you want to move what we call line height. You want to move one whole line. In this example, a line would be the one row of the table here.

If you had a tick of the scroll wheel, you would move one line. Well, with the latest Magic Mouse and Mighty Mouse, we have much more precise information that we can give you, and we've been doing this for some time. And with these more precise information, instead of scrolling a whole line, you scroll that number of points in your view, and it's a much more fluid and better feeling scrolling behavior to the user.

And NSScrollView has automatically taken advantage of this for some time, but it hasn't been public. So if you're not using an NSScrollView for some reason and you're looking directly at the deltas, we will now have a public way for you to get to this more precise information. But you can also check the has precise scrolling deltas property. If this returns no, then the scrolling delta x and scrolling delta y will contain line scrolling deltas and you need to multiply by whatever your line height is whenever you do your scrolling.

We also have a couple of new, an additional two properties on NSScrollWheelEvent which is phase and momentum phase. And let's talk about why you might use these. NSScrollView handles precise scrolling deltas as I mentioned. It also handles the phase, the momentum phase, and these are the couple events properties that we rely on to do rubber-banding and elasticity.

So if you're using it in a scroll view, you don't even have to worry about them. But perhaps you have a volume controller and when the user does a quick motion on the Magic Trackpad and this will generally in a scroll view continue to scroll the content with momentum, you don't want to do that in your volume controller.

Would that really make a lot of sense where the user moves their finger a little bit and this extra momentum scroller slams the volume all the way up to high? It's probably not what the user wants. So by looking at the phase and the momentum phase properties, you can determine which scroll events you want to filter out from your -- from interacting with your control.

Likewise, perhaps you have, let's say here, a time code in a cell or a view. And the user starts scrolling in your time code and they can change between the hours and the minutes and you automatically start updating the appropriate subpart of your view and they go into a momentum scroll by flicking on the mouse.

AppKit is going to lock those scroll events to your view. So even if the user moves the cursor outside of your view, you're going to continue to get these scroll wheel events. Well, what part of your time code should you update at that point? By looking at where the mouse was when the momentum phase started, you can then lock it to the appropriate part of your time code and continue to update just that portion of your time code. So regardless of what the user does with the mouse.

A traditional scroll wheel is completely relative. Every time there's a turn of the wheel, we get a single scroll wheel event. We don't know when the user's touching the scroll wheel. We don't know when they stop touching the scroll wheel. We don't have any of that information. So every time the user does a tick on the scroll wheel, the new phase, the momentum phase properties are gonna be NSEventPhaseNone.

And this is one interesting way for you to determine by looking at any scroll wheel event, did this event come from a gesture scrolling device, or did it come from a scroll wheel type of mouse? On the other hand, something like the Magic Mouse, we can tell exactly when the user starts touching it, but no events get sent out at this point because with one finger the user might click. We don't know what they're doing yet. But once they start moving their finger a little bit, we latch on that, ah-ha, the user is doing a scroll gesture. So scroll wheel event comes in, and it's going to have a phase of NSEventPhaseBegan.

And a quick way of looking at this is phase represents the physical gesture that the user is performing, and momentum phase is going to be the virtual gesture that the device does on behalf of the user, depending on the velocity that they ended the physical gesture with. So the user started a physical gesture here. The first event has an NSEventPhaseBegan. This is the beginning of what I call a gesture scroll sequence.

The scroll sequence has a begin, it has some change events, and it has an end. So as the user continues to move their finger across, more scroll wheel events come in, and they will have a phase of NSEventPhaseChanged. It's easy to tell that this is still a gesture scrolling sequence tied to the very first NSEventBegan event that we saw earlier.

At some point, the user is going to lift their finger, and we'll get another scroll wheel event with a phase of NSEventPhaseEnded. And you'll also have a scrolling delta of 00. If this is important to you, you'll -- the scrolling deltas for NSEventPhaseEnded is 0. At this point, the user has physically ended their gesture and life completed. continue.

But that's not quite the end of the story because, as I said earlier, the user might have some velocity when they lift their finger. And when they do that, you'll get that one NSEVENT_PHASE_ENDED event to finish off the physical gesture. It will have its delta of . And then that might be followed up with a scroll wheel event with a phase of none, but a momentum phase of . And this follows the same gesture sequence. You get a . As the momentum dies down, you get a series of NSEVENT_PHASE_CHANGED in the momentum phase property. And then finally, once the momentum has ended, you'll get a final scroll wheel event with a momentum phase of NSEVENT_PHASE_ENDED.

So that's the basics of what's going on with scrolling. If you just use an NSScrollView, a lot of it is handled for you. If you need to look at the raw scroll events, you can determine when a physical gesture is occurring, when a virtual gesture is occurring, or if it's occurring just from a scroll wheel device.

Let's move ahead to fluid swiping. This is something you've seen in Safari. We also have it in preview and in iCal and Quick Look. It's a nice way of moving between pages back and forward and it behaves similar to what you see on iOS where you get the nice one bounce back and you're scrolling with two fingers on a Magic Trackpad and you keep going a second time and you get a nice back and forward with the pages. And if you pay attention, what you'll realize is fluid swiping is just scrolling. That's all it is. It's just a new way of tracking scrolling.

Anybody that's using an In-a-Scroll view, this should look a very familiar responder chain to you. You start off with your Document View, and a scroll wheel event is going to be hit tested to your Document View. That's when the cursor is over. And your Document View doesn't do anything with the scroll wheel event.

It just gets past the responder chain, gets to the In-a-Scroll View, and the scroll view does scrolling, it does rubber banding, it does everything it needs to do, and it eats the event. Well, if fluid swiping is just a new way of tracking scrolling, we need to get the scroll event to your View Controller so that you can perform a swipe.

And instead of changing properties on the scroll view directly, in your View Controller, you just override this new In-a-Responder method, "Want scroll events for swipe tracking on axis?" And depending on the axis and what you want to do, you return "Yes." And when you do that, if we re-look at the sequence, the scroll event comes to the Document View, comes up to the scroll view, scroll view determines if it needs to do the single bounce, or if you're already at the edge, what is the user preference, it looks at all of that, then it might ask of the responder chain, "Get into your View Controller, "Want scroll events for swipe tracking on this axis, "horizontal or vertical?" And your View Controller is going to say, "Yes, I want to track a swipe." And the scroll view will then go ahead and forward that scroll wheel event right to your View Controller, and you can go ahead and start tracking it as a fluid swipe.

But that's not quite the end of the story. Turns out we have a user preference for this. The user can actually turn this off if they don't want to perform fluid swiping with scrolling to go backwards and forwards in pages. Well, NSScrollView, and it already looks at this user preference for you and it respects this user preference, and it will not forward scroll wheel events up the responder chain regardless if you want to track it as a swipe or not.

But that's not the only way that your controller might get a scroll wheel event. If you're just, if you're tracking this in just your generic window controller, you, scroll wheel events might come from the sidebar if the user is over a sidebar or a toolbar or somewhere else that there isn't a scroll view. And performing a swipe might perfectly be acceptable while the user's cursor is over the sidebar.

But in those conditions when your scroll wheel event is called, you will need to check to make sure that it is, it does have a phase that is not equal to none, so that it is coming from a gesture device. You don't know when a regular scroll wheel begins and ends, so we can't track that as a fluid swipe. And also check the user's preference with this new category method on NSEvent.

Is swipe tracking from scroll events enabled? So this is sort of a scary block of code here. I want to point out that-- don't panic-- that we have sample code that I'm going to show you in a little bit. So I'm just going to point out key portions of this. And this is kind of some stripped down coding and compressed. But when you get a scroll wheel event in your view controller, what you want to do is, on that event, call track_swipe_event_with_options dampen_amount_threshold_men_max using handler.

And then you pass it a block. And we return a bunch of properties. And this block is going to get called back. And it's going to do all the tracking for you. And it will have a consistent way of tracking a fluid swipe across all applications, as long as you do this.

So you don't have to track it manually and worry about the end of the phase of the event from the scroll wheel directly. Instead, we get you this nice block that we call back as we continue to track the scroll wheel events for you. And the first thing you see in there is We have a gesture amount.

So when you're tracking a fluid swipe, you start off at zero and you're going to go to 100% or negative 100% depending on the direction. And we give this to you in floating values between zero and one or over to negative one. And there are actually cases where you can get, you can go past one. We'll get into that in a little bit. Generally all you have to do is just move your overlay content appropriately. It's supposed to be 50% swipe, you move your content over 50% of however wide your view is.

We also have a phase value that's passed into your block every time we go into this block. And just like a scroll wheel sequence, you have a sequence here as well. You're guaranteed to get an NSEventPhaseBegin to start things off. And at that point, it's a great time for you to go ahead and show your overlay.

If you get an NSEventPhase ended, this is the physical end of the gesture. Go ahead--it means it completed successfully, so go ahead and update your model--your data model. If you get an NSEventPhase canceled, this is another form that the user has physically ended the gesture, but it didn't end successfully. Perhaps it didn't swipe far enough or fast enough, or they started going in one direction and started going back in the opposite direction.

For a number of factors that we will deal with in heuristics for you, it wasn't a successful swipe. It was a cancel. If it's a cancel, you don't need to update your data model. You don't need to do much of anything. But again, this isn't quite the end of the story because even though the user's physically ended the gesture, we need to continue to call back to your block with the appropriate animation curves in continuing to modify the gesture amount to go to 1 or -1 if it was a success or back to 0 if it was unsuccessful. So we will continue to call back into your block, and the phase will be none, and at some point, the isComplete property will be set to yes.

The isComplete parameter of the block will be set to yes. This is the last time that your block is getting called. After this, the block memory is going to be released for you, so go ahead and hide any overlay content that you have to go ahead--that is animating the fluid swipe. And with that, let's get a quick look at the demo.

So this is Picture Swiper. This demo should be already attached to this session, so you can go ahead and download the project. And I'm just using two fingers here. And you can even see I get a nice little bounce effect going, and if I were to start at the beginning, it automatically dampens the amount for me as I get to there and I have no more content to go. I'm going to zoom in a little bit here. And you can see I can scroll around. I can get to the edge.

Get to the edge and I get the one bounce and then it goes ahead and continues to scroll. This is more of a photo style that you might have seen on iOS. We also have this stack style that the sample code shows you how to do which is more like Safari does as it goes back and forward in pages in Safari.

And here's the project. What I want to point out in the project is-- oh, wrong project, sorry. Here it is. What I want to point out in the project is we have a stack swipe controller and a slideshow swipe controller. And these are the two controllers that do one style of animation or the other. And the other thing I want to point out in here is You'll see in here we have this concept of updating layers cache. It turns out that fluid swiping is very, very performance sensitive.

As soon as the user starts to perform a fluid swipe they need to have immediate feedback visually on the screen. And if you spend time creating a view, making it layer back, generating thumbnails or high resolution images of whatever your content is and getting that across to the video card and to the layers, this can severely impact that initial showing of responding to the user and it can make for a really bad feeling for the user if this is not immediately responsive. So this project shows you one way of handling that by caching your layers and keeping them up in sync. And so as soon as swiping starts we can have an immediate feedback for the user. So download the sample code, check it out.

So let's go ahead and move on to multi-image dragging. You might have seen this in the finder already as you drag around. We'll go ahead and animate to some final destination and as you go across to a different destination we can change the dragging contents so that it looks more like where it's going to end at the final location.

and this has three main benefits. One, as we have an image for each individual item that you're dragging, we can animate them independently of each other, and they can move slightly offset from the cursor so we can see what's underneath the cursor in some cases, if that makes sense.

It provides great feedback to the user, so they can see what's going to happen when I drop in this view. And third, it provides a means for you as a developer to interact with the dragging, both as the source-- it's not just a static image that you started with-- as the source, you can change the dragging image. And even as a destination, you can participate in the drag and change the dragging image as well to give the user better feedback on what will happen when they drop in your view.

And the way this works is we have a new concept of an NS dragging item, and there's the source and the destination, and we want to animate from the source set of images to the destination set of images. And it's exactly that. It's a set of images, and they're named in this case. And so we'll take the source icon and animate that to the destination icon image, and we'll take the source label image and animate that to the destination label image. And the animation looks a little something like this when it's slowed down.

In Snow Leopard, this is how you would start a drag. You would create a pasteboard, you would clear its contents, and you would write your pasteboard writers onto the pasteboard and you'd set the pasteboard all up. And then on some view you would call drag image at location offset event pasteboard source slideback.

In Lion, it looks a little bit like this. On your SumView, you say, begin dragging session with items, event, source. And unlike Snow Leopard, where a drag would start and it would run off, and when the drag was completed, it would return to that portion of your code and continue executing from there.

Begin dragging session with items, event, source returns immediately and it returns a dragging session. And at this point, you can continue to modify the properties of the dragging session. In this case, animates to starting position on cancel or fail, which is equivalent to the slide back parameter of the method in Snow Leopard. The drag actually starts for you and is handled asynchronously on the next turn of the run loop.

So when BeginDraggingSessionWithItems is called, you need to supply it a set of dragging items. And an array of dragging items-- there's the NSDraggingItem that we covered briefly earlier. We're going to go into a little more detail. And dragging items have to have an item. And the item can either be an NSPaceboardWriter or an NSPaceboardReader, depending on its context. In this case, we're creating it. So it needs to be an NSPaceboardWriter. And that means that it has to be an object that conforms to the NSPaceboardWriting protocol. A lot of objects already in the kit do this for you, such as NSString or NSURL.

And then this dragging item has a frame. And the coordinate system of the frame is dependent on its context and where the dragging item is used. And now you'll see some examples of that as we go along and I'll point that out. And then it has an array of image components, which is what I showed you earlier, where these are the named images that make up the complete image for this drag. And now we can animate these items, these individual items independently of each other.

Image components can't be set directly. That's why there's the little double asterisk there. You don't set them directly, and I will show you how to set those, but they have a name and they have a frame, along with their contents, and their frame is in the coordinate system of the NSDRAGGING item itself.

It's sort of like an NSView parent-child relationship. So the coordinate system for the image component frames is the bounds of the NSDRAGGING item itself, and this turns out to be really easy to set it up and maintain it. And every NSDRAGGING item comes with a limited lifetime guarantee. What I mean by that is I guarantee that NSDRAGGING items have a very limited lifetime.

You'll see this is a reoccurring theme in the coming up slides and I will be sure to point out exactly where this limited lifetime takes effect. So this is how you might go ahead and create an NSDragging item in code. NSDragging item Alec, a knit with pasteboard rider. The only way you can create an NSDragging item is with a pasteboard rider.

And then you go ahead and you set the dragging frame and the contents of the image. This is actually a convenience method, set dragging frame contents. What this does is it sets the appropriate dragging frame and it sets one image component that contains whatever content you supplied, whatever image, and it maps directly to the dragging frame that you set in there. And it's automatically given the name of icon for animation purposes.

You really only want to do this if you're dragging one item that contains one image, maybe a handful, two, three items. Because when a user starts to do the drag, you want to respond to that right away. If you have thousands of items and you're trying to generate thousands of images, that's just going to be too slow and have a lot of lag time for the user.

The preferred method for setting up the image components is to provide an image components provider. And this is a block that will be called back at some later time during dragging. And it is at this point that you build your components array. And you do that by simply calling NSDraggingImage component, dragging image component with key. In this case, the first one I'm creating is the icon. So I pass an NSDraggingImage component icon key.

We also define the NSDraggingImage component label key. We think icon and label are going to be the two most used names as you drag between applications. So we can animate from one item to the other fluidly and seamlessly. You go ahead. And you set your frame. And then you set your contents. Even though you're doing this in an image components provider, please update your contents here as quickly as possible or use some cache image if you already have that.

And then you add that to your components array. And then you just continue to do this for however many items that you want in your NSDragging item, however many images that you want. The key is just an inner string. Like I said, we've defined icon and label. You can put any string you want in there, like perhaps background. And we have sample code that I'm going to show you in a little bit. And that's exactly what I do is I add a third image component that's named background.

Once you have your array, your components provider block will return that. Now one of the reasons we're using a block is, say that the user is trying to drag thousands of items, or even just hundreds of items from your application. If we were to call in every single image component provider for that, that would take a long time to generate all these images. And they're not all going to be probably shown on the screen at the same time anyway.

So we look at a variety of heuristics, and we determine which of the dragging items, since you always return the dragging item frame that helps out, are the ones we're going to display on screen. And given some other heuristics, we only call in a subset of the image component provider blocks that you've passed in. It's much cheaper and much faster for you to provide 1,000 image component provider blocks than it is for you to generate 1,000 times however many images in each dragging item you have. So we do this for performance.

So back to our Begin Dragging Session with Items. We've created our collection of dragging items. One thing you might be wondering is from the Snow Leopard example of the code, we started off by creating a pasteboard. In Lion, we haven't created a pasteboard yet. And that's because when you created these dragging items, you set the item as a pasteboard rider. Begin Dragging Session with Item will go through these and automatically create the pasteboard for you and put the items on the pasteboard right out of your dragging item since the pasteboard rider is automatically for you. It works out great.

But right after you call begin dragging session with items and a dragging session is returned, the effective lifetime for those dragging items is over. Don't bother accessing the dragging items in the array anymore. There's still valid objects. They still have a retain here, so you should probably release your array. Actually, that should be an auto-released array based on that method name. And they'll be auto-released when the pool is released. But if you modify them at this point, they won't affect the drag. So effectively, their lifetime is over. Don't access them anymore.

Anybody that's written a dragging source, you've probably seen these category methods. They've been upgraded to a formal protocol. These are all optional, of course. And we've added a dragging session parameter where appropriate in the names. So when you get callback, for example, began at point or moved to point, you have the dragging session and you can modify properties of the dragging session or even enumerate and modify the dragging images.

Likewise as a destination, these are the category methods we've had before. They've been upgraded to a protocol. And we've also declared that the sender of these methods, they conform to a new protocol called NSDraggingInfo so that you know definitely that sender implements the NSDraggingInfo protocol and you can easily find out and get auto-completion in Xcode on what methods sender implements. We did add one new method called updateDraggingItemsForDrag, which is really important.

And let's take a look at a typical drag. In this setup, we have, on the left, an AcceptsDrag. On the right, excuse me. On the right, an AcceptsDrag and a userDestination. Both of these views can accept this drag. But in this example, the user's going to drop it on the userDestination view.

So the user starts the drag, it enters the AcceptsDrag. And that view is going to get a dragging entered. And you might think, oh, dragging entered this view, now would be a good time to update these images. No, you don't want to update them here, because the user's going to end up over the destination view.

And if you change the image here and change it as the user goes into the destination view, you're going to have a lot of animations going on. And it can get distracting really fast. Think of dragging across a couple of Finder windows. You're going to go across the desktop, over the sidebar, over icon view, over the desktop again, another sidebar, and maybe now into a list view. You're going to have a lot of animations happening. And it's going to be very distracting for the user.

So now's not a dragging entered. So this isn't a good time to do that. You might think, well, during dragging updated, I'll look at the mouse and figure out how fast things are moving, and I'll try and decide something. You don't need to do that. We're going to take care of that for you. So in dragging updated, don't worry about changing your images yet, because in this example, the user has exited.

and has immediately entered the other view, and that view gets a dragging entered, and as we've already discussed, we're not gonna do anything. You'll get some dragging updated, and the dragging manager is doing heuristics, uh, behind the scenes, and it just-- finally, at some point, it's gonna determine that this view is a probable drop destination. At that point, you're gonna be called, as a destination in this new method, "update dragging items for drag." It's at this point that you wanna go ahead and enumerate all of the dragging items, update their images, and we'll get a nice image animation transition.

And the way you do that is with this method on NSDraggingInfo. Drag info, enumerate dragging items with options for view, classes, search options using blocks. I also want to point out that this exact same method also exists on NSDraggingSession. So if you're doing things as a dragging source where you have a dragging session, you can also enumerate the items and change their images there.

Right away in this block, you'll get called back on the block for each dragging item. You'll see that I'm just typecasting the dragging item item as a URL. In this case, whenever you're enumerating dragging items, this is a case where the item is actually a pasteboard reader. What's going on here is you're actually enumerating the pasteboard, not the original dragging items that you began the dragging session with, because this might be a completely different process as a destination. So you're actually enumerating the items on the pasteboard and the data that is associated with it. We will go ahead and create a dragging item on the fly and use it in the block right here for you.

Now I can do this type cast because when I wanted to enumerate these dragging items, I said, well, these are the object classes that I accept. This is very similar to what you're going to see in NSPasteboard for reading objects. You read objects for classes with some options. And these are the exact same classes and options that you get there.

And in this case, I just said what I accept is an NSURL class. It's the only thing I've accepted. So in my block, I know that the dragging item item is going to be an NSURL. An NSURL is a pasteboard reader. I can go ahead and do this type cast.

At this point, create your dragging image or better yet, create an image components provider block that's going to create the dragging image for you. Do it quickly. Even in the image component provider, do it as quickly as possible. And determine what your new frame is. The dragging frame I mentioned earlier was, its coordinate system was dependent on where it's used. When you're creating a dragging item and you call begin dragging session, at that point, the view that you're calling begin dragging session on, The dragging items, dragging frames are in that coordinate space of that view.

In this case, it's whatever view that you passed in for the enumerate dragging items will automatically set up the dragging frame of the dragging item inside that coordinate space for you. If you set nil, we'll just use the screen coordinate space. Again, I'm for, so everything fits on the slide. I went ahead and used the set dragging frame contents convenience method, but in general, try and use the image components providers where possible, particularly if you're dragging lots of items.

As I mentioned, we create a dragging item on the fly as we're enumerating, from the paste board and from the associated data. So this instance of the block, each time it gets called, that is the effective lifetime for that dragging item. You don't want to retain it and try and use it outside of the block.

And if you create a components provider, you don't want to access the dragging item directly, because that will cause a retain on it inside the components provider block, which is going to get called outside of this enumeration block, and at that point it's effectively no longer valid. So always pull the item, the information that you need out of the dragging item first, and then use the information you pulled out in your image components provider block. So this is the effective lifetime, limited lifetime for the dragging item when you enumerate.

There's always been the prepare for drag operation, perform drag operation, and conclude drag operation methods have been there for when a drop occurs. And these always got called back. But now they have some new meaning applied to them. We want to have a continuous fluid drag and drop all the way to their final location of whatever the user is dragging. And you can participate in this during prepare for drag operation. What you do is you set animates to destination equal to yes.

And then when we call perform drag operation on your view, you not only update your data model, but we've set up an NSGraphics context for you with the appropriate duration, and you want to animate making a gap. You also enumerate through the dragging items one more time, and you set their final locations.

You enumerate with your view as the for view property, so you just determine where their frame is inside your view, and you set that as the dragging frame. It's really as simple as that. And then when we get back from perform drag operation, you've animated the gap. You've told us where to drop the drag image, so we move the drag image into place, and we call you back with conclude drag operation.

At this point, the drag image is actually still on screen, and you've actually just drawn a blank spot. At this point, you update your drawing model to start drawing the actual data in the right place. And you set your view as dirty. When we return from this, we're gonna drop the drag image from the screen, and we'll refresh your view if it's marked as needing display.

And if the drag image that you gave us matched exactly what you draw in the view now, it will be a completely smooth transition. The user won't see any flicker, won't see anything happen. It looks like what they were dragging just dropped right into place, seamless and very smooth. So let's give you a quick demo of that.

So this is the multi-photo frame demo that's attached to this session. I'm going to drag a couple of images here. And you'll see they have automatically updated the dragging image to what they're going to appear when I drop them, and when I drop them, they animate right into place. I can even drag another image in here. And you see this one's going to go into a landscape orientation. And my content's animated as the dropped image, the dropped file was animating into place. And everything was smooth and seamless.

It works great. And even as a destination, I might start dragging this item and as the source here, as I leave the window, I change the dragging image so it looks like a window. So when I drop, I get the window there. Or if I come back in here and let go, it'll slide right back to where it was supposed to go. So these are a couple of things that the, you know, Project shows you how to do. I'm running a little low on time. Close the picture swiper. So here, the source code's got lots of comments in here for you.

To move to point, this is the case where as a dragging source I'm watching the mouse and I get the point right out of, right from here and I convert it from its screen location. And this is where I determine when it goes in and out and I go ahead and enumerate the dragging items and set up the window drag image.

I just go ahead and call the convenience method because I have the one image there. When you come back into the window I go ahead and enumerate the dragging items again and I set back up their original images and the dragging controller Here it is. The drag controller set up at the image component providers will actually bring back the image component, will actually generate the image components for me.

And this is the interesting part I wanted to show you real quick. Here's where I'm changing that animates to starting position on cancel or fail, depending on if the cursor is currently in the view or not. Dropping on the desktop isn't a valid drop. I kind of faked that in this example because I wanted to display a window.

The finder isn't accepting the drop, but I want to display the window, so it's always going to be a cancel. If it cancels on the desktop, I don't want to slide back the animation. But if I'm in the view, I want to go ahead and toggle the animates position on cancel or fail back to yes so that it does slide back into position. So this is an example of how you can modify the dragging session properties during the drag.

So multi-image dragging, there's a lot to it. And it turns out that we can do a lot of this for you in some cases. And in this table view in the delegate methods there, there are some APIs-- and particularly in the view-based table view, if you wire things up appropriately in your nib, it makes it real easy, it makes it a breeze.

Corbin's giving a talk on view-based table views later, and I encourage you to come. He'll show you how easy it is to do multi-image dragging in table views. Likewise, NSCollectionView also has a similar set of delegate APIs as the view-based table view to make doing multi-image dragging very, very easy for you to do.

So that's the talk. We've covered the new content-focused scrolling. We looked at the redesign of scrollers and how you can participate in that, particularly if you have excess reviews or if you are providing your own scroller subclasses. We've looked at fluid swiping, which is just a new way of tracking gesture scroll wheel events.

Please use the track swipe method to do the tracking. That way, all applications will get the same animation curves, will get the same heuristics applied for determining if the user's gesture was a success or a failure. And we ended it off with the new multi-image dragging, which is going to provide lots of great feedback for the user and provide some great new capabilities for both source and destinations of drags.

We have full screen and animation changes. There's another related session. This one has already occurred. So if you're watching this on the video, you might want to check that session out as well. We have view-based in its table view, basic to advanced that Corbin is giving. This is the one I alluded to a little bit earlier. It's tomorrow at 10:15 AM. I highly recommend that you check that one out.

Bill Dudney's Application Frameworks Evangelist. Please read the application kit release notes. We've got information on the heuristics, on automatically choosing which scroller style to use, and also more information on fluid swiping and multi-image dragging and the heuristics that are used there as well. And always check out the documentation. Thank you all, and I hope to see you in the lab.