Hello! And welcome to the strawberry-themed Core Animation session. The animation you see in the background here has not been made with Core Animation, but it could be. I bet it's like 500 lines of code. Anyway, yeah, that's me. So let me talk a bit about the entertainment for the coming hour. So I will give you a brief introduction of what Core Animation is, an architectural overview, what is in the system, and what are the key components that make it work. And we're going to talk about layers and their APIs and the animation model.

And towards the end, we're going to talk a bit about how it fits in with the higher-level frameworks like AppKit and how Interface Builder helps you to get to these features a bit easier than writing code. Kind of the motivation. So it turns out that doing these composite animated dynamic UIs is kind of hard. And what you think about-- somebody got that joke. OK.

So if you think about, say, the Keynote Canvas that you're seeing here, or iChat U+3 where you see a picture here, or Front Row, you tend to have a lot of images and text moving around. You try to get smooth animation and seamless compositing and these kind of things.

And you have to do quite a bit of work to get that. So we thought, well, that is the kind of type of work that the OS could do for you. Let me give you a short demo to give you a kind of visual of what I'm talking about.

Okay, so this is kind of a toy app. It's a recipes browser. So you select a dessert, then move over, and then scroll through your list of something, and you scroll further, and this kind of thing. And you notice it's smooth animation. There are definitely several layers being composited here. That's all for the demo for now. Just keep that image in mind as I'm going forward.

So a bunch of examples, yeah, you can read those. It is used in Leopard for a slideshow, the time machine, the workspaces, preferences pane, and the composition picker, the image, what is it, picture taker panel now? These kind of things. Here's kind of the architecture diagram how this all fits in.

So Core Animation sits primarily on top of Core Image, and which in turn then sits on top of OpenGL, and some of it sits on top of Quartz. So this is kind of an interesting point that Core Animation, every single frame, will build a filter chain in Core Image that then renders its output. On top of Core Animation, there are essentially the clients. So the dot dot dot is your app.

Cocoa sits on top of it, QuickTime sits on top of it, and as I mentioned before, a bunch of system apps are using it. Okay, so this is kind of the goals for the framework slide. So the goal is really to get high-performance compositing animation, which really means 60 frames per second, or whatever the refresh rate of your display is.

It's also important to have something that one can, you know, is in kind of a manageable state to work with, and for that we chose an NSU-like recursive abstraction. So you have a layer and it has a bunch of sub-layers and it kind of does what you expect.

So if you move that layer, all its children layer move with it, these kind of things. Also important is to have an abstract interface for animations. And what that really means, you don't have a callback that says, "Okay, what do I do this frame, and then this time, this time, and once more." So nothing that really runs at 60 frames per second, you want to have a sufficiently abstract representation of that, which is, "Okay, this object needs to move from A to B, and it takes half a second." So the key message on this slide is the animation is done for you by Core Animation.

And it happens on a separate thread. The key there is if you do animation in your run loop, well, you're kind of at the mercy of the other workload that the run loop has to do. Every now and then an application goes and hits the disk and reads the database and stuff like that, and that's really not good for smooth animation. So that's why all the animation happens on a separate thread.

So how to get performance? Well, first rule is minimize redraw. And what I mean with that is the layer has to produce content. Well, there's a function being called, and you're doing your drawing essentially. And the key is you do that only once. Essentially, you get a key when the layer content is needed for the first time, you get that callback, and from then on, The contents of the layer is stored in a buffer, so all the animation can happen without application code. That ties into my second bullet point here, which is to minimize application code during resize and re-layout.

If you resize your window, most of the content actually doesn't need to be redrawn, it just needs to be repositioned. Imagine you have a title and the title just slides, it has to be centered in the window, but other than that, having bits for the title is perfectly sufficient.

So in the case where essentially the application doesn't do anything special, so it doesn't process an event, then essentially there should be no application code running at the frame rate. So everything has been described in advance, all the motion and so on, and the application just does nothing and waits for events. And all these together then ensure a consistent frame rate, well, as you can expect. So let me talk a bit more about how that actually happens under the covers.

So first, there is a concept we call the layer tree, which is the representation that the application sees. So you have layers, you have sub-layers, and you have the nice visual here in orange. And that's the application-exposed API. So this is the data structure you work with. And it has this concept of implicit animation and implicit layout, which I will explain a bit later.

Kind of the man behind the curtain is the green tree here, which we call the render tree, which is a copy of that data structure. And that's the data structure that is being traversed at 60 frames a second. And it all does the explicit animation and is managed by the runtime. So your application will never see the render tree, but it's kind of important to know that it's there. Because this is what it does. So we have a kind of a visual here. On the left side, imagine this is your code.

And you want to change the position of a layer. So you essentially say, set frame origin to some xy value. And that is instantaneous from your application's perspective. So your layer data structure has updated. It's now at position xy. But the render tree is doing the animation from the original point to the new location.

Similarly, if you add an animation to the new location, it's going to do the same thing. So if you want to remove a layer, what happens is there is, for example, a fade-in animation going on. And the third example I have is if you replace an entire subtree with a new subtree, you get a transition effect.

So, the first bullet point is kind of the most important piece of the entire session. If you change the properties of your layer, it triggers implicit animation. And the key is, On your side, that change is instantaneous, and the fact that there is animation going on happens completely transparently to you.

So there's also an ability to batch multiple animations together. Essentially, you say something like "begin transaction" and add two layers, remove two others, and change the content of a fifth one. And then you say "commit transaction" and then all of these animations happen at the same time. So this ensures that even though all the rendering is done on a second thread, that they're never part of a partial update visible.

I'd also like to make the point that layers are lightweight. That kind of sounds odd if you actually have bits around representing the bitmap, but what I mean with that is the actual layer data structure is around 100 bytes. And the reason for that is you can actually build systems that have fairly large numbers of layers, like in the thousands. So, the example I just showed, the recipe browser, that's in the dozens, but the example you saw in the keynote with the album art building the city, it's about 6,000 layers.

So, at that point, even though most of these layers reference the same bits, there aren't actually 6,000 albums in that demo, but the state of the layers needs to be present, so having that small is kind of important. And yeah, all of this is configurable. So when you set the position, you get the slide animation for free by a week. But you can specify durations, timing curves, motion paths, transition types, and these kind of things.

And yeah, all of this is configurable. So when you set the position, you get the slide animation for free by a week. The entire scene is described as a Core Image processing graph essentially, and that in turn sits on top of OpenGL. There is also a second render implementation available, which is essentially rendering into a Core Graphics context.

What that enables is you can render your scene into a PDF file, which then is your spool file format, and you can actually print this stuff. There's a caveat here: Core Image filters at this point cannot be expressed in a PDF, so these layers that have Core Image filters on it will get rasterized.

Okay, a word about the compositing model. It's kind of what you expect. It's back to front. So if you have a bunch of pure layers, the first one in the array is the one that goes to the back, the next one goes on top of it, and the next one goes on top of the other two, and so on. And because everything is sitting on top of Core Image, you can add filter effects.

There are actually three different hooks on the layer data structure to add filter effects. Let me give you an example of those. So the most obvious one is the content filter, which is a filter that gets run on the content of the layer. And so I tried to visualize that here by having a text layer which has a distortion on it, and underneath there is the flower picture layer.

Similarly, there is a background filter, which is the filter that gets applied before the layer is composited into the scene. It's reasonably visible on the screen. An example here is you can put a blur underneath that Seattle 2002 badge to essentially reduce the contrast of the scene underneath and give it a bit more polished look.

And the third filter is the compositing filter itself. So you specify how the contents of your layer is merged into the rest of the scene. And most likely you're going to use the default, which is Source Over, which is good for 95% of all cases. There are some cases where it can change, and I have an example here. So if you want to add this mirror flare, then this is an object that kind of emits light. So it's better to use the addition compositing mode instead of source over.

Okay, in the keynote you saw a lot of 3D, which was kind of misleading. So although you can 3D, and let me talk a bit about what that means, the concept is really a layer can have a transform, perspective transform. So it's the concept of you have a bunch of planes stacked, somehow arranged in space. And Core Animation will do the depth sorting for you. However, it will not handle layer intersections. So if you have two layers that intersect, then you will get corrupt rendering. Thank you.

That essentially makes sure that transparency works as expected, but I'd really like to stress the latest point: it's not a 3D engine, it really works with layers and not with 3D objects. There is no Z-buffer, and if you're trying to implement Doom 3 with this, you're probably going to be in trouble. With that, I would like to go back to the strawberry demo.

and explain a bit more about what this application is actually doing. So at this point, what you notice, there is a selection layer which has this little pulsing glow on it. So this is the CI Bloom filter that is running there and there is an animation set up which just pulses it up and down every once a second. So the application is not doing anything once it's set up that animation, it's just running by itself.

What the application is doing is waiting for events. And if I hit the arrow down key, all it is doing here, it is setting the XY position of that selection layer to be on the new item. So the animation that's happening is implicit, it's done in the render tree and as far as the application is concerned, as soon as I hit that key, the layer is at the new position.

That allows you to do things, let me add the shift key here to make a bit of slow motion. So if I actually Q up two key upstrokes or QD two key downstrokes, you see there's a single animation going on because from the application perspective, it set it to the second position and then right after that set it to the third position. While the render tree is interpolating from the current position to whatever the goal state is so it will slide two elements up in this case.

Similarly, if I move sideways, what's happening here is there's a transition effect going on. So there's a subtree which makes up these layers, and there's a subtree which makes up these layers, and essentially all I'm doing is I'm replacing the original subtree with a new subtree, and yet transition animation is set to slide. That's how you get this effect. Note that the title actually is doing a crossfade.

Let me do that in slow motion to make it obvious. So actually two different-- Transition effects are set for that single keystroke. And one interesting detail, the slide animation needs to know whether it has to slow the slide left or right. So that's what happens as well. So once we get the keystroke, we set the transition to be left or right to give us the illusion that this is actually some kind of spatial arrangement.

One pane over, what we have here an image with alpha, we have resizing, we mentioned that. So I can just take this window and go resize it and you see text is reflowing. And what's happening is most of these layers don't get redrawn on a resize because in fact the title just moves to a new position, the image just moves to a new position and so on. It's the center layer which has the text which needs to reflow. So that one gets an event to redraw and everything else is, oops that was an interesting glitch. So everything else is just working.

Last item I would like to show is you can also put movies in layers. So, I don't know, it's a masterpiece. So note that the movie has rounded corners and they're nicely anti-aliased, so you can add a mask to a movie and essentially things just happen as they should. It's going to go on like this and it's not particularly interesting. So with that, the demo is over.

I'm going to invite John Harper up to talk about a really important thing, which is code. OK, so now we're going to talk about the APIs involved here. So the key thing to remember is that core animation is basically a property-based kind of data-driven model, which means that you set up this tree of these objects, which Ralf was talking about, and you set a bunch of properties. Maybe set some animations up and let it run.

So you've probably seen the new property stuff in Objective C2. And we take a huge amount of use of that. And that basically means you can say let or something instead of using these weird brackets. And the other important thing is, since we have this database model, we have kind of the idea of a transaction.

And a transaction basically is a way of batching up changes to the render tree such that you never see kind of little partial updates or anything like that. So the idea is that you basically have one transaction per event loop, or per run through the event loop. And then everything is committed from the layer tree to the render tree in one go.

And you just see this kind of nice atomic update. So you can see that you can actually run through the event loop. And then everything is committed from the layer tree to the render tree in one go. And you just see this kind of nice atomic update. on screen. OK.

So now we're going to go through basically the different things that make up a layer. These are all basically Objective C properties. And they break down into a few groups, which we're going to step through. First of all, you give your layer some geometry. And this is pretty similar to NSView, where you have a bound direct.

In our case, the frame is slightly different in that it's based as a position and a transform. And the position is basically a point in the super layer where the layer will show up. We also allow you to have kind of rounded corners and this kind of 4 by 4 matrix. And this OK transform is basically like CG Afro and transform, except it's a 4 by 4.

OK, so once you've set up the geometry, if you think of these kind of multiple components forming the Les image, the next point is the background. And this is an arbitrary CG color, so you can have things like shadings, which are gradients, or patterns, or just solid colors.

Once you have the background, on top of the background, you get the content. And typically the content is a CG image, or a few other types of object. This little guy here is too small to fill his actual bounds. So we've aligned them to the bottom right-hand side of the bounds.

And we've done that through this contents gravity property. And there's a bunch of options. You can just align it, or tell it to resize, or whatever you feel is the right thing. So as Ralph mentioned, it's a fully recursive compositing model, so every layer has a set of sub-layers.

And you can either just set this array directly by saying layer.sublayers equals something, or we give you a bunch of convenient methods to say, kind of add sub-layer, remove sub-layer, things like that. OK, finally, we let you put a border, which is just a stroked line, basically, on top of the content.

This is kind of nice, because often you want to have some kind of border around your content. And you want to be able to animate it, so it's just nice not to have to draw it with CG, or something like that. Again, this can be any type of color object.

So you can have gradient-based borders, or whatever. OK, so now we've basically made up the content for the layers. We've got the background, and the content, the actual image content, and the sub-layers, and the border. We basically turn that into a single CI image, and then kind of pass it through these next stages, the first of which is we let you apply filters, as Rob showed in some of his slides. So in this case, it's just a simple morphology filter to give it something that stands up on the stage.

The next stage after filters is you can apply a shadow, typically not a red one. But in the shadow, you have these various parameters. It's pretty simple stuff, just to blur the layer and colorize to Southwood Channel. Okay, finally, you get to supply a global opacity. So obviously, this is proper group opacity and you don't have to worry about any of that stuff.

OK, so once we've finished this filtered layer and we've applied the opacity on the filter, we use the compositing filter to put this into the backdrop. But I guess the one interesting thing is that it's not the backdrop. It's potentially the filtered backdrop. And we'll come to what that means on the next slide, where you can apply a mask to the layer.

And a mask is not just something that cuts out the shape, it's actually dissolving the content of the previous stage through the mask into the unfiltered backdrop. So you can have things like, this is basically how you apply your background blurs, because you can blur the entire backdrop and then mask out everything that's outside the layer itself.

And there are two ways to specify the mask. You can either just say, give me a mask the size of my layer, or supply another kind of sub-layer, which can be anything, we just take it as alpha channel. So this means you can actually have animating layers, animating masks, or Quartz Composer masks, or whatever you want.

Okay, so that was basically the layer kind of-- the layer as it's represented in the render tree. And this is mostly just the OK layer, which is the base class. OK obviously doesn't have a Core Animation, but...

Okay, so now we know a little bit about what a layer is, we can look at how you actually use them.

So obviously first of all, it's just like any Objective C object, you allocate one. And then you configure it to basically look like, or rather to have the properties that will end up making it look like what you want. So you would, for example, set the background color or set the border width or the shadow or whatever.

And then once you're happy with that, you add it to a parent layer, which is kind of like adding it as one of the sub-layers of the parent sub-layer's property. And once it's added to a super layer, then at some point it will be committed into the render tree, typically when the event loop next iterates around.

And finally, we add it to the lettering. We've already created a super layer, and adding it to this layer will make sure it gets committed. OK, what's next? OK, so-- Like I said, layout is this thing that happens as one of the steps of the commit process. And typically you just get to set the position and size of the sublays of the layer. But basically it's just a hook, so if you want to do something else, that's fine.

And so basically, whenever you change the size of a layer, we mark that the layer needs to be re-layered. That's a word. Or you can do this explicitly by saying set needs layout. And then what that does is it kind of marks the layer as saying, next time you commit it, call this layout sub-layers method. And the layout sub-layers are something you can override in a subclass.

And then basically just kind of look at the sub-layers and say, do any of them need to be repositioned? And if so, move them around. And the other thing is that-- If every layer subclass was to define its own layout behavior, you end up with a lot of duplicated code. For example, lots of things have something that looks like a grid, or something that looks like a box, or whatever. So we have this kind of layout manager idea, which is basically layout via delegation.

So every layer has a LayoutManager property, which you can set to some other object. And then when layout happens on this layer, the default sublayer's implementation will just call the LayoutManager. And this is a nice way of, you know, if you have a bunch of things that you want to do grid layout or circles or whatever you want, you can kind of encapsulate those algorithms in a single class and then just kind of add them to each layer.

And currently we only provide one default LayoutManager, sorry, one built-in LayoutManager. And it's a constraint-based system. So what that means is you can add the constraint LayoutManager to your superlayer, and then on every layer you can describe the relationships between itself and its siblings and its superlayer. So for example, you can say that this layer has to be, sorry, the right edge of this layer is aligned to the left edge of a sibling called something. So this is very, we found this to be very kind of flexible. And this was how we were able to do it. So the graphs resizing thing was actually working. And finally, if you want to use the app get style stuff, we have that as well.

Okay, so next we do drawing. And as I showed in the example, if you want to, you don't have to do any of this kind of subclassing stuff. You can just set the contents of the layer to be an image. And that's pretty straightforward. Otherwise, we have basically the same method which you have for layout and very similar to how NSView works, which is you can say setNeedsDisplay and then sometime later the display method will be called. And by default, that creates some kind of CG context which matches the type of data we would prefer to render. And then you get the chance to render whatever you want into the CG context.

So the next point is that there are some types of layers which need to render every time they, or rather need to redraw every time their size changes. So since that's such a common thing, we let you basically just set this needsDisplayOnBalanceChange property to true. And when you do that, we automatically call the setNeedsDisplay method.

Okay, so the next big thing obviously is animation. As Ralph said, all our animation is declarative. What that basically means is you describe what you want it to do, not provide some code that does it. So first of all, you create an animation object. There's a number of built-in classes which describe types of animations, like From To or Keyframe or whatever.

So first of all, you create your animation, you set the properties which describe what you want it to do. For example, what property you want it to change, the from, the two values, the duration, things like that. And then you add it to allow using one of these methods. And then after that, it's all autonomous. You don't have to worry about it. So basically, we have two ways of really triggering animations. The underlying model is the same. We have these animation objects. But you can either have it set up so that-- excuse me.

So basically, we have two ways of really triggering animations. I mean, the underlying model is the same. We have these animation objects. But you can either have it set up so that-- excuse me.

Okay, some more animation. So like I mentioned, there are a number of different animation types.

Firstly, we have the Basic Animation, which is a way of specifying a from value and a to value and a property to change, and some other timing parameters. And similar to that, there's a Keyframe Animation, and these two are very closely related, except that the keyframes are an array of values rather than just two endpoints. So this lets you do very kind of intricate, kind of multi-frame animations. So this is how a lot of the things on the city demo were done.

And I guess one of the big things we haven't talked about at all yet is timing. And Lackit is a composition model not only in space but also in time. So every object in the system is part of this hierarchical timing model. And what that basically means is that everything has a begin time, which is relative to the timeline of its parent object.

So for Lairs, every Layer can have a start time, and then that's some number of seconds since its parent started. Something which should repeat, and a number of other properties. And obviously, animations fit this nicely as well, because they are just kind of a sub-object of the Layer in some sense.

So the animation's timeline is relative to the Lairs timeline. And animations also have an extra feature, which you can kind of specify a timing curve. So you get your ease in, ease out, or you can basically actually specify anything that fits a Bezier spline, one segment Bezier spline. OK. And as I said, let's also do that. So we're going to do a little bit of a simulation here. So here's some actual code, which is trying to animate the opacity property of a Layer.

Simple thing you can do with the animation system. So basically, we're going to create the animation. We're going to set the property we want to change. We're going to set the values, which is basically 1 for opaque and 0 for transparent. And we're going to say do this over the duration of a second. And then once we've done that, we just kind of add it to the Layer. And the key argument is kind of important. Basically, the animation system for each, or rather the set of animations attached to each Layer is kind of a dictionary.

So every Layer can only have one animation per key. And this is kind of crucial for the implicit animations because, as Ralph showed, if you hit the key more than once, and the Layer moves five times, one animation moving from the current screen value to the target state, you don't want to have five animations queued up. So basically, that's what the key is for. OK. What else? All right.

So we haven't talked at all about how you actually use these things. We've told you how to create a tree when you have to actually take the root of the tree and make it visible. And in Leopard, NSView will basically support rendering from Layers directly, which you'll hear more about later.

But for now, what we really need to know about is that you can take a Layer and assign it as the kind of the visual representation of a View. And then the View system will take that and do the most optimal thing to get it to the screen. And it's going to do as much as it needs to.

So there's these two methods. One, set the Layer. And then there's this other one called setOnceLayer. And it's the setOnceLayer which actually turns on the rendering model. OK. So now I think I'm going to do a couple of demos, which just basically just show a few features of the system. OK. So this one.

So first of all, this little application, which is just a red window. And we can open a bunch of icons in here. And so what we did is we loaded a bunch of images, created a number of layers, set each image as the content of each layer, and then added this kind of rotation animation just to give it some life.

And then we have a mouse-down, mouse-drag event handler which looks for the thing under the mouse and is going to set the transform to do a scaling effect on the layer, so you get this kind of Doc Magnify type thing. The other thing is that we're using the layout system to do all the positioning. So we have this layout manager which packs the layers into a 1D box.

So we change the transform and then at some point the layout manager will be called to reposition everything. So that's kind of nice, and I can do this. The other thing obviously is we have a selection filter which is a CI filter generator type thing, which has a Gaussian Blur and some kind of exposure adjusting just to give it some pop.

And I guess the other thing is that we can change the layout manager. That's the nice thing about these things is I can set it to kind of instead of flowing horizontally, I can just say wrap across the window. So I can do this. Obviously I'm still using the exact same code to set the scale factor and the filters and all that kind of stuff. And so as an experiment we can kind of add a bunch more icons and see what we can get going. I needed to copy these a few times.

I haven't actually tried it on this machine, so I hope it works. OK, so we have a 50 or so. You can get a lot more than this, but it's not really stressing it at all. OK. So next demo is totally different, and this one has 3D, which I guess you saw a bit of this in Peter's part in Bertrand's talk. But this is basically a 3D file browser of some sort.

So you can kind of position these things in space and rotate them. And basically, it's planes in space, as we like to say. And what that means is that there are a number of layers, one layer per item in the stack. And each of those has a number of sub layers. Obviously, each sub layer has a border color, which is yellow, and some text. The text is being resized as we select and unselect.

The animation is a mix of implicit and explicit animations to move the thing along the z-axis. The other thing I should mention about animations is that there's also a notification system so you can get messages when they start and finish. That's what we're using in this case to know that at some point the layer has actually moved out of the scene and we can delete it. I think that's about all I wanted to share. I think I'm going to hand it back to Ralph. - Okay.

Okay, so first I would like to mention that you heard the word "layer kit" in this presentation sometimes, which was kind of referring to Core Animation. The reason for that is that was the code name for this project till last Friday and, you know, the context switch in my mind is not happening that quickly. That's also the reason why all the APIs start with LK, just in case you wonder. Okay, a word about AppKit.

So in Leopard, AppKit has a mode where you can essentially have NSViews backed by LK layers. And what this essentially means is it's a flag on a view that says "host this on a layer" and then that view and all its subviews get hosted on LK layers. And that has really nice effects that can make use of all of the Core Animation animation. That didn't sound right.

Usually that is a one-to-one mapping, so every view gets a layer. And for you, essentially what changes is instead of getting the drawRect call every single time something happens, you get it once and then the content of that view is cached. In some cases you don't want that, so imagine you want to pop up some, you drag an object around and you want to pop up a snapping guide, then that would be a good example of taking a layer which is just that guide, add it to the view temporarily to give some visual feedback and then once on mouse up you would remove it again. And go to session 132, Cocoa Animation Techniques, tomorrow at 3:30, which talks all about how to get this stuff into Cocoa.

Unfortunately, it's at the same time as the Core Image session, so if you're interested in making new filters to add effect to your layers, then you're in a pickle. Okay. So in Interface Builder that's actually exposed in a checkbox, so it's this once-layer checkbox that you have to add to the view and then from then on everything happens pretty much automatically. And with that, we'll go to another demo.

Okay, so let me go back to my strawberries here. One thing I did forget to show in the last demo is actually that LK scroll layer. Which is used here. So there's a group layer, which is an LK scroll layer that manages all these different entries and the entries are actually created on demand so they're not visible at the beginning of the app. Okay, another example I would like to make here is... So this is a movie playing on a layer and it has the audio spectrum, gets represented by our, by my boss essentially. It gets squeezed and scratched appropriately.

So the kind of important part of this piece is that it's actually the controls that are on top of this view, the frequency and background filter and... So the kind of important part of this piece is that it's actually the controls that are on top of this view, the frequency and background filter and... So the kind of important part of this piece is that it's actually the controls that are on top of this view, the frequency and background filter and...

So the kind of important part of this piece is that it's actually the controls that are on top of this view, the frequency and background filter and... image image units, because Core Animation will load image units and you can just specify any filter that you have as a plugin. So you saw the glow effect in the strawberries demo. You could have arbitrary filters going in there.