Hello and welcome to Integrating and Extending Quartz Composer. My name is Troy and I'm here with the rest of the Quartz Composer team to give you a sneak peek at what we've been working on in Snow Leopard. So today we're going to talk about Quartz Composer, obviously. I'm going to start with a brief introduction.

We don't have a lot of time today, so it's going to be brief. And if this is your first interaction with Quartz Composer, I invite you to just sit back and relax, enjoy the show, and afterwards take a look at one of the many getting started materials found online. Also, we're having a lab session after this session, so you can come and ask us any questions you have.

So then I'm going to talk about some reuse in Quartz Composer and how you can get the most out of the workflows in Quartz Composer. And I'm going to talk about some of the performance improvements we were making in Snow Leopard. Then I'm going to have Alessandro come up on stage and talk about the new interaction model we have in Quartz Composer. And Kevin is going to wrap things up with integration with OpenCL and how Quartz Composer is the best environment for you to create those OpenCL kernels.

So what is Quartz Composer? Quartz Composer is a visual programming environment which is embedded in the system in the framework. It's also the name of an editor that we ship with the developer tools for free. So you might be confused if you think that, oh, I don't have a Quartz Composer editor, I can't show my compositions to my users.

But that's not the case because it ships with the operating system and you can view it in many different places across the system. For example, here is a composition embedded directly in Keynote. This is a very practical use of Quartz Composer because it was the basis for Time Machine and it's also really great for hypnotizing developers.

This composition was created with this set of patches. Now a patch is a functional block which is used to define a composition. And as you can see, we're working on updating the UI to make it more functional and easier to use. So we hope that you guys enjoy that.

Quartz Composer is a patch-based abstraction of developer APIs. What that means is there's all these things in the system that you want to get access to, and Quartz Composer is your way to use them without writing any code. So for example, we have graphics technologies like Core Image, QuickTime, and it's all based on OpenGL.

We also have communication type technologies such as networking and XML. And we have patches which talk directly to the input devices in the computer, such as the keyboard, the mouse, and the video input. So this makes it really easy for you to create compositions because we have so many patches.

Now, when you create these compositions, they become a file format. We call them compositions. They end in .qtz. And that allows you to integrate them easily within the operating system. Here at Apple, we've been hard at work integrating them in a lot of applications that Apple writes. And there's a lot of third-party development that's going on in Quartz Composer also. I'd like to point out, like, for example, Podcast Producer is really, really useful for universities and institutions to apply their branding to podcasts before they get distributed in iTunes U, for example.

Another application you might be familiar with is Photo Booth. It's a really fun example of Quartz Composer, but some of the most useful examples of Quartz Composer being used are the ones that are being developed by developers like yourselves. So we have companies today that are using Quartz Composer for image processing, for example, Photo Booth type effects in their applications.

Also, there's broadcast television stations using Quartz Composer to process their images before they're distributed. We also have medical and scientific applications that are using Quartz Composer to display large amounts of data. We have medical companies and we have schools that are using Quartz Composer for scientific uses like astrology and that kind of stuff.

Quartz Composer is also really great for motion graphics. We have VJs using Quartz Composer for their live performances in clubs because Quartz Composer has a really great real-time rendering system. Also, you might be aware of Apple stores which use Quartz Composer in their digital signage to tell you when Genius is ready for your appointment.

Also, you should notice that Quartz Composer is not only for real-time graphics because you can use it as a processing engine, either to process your images and create offline representations of those images, or you can just process data and not even use the rendering pipeline of Quartz Composer at all. So let's talk about what Quartz Composer can do for you in reusing the various components that you tend to write.

As you may know, reuse is very important, whether you're an artist and you want to reuse components in your various artistic designs, or you want -- if you're a computer programmer, you know that reuse is important for your frameworks and your classes. So Quartz Composer is good for that too. And one of the base functionalities of Quartz Composer is the patches, and we've written over 250 patches for you to use in your compositions.

That's code that you don't have to write because we've already done the hard work of writing, testing, and implementing those patches. And if that's not enough for you, we also provide a QC plugin API which shipped on Leopard which allows you to write your own custom QC plugins.

There's also, for some of the stylistic effects that you want to reuse, there's Composition Loader, which allows you to load at runtime a composition which you may not know the path to when you're developing your application. And we also have the QC Composition Repository, which allows you to get access to a lot of effects that Apple has written and also some effects that you guys are writing and installing on the computers.

So that's a really good way to have reuse in your applications. And finally, for development time reusability, there's virtual patches. And these are small patches which reference external compositions in a way that's more functional rather than stylistic. So to give you an example how virtual patches work, because it is pretty abstract, I'm going to go to a demo.

So on your Leopard machines or Snow Leopard, we're going to have a lot of examples in the developer examples. So let's go ahead and go to your developer examples inside Quartz Composer. We have a compositions directory, and inside that is a conceptual compositions directory. I'm going to choose Image TV.

Now even though we have most of these compositions on your computer, we don't have a lot of time for hands-on demonstrations this year. So go ahead and just pay attention up here and you can go back to those compositions later on your own time. Now that this composition is fully loaded, you can see that it's a wall of images which is used to replicate the effect of Apple TV when it first starts up.

It's a pretty neat effect, I think. But what I'm really interested in today is this progress bar because it's a fairly modular component and it can be reused in other compositions. So as an example of composition that could be reused in, let's go back to the conceptual examples and open up Image Flow.

Now this composition you may also be familiar with because it replicates the effect of CoverFlow in the system. CoverFlow is using OpenGL but Quartz Composer is based on OpenGL so there's no reason you can't implement an effect as cool as that in Quartz Composer. But as you can see, even though it loads images from the desktop pictures folder, there's no progress bar in this. So let's go ahead and add one. First of all, let's take a look at this, the image TV composition and figure out how to read it.

First thing I always do when I look at a composition is figure out where the renderers are because that's actually what's doing the hard work and pushing pixels to the screen. So the renderers now in Snow Leopard are drawn in blue. And they are usually on the right side of the composition because input goes from patches. They're inputs into the left side and outputs on the right side. So for a renderer that has no outputs, all the inputs are on the left side of the patch. So let's go ahead and zoom in on one of these renderers.

First thing a composition does is clears the composition. This provides the black background that you see and kind of makes sure there's no graphic corruption. After that, we render a 3D transformation. Now, we don't know actually how the wall is drawn because that's not important. It's encapsulated inside this 3D transformation. And all we know is we have this block, this blob on screen. Whatever is inside is rendering can be animated by using -- by connecting these rotation translation inputs to, for example, this Timelines patch. And so the Timelines are what drive the whole composition.

On layer three, we have a sprite which provides the fade-in effect at the beginning of the composition. And just below that is this macro patch which contains all the necessary components for the progress bar. I'm going to go ahead and double click on the macro patch and see what patches are inside.

You can see it's composed of two sprites. The sprite renders a rectangle on screen. The lower sprite renders just a gray square. And the upper sprite renders with a width that is determined by the number of images loaded. So if I go back to the root macro patch by clicking on the navigation bar, you can see once again this macro patch, which has square corners because that's how we denote patches which contain other patches.

New in Snow Leopard, we can unencapsulate this macro patch and make it all those internal patches right here on the root level. And that's done by selecting the patch, holding down the Control key and clicking, and you can choose Explode Macro. That should make it really easy for you to refactor your compositions because you can see all the connections are recreated and you can now select a different set of patches to create a macro from.

Now, I said I wanted to use this progress bar in another composition, so that sounds like an excellent chance to use a virtual patch. And a virtual patch will package these patches up like a macro, but it will save it to my library folder and allow me to insert it into other compositions. So I'm going to choose the editor menu, say Create Virtual Patch, and name it Progress Bar.

Now Quartz Composer has gone behind the scenes, saved all those patches to my library folder and replaced it with this patch, which no longer has rounded corners because this is no longer a macro. It's actually a real patch as far as Quartz Composer is concerned. And it's completely separated from the system as far as the subgraph is concerned. So all you have to know is that this is a real patch now in the composition. I'm going to close this down. It's still going to be open over here. And I'm going to open up the image flow once again.

Now when I want to insert a patch, all I need to do is open the Patch Creator, type in progress bar, and hit return and it will insert into the workspace. This macro here is what does all the heavy lifting of downloading and processing the images. For example, in this composition, it processed a second copy of the image to create the reflection, but that's all done under the hood because it's a macro and we don't have to worry about the details.

All we need to know is the download progress output of this macro patch can be connected to the input of this progress bar. Now when I go to full screen and reload it, you can see that the images load and the progress bar fills at the same time.

Now to show you the real power of a virtual patch, I'm going to open up that previously opened image, Image TV composition. and show you how I can change the virtual patch in one place and have that modification affect all the open compositions. So I'm going to go to my preferences. Virtual patches are managed in the Clips Manager. We still have clips, but they have slightly different meanings.

So I'm going to open up the progress bar here in the Clips Manager. And down here are all those patches that we've been using over and over again. Before the session, I prepared a small asset, which is just a small little image that provides an Aqua gradient. So I'm going to go ahead and aquify all these progress bars.

I've drag and dropped an image from the finder into the workspace and that creates an image importer automatically. So that's a pretty easy way to get images in your composition. Now I just need to click on an image, connect it to the image input of the sprite which draws the progress bar and save this virtual patch. And you'll notice the two open compositions will automatically have their will automatically have their progress bars updated to this Aqua look. Now when I restart, the progress bar is automatically aquified. That's pretty good instant savings right there.

It's important to notice that compositions that have virtual patches are like any other assets. So you do need to save those--you need to distribute those with your composition. If you have, for example, movies or downloaded images, it's going to be the same story. So make sure that you have all your assets with your composition when you distribute it.

So let's go ahead and go back to the slides. We really think that that will save a lot of time for creating compositions, but what are we going to do for your compositions once they're created? Well, we've been working really hard in Snow Leopard to make performance our number one priority, and we think we've done a pretty good job. So all the compositions that we've been writing, or that we've been testing, are faster, as fast or faster, than their Leopard counterparts. and with some compositions such as high iteration compositions that are up to eight times faster. So that's quite a bit of savings.

We also have a certain class of compositions, which, when they're embedded in applications, are really useful, and those are the event-driven compositions, because Quartz Composer was initially designed as an animated graphics technology, and so it's really good at that, but your applications don't always animate, so Quartz Composer has developed a system which allows Quartz Composer to analyze the graph and figure out when the composition doesn't need to render, and we call that Idle State Detection. And to show how that works, I'm going to go to another demo real quick.

So I've created a composition which is arbitrarily hard on the GPU because the teapot's object in Quartz Composer is not optimized in any way. So just because this is showing high CPU usage, that's by design. When I click on this and rotate it, I can get the composition to animate.

To help me out, I've created this small application, which, by the way, uses Quartz Composer as the visualization to give a CPU monitor with large types so that you guys can all see it. So I spin this around, you can see that the CPU process power for Quartz Composer is hovering around 81%. But when it stops, the CPU power drops almost to nothing. And that's because of the idle state detection that we've been implementing.

It's important to notice that Quartz Composer is doing all the analyzing in the background, so you don't have to do anything special to set that up. And to prove that, I'm going to open up the composition we were working with earlier, this image flow composition, which is exactly the same as the one on your Leopard machine, so we haven't done any work to optimize it, and show you that it is exactly the same story here.

It's perfectly performant and responds nicely to the keyboard. And when I open up the CPU meter, You can see that the processing power required when it's animating is about 15% of the CPU. When I take my hands off the keyboard and let it sit for a second, it drops down to about 0.4% of the CPU. So that's pretty good savings.

This is going to come for free when you're using QC view as we are in the editor because we control the timers and we tell the composition when to animate. But what are you going to do if you have a lower level API like QC Renderer? If I can go back to slides, I'll explain how that works.

So this is completely optional because QCRenderer, as you know, just asks Quartz Composer to render a frame and it gives you the rendered results. But if you want to check, if you want to ask the composition if it needs to render right now, you can ask it this question, next recommended time for time.

And if the time it returns is now, then you go ahead and render that -- you call the render at time method. But if that time it returns back is in the future, then you can forgo rendering for this iteration of your timer loop or your display link, whatever you're using. And so that provides you with the optimum performance for those idle state opportunities.

Now, I said that QuartzComposer analyzes the graph and asks all the patches how -- if it needs to render. So if you're a patch writer, you may also want to respond. So there's a new API for that as well. If you override this method on your time-dependent patches and tell the renderer that, yes, you do want to render this time, you can return the current time. But if you don't want to render at this frame, then you can return infinity and QuartzComposer will take that into consideration when it's analyzing the graph.

Now, please note that this is completely optional. So if you don't implement this, none of your patches are going to break. They're going to work exactly the same as before. And this is also completely unnecessary if your patch just depends on the inputs because it's going to do the right thing anyway. This is only if, for example, you're writing a network-dependent patch that checks the Internet once per minute or something. So the time you return could be that time remaining until that minute is up. it up.

So this is really easy for you to implement. And in most cases, you're not going to need to do any work at all. And this is really great for your event-driven applications because not only does it save processing time, it'll save batteries-- battery time as well, which is really important for those Apple portables. So to tell you how to take advantage of this new API and the new idle state optimization, we're going to get Alessandro up here to talk about the interaction model in Quartz Composer.

Hey, thanks Troy. Alright, so as Troy mentioned, I'm going to talk a little bit about interaction in Quartz Composer. So one of the reasons that we started to look at interaction in Quartz Composer is that we found that a lot of developers were creating these rich, interactive, media-based compositions. A lot of them being full-screen compositions wherein you'd actually have to create these on-screen type controls and it was actually rather difficult.

So what we did is we went ahead and created two modes of interaction in Quartz Composer, the first one being interactive placement. And what this is, is this is an actual rendering mode and it allows you to just quickly and easily set up your compositions, place a bunch of images and get going.

And then the next type of interaction that we added was interactive compositions. In this case, this interaction patch. And what that allows you to do is actually make your compositions interactive. So I can sit up here and talk about interaction, or I could just show you. So in this case, what we're going to end up building is this interactive photo browsing composition. If I can go to Demo.

All right, so I'll just get set up here. So the first thing that we'll look at is interactive placement. And again, this is just a quick and easy way to get your compositions set up. Okay. So, you know, as usual, we'll clear the screen. And the next thing I'm going to do is I'm just going to grab three images here and drag them from the Finder into the workspace. We'll create three image importer nodes. And I'll just use a billboard to display these. So here I have my first image.

My second image and my third image. And you'll notice that these images are all stacked one on top of the other. So traditionally in Quartz Composer, what I could do is I can inspect one of these billboards. And I would have to independently position them in the X and the Y dimension.

So in this case, like this. And you can do it. And you've probably seen some pretty great examples of some of the things that you can do in Quartz Composer, but it's not necessarily optimal. So what we did is we added this new interactive placement rendering mode, which you can see up here as this little cross. So if I just click on that, now I can go ahead and actually just position these as I want.

So, you know, I mean, kind of an obvious thing, but you'll notice that as I move them around, the X and the Y position actually changes. So what I can do is I can just stop, restart the composition, save, reopen, they get re-serialized into the composition. So again, you know, just a quick and easy way of kind of setting up your compositions. But, you know, when your compositions are actually being executed within whatever environment they're being executed in, this isn't going to work.

This is really just for getting set up. It's an editor feature. What you want to do is you want to then leverage the interaction patch. So I've gone back to here to the normal rendering mode. I'll just get cleaned up here a little bit, and I'm going to add our new interaction patch.

So here's the interaction patch. And what I'm going to do is just connect the new interaction port, which is going to get data back and forth between the billboard and the interaction patch to do a little bit of hit detection, stuff like that. And I can just connect the x position and the y position of my interaction patch to my image. You'll notice that it actually snaps into a certain place.

And the reason why is because the initial position for the interaction patch is going to be at 0, 0. And if I wanted to actually set up my interaction patch or the components of my composition, which are going to be interactive, I could just change the offset. So now this is interactive. I can just grab it and move it around. The idea here being, if I stop, restart my composition, it's going to snap back to that original offset. So you then use the offset to kind of position your objects.

So the thing that's really powerful here is we could have just made interaction work in the same way that we do for the interactive placement, but by adding this patch, we really kind of have created a new type of interaction model. So just like in Quartz Composer with everything else, you can actually modulate these values in line. So here I have my X position and my Y position. They're going into my billboard.

I can just stick patches in between to change that behavior. So as an example, why would you want to do that? Okay, so a very simple example is one of the features that we added to the interaction patch is the ability to make these things throwable. So I can just make this throwable, grab it, and throw it away. So now it's gone, right? That's great, except somebody might not have wanted that to have been thrown away.

So what you can do is you can stop and restart your application, which nobody's going to do. Or you can actually, say, set the range of these values coming out of the interaction patch. And you do that with a range patch. So here I have a range. I can just set the range min to minus 0.5 and the range max to 0.5. Just duplicate that. Troy Koelling, Alessandro Sabatelli, Kevin Quennesson And I'll modulate these x and y values coming out of the interaction patch.

Set those into the billboard. And now you'll notice that it snapped back. Now I can throw this, but it'll be pinned. So that's just a very simple example of how you can modulate interaction in Quartz Composer. But obviously, you can do some pretty advanced things with it. And to give you an example, I'm going to build that photo browsing application for you.

All right, I'll just get set up here again. So in that photo browsing application, the first thing that we're going to do is just kind of get our environment set up. So I'm just going to paint the background with a simple gradient. To do that, I'm going to use a billboard. Oops. Let's try that again.

And for this, I've just prepared a simple gradient that I made in my favorite image editing tool. And so in this case, I actually want the gradient to be sized to the view or the rendering destination. So to do that, I'm just going to set my billboard to have some custom dimensions.

And I'll just use the rendering destination dimensions. And the reason that I want to do this is because when I'm creating this composition, I can actually just go ahead and manually size it. But the thing is I don't know where it's actually going to be shown. So it could be shown on my MacBook, or it can be shown on somebody's 30-inch monitor, in which case I want the gradient to be sized to whatever destination it's actually displayed on. So in this case, if I then go ahead and resize the viewer or go full screen, you'll notice that it scales appropriately.

So I'm just going to pick these patches and I'm going to create a macro and I'll just call that background. Again, it's always a good idea when creating these compositions to constantly refactor and keep things kind of clean. The next thing I'm going to do is I'm just going to display this other image here along the bottom. This is going to be the image bar at the bottom that's going to hold each one of my images. Again, I'll just use a billboard. Connect my image. I'll set a custom size for that as well.

So I'll make it, I don't know, 0.3 high. And I'm going to make it really wide, and that'll be apparent in a minute. So let's say like 15 or something. So in this case, now I have it and it's just sitting there and I want to anchor to the bottom. So I'll just use an anchor patch.

Anchor position. In this case, I'll tell the anchor position that the height is 0.3 and I will use that to pin the Y position by setting the anchor point to bottom center. So again, resolution independent. I want this always pinned at the bottom. I can now redrag and change the size of my viewer, go full screen, the bars are at the bottom.

Clean up a little bit. OK, so the next thing that I want to do is now that I have this little image bar here at the bottom, I want to get a bunch of images and display them. So to do that, I'm going to use a similar macro to what Troy used, which is download and process images.

But in this case, I've actually changed it a little bit to apply some different effects to these images. And what this does is it simply goes out to the Finder and grabs a directory of images and then iterates over that list of image paths and downloads those images and applies a CI filter to add a little bit of a border, a shadow, and then stores them in a structure.

Okay, so considering that they're now in a structure, what I need to do is use an iterator to display them. So I'll just make sure that my directory... is set. I have a bunch of images here on my desktop. I'm going to use an iterator. You'll notice that the iterator is non-consuming for those of you that are familiar with it. An iterator, all that it's going to do is it's going to iterate a certain number of times.

So it'll loop over some data. So in this case, what I'm going to have it do is I'm going to have it draw a billboard a certain number of times. I'm going to grab a structure index member, which I can use to look up into a structure, publish that structure port, set the index using the iterator variables, which is a special patch which can be used inside of an iterator. In this case, I'm going to get the current index. And that's going to give me the current index of the current iteration. I'll connect that to my image.

Clean up. Now I can just connect the image structure to my structure here. And you'll notice it'll load a bunch of dog images. You'll also notice that they each have a little bit of a black background. That's because I didn't set the blending mode of my billboard. I'll just go inside the iterator, set the blending mode, in this case, to over. And I have a bunch of images laid on top of one or the other. It's also a good idea to set the loop count, the number of iterations that you perform within your iterator.

So I can just use a structure count. So in this case, if I just hover over the image structure, you'll see that there's 24 members, but I'm only doing 10 iterations. So right now, I only have 10 images being displayed. So now I can just set that to be dynamic. In this case, now it'll be 24. So again, I have 24 images just overlaid one on top of the other. So what I want to do next is make these images a little bit smaller and kind of arrange them so that they're offset each in space.

So to make them smaller, I can just go in and change the width. And this width is going to apply to every single one of these iterations because I have not differentiated it in any way. I'll just set them to be 0.25. And that's just a little less than that bar height that we set so that they'll sit within that bar. And now we want to spread them out.

So the question is, is how do we differentiate these images or their positions? And we can do that just with the current index. And that's really an identifier or a means by which to differentiate items within your iterator. So in this case, I'm just going to say take some value, which is going to be something like a little bit more than their actual size.

So I have a little bit of a border. So I'm going to take something like 0.3 and use that to multiply by, say, the index. So it'll be 0.3 times 0. So that'll be 0.3 times 1. That'll be 0.3, 0.6, so on and so forth. So I can just use a simple mathematical expression. And I will change the expression to be exactly that. So it'll just be 0.3 times index.

And now you'll notice that they're arranged independently. Each one is offset. So in this case, I also kind of want to slide them a little bit over, make this a little bit bigger so you can see. I want to slide them over to the left and then I'm also going to slide them down. So to slide them over to the left, I can just add a little bit of an offset here. So say, I don't know, plus the start position. And, you know, I don't know what that is, but I can just eyeball it, which is kind of nice.

So we'll just move it over here. And now in order to move down, I can do the same thing. But like I said, we're kind of creating this so it'll work in resolution independent modes or in resolution independent environments. So instead of just eyeballing it and sliding it down, I'll just publish the Y position here and use the position that we've generated from our anchor position to position that bar at the bottom to anchor our images at the bottom.

So now if I resize this and go full screen, you'll notice that they're always pinned there at the bottom. So that's basically the layout for our composition. All we did is we just positioned a gradient there at the background, bar at the bottom, and then a bunch of images using an iterator.

The next step is to add some interactivity to each of these little billboards so that we can go ahead and drag them around and kind of position them and create some kind of photo browsing narrative. And to do that, I'm going to use the interaction patch. I'll just grab the interaction patch, connect that to my billboard, and then I'll connect the X and the Y position. But you'll notice that I already have these X and Y positions connected. So again, if you'll recall, I can just input these positions and use the interaction patch to actually modulate the initial position of all of my images. So I'll just connect the X position.

So what that'll do is that'll refactor my composition, put this input splitter in here that has its port published and connected to the level above. And now I can just connect that to my interaction patch and the interaction patch to my billboard. Again, I'll just clean up a little bit. Now if I go full screen, you'll notice I can just drag these images.

One thing you'll notice is that I've arranged these images at the bottom and there happen to be 24 and this one's cut off. So there's a whole bunch of images that keep going off into the distance. So what I can do is actually make this bar here at the bottom draggable and then use that position to offset the positions of each one of my images. So again, it's just an interaction patch. Here's that image bar, which I can just relabel so you guys know.

And in this case, all I need to do is be able to move it in the X dimension. There's nothing special that you have to do other than just connect the X position. So now I have my X position. And I'm going to just say add another offset to the positioning of the offset X. I have my individual images within the iterator. So we'll say bar position. And I'll just publish that.

And connect the X position here to my bar position. So now I can literally just grab my bar, move it around, and continue on. I can even say make that throwable. You'll notice I'll get into the same situation whereby I can end up kind of sliding this off into the distance. Here you'll see the edge of my bar, not necessarily super clean. Again, we can just apply the same principle. We can just range. I'll just range that value.

Again, I can just kind of eyeball it. So in this case, my exposition is... Let's see, for my last image, negative 5.2, you know, and at the start it'll actually be zero. So the min will be negative 5.2. The max will be 0. And I'll just put that in line. So now I can just kind of throw it to the end there and nothing will pop off.

Okay, so for the last thing is I want to make these images actually kind of grow so that when I place them up top, what I'm doing is, again, I'm kind of creating this narrative. And I'd actually like the model here to be that I have a bunch of images at the bottom that represent, say, my library of images, and then at the top, some selects that I've chosen.

So what I can do is in order to make these, say, grow when I drag them up, is I can just test the position of the actual image against some value and then say if it is larger than that value in y, I can just switch the sizes. So why don't we go and do that? So in order to do that, what I'm going to do is I'm just going to use a conditional to test the value.

And I'm going to be testing the Y position. And I want to say if that Y position is greater than-- in this case, we have the Y position of our bar, which again is always going to change and always be pinned to the bottom of our view. And that's actually going to be this little line here in the middle, which represents the middle of our bar image.

So I want it to be a little bit above so that it doesn't just pop as soon as I grab it. So I'll just grab a little math patch. You'll notice we've actually made it a little bit smaller, which is nice and easier to work with. And I'll say, I don't know, add a .2 to that.

So now if my Y position is greater than the bar position plus 0.2, I want to use that to switch between two values. And to do that in Quartz Composer, we have a multiplexer. For those of you familiar with DSP tech terminology, it's basically a switch. And I'll change the type, which is generally an important thing to do in Quartz Composer.

And the reason why, particularly in this case, is because I want to put values on the input ports. So I'll set the type to number, and then I can go ahead and just set the value. So for this particular example, I'm saying if the Y position is greater than some other position, then in that case it'll be true and it'll use source number one. So I want that to be a value of one and the current size to be 0.25.

And now if I connect that to my width, you'll notice that those images up top just kind of get larger and the ones at the bottom stay smaller. So I can just drag these back down. And then you have a pretty simple way of kind of switching between two values based on the interactive value of the Y position.

So that's what makes it really easy and nice to kind of, in this case, just kind of place a bunch of images. That guy's hair is just crazy. This guy's pretty cool. The monkey riding the dog. But you'll notice that when I do pull them up from the bottom that it's discontinuous. It'll just pop in size.

As soon as I break that value, somewhere right around there, it just instantly switches between these two values. So, probably won't be pleasing to most of you, so what we can do is we can just smooth that value out over time. And to do that, we'll just use the smooth patch. So I'll just grab a smooth patch.

Connect that value, I'll set the duration to be 0.2 and the type to be sinusoidal in and out. Connect that to my width. So now if I go full screen, You'll notice that when I drag them from the bottom, they smoothly interpolate between those two values. So that should give you an idea of what it would be like to kind of build an interactive composition in Quartz Composer. If we could go back to slides, please.

So what we talked about is interactive placement. Kind of an obvious thing. Just a quick and easy way to set up your compositions. And then finally, hopefully you guys get a little bit of an idea of some of the power that you get from the interactive model that we've added to Quartz Composer. Again, it's just another patch. And it seems very simple. But in that case, for example, just to move in one dimension, it's just a matter of just connecting that dimension.

So certain things are just kind of obvious. And if you want to then play with those values over time, it takes all that information into account. And you can really start to create some very powerful and expressive interactive models. So we're really excited about it. We've started creating some really cool stuff. And we're looking forward to some of the stuff that you guys will create. So next I'm going to bring up Kevin. And he's going to talk a little bit about OpenCL integration in Quartz Composer.

All right. Thanks, Alessandro. So it's pretty crazy. It's pretty amazing what you can do in 20 minutes right now. So, these days. Alessandro, from scratch, has been building an interactive for the browser that leverages both OpenCL and Core Animation. So interactive that is pretty functional, so you can just take the composition and putting in Xcode and have an application that you can ship. And so this with no code and in 20 minutes. So imagine what you can do in one day, or maybe two days, maybe a week. So lots of potential there. So these with just one patch, the interaction patch.

And so with the idle state improvements that we've been making in Quartz Composer, the application that you would ship will also be very efficient on all hardware. And what it means is that Quartz Composer will automatically analyze the graph to understand its time dependency so that only what needs to be executed is executed when it needs to. So if nothing happens, for instance, if you're in your photo browser and nothing moves, nothing will get executed. The graph will be skipped.

The display won't be flushed. The CPU won't do anything. And the GPU won't do anything. So it makes a very good solution and a very powerful solution to create a very stunning interactive visualization that works all across the hardware and are very efficient, in particular on laptops. So just with one patch, the interaction patch that we added to Quartz Composer. Another very important patch that we're adding first in the lab is OpenCL. So OpenCL is this open specification and programming language that is built to leverage the the huge computational powers of the GPUs.

And so we added an OpenCL patch in Quartz Composer that is similar to the other programmable patches that you've seen in Quartz Composer. So meaning that you have the patch and you can edit the settings and write up your OpenCL kernel. And Quartz Composer will automatically parse the code to find input ports and output ports.

And so as for other programmable patches, compilations will be made in the background at runtime. You have error feedback. So it's a very nice and very easy way to bring the computational power of OpenCL in Quartz Composer, but also much more. But before going to that, I'm going to show you a demo of what this patch can do.

Can I go to demo machine, please? So I'm going to show you a demo that I've been showing in the OpenCL session this morning, but I'm going to go more in details into what's behind this composition. So here's the NBody simulation. So 16,384 particles surrounding here in cross-composer. Here's the composition.

And so what we've been doing, we have an OpenCL kernel patch here that if we inspect a setting, we see that this is just this code, this OpenCL kernel, that we've been-- so we haven't been doing any OpenCL programming. We just took this kernel that is available for other demos. We pasted it in here. And then you just need to connect the output of this patch to some renderer here that does the point drawing. And so that's it. So you already immediately can visualize what this OpenCL kernel does.

And so to show you the power of the real time compilation, I've commented here a line that multiplying the time set by five, so I can increase that time set and the simulation picks it up, comment it out, the simulation picks it up. I can also put some error and the simulation picks it up, second step, and we start again where we were.

So it's a great way to understand what's going on and to iterate very quickly. But because you're in Quark's Composer you also have access to the power of Quark's Composer for visualizations. And in particular, it's very easy, and I'm going to show you how, to add some motion blur here.

So let's say 100 trails. So that is three dimensional, so you can rotate and see how the simulation evolves over time in a very nice way. And to do that with Quark's Composer, it's very easy. So we just feed up the result of that kernel into, so here, three transformations. So here we just have CI to generate the little texture that is applied on each of the dots. And here we have a QPatch. So QPatch will stack up whatever results have been given for the last number of times.

So here we have, we can set that time here in the parameters. So here for instance, 110 times. So it's going to stack up 110 results. And then just iterator, we simply iterate over the structure and draw each of these set of points with an alpha that deems over time to generate this nice, you know, 3D visualization. So with no code at all.

And Quark's Composer has a strong and very powerful OpenCL pipeline and is going to cache everything in the background. You don't have to worry about doing OpenGL attachment, vertex buffer object, all these things will be cached by Quark's Composer and reused optimally. So here for instance, you have 100.

So, 200 now. And let me start again. Up. To show you because it's nice. Here you have -- so, 200 times, 16,000 points being drawn by Quartz Composer. It's still smooth and responsive and very nice. You can also interact with it. You can zoom in, zoom out. So, it's very simple to get from something very simple and to put it very far using Quartz Composer. In particular, this brand-new and very exciting OpenCL technology. So, let me go back to slides, please.

Thanks. But so OpenGL is not only for scientists. And so it means a lot for Cross Composer. It's very important for Cross Composer. I'm going to try to tell you how. So it's a great complement of the programmable patches that we had so far. And so programmable patches, I mean, so we had GLSL that is good for doing -- so rendering time, vertex and fragment processing, so in OpenGL. CoreImage that is great for doing -- and very great for doing image processing on the GPU or on the CPU and that handles filter chains very well, does concatenation and does color syncing. So it's very good for image processing.

But it's not good for other sorts of data, so other sorts of tasks, for instance, general data computation, because it's not built for that. And so for these sorts of tasks where you just have arrays of number, you want to do some processing on them, some computation, get the maximum, sort them, that sort of things.

So you had to roll back onto JavaScript in Cross Composer that -- so you have the overhead of the language and it's not -- so it's very much faster in Snow Leopard, but still it's not the optimal way to deal with large amounts of data -- large amounts of data. And so what OpenCL brings to Cross Composer is this ability for any kind of object that Cross Composer handles, and by objects I mean images or structure, to do the expensive computation to offload it on the GPU.

So -- and that's -- and that's a great match for Cross Composer because the big advantage and the great quality of Cross Composer has always been to be able to have this graph piling that runs on the CPU and that leverages the GPU for OpenCL -- for OpenGL, sorry, for core image whenever it can.

And so there was still missing parts that it couldn't fulfill on the GPU, but now OpenCL is going to allow you to offload all the expensive stuff, all the expensive computation onto the GPU. And then the CPU remains for handling, you know, events, networking and so on, or interaction.

So the cheap stuff, the threaded stuff, all these things can be on the CPU, and the GPU can be leveraged whenever it makes sense. And so that's a great marriage and that's a great dual solution that you have here to leverage both the system very optimally and very simply.

And so you can process images now, but you can process now geometry, so structures, and do some effects on them. So we added a new mesh patch that takes some set of points. And so we show that in the NBody demo, but that can also draw triangles or triangle strips. And so I'm going to show you the sort of things we can do with this new pipeline in Quartz Composer. So let me go back to demo machine. Thank you.

So that's also some demo I showed this morning in the OpenCL session, but I'm going to go more in detail to understand what's behind that presentation. Here you have a cloth simulated in real time via OpenCL. So all on the GPU. There's no download. So all the computation is on the GPU. The CPU is doing pretty much nothing in here.

So you can play with it, and it's kind of fun. So it's kind of elastic, and you can move it around and everything. But because you're in Quartz Composer, so first of all, you can bring in images very easily to texture map on that cloth. But you can also apply some core image effect on that image, and it's alpha blended on the background. OK, so that was the cloud simulation demo.

And so to summarize, so we tried, I believe you see where we're trying to go and the problem we're trying to solve. So we tried to make QC be a very fantastic tool for doing stunning new sorts of very innovative data visualization and interactive data visualizations. So and this works very well with this idle state so that we'll make that visualization very efficient over multiple broad range of hardware and the interaction that allows you to very innovative and very novel ways interact very easily with that UI.

And then OpenCL is there to help you leveraging the GPU whenever it needs to bring the whole compute power of the GPU to your competition to broaden your range of processing. So you can process mesh, you can do simulations, you can do a lot of things, rigid body and so on.

You can take these kernels, bring it into Quartz Composer and start doing fun stuff with the rest of Quartz Composer at your hands. And so reusability then is very important because a lot of people use Quartz Composer and they don't have to be scientists. So they can be a graphic artist or they can be regular developers.

And so They might not want to do some OpenCL simulations because it's still some code, and you might not want to design that pipeline because you might look a little complicated, or you just don't want to bother about it. So what you can do with the Quartz Composer metaphor is that you can actually wrap that up inside a virtual patch, and then you can ask somebody else to do it, or you can get it on the Internet, get a virtual patch that does whatever you want, that leverages the CPU with OpenCL very efficiently, so it creates stunning new sorts of interactions, and you can bring it into your application and do whatever you want to do with it.

So if you're a novice, you can just do some very nice artistic applications, artistic visualizations. And so to show you what I'm talking about, well, that's -- I'm going to -- to show you what I'm talking about, I'm going to talk to you about this code simulation patch. So we just took the code simulation and put it within a virtual patch.

And we exposed the necessary parameters, so input ports and output ports, so that code simulation virtual patch is taking a mesh, and we exposed some X and Y position, some parameters for the width and the resolution of the mesh, and it returns from that non-transform mesh, it returns a close simulated mesh. And so if you're an artist, you don't need to know the internals, you need to know all the stuff I just showed you before.

Just get that patch, put it into whatever you add, and, you know, start playing with it and start putting that to new limits. All right. And so that's an example that I want to show you now in this latest demo, in this last OK. So here is this composition that Alessandro has been doing for you, demonstrating-- so creating in front of you in 20 minutes.

And so we added some OpenCL, so we added this virtual patch within that composition. So very simply, just put the patch and insert it. And then you can do some fun stuff. You can just drag this thing around. And they have the close simulation going on here, so you can move it around, play with it, drag it. It's kind of elastic. But you know, it's sort of fun.

So you realize that OpenCL can bring to your application some new source and some new way to interact with it and some new control and new power. So you can take that thing, you can bring that up and drag it around, you can take that other thing, drag it around. You can also do real page curl.

And that's what is cool. You can just take that thing and say, oh, I want to review what's below. I can do real page curl. I can just want to see the other things. So you don't need to worry about what's the function I need to do so that the page curl looks right in 2D.

You can just have that. You have this paper simulation. And then it's even simpler, because you have the close. You just drag the two corners, which is only what you need to do, just drag the corner, pull them up, and that makes you a real, physically simulated page curl.

And you can drag stuff around and everything and play with them. And yeah, again, so I say, oh, I want to see my dog. And you can see a dog. And then you can reveal your dog. So OK. So you see, lots of potential. That's a part of there. Thanks to Quartz Composer, that can really bring all these technologies together.

And in particular, OpenCL is going to give you much. So thanks. So that was it. So to know more about Quartz Composer, Alan Schaeffer is our graphics evangelist. So please email him. Please also come to our lab. Right after this session, we're going to be downstairs, be able to answer any questions you might have.

So documentation is available. You also have a very strong and very good mailing list. We are very active, so a lot of people can ask questions and get it answered. So it's a very good place to look at. But right now, of course, everything we discuss here is still confidential. And so the editor is also going to help when you choose a very convenient. OK, well, thank you. And so we're going to have some Q&A now.