Good afternoon everyone, welcome to the Quartz Composer hands-on session. Let's get started immediately. We've got a lot of content to see today. So first of all, what is Quartz Composer? I assume that most of you are actually familiarized by now about this motion graphics technology, especially since the demos yesterday during the graphics and media keynote. But let's just go over this quick overview once again.

So Quartz Composer is an operating system-level motion graphics technology, and operating system level is very important. It's not like a side technology or something like that. It's sort of integrated into Tiger. It was built around two key things. OpenGL to get high performance and real-time graphics. And at the same time it was built about this idea of integrating all of our graphics technologies through a common technology. What you do with Quartz Composer is create what we call Compositions.

Compositions basically render visuals on screen. Now, those Compositions are not necessarily black boxes. They actually can be interactive with external user devices, the basic ones being the mouse or the keyboard. But also some, they can also communicate with the host application because those Compositions can have what we call input parameters or produce output results which are not necessarily visuals. The way you create Compositions in Tiger is by using our full features graphical editor, part of the developer tool.

We also try to make sure that Composition would be easily supported by various applications in Tiger. And for that, we have QuickTime compatibility that is Tiger only, and we'll look at that in details later during this presentation. And we also have a very easy to use API so that you can integrate Quartz Composer into your applications.

Pierre-Olivié Latour: So as I said earlier, Quartz Composer is really integrating several technologies. If you look at this simple graph here, we have most of our graphics technology from OpenGL and the new one we introduced in Tiger, Core Image, Core Video and such. And also some technologies that are not necessarily graphics like simply JavaScript or some basic spotlight queries. And all those technologies are kind of bridged through Quartz Composer.

And then the Quartz Composer system is used by a variety of hosts. Right now we have three hosts and we look at those in detail during the presentation. There's going to be the screen saver engine. There's going to be QuickTime and finally your applications. So what's interesting here is that from a developer point of view, instead of having to learn all the various APIs for all those technologies, you can simply learn the Quartz Composer graphical editing system.

And learn the little -- the easy to use playback API and leverage all those APIs and don't have to take care about how they need to interact together, how you connect them together and just use that very easily in your application. But I did actually found on the internet a pretty nice description of what Quartz Composer could be.

Now that said, what are we going to learn today? This hands-on session is divided into four tutorials. The first one will be creating a parameterized Composition using Quartz Composer. So it's a little more than your basic Composition. It's one that has actually input parameters. Then we're going to look at where and how we can use this Composition straight out of the box in Tiger.

We're also going to look at using the Playback API so that you can have even better integration of Quartz Composer inside your applications. And we'll finish by building what I called a kind of a complete Quartz Composer application from scratch that really takes advantage of the technology. And hopefully after these four tutorials you'll be able to add Quartz Composer support to your applications and create new applications with compelling motion graphics visual. It's kind of important because some people outside, some users are really waiting for you to come up with those applications.

Okay. Let's start with tutorial number one, which is, as I said, creating a parameterized Composition. So the first thing we're going to do is obviously launch Quartz Composer. And for that you simply go to the Developer folder and then the Applications folder, Graphics Tools folder, and you will find the editing application right there. Double-click to launch. You can click cancel on the Assistant panel. We're actually not going to use the Assistant for this tutorial. And let's close that window as well. Okay. Now in Quartz Composer you can create a blank document simply by doing New in the File menu.

So here we have the typical Quartz Composer environment with the viewer window over there that actually shows what the Composition is rendering, its output visuals in a way. And here we have the editing window, which is where you assemble the various Quartz Composer components that we actually call patches and you connect them together and which are gonna build a workflow that's eventually gonna create visuals.

So let's get started and create some visuals. The first thing we're going to do is simply paint the background, I mean, with a gradient, because right now there is absolutely nothing rendered by the Composition. As you can see, it's indicated by the fact there is a simple gray and white checkerboard.

If you need to find a given patch to do some visual operations, the easiest way is actually to use that search field in the editor window. So if I want to paint the background with a gradient, I can simply type the keyword gradient, and then it's going to restrict the list of patches to those who are related to gradients. The one we're going to use is the renderer gradient.

So let's pick up some kind of light blue. And obviously, to edit the middle and end color, we could do as well and double click on each of those circles. But there is an easier way when you want to edit multiple parameters. You actually select the gradient patch, and then you click on the Expector button.

Now in the Inspector button, you can obviously inspect the various aspects of the patch that is currently selected. And you want to look at the input parameters pane. Here you can see all the parameters of the patch at once. So we're just going to take the current color, the current start color, and drag and drop it on the middle color and end color.

Now when I click on the middle color, I can see the usual color picker and I only have to change one parameter which would be the kind of luminosity of the color. And this way I can create my gradient, my blue gradient of three colors. So we have our first step, simply painting the background with a blue gradient.

Now we're going to add some animation to it. An easy way to add animation is to use a particle system. So let's look in the list of patches for particle system and create an instance of it, like for the gradient, simply by drag and dropping it in the workspace.

So a particle system is, as the name implies, a system of particles that evolve in time according to very simple physics law, depending on the speed of the particles, their acceleration, the gravity of the environment, and such. Right now there are absolutely no images that are displayed on the particles, so you just see a bunch of white squares. So we're going to load an image.

And we can simply drag and drop that image file into the workspace. What it does is it actually creates on the fly an image patch that contains, that embeds the data of that particle image. Now what we want to do is to have the image parameter of the particle system be that image we just loaded. So we just create a connection by clicking on the output here and connecting it to image.

Let's get the viewer in foreground. Now we can set the parameters of the viewer itself. So just click on the particle system, make sure it's selected. And then display the input parameters pane, and there we can change the, at the bottom of it, we can change the blending operation to set it to add, which creates a nice saturation effect because all the images are actually being added on top of each other. So you get this white saturation effect. We're also gonna change one parameter, which is the attraction at the very bottom, which is currently set to 0.5, and we're gonna set it to zero.

So now we have a nice kind of star field effect. Next step is going to be to add a simple logo to hide the origin point of that star field. So, same step. You use the Apple logo.png file that is in the materials folder for this tutorial. And we have that image in the workspace and we want to display it in the viewer. The simplest way to just display an image into the viewing area is to use a billboard.

So let's create an instance of that billboard patch. And the billboard is simply a 2D quad that is always facing the viewer that has an image on it. So we configured the billboard to use the image we just loaded. And we need to set the blending parameter of the billboard to over so that we actually have some transparency. And at the same time, we can change the width to be something like 0.5. Sorry, that's better.

So we're slowly getting there and our next step will be to actually add some text there. So how do you create, how do you display, how do you create and display text in Quartz Composer? The way it actually works is by creating an image that contains the text you want to display. For that we have this image with string patch. As the name implies, you pass in some text as a string and then it creates an image that you can display afterwards.

I'm going to show you a second method to display an image. We've seen the billboard, which is the simplest one, always facing the viewer for very basic 2D image display. Another way to display an image on screen is to use the Sprite. So Sprite is like an advanced version of the billboard, except it's 3D. You can position it in a 3D space and send for the orientation.

So it has as well an image input that I'm going to set to display that image. And then I'm going to set the blending to be over as well so that we get transparency. Now you will notice that the image does not appear right. I mean the quality is poor and it appears distorted. What's the reason? Well, imagery string just creates an image of a given dimensions.

Here, 152 pixels by 31 pixels. And Sprite has two input parameters called width and height that actually display determines the area in which the image is going to be displayed. So if you want to have high quality, you need to somehow interconnect those two things so that the display area of Sprite matches the optimal display size of the image we just created from the text.

Hopefully there are those two parameters called display width and display height. And we can just connect them to the width and height of the sprite, And now, we really have the text that is displayed at its optimal size. We need to make it bigger, though. So if you look, if you select the imagery string patch and look in the input parameters pane, you can set here the glyph size. All the units in Quartz Composer, I expressed all the dimensions in Quartz Composer I expressed using what we call Quartz Composer units.

And the Quartz Composer coordinate system is with the 00 at the center of the viewer. Then we have minus 1 on the left, plus 1 on the right, so that's the X axis which is horizontal. And then we got the Y axis which is vertical. And the range of the Y axis actually depends on your aspect ratio.

Here we have something like a 4 by 3 aspect ratio. So the maximum dimension on the Y axis is going to be like 075 and the minimum minus 075. And eventually we have the Z axis that is coming towards the viewer. It's kind of an OpenGL referential system.

So anyway, that means that the size of the font we're using to generate that text is expressed in units. Here instead of 01, I'm going to set it to 02. Some parameters of the patches are not animatable. When you need to edit those, you actually use the inspector and the settings pane. For the image-restoring patch, we actually set up the font to use in the settings pane. So here we're going to change it to, let's see, Ariel Bold would be nice.

Obviously here we need to kind of clean up the viewing area so that we don't have the text on top of the logo and such. It's fairly easy to do. We just select the sprite patch and change the Y parameter to, let's see, minus zero two. That should be enough. And then we can set up the Apple logo to be a little higher, 0.2. Remember it's expressed in units and the Y position as well of the particle system, 0.2. So let's save what we have for now.

Okay. Now we're going to add some kind of halo effect on the text itself and use some core image filters to achieve that. So you see the workspace here is not really crowded yet, but when we add patch going along through the demo, it's gonna become more and more kind of complex to figure out what exactly is happening.

So a good habit is to create what we call macros. Therefore you can partition your workspace into sub functions in a way. So we're gonna put the text into a macro. So what you do is simply select the two patches. Then in the edit menu, you can cut them.

You can create, using the Create Macro button in the toolbar, a blank macro patch. And the way we actually edit a macro patch is simply by double-clicking on it. What we see now in the workspace is the inside of the Macropatch. To know where you are in the workspace when you navigate in the Composition, we have this hierarchy browser here. And you can see root Macropatch, Macropatch. And now what I can do is simply paste the Oops, there we go.

Make sure the focus is on the editor itself, the editing area. And here I can pass the two patches that I just cut. They're not visible in the workspace because they got actually pasted at their original position, which was a little lower in the area. So we said we want to create some kind of hello effect on the text. Simple way to do that is simply to blur the text. So let's look for what kind of core image filters we have to do blurring operations. The basic one is the Gaussian Blur.

So let's take our image and send it through the Gaussian Blur and display the result. So I'm just going to make a copy of that sprite because it already has all the parameters I want. So you press the Option key and you can drag and drop the sprite and create a copy of it. And we want it to display that image.

Same problem as earlier, the width and height are not necessarily correct. The text doesn't really match correctly. So we could just connect the display width and display height to the width and height input there. But it's not going to work perfectly. Why is that? It's because if you look at the original image, here it has dimensions of 300 pixels by 58, approximately.

But after the blurring operation, because the blur expands the image, it's now a little bigger. by around 100 pixels. So the display width and height are not going to be exactly accurate. So how do you get those accurate values? There's hopefully a tool for that called Image Dimensions.

Let me organize the workspace. Here we go. So the image dimensions simply takes an image and returns various dimensions of that image expressed in Quartz Composer units and its pixel as well, in case you really need those. And we can simply take the width and height and connect those.

And now we have the blurred version of the text that is displayed at the appropriate size and that really matches the text that is underneath. The last step to create the real halo effect is to change one setting in the sprite parameter to have -- so that the blending is add instead of over. And now you have the saturation effect as the blurred version of the text is added on top of the non-blurred version.

Last step is going to be to animate this hello effect. What we would like to have is the hello to kind of pulse. And let's see, how can we achieve a pulse effect? Well, simply with the color. The color on the sprite is a color that actually multiplies the image.

If I click on the color here, I can see that by changing its opacity, I can create my pulse effect that I'm looking for. So how can I create that kind of on the fly in the workspace? What you need here is a little patch called RGB Color, which simply takes four values for the RGB and alpha components and outputs a color.

And now I can look at the alpha component and create my blinking effect. So what we need to do, it's obvious by now, is simply animate that alpha value. You have plenty of ways to create animations in Quartz Composer. The one we're going to use here is the LFO, which stands for low-frequency oscillator. It simply generates a wave that oscillates around a given value.

By default it's going to oscillate with an amplitude of 0.5 around the 0.5 point. Therefore it's going to oscillate between 0 and 1, which is what we want because the alpha value here actually expects something between 0 and 1. So now I have my pulsing effect, and I'm just going to slow it down a little by setting the period to 2.

And we can save the results. To go back to the root macro patch, you just press the Edit Parent button here, or you can use the hierarchy browser on the top of the editor to navigate. So we have our Composition, so basic Composition, and we now want to turn it into a parameterized Composition.

So that you've seen here, we have probably about 10 patches, each of those can have like 10 parameters or something, so it's already 100 parameters. But they're all about those 100 parameters, not all of them matters, only a few of them are important. So let's extract them and make them input parameters of the Composition.

How do we do that? Well, let's start with the simplest one to do. We have, typically in that Composition, three parameters. It's the Apple logo, the text, and the overall color of the background. The simplest one to turn into a parameter of the Composition is the Apple logo.

So you actually control click on the billboard to display the contextual menu, and here you have what we call the publishing system for the inputs and outputs. What we're going to do is take that image input of the billboard and publish it as a parameter of the upper level. So I just do that, I'm prompted for a name, and I just keep the one there, image.

The connection is broken, the input is now a dark circle to indicate that it's published, and the input becomes an input of the upper level. The upper level here is the Composition. So if you go in the Viewer window and expand the toolbar and then click on the Parameters button, you can see and edit, actually, the parameters of the Composition itself. And it now has a parameter called Image that expects an image.

And I can just take an image, drag and drop it there, and retrieve my original result, visual result. Let's get rid of the one that is embedded into the Composition. Let's do the next one now. It's going to be the text. So we double click on the macro patch.

And the text we are displaying is actually here. It's the string input of ImageWithString. So same operation, Control-click on ImageWithString. We're going to change the name of that input to text. Now we can go back to the parent level, clicking on the Edit Parent button in the toolbar. And you will notice that the macro patch now has a new input called text.

And we, it is not yet a parameter of the Composition because we only publish to the immediate upper level. So to make it a parameter of the Composition, we need to publish it once again, same operation, control click, published inputs, and then text. And now it has become a parameter here. We can change it, as you can see.

A real parameter is a parameter of the composition. Our last parameter is going to be the colors of the background. Here we have something interesting because this gradient is defined by three colors and obviously it wouldn't be practical to publish those three parameters. While what really matters is a single color and then is from which we build two other colors to use for the gradient.

We have a convenience tool for that called a Splitter. It's basically a one-in-one-out patch that you can use to unify several inputs. So we create one of those and then in the settings panel, because we know we're going to transmit colors through that splitter, we can force its type to be color, which is going to be more practical for later. And now I can just connect my color to my input, to my start color of the gradient patch.

We need to build the middle and the end color as variations from that single color. We have for that a modifier called Color Transformation, which allows you to take a color and modify its hue, saturation, and luminosity. So we can just take the original color, send it through the color transformation, and create the middle color. Now we'll create a copy of it with the Option key and do a similar operation for the end color.

We can go back to the very original color, set it to approximately the same kind of blue we had, something like this. And then we can change the middle color to be the same color but with a luminosity of 0.85, something like that. And the end color to have a luminosity of 0.5.

And now we're back to our gradient. Last operation, make that single color a parameters of the composition and for that, published input and rename it color. So here we go, we have our final composition with the three parameters that actually matters. We're gonna save that. And let's go back to slides, please.

Things to remember. What have we learned so far? Well, Quartz Composer, and that's very important, uses units and not pixels. When you position elements on screen. There is one exception. It's when you use core image filters. The reason is that core image filters actually process pixels inside the image.

So, here, when you want to apply a given effect at a given position inside the image, you specify those coordinates using pixels. But everywhere else, it's units. You can use the macros. You've seen it's fairly simple to organize the workspace and make it less cluttered and easier to understand when you give your composition to someone else.

Publishing inputs and outputs allows you to really build kind of sub-functions inside a Composition using macros. There is the splitter patch when you need to control several inputs at once, which is a very convenient tool. And eventually we've seen how we can create parameters of the Composition. It's for the Composition. It's all about publishing the parameters at the root patch level, parameters of the patch, and they become parameters of the Composition itself.

Now we can go to the second step of our tutorial, which is we created that Composition. What can we do straight out of the box in Tiger with this Composition? Back to Demo Station, please. I'm just going to close. the composition windows. Okay. So the first thing we can do is simply use that composition as a screen saver.

So if you go into the library folder of your home directory and then to the screen savers folder which is not created by default so we have to create it screen savers and I can just drag and drop my QDZ file inside and now when I go to the system preferences in the screen saver pane I can see my new screen saver called demo. It works fine. It's just a composition I just created. What's interesting is that the parameters of the Composition become actually options of the screen saver. So here we have our settings.

Okay, and I can change the image, the color, everything I want. And those values are actually saved as preferences inside - Inside the preference file for the screen server system. It's not changing the Composition in any way. The Composition is completely preserved. Just the input parameters are changed. The second thing I can do is take that Composition file and actually open it with QuickTime. You can simply drag and drop it on the QuickTime player.

So here we have our Composition playing. And it's interesting to notice that when I rescale the Composition here, I mean, I should say the movie, it's not the Composition that is being rendered and then the result is actually scaled by QuickTime. The Composition is really rendered at the final resolution. So when I go full screen here, I still have perfect quality text, very sharp and accurate resolution.

If you can use it in QuickTime then, I mean in a QuickTime player, then you can use it abutely in QuickTime-aware applications. For example, iMovie. So here I have a basic iMovie project, iMovie HD project, 720p resolution, and I can just take my Composition file and drag and drop it into this pen area of iMovie.

Now we have a specificity here to iMovie, which is iMovie needs to convert whatever assets you import that are in a DV file into DV files. So it's actually using QuickTime to rasterize the Composition on the fly as a DV file. And here we go, you got an HD resolution of the Composition.

Pierre-Olivier Latour is a great example of how you can integrate into your iMovie projects. What I actually want to show you is that iMovie file we have here, if we look at the inside, you will find in the media folder the rasterized version of the Composition. If we look at its size, it's actually 30 megabytes approximately.

Now if we look at the original Composition file, it's 16 kilobytes. Actually, it's even 14 kilobytes. So that is one of the key things about Quartz Composer, is the fact you create Composition that purely describes the visuals. They're not rasterized versions, and therefore they're very small, and they can render at various resolution. Back to slides, please.

I would like to look a little more at the QuickTime support for Quartz Composer in Mac OS X Tiger. First of all, it's Mac OS X Tiger. That means no Windows, no Panther, Jaguar, whatever. We have a few limitations there so far. No user events. That means if you create a Composition that uses the keyboard or the mouse, those events are not going to be transmitted to the Composition when running in a QuickTime environment.

We also have a restriction on Internet downloads. If you have a Composition that downloads an RSS feed, for example, it's not going to work inside QuickTime for security reasons. And eventually the Composition -- the parameters, sorry, of the Composition are not editable inside the QuickTime environment in their user interface.

Technically, how does that work, the whole thing of Quartz Composer inside QuickTime? The way it works is you take a Quartz Composer file and you drag and drop it on QuickTime and then it creates a movie that wraps that Composition and stores it into a Quartz Composer movie track, which has the type QDZ. It's a track that's going to render using progressive frames and square pixels. And important, the alpha channel is completely valid, which means if you create a Composition that uses the alpha channel, you can then composite the results safely in QuickTime.

Because QuickTime, when it uses Quartz Composer contents, is kind of using OpenGL, performance can be affected depending on which API you use inside QuickTime. So it's very important that if you intend to have Quartz Composer contents inside QuickTime, you use the new QuickTime 7 APIs for the visual context and the core video APIs. If you are kind of a higher level developer, you simply have to use the Qt Kit for Cocoa applications or the HMI MovieView for Carbon applications. And this will ensure optimal performance of Quartz Composer contents rendering inside a QuickTime environment.

But you can go even further because we've seen, let me explain that, we've seen when you take a Composition file and you actually drag and drop in on QuickTime, it has a default size and duration as it opens in the viewer with like 640 and 30 seconds duration, which are the default. This is necessary because QuickTime does not really have a concept of infinite size and no frame rate contents. So we need to set up some defaults there.

If you need to change those defaults, you can do that through the terminal, but the easiest way is actually to do it on a per Composition file basis. When you are in Quartz Composer, you can select the export as QuickTime movie command from the file menu, and there you will be prompted to enter a specific size and duration for the QuickTime movie that will wrap the Quartz Composer Composition file.

This is essential when you are actually a QuickTime developer and you want your users to be able to take the Composition files and drag and drop them on your application icon and use the contents inside your application. Problem is, you might not be satisfied with 640, 480, and 30 seconds. You might want to have 1024 by 768 for five seconds or something.

The best way to do that is actually to override the defaults at your application level. It's the same operation as if you would do it on the terminal, except you don't need to go through the terminal. It's just two lines of code if you're Cocoa applications, and it's also very simple if you're a Carbon application. For that, you use the NSUserDefaults or the CFPreferences.

You can do part of that using the QuickTime API as well. That would be changing the dimension of the movie using setMovieMatrix or setMovieBox. Once the movie has been created and it contains that Quartz Composer contents. Unfortunately, there is no easy way to set up the duration this way, so that's why I recommend using the NSUserDefaults or CFPreferences approach.

Now that we can import Quartz Composer contents into QuickTime, that opens kind of new doors. For example, here I have a Quartz Composer Composition that does some live video that is simply running as a QuickTime movie inside Keynote. And we have a bunch of core image filters that are applied in real time to create that effect and create the halo, this kind of interlaced green overall thing.

You can see the audio input and the spectrum that is displayed at the very top of that movie. And that's a movie that is ridiculously small as well. That's kind of a far stretch for the technology, but it just shows that it opens brand new door going through QuickTime and Quartz Composer. And that excites some people.

What did we learn through that tutorial? Quartz Composer, obvious use of the Composition files immediate use. Just run and drop that in the screen server folder and off you go. We also have native QuickTime support for Quartz Composer Composition inside Tiger. It's the best way, the simplest way to add motion graphics to existing QuickTime application, because once again, the Composition files are gonna be resolution-independent and frame-rate-independent. No more HD, SD, PAL, NTSC, all those things for the same assets. If you can create the asset using Quartz Composer, you will have a small file that you can just put in your application resources folder and that's it.

So if you're a QuickTime developer, make sure you allow your users to select .qdz files and make sure you use the QuickTime 7 APIs whenever possible to ensure best possible performances. We're now gonna start tutorial number three. And we're going to see how we can use the Quartz Composer Compositions inside our applications.

As a simple application, typical application, I decided to use TextEdit. whose source code is available inside the developer folder on your hard drive and then it's examples and app kit. It's there. So let's just open the project in its code. What we would like to do with Text-Setit is add some Quartz Composer contents to it.

I'm gonna do something fairly simple that really demonstrate what you can do is replace the current About box in Quartz Composer by a new one that displays some motion graphics instead of the simple boring one with the copyright, and that's it. First thing we need to do is edit, obviously, the interface file of TextEdit. The one that we're interested in is edit.neb.

So now we can look at it in Interface Builder. And before we start working with Quartz Composer inside Interface Builder, there is something you need to do once and for all. It's to go into the Preferences of Interface Builder, then go to the Palettes tab, then click Add, and go to Developer, Extras, Palettes, and Quartz Composer. This will load the Quartz Composer palette inside Interface Builder so that you can use the QC view and the QC patch controller, which are the two Quartz Composer elements.

So let's create that new About box. For that we just need a window. That's it. In that window, we're going to put from the Quartz Composer palette a QC view. QC view is the basic way when you need to render some Quartz Composer contents into a Cocoa application. It's a subclass of NSView that takes care of everything.

So we just resize it appropriately, then we can set up its parameter. And for that, we need the Interface Builder Inspector from the Tools menu. And here in the attributes of the QC view, we can specify which Composition should be rendered. So we just click on the Load button and select the one we created.

Now the last step regarding the modification of the Nib file is to make sure that when you click If you want to create a new window for the About Texted It, it displays that new window instead of the standard About Panel. You select About Texted It in the menu bar, and then in the Connections panel of the Inspector, Target Action Subpanel, we can see that it's currently connected to Order Front Standard About Panel.

To break that connection, create a new one. For that, you Control-click on About Texted It and drag a new connection to the panel and select Make Key and Order Front. What we just did is modify the Nib file so that when you select About Texted It, it sends the message Make Key and Order Front to that panel. So we can save the modified .need file and just run text to it.

So here we're getting an error. It basically says that it's not able to find the QC view class when unarchiving the Nib file. It's a common error. The reason is it's missing the framework, the Quartz Composer framework. We need to add it to the project when you use Quartz Composer contents.

For that, we're going to go to the external frameworks folder here and then select project, add to project. And the Quartz Composer framework is actually part of an umbrella framework called Quartz Framework. And it has all the framework in system library frameworks and then Quartz Framework. And we add it. I think I need to do a clean build.

Okay, and I should be able to run the build application. Now I went and select About Text Edit. I do have my brand new Motion Graphics fantastic About Panel. We can do more. Remember that Composition, the one we created, is a parameterized Composition. It has three parameters: the color, the text, and the image. So let's set those parameters from the application. We're simply going to modify two files, controller.h and controller.m, which defines, as the name implies, the controller for that application.

And we're going to communicate with the QC view from inside the application. So first thing is to add an outlet member to that interface, to the interface of the controller class that actually points to the QC view. Oops. And don't forget to include the Quartz framework as well.

Okay, we can save the results. Go back in IAB and take that controller, that modified controller, that edge file, and drag and drop it in the area of the Nib file. And then we click on Merge. That basically updates the definition of the controller class inside the Nib file.

And that means I can now control click on the controller and create a connection to the QC view inside my panel. - And save the result. Now back in my hosting application, I have this QC view member of my controller class that points to the QC view inside the about window.

And a good place to talk to the QC view is the, let's see, application did finish launching. It's perfect, just as a setup time. So here I can take my QC view. and set the parameters of the Composition. The methods are fairly simple. Let's see, set value for input key is the one you want.

Let's see, the first one we're going to set is the text input, the text input parameter of the Composition, and we're going to give it the value of the current username, let's say. should be full username, here we go. Then we're also gonna set the background color to something different. So let's see, any color, red color would be fine, some kind of red.

And we'll finish by changing the image displayed by the about box to use, well the best here is to use the icon of the application. And there is a method to retrieve it called application something. Here we go, application icon image. Okay. And now we can build and run. Here we go.

So that was fairly simple. Back to the slides, please. Let's have a deeper look at the QCview API for a minute. We've seen that if you want to set parameters, input parameters of the Composition, it's simply a matter of calling setValueForInputKey where you pass the key that identifies uniquely that input parameter.

If the Composition were to produce some results that wouldn't be visuals and output some values, you would retrieve those simply by calling valueForOutputKey. And eventually, I actually forget to mention that is when you go into Interface Builder and you set up the QCview to render a given Composition file, the Composition file is actually saved inside the Nib file. You don't need to move the .qdz file around. Now, in some cases, you want to load that Composition file dynamically. It's also quite easy to do. It's two lines of code. Just calling the method loadCompositionFromFile on the QCview.

When you call set value for input key or value for output key, what is the correspondence between the type of values that are actually moved around in Quartz Composer and the app kit values? Well, here you have the table and it's fairly straightforward. You've seen for color input in Quartz Composer, you can pass an NSColor. For a string input in Quartz Composer, you just pass anything pretty much, an NSString or anything. For an image, you can pass an NSImage, a Core Graphics Image, a Core Image Image as well, or even Core Video Image buffers. It's really flexible.

Things to remember, that's pretty sad. Now things to really remember, we have Quartz Composer, which runs only on Quartz Extreme configurations. Tiger does support a few, there aren't that many, but a few machines that are not Quartz Extreme compatible. So if you intend to run Quartz Composer on, add some Quartz Composer contents to your application and have your application run on those machines, you need to check for the support of Quartz Extreme. This is an easy way to do it, just one method, part of Core Graphics, which is written true if basically Quartz Extreme is available on that machine.

We've also seen how simple it was to communicate data from the hosting application to the Composition. So that means that when you create a Composition and you're stuck because you don't know Quartz Composer enough yet and you're not sure how to create the data the way you want or process it the way you want, try to simply take that out of the Composition and put it in the hosting application. You do that processing using C code or Objective-C, whatever you prefer, and then simply pass the result to Quartz Composer.

Something quite important as well, it's the fact that when you call setValueForInputKey or valueForOutputKey, as the name of those methods imply, you're actually identifying the inputs and outputs by the keys, which is not necessarily the name. The name can be localized. So an input, if for example, you create -- you publish an input in Quartz Composer and you give it the name, My, and space color, this is not going to be the actual key. Because there are some restrictions of the kind of characters that can be used and everything. So the real name is likely going to be My_color.

To know what is actually the key for an input port or output port and not its name, you can use the tooltips for that in the editor. You just hover the mouse and you will see in a tooltip the real key. And the common pitfalls here when you're using Quartz Composer in your applications, don't forget to add the Quartz framework in your Xcode project and include the header as well.

We can now move on to the last tutorial of this session. You're not expected to follow that tutorial and doing it as I'm gonna do it because it's a fairly big thing. And we're actually gonna look at the overview. The idea is to build an application from scratch that really leverages the Quartz Composer power. And a nice example with that would be to build our own kind of TV channel system. So let's look at that.

In Quartz Composer Editor, we're obviously going to start with a blank composition. Now, you know those TV channels, news channels, they all have something that immediately would make ours look professional. It's that crawler at the bottom that just goes very fast and displays tons of information that nobody can really read anyway.

So let's exactly reproduce that so that we can have something that looks professional. We need some data first, some data to display. In Quartz Composer you have RSS support, which means you can give the URL of an RSS feed and it's going to download it for you, process it, parse it, and give you some data you can display. Okay, let me organize my windows.

This is the patch that downloads the given RSS feed and processes it. We need to extract the information we want to display and then display it on screen. So I'm going to do those few steps and then explain to you exactly how it goes. - There we go. Then we're gonna have some text we need to display. So as usual, we use image with string in combination with, let's say, a billboard. That would be fine.

So how does that little pipeline work? At the start of the pipe, we have the RSS feed patch. It downloads the RSS and parses it, as I said. It outputs on its article list output a structure actually describing the feed contents. This structure is itself made of substructure. We have one substructure per article.

The way you manipulate structures in Quartz Composer is by using a few patches like StructureIndexMember and StructureKeyMember. They allow you to simply retrieve an element in a structure using its index or using its identifying key. Pierre-Olivié Latour Pierre-Olivié Latour Then we can create an image that contains that title and eventually display that image on screen using the billboard. It's a fairly straightforward process, as you've seen. We're just going to fix the, well actually no, let's not fix the transparency. Displaying one news is one scene. What we want to do here is display all the news and have them scroll.

We obviously need to have that index change with time. There are plenty of ways to do that. I'm going to do it using actually the current time in the system that you can retrieve this patch time and then doing a few mathematical operations on it. The idea is I take the current time, Then I can divide that value, for example, by five so that it goes -- the current time goes way slower.

And use that as the index. Now that may not be enough because the index here has to be in a given range. If you try to, here the current value is like 45, but there are no items at index 45. So we need to know the number of items in a structure.

You're right, thanks. Here we go. And then our last operation here is to actually take that value and make sure it doesn't go above the number of elements in the, the number of articles. So let's just do a modulo. So now we have our display system that displays an item from the RSS feed that changes every five seconds.

There is one last thing to do here. It's when you convert a floating point value to an integral value in Quartz Composer, it runs to the closest value. So when you have, for example, here 7.2, it's gonna run to seven, and when you have 7.6, it's gonna run to eight. In our case, so that we know exactly when the item changes, we just wanna make sure we always round, we do the rounding ourself and always take the floor value.

Okay, last step is to make that text scroll on the window. It's fairly simple to do. We're going to need to change the Y position with the time. The X position, sorry. When we need to have a value go from a given value to another value, there is interpolation patch.

We just specified a start value here, let's say 2, so that we start on the far right of the viewer and we end on -2, which is the far left, and a duration which has to match the one we defined earlier, which is 5 seconds. And we want it to loop, that's fine.

So now I have my kind of RSS scrolling crawler. After a few more steps of iterations, we will end up with the final RSS crawler, which kind of looks like this. Whoops, let me... Constraint-Aspect Ratio. Here we go. Which will look like this. With a nice shadow, a background, and everything.

So we're going to start directly from this one and go on with building our prototype. So I create a blank document and I just load my RSS patch that I created previously. And it actually has an image input so that I can specify a logo to put between the news titles. That's one part of the contents. Now we need to really build the rest, like the background, the weatherman, and everything. I like gradients, so we're going to use a gradient for our background. Some kind of usual blue gradient.

Pierre-Olivié Latour: Okay. Here we have something interesting. We don't see the crawler anymore. It's simply due to the fact the gradient is rendered after the crawler was drawn. How do you change the rendering order of items in Quartz Composer? Well, you use the contextual menu on a patch and then you will have a rendering layer submenu where you can select the rendering layer. So here we changed it to be one.

Let's create a title bar at the top of our TV system. If you need to draw some kind of colored shape, which is a square shape, Sprite is perfect for that. We can use it to display some kind of orange shape. We need to change the width to be two so that it's full width, and the height could be zero, one should be enough. Then we can simply position it like this. We need to add, obviously, the logo for our TV channel.

Here we go. Display it using the billboard as we've seen earlier. Simplest way to display a 2D image. And then we need some blending. Change the size to, let's see, whoops, that's a little small. Something like that. And then position it at the correct place. Okay, we need the name of our TV, so let's create an image that contains some text and display it using a billboard.

So, same step as we've seen earlier. Combination of billboard and image with string. Except we want it to be a little more on the right, like this. Okay, it's looking pretty good. I'm just going to move things on the workspace so that it's easier to follow. Now it's time to add our weatherman. Okay. So I got an image here I'm going to use and display it using the billboard as well.

Make that a little smaller, add the transparency as usual, and position it. We need to fix the rendering layer so that it's just on top of the background below the RSS feed. Okay, this is starting to look like something, but we obviously need some realistic environment. We'll start with our fabulous weatherman.

Set the width to 2 so that it's full size. Then we can set it on the background. Almost realistic, except usually when you film someone like this, the background is blurred because the focus is on the weatherman. So let's use a Core Image Filter, the same one we've used earlier, which is the blur. Kind of blur that image.

And what you see is actually where Blur Extend, it's kind of gonna sample outside and it's black by default. So these are the things you actually fix when you have the concept pretty much ready. But for now it's enough. So you continue, same as earlier, when we built the RSS Scroller, you continue iterating a few steps, getting your ideas right inside Quartz Composer, and eventually you end up with the, shall we call that kind of final result? Okay, something very realistic.

So here we have our final Composition that is live, obviously. You can see it's the current time here. And it's nicely organized. It has been divided into several macros. Each of the macro is performing one operation like drawing the title, drawing the info box here, the yellow info box, the RSS feed, the caption here, and everything.

So at that point, we started with the idea of let's kind of build a TV channel thing, and we experimented with Quartz Composer, getting a prototype of how the final result could look like. And it's now time to build the application around it. Can we go back to slides, please? So we start from that point. We got our Composition, kind of a monolithic object that does everything.

Is that really practical to manipulate? Well, actually not, because if you want to have a graphic artist, for example, modify just the background or the scrolling thing at the bottom, it's easier to work on separate pieces. So what we're going to do is divide this Composition into five sub-Compositions, one for each visual part, like the background, the title and such.

In our hosting application, we will need to recompose those various Compositions. The way to do that is you use the, you've seen so far the QC view, which is one API for Quartz Composer, and there is another API called the QC renderer, which is lower level. It allows you more control over how the rendering is done and when it is done. So here we're going to use one QC renderer for each of those sub-Composition files. The QC renderer is going to render into an OpenGL context, so this is fully accelerated.

Then because we're going to call those five QC renderers in order, we're actually going to end up inside the OpenGL context with the same image pretty much as the one we had at the beginning. So if we connect that OpenGL context to an on-screen display, well, you're pretty much done because you see on screen the final result. If your video output of your computer is connected to some kind of TV system, then the result is, you can use the result immediately. But we can do more than that.

We can actually take the frames that are inside the OpenGL context and process them through QuickTime. Let's have a look at that. So as I said earlier, we have on one side the OpenGL context where the rendering happens, and this is optimized, hardware accelerated, and it runs on the GPU. When we want to do frame processing, like compress them and using QuickTime and such, this happens on the CPU. So the first step is to bring back the frames from the GPU to the CPU.

For that, we're going to use a component that I call the OpenGL Frame Reader. This component takes the OpenGL context, read the frames inside, and produces CV pixel buffer. CV pixel buffers are core video pixel buffers that are like simple memory wrappers around a block of pixels in memory. It's your usual base address row bytes thing.

Then we can take those CV pixel buffers and pass them to the new QuickTime 7 compression session APIs. They're going to take the contents of those buffers and compress them using whatever codec and quality you want. So we have a quick time movie. We can also, if the compression is DV, actually send those frames directly on the FireWire DV port. And then if you have any kind of DV hardware that has a video input, I mean that has a DV input, then the frames are ready to be used on that hardware.

Let me go back to the demo station, please. I'm going to show you now that we have the idea, we build a prototype in Quartz Composer, we have a clear idea how to build the architecture of the hosting application, we can look at the final project, which is right here.

Okay. So as we said earlier, we have obviously the part of the application that's going to take care of creating the QC renderers and rendering them in the correct order and such. So that part is in your usual app controller class of the Cocoa application. Then we have the frame processing part. There are a few classes here.

The first one is the one we talked about that takes the contents inside the OpenGL context, downloads it, and outputs core video frames. It's the frame reader class. Then we have another class that is going to take those frames and compress them using the QuickTime Compression Session APIs. So that's the frame compressor class. This is actually an abstract class in the way this application is designed.

So what I did is subclassed it and create two subclasses. One that takes the resulting compressed frames and sends them on the FireWire DV port, and the other one that simply appends them to a movie file. And the last thing we need is two classes to allow us to build UI dynamically on the fly given a Composition so that you can edit the Composition parameters inside the application. Let's look at the final result.

So this is our final application, Quartz Composer TV. You can see here, let me just get, we're not really in a nice environment to do keying, so what I'm going to do is use actually a movie. Okay, left here. And I configured the Composition that takes care of the contents to actually have an input where you can say, take the live video input or take just a movie to operate on for demonstration purposes. And we can do also some basic keying.

Get rid of the green and here we go. And whoops, sorry about that. And we obviously need to fix kind of the position and scale and everything so that your weatherman is perfectly positioned. And all the parameters here of the various compositions are available through that UI, which was built on the fly using the classes I described to you earlier. With that result, we can output it to a live FireWire TV stream or to a QuickTime movie. So I'm going to take the second case as an example and just render in 320 to 240. That would be enough.

And then select QuickTime Movie. Set it on the desktop. Okay, so we can select H.264 compression, 15 frames per second, that's enough for demonstration. And now the rendering that happens inside Quartz Composer TV is visible on screen. The frames are grabbed, downloaded from the video card, processed through QuickTime, compressed as H.264, and eventually written to disk inside a movie.

So H.264 is a pretty expensive compression, so in real time you won't get like 60 frames per second or something like that. But it's a good technological demonstration. Now I can stop the export and look at the result. So here I have my movie that was generated in real time. I have 15 frames per second, about.

So what we've seen so far, it's a pretty big example of what you can do with Quartz Composer. I tried to design it so that you really can reuse everything from it. The sample code is unfortunately not available in your CDs right now, but it's going to be available in the next few days on the usual developer website and the sample code area.

Can we go back to slides, please? Things to remember. It's fairly easy to prototype in Quartz Composer. Experiment with ideas, have some graphical display with it, get an idea of what you want, what's the final result you would like. But at some point, you're gonna have to switch to implementation, and you're gonna have to consider some design choices.

Do you wanna have a very big composition that you manipulate, or do you want to explode it like I did in several subcompositions? Might be easier. You also need to figure out where you put the line between the features done inside Quartz Composer and the features done inside your host application. That's fairly important.

For those of you who are OpenGL developers, you might be interested in knowing that the QC renderer actually renders in the current viewport. So that allows you to somehow render the Composition in a sub area of your OpenGL context in case you're interested in that. And in case you want to do your own OpenGL drawing on top or below the QC renderer drawing, you can definitely do that. And the QC renderer is saving, restoring all the OpenGL stacks, stats, except the very simple ones, the current ones. I provide as part of this sample application, a class that allows you to build UI on the fly so that you can edit the parameters and save them.

So that's pretty useful and you can reuse that easily into your own applications. One thing we did not cover today, but that is definitely worth having a look at is the Cocoa binding features. Quartz Composer is completely compatible with Cocoa bindings. The way it works is through a QC view and a QC patch controller in Interface Builder.

You will find into the Quartz Composer programming guide, a tutorial on how to use Cocoa bindings. And in simple cases that wouldn't have worked in a kind of very complex application, like, I mean, very complex, fairly complex application like this, but in a simple case, you can do, you can build UI even faster directly in Interface Builder and have the UI control directly the composition parameters.

More information about Quartz Composer you will find on the usual developer.apple.com/wwdc site. All the pointers to the documentations, new sample code that was released these days, and other resources. We have a few sessions that are related to Quartz Composer. The first one being the Quartz Composer Lab, when you can come and ask questions about Quartz Composer to developers. It's actually following this session. It's today at 3:30. And then we have other labs where you will be able to ask Quartz Composer content as well, like the OpenGL and Quartz Composer Lab on Friday morning.

Who to contact when it comes to questions about Quartz Composer? The best contact is Travis, who is our graphics and imaging evangelist. You can also contact me directly, but there is actually a mailing list, which is a much better way to ask questions about Quartz Composer. Contact me directly or contact Travis. You will find this mailing list on list.apple.com. It's called the Quartz Composer Dev List.