Good afternoon, everyone. Welcome to the Chorus Composer hands-on session. Let's get started immediately. We've got a lot of content to see today. So first of all, what is Quartz Composer? I assume that most of you are actually familiarized by now about this motion graphics technology, especially since the demos yesterday during the graphics and media keynote. But let's just go over this quick overview once again. So Quartz Composer is an operating system-level motion graphics technology. And operating system level is very important. It's not like a side technology or something like that. It's integrated into Tiger. It was built around two key things. Open GL to get high performance and realtime graphics. And at the same time it was built about this idea of integrating all of our graphics technologies through a common technology. What you do with Squares Composer is create what we call compositions. Compositions basically render visuals on screen. Now those compositions are not necessarily black boxes. They actually can be interactive with external user devices, the basic ones being the mouse or the keyboard. But they can also communicate with the host application, because those compositions can have what we call input parameters or produce output results which are not necessarily visuals. The way you create compositions in Tiger is by using our full features graphical editor, part the developer tool. We also tried to make sure that composition would be easily supported by various applications in Tiger, and for that, we have QuickTime compatibility that is Tiger only, and we'll look at that in details later during this presentation. And we also have a very easy to use API so that you can integrate Quartz Composer into your applications.

So, as I said earlier, Quartz Composer is really integrating several technologies. If you look at this simple graph here, we have most of our graphics technology from OpenGL and the new one we introduced in Tiger, Core Image, Core Video and such. And also some technologies that are not necessarily graphics, like simply JavaScript or some basic spotlight queries. And all those technologies are kind of bridged through Quartz Composer. And then the Quartz Composer system is used by a variety of hosts. Right now we have three hosts and we look at those in detail during the presentation.

There's going to be the screen saver engine. There's going to be QuickTime and finally your applications. So what's interesting here is that from a developer point of view, instead of having to learn all the various APIs for all those technologies, you can simply learn the Quartz Composer graphical editing system and learn the easy to use playback API and leverage all those APIs and don't have to take care about how they need to interact together, how you connect them together and just use that very easily in your application. But I did actually found on the internet pretty nice description of what Chorus Composer could be.

Now, that said, what are we going to learn today? This hands-on session is divided into four tutorials. The first one would be creating a parameterized composition using Quartz Composer. So it's a little more than your basic composition. It's one that has actually input parameters. Then we're going to look at where and how we can use this composition straight out of the box in Tiger.

We're also going to look at using the playback API so that you can have even better integration of Quartz Composer inside your applications. And we'll finish by building what I called a kind of a complete Quartz Composer application from scratch that really takes advantage of the technology. And hopefully after these four tutorials, you'll be able to add Quartz Composer support to your applications and create new applications with compelling motion graphics visual. It's kind of important because some people outside, some users are really waiting for you to come up with those applications.

OK. Let's start with tutorial number one, which is, as I said, creating a parameterized composition. So the first thing we're going to do is obviously launch Course Composer. And for that, you simply go to the developer folder and then the applications folder, graphics tools folder, and you will find the editing application right there. You should double click to launch. You can click cancel on the assistant panel. We're actually not going to use the assistant for this tutorial. And let's close that window as well. Okay. Now, in Quartz Composer, you can create a blank document simply by doing New in the File menu.

So here we have the typical Quartz Composer environment with the viewer window over there that actually shows what the composition is rendering, its output visuals in a way. And here we have the editing window, which is where you assemble the various Quartz Composer components that we actually call patches, and you connect them together, and which are gonna build a workflow that's eventually gonna create visuals.

So let's get started and create some visuals. The first thing we're going to do is simply paint the background with a gradient. Because right now, there is absolutely nothing rendered by the composition. As you can see, it's indicated by the fact there is a simple gray and white checkerboard.

If you need to find a given patch to do some visual operations, the easiest way is actually to use that search field in the editor window. So if I want to paint the background with a gradient, I can simply type the keyword "gradient," and then it's going to restrict the list of patches to those who are related to gradients. The one we're going to use is the renderer gradient.

Now, we have on screen a gradient, a vertical gradient that goes from black to white, and we're going to change those colors. Here you see the patch of the gradient patch, and you can see all its input parameters. The one we are interested in is the start color. To change it, you can simply double click on the dot, and you will have an editor that appears here because this parameter is a color, we actually get the usual color picker. If it was a number, you would get a text field where you can enter a number and such.

So let's pick up some kind of light blue. And obviously, to edit the middle and end color, we could do as well and double click on each of those circles. But there is an easier way when you want to edit multiple parameters. You actually select the gradient patch, and then you click on the expector button.

Now, in the Inspector button, you can obviously inspect the various aspects of the patch that is currently selected, and you want to look at the input parameters pane. Here you can see all the parameters of the patch at once. So we're just going to take the current color, the current start color, and drag and drop it on the middle color and end color.

Now when I click on the middle color, I can see the usual color picker, and I only have to change one parameter, which would be the kind of luminosity of the color. And this way I can create my gradient, my blue gradient of three colors. So we have our first step, simply painting the background with a blue gradient. Now we're going to add some animation to it. An easy way to add animation is to use a particle system. So let's look in the list of patches for particle system and create an instance of it, like for the gradient, simply by drawing and dropping it in the workspace.

So a particle system is, as the name implies, a system of particles that evolve in time according to very simple physics law, depending on the speed of the particles, their acceleration, the gravity of the environment, and such. Right now there are absolutely no images that are displayed on the particles, so you just see a bunch of white squares. So we're going to load an image. Okay. You will find on your archive, you download it from the WWDC website, the various materials that are required for this tutorials. So the one we want is particle.png in the Quartz Composer WWDC composition folder.

And we can simply drag and drop that image file into the workspace. What it does is it actually creates on the fly an image patch that contains, that embeds the data of that particle image. Now, what we want to do is to have the image parameter of the particle system be that image we just loaded. So we just create a connection by clicking on the output here and connecting it to image.

Let's get the viewer in the foreground. OK. Now we can set the parameters of the viewer itself. So just click on the particle system. Make sure it's selected. and then display the input parameters pane, and there we can change the, at the bottom of it, we can change the blending operation to set it to add, which creates a nice saturation effect because all the images are actually being added on top of each other. So you get this white saturation effect. We're also gonna change one parameter, which is the attraction at the very bottom, which is currently set to 0.5, and we're gonna set it to zero.

So now we have a nice kind of star field effect. Next step is going to be to add a simple logo to hide the origin point of that star field. So same step, you use the Apple logo.png file that is in the materials folder for this tutorial. And we have that image in the workspace, and we want to display it in the viewer. The simplest way to just display an image into the viewing area is to use a billboard.

So let's create an instance of that billboard patch. And the billboard is simply a 2D quad that is always facing the viewer that has an image on it. So we configured the billboard to use the image we just loaded. And we need to set the blending parameter of the billboard to over so that we actually have some transparency. And at the same time, we can change the width to be something like 0.5. Sorry, that's better.

So we're slowly getting there, and our next step will be to actually add some text there. So how do you create, how do you display, how do you create and display text in Quartz Composer? The way it actually works is by creating an image that contains the text you want to display. For that, we have this image with string patch. As the name implies, you pass in some text as a string, and then it creates an image that you can display afterwards.

I'm going to show you a second method to display an image. We've seen the billboard, which is the simplest one, always facing the viewer for very basic 2D image display. Another way to display an image on screen is to use the sprite. So sprite is like an advanced version of the billboard except it's 3D. You can position it in a 3D space and send for the orientation.

So it has as well an image input that I'm going to set to display that image. And then I'm going to set the blending to be over as well so that we get transparency. Now, you will notice that the image does not appear right. I mean, the quality is poor and it appears distorted. What's the reason?

Well, imagery string just creates an image of a given dimensions. Here, 152 pixels by 31 pixels. And sprite has two input parameters called width and height that actually display determines the area in which the image gonna display it. So if you want to have file quality, you need to somehow interconnect those two things so that the display area of sprite matches the optimal display size of the image which was created from the text. Hopefully there are those two parameters called display width and display height, and we can just connect them to the width and height of the sprite.

And now we really have the text that is displayed at its optimal size. We need to make it bigger, though. So if you look -- if you select the imagery string patch and look in the input parameters pane, you can set here the glyph size. All the units in Quartz Composer are expressed -- all the dimensions in Quartz Composer are expressed using what we call Quartz Composer units. And the Quartz Composer coordinate system is with the 00 at the center of the viewer.

Then we have minus 1 on the left, plus 1 on the right, so that's the X axis which is horizontal. And then we got the Y axis which is vertical. And the range of the Y axis actually depends on your aspect ratio. Here we have something like a 4 by 3 aspect ratio. So the maximum dimension on the Y axis is going to be like 075 and the minimum minus 075. And eventually we have the Z axis that is coming towards the viewer. It's kind of an OpenGL referential system.

So, anyway, that means that the size of the font we're using to generate that text is expressed in units. Here instead of 01, I'm going to set it to 02. Some parameters of the patches are not animatable. When you need to edit those, you actually use the inspector and the settings pane. For the image restring patch, we actually use the -- we actually set up the font to use in the settings pane. So here we're gonna change it to, let's see, Arial bold would be nice. Obviously, here we need to kind of clean up the viewing area so that we don't have the text on top of the logo and such. It's fairly easy to do. We just select the sprite patch and change the Y parameter to, let's see, -02. That should be enough. And then we can set up the Apple logo to be a little higher, 0.2. Remember, it's expressed in units, and the y position as well of the particle system, 0.2. So let's save what we have for now.

Okay. Now we're going to add some kind of halo effect on the text itself and use some core image filters to achieve that. So you see the workspace here is not really crowded yet, but when we add patch going along through a demo, it's gonna become more and more kind of complex to figure out what exactly is happening. So a good habit is to create what we call macros. Therefore, you can partition your workspace into sub functions in a way. So we're gonna put the text into a macro. So what you do is simply select the two patches. Then in the edit menu, you can cut them.

you can create, using the Create Macro button in the toolbar, a blank macro patch. And the way we actually edit a macro patch is simply by double-clicking on it. What we see now in the workspace is the inside of the macro patch. To know where you are in the workspace when you navigate in the composition, we have this hierarchy browser here. And you can see root macro patch, macro patch. And now what I can do is simply paste the -- Oops. There we go. Make sure the focus is on the editor itself, the editing area. And here I can pass the two patches that I just cut. They're not visible in the workspace because they got actually pasted at their original position, which was a little lower in the area. So we said we want to create some kind of hello effect on the text. Simple way to do that is simply to blur the text. So let's look for what kind of core image filters we have to do blurring operations. The basic one is the Gaussian blur.

So let's take our image and send it through the Gaussian blur and display the result. So I'm just going to make a copy of that sprite because it already has all the parameters I want. So you press the Option key, and you can drag and drop the sprite and create a copy of it. And we want it to display that image.

Same problem as earlier, the width and height are not necessarily correct. The text doesn't really match correctly. So we could just connect the display width and display height to the width and height input there. But it's not going to work perfectly. Why is that? It's because if you look at the original image, here it has dimensions of 300 pixels by 58 approximately. But after the blurring operation, because the blur expands the image, it's now a little bigger. by around 100 pixels. So the display width and height are not going to be exactly accurate. So how do you get those accurate values? There is hopefully a tool for that called image dimensions.

organize the workspace. Here we go. So the image dimension simply takes an image and returns various dimensions of that image expressed in Quartz Composer units and its pixel as well in case you really need those. And we can simply take the width and height and connect those. And now we have the blurred version of the text that is displayed at the appropriate size and that really matches the text that is underneath. The last step to create the real halo effect is to change one setting in the sprite parameter to have so that the blending is add instead of over. And now you have the saturation effect as the blurred version of the text is added on top of the non-blurred version.

Last step is going to be to animate this hello effect. What we would like to have is the hello to kind of pulse. And let's see, how can we achieve a pulse effect? Well, simply with the color. The color on the sprite is a color that actually multiplies the image. If I click on the color here, I can see that by changing its opacity, I can create my pulse effect that I'm looking for. So how can I create that kind of on the fly in the workspace? What you need here is a little patch called RGB color, which simply takes four values for the RGB and alpha components and outputs a color.

And now I can look at the alpha component and create my blinking effect. So what we need to do, it's obvious by now, it's simply animate that alpha value. You have plenty of ways to create animations in the course composer. The one we're gonna use here is the LFO, which stands for low-frequency oscillator. It simply generates a wave that oscillates around a given value.

By default, it's gonna oscillate with an amplitude of 0.5 around the 0.5 point. Therefore, it's going to OCI between 0 and 1, which is what we want because the alpha value here actually expects something between 0 and 1. So now I have my pulsing effect, and I'm just going to slow it down a little by setting the period to 2.

and we can save the result. To go back to the root macro patch, you just press the Edit Parent button here, or you can use the Yorki browser on the top of the editor to navigate. So we have our composition, so basic composition, and we now want to turn it into a parameterized composition. So that you've seen here, we have probably about 10 patches. Each of those can have like 10 parameters or something. So it's already a hundred parameters, but they're all about those hundred parameters. Not all of them matters. Only a few of them are important. So let's extract them and make them input parameters of the composition.

How do we do that? Well, let's start with the simplest one to do. We have, typically in that composition, three parameters. It's the Apple logo, the text, and the overall color of the background. The simplest one to turn into a parameter of the composition is the Apple logo. So you actually control click on the billboard to display the contextual menu. And here you have what we call the publishing system for the inputs and outputs. What we're gonna do is take that image input of the billboard and publish it as a parameter of the upper level. So I just do that, I'm prompted for a name, and I just keep the one there, image.

The connection is broken, the input is now a dark circle to indicate that it's published, and the input becomes an input of the upper level. The upper level here is the composition. So if you go in the viewer window and expand the toolbar and then click on the parameters button, you can see and edit, actually, the parameters of the composition itself. And it now has a parameter called image that expects an image, and I can just take an image, drag and drop it there, and retrieve my original result, visual result. Let's get rid of the one that is embedded into the composition. Let's do the next one now. It's gonna be the text. So we double click on the macro patch.

And the text we are displaying is actually here. It's the string input of ImageWithString. So same operation, Control-click on ImageWithString. we're gonna change the name of that input to text. Now we can go back to the parent level, clicking on the Edit Parent button in the toolbar. And you will notice that the macro patch now has a new input called text.

And it is not yet a parameter of the composition because we only published to the immediate upper level. So to make it a parameter of the composition, we need to publish it once again, same operation, control click, published inputs, and then text. And now it has become a parameter here. And we can change it, as you can see. A real parameter is a parameter of the composition. Our last parameter is going to be the colors of the background. Here we have something interesting, because this gradient is defined by three colors, and obviously it wouldn't be practical to publish those three parameters, while what really matters is a single color, and then from which we build two other colors to use for the gradient.

We have a convenience tool for that called a splitter. It's basically a one-in, one-out patch that you can use to unify several inputs. So we create one of those, and then in the settings panel, because we know we're going to transmit colors through that splitter, we can force its type to be color, which is going to be more practical for later. And now I can just connect my color to my input, to my start color of the gradient patch.

We need to build the middle and the end color as variations from that single color. We have for that a modifier called color transformation, which allows you to take a color and modify its hue, saturation, and luminosity. So we can just take the original color, send it through the color transformation, and create the middle color. Now we'll create a copy of it with the Option key and do a similar operation for the end color.

We can go back to the very original color, set it to approximately the same kind of blue we had, like something like this. And then we can change, say, the middle color to be the same color, but with a luminosity of 0.85, something like that, and the end color to have a luminosity of 0.5. And now we're back to our gradient. Last operation, make that single color parameters of the composition, and for that, published input and rename it color. So here we go, we have our final composition with the three parameters that actually matters. We're gonna save that. And let's go back to slides, please.

Things to remember. What have we learned so far? Well, Quartz Composer, and that's very important, uses units and not pixels when you position elements on screen. There is one exception. It's when you use core image filters. The reason is that core image filters actually process pixels inside the image. So, here, when you want to apply a given effect at a given position inside the image, you specify those coordinates using pixels. But everywhere else, it's units. You can use the With the macros, you've seen it's fairly simple to organize the workspace and make it less cluttered and easier to understand when you give your composition to someone else.

Publishing inputs and outputs allows you to really build kind of sub functions inside the composition using macros. There is the splitter patch when you need to control several inputs at once, which is a very convenient tool. And eventually we've seen how we can create parameters of the composition. It's for the composition. It's all about publishing the parameters at the root patch level, parameters of the patch, and they become parameters of the composition itself.

Now we can go to the second step of our tutorial, which is we created that composition. What can we do straight out of the box in Tiger with this composition? Back to Demo Station, please. I'm just gonna close. the composition windows. Okay. So the first thing we can do is simply use that composition as a screen saver. So if you go into the library folder of your home directory and then to the screen savers folder, which is not created by default, so we have to create it. Screen savers. Okay. And I can just drag and drop my QDZ file inside. And now when I go to the system preferences in a screen server pane, I can see my new screen server called demo. And it works fine, it's just a composition I just created. What's interesting is that the parameters of the composition become actually options of the screensaver. So here we have our settings.

Okay, and I can change the image, the color, everything I want. And those values are actually saved as preferences inside inside the preference file for the screen server system. It's not changing the composition in any way. The composition is completely preserved. Just the input parameters are changed. Second thing I can do is take that composition file and actually open it with QuickTime. You can simply drag and drop it on the QuickTime player.

So here we have our composition playing. And it's interesting to notice that when I rescale the composition here, I mean, I should say the movie, it's not the composition that is being rendered and then the result is actually scaled by QuickTime. The composition is really rendered at the final resolution. So when I go full screen here, I still have perfect quality text, very sharp and accurate resolution.

If you can use it in QuickTime, then, I mean in a QuickTime player, then you can use it abusively in QuickTime-aware applications. For example, iMovie. So here I have a basic iMovie project, iMovie HD project, 720p resolution, and I can just take my composition file and drag and drop it into this pen area of iMovie.

Now, we have a specificity here to iMovie, which is iMovie needs to convert whatever assets you import that are in a DV file into DV file. So it's actually using QuickTime to rasterize the composition on the fly as a DV file. And here we go. You got an HD resolution of the composition.

that you can integrate into your iMovie projects. What I actually want to show you is that iMovie file we have here, if we look at the inside, you will find in the media folder the rasterized version of the composition. If we look at its size, it's actually 30 megabytes approximately.

Now, if we look at the original composition file, it's 16 kilobytes. Actually, it's even 14 kilobytes. So that is one of the key things about Quartz Composer, is the fact you create composition that purely describes the visuals. They're not rasterized versions, and therefore they're very small, and they can render at various resolution. Back to slides, please.

I would like to look a little more at the QuickTime support for Quartz Composer in Mac OS X Tiger. First of all, it's Mac OS X Tiger. That means no Windows, no Panther, Jaguar, whatever. We have a few limitations there so far. No user events. That means if you create a composition that uses the keyboard or the mouse, those events are not going to be transmitted to the composition when running in a QuickTime environment. We also have a restriction on internet downloads.

If you have a composition that downloads an RSS feed, for example, it's not going to work inside QuickTime for security reasons. And eventually, the parameters of the composition are not editable inside the QuickTime environment. in their user interface. Technically, how does that work, the whole thing of Quartz Composer inside QuickTime? The way it works is you take a Quartz Composer file and you drag and drop it on QuickTime, and then it creates a movie that wraps that composition and stores it into a Quartz Composer movie track, which has the type QDZ. It's a track that's going to render using progressive frames and square pixels, and important, the alpha channel is completely valid, which means if you create a composition that uses the alpha channel, you can then composite the results safely in quick time.

Because QuickTime, when it uses Quartz Composer contents, it's kind of using OpenGL, performance can be affected depending on which API you use inside QuickTime. So it's very important that if you intend to have Quartz Composer contents inside QuickTime, you use the new QuickTime 7 APIs for the visual context and the core video APIs. If you are kind of a higher level developer, you simply have to use the Qt Kit for Cocoa applications or the movie view for Carbon applications. And this will ensure optimal performance of Quartz Composer contents rendering inside a QuickTime environment. But you can go even further because we've seen, let me explain that. We've seen when you take a composition file and you actually drag and drop in on QuickTime, it has a default size and duration as it opens in the viewer with like 640 and 30 seconds duration, which are the default. This is necessary because QuickTime does not really have a concept of infinite size and no frame rate contents. So we need to set up some defaults there. If you need to change those defaults, you can do that through the terminal, but the easiest way is actually to do it on a per composition file basis. When you are in Quartz Composer, you can select the export as QuickTimeMovie command from the file menu, and there you will be prompted to enter a specific size and duration for the QuickTimeMovie that will wrap the Quartz Composer composition file.

This is essential when you are actually a QuickTime developer, and you want your users to be able to take the composition files and drag and drop them on your application icon and use the contents inside your application. Problem is you might not be satisfied with 640, 480, and 30 seconds. You might want to have 1024 by 768 for five seconds or something. The best way to do that is actually to override the defaults on your application level. It's the same operation as if you would do it on the terminal, except you don't need to go through the terminal. It's just two lines of code if you're Cocoa applications, and it's also very simple if you're a Carbon application.

For that, you use the NSUserDefaults or the CFPreferences. You can do part of that using the QuickTime API as well. That would be changing the dimension of the movie using setMovieMatrix or setMovieBox. Once the movie has been created and it contains that Quartz Composer contents. Unfortunately, there is no easy way to set up the duration this way. So that's why I recommend using the NSUserDefaults or CFPreferences approach.

Now that we can import QuickTime contents into Quartz -- Quartz Composer contents into QuickTime, that opens kind of new doors. For example, here I have a Quartz Composer composition that does some live video that is simply running as a QuickTime movie inside Keynote. And we have a bunch of core image filters that are applied in real time to create that effect and create the halo, this kind of interlaced green overall thing. You can see the audio input and the spectrum that is displayed at the very top of that movie. And that's a movie that is ridiculously small as well. That's kind of a far stretch for the technology, but it just shows that it opens brand new door going through QuickTime and Chorus Composer. And that excites some people.

What did we learn through that tutorial? Quartz Composer, obvious use of the composition files, immediate use, just run and drop that in the screen server folder and off you go. We also have native QuickTime support for Quartz Composer Composition inside Tiger. It's the best way, the simplest way to add motion graphics to existing QuickTime applications. Because once again, the Composition files are going to be resolution independent and frame rate independent. No more HD, SD, PAL, NTSC, all those things for the same assets. If you can create the asset using Quartz Composer, you will have a small file that you can just put in your application resources folder and that's it. So if you're a QuickTime developer, make sure you allow your users to select.QDZ files and make sure you use the QuickTime 7 APIs whenever possible to ensure best possible performances. We're now going to start tutorial number three. And we're gonna see how we can use the Quartz Composer compositions inside our applications.

As a simple application, typical application, I decided to use TextEdit. whose source code is available inside the developer folder on your hard drive, and then it's examples and app kit. It's there. So let's just open the project in Xcode. What we would like to do with Text-Setit is add some Quartz Composer contents to it. I'm gonna do something fairly simple that really demonstrate what you can do is replace the current About box in Quartz Composer by a new one that displays some motion graphics instead of the simple boring one with the copyright, and that's it. First thing we need to do is edit, obviously, the interface file of TextEdit. The one that we're interested in is edit.neb.

So now we can look at it in Interface Builder. And before we start working with Quartz Composer inside Interface Builder, there is something you need to do once and for all. It's to go into the preferences of Interface Builder, then go to the palettes tab, then click add and go to developer, extras, palettes, and Quartz Composer. This will load the Quartz Composer palette inside Interface Builder so that you can use the QC view and the QC patch controller which are the two Quartz Composer elements.

So let's create that new about box. For that, we just need a window. That's it. In that window, we're gonna put, from the Quartz Composer palette, a QC view. QC view is the basic way when you need to render some Quartz Composer contents into a Cocoa application. It's a subclass of NSView that takes care of everything.

So we just resize it appropriately, then we can set up its parameter. And for that, we need the interface builder inspector from the tools menu. And here in the attributes of the QC view, we can specify which composition should be rendered. So we just click on the load button and select the one we created.

Now, the last step regarding the modification of the Nib file is to make sure that when you click Texted it, about texted it, it actually displays that new window instead of the standard about panel. So you actually select about texted it in the menu bar. And then in the connections panel of the inspector, target action sub panel, we can see that it's currently connected to order front standard about panel. And we need to break that connection and create a new one.

So for that, you control click on about text edit and draw a new connection to the panel and select make key and order from. What we just did is modified the Nib file so that when you select about text edit, it sends the message, make key and order from to that panel. So we can save the modified need file and just run TextedIt.

So here we're getting an error. It basically says that it's not able to find the QC view class when archiving the Nib file. It's a common error. The reason is it's missing the framework, the Quark's Composer framework. We need to add it to the project when you use Quark's Composer contents.

For that, we're gonna go to the external frameworks folder here and then select project, add to project. And the Quark's Composer framework is actually part of an umbrella framework called Quartz Framework. And it is as all the framework in system library frameworks and then Quartz Framework. And go, and we add it. I think I need to do a clean build.

Okay, and I should be able to run the build application. Now when I select about TextEdit, I do have my brand new motion graphics, fantastic about panel. But we can do more. Remember that composition, the one we created, is a parameterized composition. It has three parameters, the color, the text, and the image. So let's set those parameters from the application.

We're simply going to modify two files, controller.h and controller.m, which defines, as the name implies, the controller for that application. And we're gonna communicate with the QC view from inside the application. So first thing is to add an outlet member to the interface of the controller class that actually points to the QC view. And don't forget to include the course framework as well.

Okay, we can save the results. Go back in IAB and take that controller, that modified controller, that edge file, and drag and drop it in the area of the Nib file. And then we click on merge. That basically updates the definition of the controller class inside the Nib file. And that means I can now control click on the controller and create a connection to the QC view inside my panel. and save the result. Now back in my hosting application, I have this QC view member of my controller class that points to the QC view inside the about window.

And a good place to talk to the QC view is the, let's see, application did finish launching. It's perfect, just as a setup time. So here I can take my QC view, and set the parameters of the composition. The methods are fairly simple. Let's see, set value for input key is the one you want.

Let's see, the first one we're going to set is the text input, the text input parameter of the composition, and we're going to give it the value of the current user name, let's Should be full username, here we go. Then we're also gonna set the background color to something different. So let's see, NSColor, red color would be fine, some kind of red.

And we'll finish by changing the image displayed by the about box to use-- well, the best here is to use the icon of the application. And there is a method to retrieve it called application something. Here we go, application icon image. OK. And now we can build and run. Here we go.

So that was fairly simple. Back to the slides, please. Let's have a deeper look at the QCview API for a minute. We've seen that if you want to set parameters, input parameters of the composition, it's simply a matter of calling set value for input key, where you pass the key of the, that identifies uniquely that input parameter. If the composition were to produce some results that wouldn't be visuals and output some values, you would retrieve those simply by calling value for output key. And eventually, I actually forget to mention that, is when you go into Interface Builder and you set up the QCview to render a given composition file, the composition file is actually saved inside the Nib file. You don't need to move the.qdz file around. Now, in some cases, you want to load that composition file dynamically. It's also quite easy to do. It's two lines of code, just calling the method loadCompositionFromFile on the QCview.

When you call set value for input key or value for output key, what is the correspondence between the type of values that are actually moved around in Quartz Composer and the app kit values? Here you have the table and it's fairly straightforward. You've seen for color input in Quartz Composer, you can pass an NS color. For a string input in Quartz Composer, you just pass anything pretty much, an NSString or anything. For an image, you can pass an NSImage, Core Graphics Image, Core Image Image as well, or even Core Video Image buffers. It's really flexible.

Things to remember, that's pretty simple. Now, things to really remember, we have Quartz Composer, which runs only on Quartz Extreme configurations. Tiger does support a few, there aren't that many, but a few machines that are not Quartz Extreme compatible. So if you intend to run Quartz Composer on, add some Quartz Composer contents to your application and have your application run on those machines, you need to check for the support of Quartz Extreme. This is an easy way to do it, just one method. part of Core Graphics, which is written true, if basically Quartz Extreme is available on that machine.

We've also seen how simple it was to communicate data from the hosting application to the composition. So that means that when you create a composition and you're stuck because you don't know Quartz Composer enough yet, and you're not sure how to create the data the way you want or process it the way you want, try to simply take that out of the composition and put it in the hosting application. You do that processing using C code or Objective-C, whatever you prefer, and then simply pass the result to Quartz Composer.

Something quite important as well, it's the fact that when you call set value for input key or value for output key, as the name of those methods imply, you are actually identifying the inputs and outputs by the keys, which is not necessarily the name. The name can be localized. So an input, for example, you create -- you publish an input in Quartz Composer and you give it the name my and then space color, this is not going to be the actual key because there are some restriction of the kind of characters that can be used on everything. So the real name is likely gonna be my_color.

To know what is actually the key for an input port or output port and not its name, you can use the tool tips for that in the editor. You just hover the mouse and you will see in a tool tip the real key. And the common pitfalls here when you're using Quartz Composer in your applications, don't forget to add the Quartz framework in your Xcode project and include the header as well.

We can now move on to the last tutorial of our session. You're not expected to follow that tutorial and doing it as I'm gonna do it because it's a fairly big thing. And we're actually gonna look at the overview. The idea is to build an application from scratch that really leverages the Quartz Composer power. And a nice example with that would be to build our own kind of TV channel system. So let's look at that.

into Chorus Composer Editor, we are obviously gonna start with a blank composition. Now, you know those TV channels, news channel, they all have something that immediately would make ours look professional. It's that crawler at the bottom that just goes very fast and display tons of information that nobody can really read anyway. So let's exactly reproduce that so that we can have something that looks professional. We need some data first, some data to display. In a Chorus Composer, you have RSS support, which means you can give the URL of an RSS feed and it's gonna download it for you, process it, parse it, and give you some data you can display. Okay, let me organize my windows.

This is the patch that downloads the given RSS feed and processes it. We need to extract the information we want to display and then display it on screen. So I'm going to do those few steps and then explain to you exactly how it goes. There we go. Then we're gonna have some text we need to display. So as usual, we use image with string in combination with, let's say a billboard. That would be fine.

So, how does that little pipeline work? At the start of the pipe, we have the RSS feed patch. It downloads the RSS and parses it, as I said. It outputs on its article list output a structure actually describing the feed contents. This structure is itself made of substructure. We have one substructure per article.

The way you manipulate structures in Quartz Composer is by using a few patches like structure index member and structure key member. They allow you to simply retrieve an element in a structure using its index or using its identifying key. So here, the first step is to, among the list of articles, to retrieve one at a given index. So here we're retrieving article number zero. Then we get this, a substructure, which contains in that case four elements, the title of the article, the description, the link, and its time, basically, timestamp. The one we're interested in is the title. So we use structure key member to extract the title member of that structure. we retrieve it here on the output, then we can create an image that content that title and eventually display that image on screen using the billboard. It's a fairly straightforward process, as you've seen. We're just gonna fix the, well, actually, no, let's not fix the transparency.

displaying one uses one saying what you want to do here is display all the news and have them scroll we obviously need to have that index change with time. There are plenty of ways to do that. I'm gonna do it using actually the current time in the system that you can retrieve this patch time, and then doing a few mathematical operations on it. The idea is I take the current time, then I can divide that value, for example, by five, so that it goes, the current time goes way slower.

and use that as the index. Now, that may not be enough, because the index here has to be in a given range. If you try to, here, the current value is like 45, but there are no items at index 45. So we need to know the number of items in a structure.

You're right, thanks. Here we go. And then our last operation here is to actually take that value and make sure it doesn't go above the number of elements in the, the number of articles. So let's just do a modulo. So now we have our display system that displays an item from the RSS feed that change every five seconds.

There is one last thing to do here. It's when you convert a floating point value to an integral value in Quartz Composer, it runs to the closest value. So when you have, for example, here, 7.2, it's gonna run to seven, and when you have 7.6, it's gonna run to eight. In our case, so that we know exactly when the item changes, we just wanna make sure we always round, we do the rounding ourself and always take the floor value.

Okay. Last step is to make that text scroll on the window. It's fairly simple to do. We're going to need to change the Y position with the time. The X position, sorry. Thank you. When we need to have a value go from a given value to another value, there is interpolation patch.

We just specified a start value here, let's say two, so that we start on the far right of the viewer, and we end on minus two, which is the far left, and a duration which has to match the one we defined earlier, which is five seconds. And we want it to loop, that's fine.

So now I have my kind of RSS scrolling crawler. After a few more steps of iterations, we will end up with the final RSS crawler, which kind of looks like this. Whoops, let me... Constraint aspect ratio. Here we go. - Which will look like this. - With a nice shadow, a background, and everything.

So we're going to start directly from this one and go on with building our prototype. So I create a blank document and I just load my RSS patch that I created previously. And it actually has an image input so that I can specify a logo to put between the news titles. Okay. That's one part of the contents. Now we need to really build the rest, like the background, the weatherman, and everything. I like gradients, so we're gonna use a gradient for our background. Some kind of usual blue gradient.

Here we have something interesting. We don't see the crawler anymore. It's simply due to the fact the gradient is rendered after the crawler was drawn. How do you change the rendering order of items in Quartz Composer? Well, you use the contextual menu on a patch, and then you will have a rendering layer sub-menu where you can select the rendering layer. So here we changed it to be one.

Let's create a title bar at the top of our TV system. If you need to draw some kind of colored shape, which is a square shape, Sprite is perfect for that. We can use it to display some kind of orange shape. We need to change the width to be two so that it's full width, and the height could be zero, one should be enough. Then we can simply position it like this. We need to add, obviously, the logo for our TV channel.

Here we go. Display it using the billboard as we've seen earlier. Simplest way to display a 2D image. And then we need some blending. Change the size to, let's see, whoops, that's a little small. Something like that. And then position it at the correct place. Okay, we need the name of our TV. So let's create an image that contains some text and display it using a billboard.

So same step as we've seen earlier. Combination of billboard and image with string. except we want it to be a little more on the right, like this. Okay, it's looking pretty good. Thank you. I'm just going to move things on the workspace so that it's easier to follow. All right. Now it's time to add our weatherman. Okay. So I got an image here I'm gonna use and display it using the billboard as well.

make that a little smaller, add the transparency as usual, and position it. We need to fix the rendering layer so that it's just on top of the background below the RSS feed. Okay, this is starting to look like something, but we obviously need some realistic environment. behind our fabulous weatherman. So let's see. That should do. Set the width to two so that it's full size. Then we can set it on the background. It would be like this. Okay. Almost realistic, except usually when you film someone like this, the background is blurred because the focus is on the weatherman. So let's use a Core Image Filter, the same one we've used earlier, which is the blur. Kind of blur that image.

And what you see is actually where blur extend, it's kind of gonna sample outside and it's black by default. So these are the things you actually fix when you have the concept pretty much ready. But for now it's enough. So you continue same as earlier when we built the RSS scroller, you continue iterating a few steps, getting your ideas right inside Quartz Composer, and eventually you end up with the, shall we call that kind of final result? OK. Something very realistic.

So here we have our final composition that is live, obviously. You can see it's the current time here. And it's nicely organized. It has been divided into several macros. Each of the macro is performing one operation, like drawing the title, drawing the info box here, the yellow info box, the RSS feed, the caption here, and everything. So at that point, we started with the idea of, let's build a TV channel thing, and we experimented with Quartz Composer, getting a prototype of how the final result could look like. And it's now time to build the application around it. Can we go back to slides, please?

So we start from that point. We got our composition, kind of a monolithic object that does everything. Is that really practical to manipulate? Well, actually not, because if you want to have a graphic artist, for example, modify just the background or the scrolling thing at the bottom, it's easier to work on separate pieces. So what we're going to do is divide as composition into five sub-compositions, one for each visual part, like the background, the title, and such. In our hosting application, we will need to recompose those various compositions.

The way to do that is you use the, you've seen so far the QC view, which is one API for course composer. And there is another API called the QC renderer, which is lower level. It allows you more control over how the rendering is done and when it is done. So here we're gonna use one QC renderer for each of those subcomposition files. The QC renderer is going to render into an OpenGL context, so this is fully accelerated.

then because we're gonna call those five QC renderers in order, we actually gonna end up inside the OpenGL context with the same image pretty much as the one we had at the beginning. So if we connect that OpenGL context to an onscreen display, well, you're pretty much done because you see on screen the final result. If your video output of your computer is connected to some kind of TV system, then the result is, you can use the result immediately.

But we can do more than that. We can actually take the frames that are inside the OpenGL context and process them through QuickTime. Let's have a look at that. So as said earlier, we have on one side the OpenGL context where the rendering happens and this is optimized, hardware accelerated, and it runs on the GPU. When we want to do frame processing, like compress them and using QuickTime and such, this happens on the CPU. So the first step is to bring back the frames from the GPU to the CPU. For that, we're going to use a component that I call the OpenGL frame reader. This component takes the OpenGL context, read the frames inside and produces CV pixel buffer. CV pixel buffers are core video pixel buffers that are like simple memory wrappers around a block of pixels in memory. It's your usual base address row bytes thing.

Then we can take those CV pixel buffers and pass them to the new QuickTime 7 compression session APIs. They're going to take the contents of those buffers and compress them using whatever codec and quality you want. And this component outputs ICM encoded frames. We have now compressed frames. We can simply write them to disk into a QuickTime movie. Just app on the frames and here you go, you have a QuickTime movie. We can also, if the compression is DV, actually send those frames directly on the Fire Warrior DV port. And then if you have any kind of DV hardware that has a video input, I mean that has a DV input, then the frames are ready to be used on that hardware.

Let me go back to the demo station, please. I'm going to show you now that we have the idea, we build a prototype in Quartz Composer, we have a clear idea how to build the architecture of the hosting application, we can look at the final project, which is right here.

Okay. So as we said earlier, we have obviously the part of the application that's going to take care of creating the QC renderers and rendering them in the correct order and such. So that part is in the usual app controller class of the Cocoa application. Then we have the frame processing part. There are a few classes here. The first one is the one we talked about that takes the contents inside the OpenGL context, and outputs core video frames. It's the frame reader class. Then we have another class that is gonna take those frames and compress them using the QuickTime Compression session APIs. So that's the frame compressor class. This is actually an abstract class in the way this application is designed. So what I did is subclassed it and create two subclasses, one that takes the resulting compressed frames and send them on the FireWire DV port, and the other one that simply append them to a movie file. And the last thing we need is two classes to allow us to build UI dynamically on the fly, given a composition, so that you can edit the composition parameters inside the application. Let's look at the final result. Okay.

So, this is our final application, Watch Composer TV. You can see here, let me just get -- we're not really in a nice environment to do keying, so what I'm going to do is use actually a movie. And I configure the composition that takes care of the contents to actually have an input where you can say, take the live video input or take just a movie to operate on for demonstration purposes. And we can do also some basic keying.

get rid of the green, and here we go. And whoops, sorry about that. And we obviously need to fix kind of the position and scale and everything, so that your weatherman is perfectly positioned. And all the parameters here of the various compositions are available through that UI, which was built on the fly using the classes I described to you earlier. With that result, we can output it to a live FireWire TV stream or to a QuickTime movie. So I'm going to take the second case as an example and just render in 320 to 240.

and then select QuickTime Movie, save it on the desktop. Okay, so we can select H.264 compression, 15 frames per second, that's enough for demonstration. And now the rendering that happens inside Quartz Composer TV is visible on screen. The frames are grabbed, downloaded from the video card, processed through QuickTime, compressed as H.264, and eventually written to disk inside a movie.

So H.264 is a pretty expensive compression, so in real time you won't get like 60 frames per second or something like that. But it's a good technological demonstration. Now I can stop the export and look at the result. Subtitles by the Amara.org community So here I have my movie that was generated in real time, at 15 frames per second, about.

So what we've seen so far, it's a pretty big example of what you can do with Quartz Composer. I tried to design it so that you really can reuse everything from it. The sample code is unfortunately not available in your CDs right now, but it's gonna be available in the next few days on the usual developer website and a sample code area. Can we go back to slides, please?

Things to remember. It's fairly easy to prototype in Quartz Composer. Experiment with ideas, have some graphical artists play with it, get an idea of what you want, what's the final result you would like. But at some point, you're gonna have to switch to implementation and you're gonna have to consider some design choices. Do you wanna have a very big composition that you manipulate or do you wanna explode it like I did in several sub-compositions? Might be easier. You also need to figure out where you put the line between the features done inside Quartz Composer and the features done inside your host application. That's fairly important.

For those of you who are OpenGL developers, you might be interested in knowing that the QC renderer actually renders in the current viewport, so that allows you to somehow render the composition in the sub-area of your OpenGL context in case you're interested in that. And in case you want to do your own OpenGL drawing on top or below the QC renderer drawing, you can definitely do that, and the QC renderer is saving, restoring all the OpenGL stats except the very simple ones, the current ones. I provide as part of this sample application a class that allows you to build UI on the fly so you can edit the parameters and save them. So that's pretty useful and you can reuse that easily into your own applications. One thing we did not cover today but that is definitely worth having a look at is the Cocoa binding features.

Quartz Composer is completely compatible with Cocoa bindings. The way it works is through a QC view and a QC patch controller in interface builder. You will find in the course programming guide a tutorial on how to use Cocoa bindings. In simple cases that wouldn't have worked in a very complex application like this, but in a simple case, you can build UI even faster directly in interface builder and have the UI control directly the composition parameters.

More information about Quartz Composer, you will find on the usual developer.apple.com/wwdc site, all the pointers to the documentations, new sample code that was released these days and other resources. We have a few sessions that are related to Quartz Composer. The first one being the Quartz Composer Lab, when you can come and ask questions about Quartz Composer to developers. It's actually following this session. It's today at 3:30. And then we have other labs where you will be able to ask Quartz Composer contents as well, like the OpenGL and Quartz Composer Lab on Friday morning.

Who to contact when it comes to questions about Quartz Composer? The best contact is Travis, who is our graphics and imaging evangelist. You can also contact me directly, but there is actually -- oh, we're missing that slide. Okay. There is actually a mailing list, which is a much better way to ask questions about Quartz Composer. Contact me directly or contact Travis. You will find this mailing list on list.apple.com. It's called the Quartz Composer Dev List.