Good afternoon. Welcome to the second day of WWDC. My name is Derek Clegg, and I'm a Quartz engineer. I've been working in Quartz for quite some time now, hence my title. Today I'm going to talk about the new things that we've added to Tiger to help you as a developer do more cool stuff.

In addition, we're going to have my colleague John Burkey come up for the second half to show you more about Quartz 2D and how it's actually leveraging the Quartz Extreme part of the system to do even more cool stuff. And then along the way we're going to have a number of demos.

So the first thing that you, of course, need to remember, as we've emphasized already in other sessions, is that our focus in Mac OS X is Quartz 2D. It's no longer Quick Draw. Quick Draw is a great technology, served us really well for 20 years. It's crystals going dark. It's time to sort of move to Quartz 2D.

And so we really want you to think toward doing that if you haven't already. And of course, if you have done that, we certainly want to continue supporting you and making more and more stuff part of Quartz 2D to help you out. There's a special session on Friday at 2, transitioning to Quartz 2D, which will go into this in more depth and help those of you who haven't already started the transition make the transition.

Now, I'm also not going to talk today too much about the details of Quartz2D, the architecture, the basic types, how all that fits together. There's a session that I gave last year at WWDC that talks a lot about all of that information, so you can certainly pick up the DVD and check that out. And in addition, perhaps more importantly, this year we've added a lot of really great documentation. There's some brand new stuff that's all about drawing with Quartz2D, the transition to Quartz2D, there's a full reference of all the APIs, and there's a lot more.

So if you have any issues like that, sort of basic understanding, I really urge you to go to the documentation, it's great, and of course, check out the DVD last year as well. So my focus today is going to be all the new APIs that we've added in Tiger, sort of the new things that are coming down the road that you can expect to start using.

Now just though, to give you some context, we have a CG context, is the basic type in Quartz2D, that's what everything uses to do drawing. So CG context is sort of... I'm going to go back to the coin of the realm in a certain way. It's by nature, by its architecture, device independent.

What that means is that you use a CG context if you're talking to a printer, or to the screen, or to a bitmap, sort of the same basic type for all of those things. So it sort of abstracts the drawing environment for you. In addition, very importantly, it's resolution independent. Use a CG context with the same APIs to draw to a 100 DPI screen or a 1600 DPI printer.

And in addition, it's stateful. It maintains state for you. It has what we call the graphic state, which is things like the color, what color you're going to fill shape in, or whether your line is dashed or not, things like that. And most of the APIs in Quartz take a CG context and work with that.

So what are the new APIs that we're going to be adding for Tiger? Well, there are going to be some new drawing APIs that work with CG context directly, a new thing, a layer API, which is sort of a more efficient representation of off-screen bitmap drawing, and some new PDF input and output APIs for those of you who are used to using those. Okay, so this is sort of the list of what I'm going to cover for the drawing APIs. There's a lot of things, so let's just go into each one independently.

So the first is a new clipping API. The idea here, very straightforward concept. You have a mask, a soft mask of some sort, and you want to use that as a clipping region. In the past, in Panther, clipping was always sort of path based. You could clip to a circle or clip to rectangles or to a much more complex path, but it wasn't very easy to do a clip directly to a mask. So we've added API in Tiger to let you clip to a mask. And it works just like every other clipping region, clipping in Quartz.

It's intersected with the current clipping region. It obeys G-save-G-restore semantics. So in that respect, it's identical. It's just that now you sort of get the power of a soft mask for your drawing operations in addition to sort of the standard clip path concept that we've already had. So we think this will be very useful for certain people who need to be able to do this type of operation.

We've also added some new path APIs. And the idea, the sort of the overarching theme of some of the things we're going to be talking about is make simple things simple. Quartz is a very, very powerful API. It has lots of things available in it. But the problem is that sometimes a little bit, the water gets deep really fast. It's hard to know where to start sometimes. We've had people say, well, if I want to draw a circle, I have to learn about Bezier paths and cubic equations. And I don't know what to do about that.

So that's a bad thing. We want to make it easy for people to do simple things. So one of the goals for some of the APIs we've added is make simple things even simpler. So for Tiger, we're adding APIs to let you do circles just directly without having to sort of worry about Bezier paths and so on. Of course, ellipses are just a variation of a circle. We've also added APIs to let you do lots of lines stroked very, very rapidly. And you'll see a very good demo of how that's used a little bit later when John comes up to talk.

But of course, we want to keep simple things simple. But we also don't want to neglect people who have more advanced needs. And so one of the things we've added for paths is hit testing. So you can find out if a point is inside a path. So you can do, you know, you want to drag. Oh, yay. And also, those are some of the advanced developers. And also, we want to be able to allow you to replace the current path with a stroked version of the path, again, for hit testing or for special shading, stuff like that.

Now again, along the same theme of keeping simple things simple, we've added some APIs to let you get access to some of the basic fundamental color sync color spaces. Color sync has sort of three fundamental color spaces, the generic gray, generic RGB, generic CMYK. And it used to be in Panther sort of difficult to get access to these.

You'd have to sort of go to color sync and muck around in color sync, come back to core graphics, muck around in core graphics, and after you've done all that mucking around, you finally, oh look, you've got a color space. So what we've done is made it very, very simple to get access to the color space without having to go through all of that sort of gyration. And that way you can guarantee that you can easily get a color space that gives you high fidelity color everywhere.

Now that's important because our future direction is high quality color, high fidelity color everywhere across all devices. We are no longer trying to model sort of the device idea where you get a different color if you're printing versus a different color versus on your monitor. We want to make sure that you get a high quality match color everywhere you go. To that end, we want you to avoid using device color spaces and to help you with that, we're actually going to replace the use of device color spaces automatically for you with the generic RGB, generic gray, and so on.

So that even if you don't care too much about color, and a lot of people sort of are willing to sort of let the system handle a lot of that for them, you'll still get the same results on all monitors, on all printers, without actually having to do anything very difficult yourself. Now an example of how easy it is though to transition to use the sort of not used device color but instead use the more specific color spaces.

In the case of the CIGI ColorSpace Create, you might use the function CIGI ColorSpace Create Device RGB by its name. It's a device color space. The results will look different on different output devices. That's a bad way. Don't do that anymore. The good way, the tiger way, is to call CIGI ColorSpace Create with name and you pass in the name of the color space, in this case generic RGB. So it's very simple. Not a lot of code changes, not much difference, and what you end up with is a color that will look the same across all platforms.

So this is definitely the direction we're headed, and so we want you to be able to come along with us in that. For image APIs, we've added a number of new ways to do image drawing and work with images, and so I'll go into each one of these independently. So the first is chroma key masking. This is, you know, some people call this blue screen. It's a very straightforward idea. You have someone stand in front of a blue screen, and here we have two men shaking hands.

And then when you display it, you subtract out the blue color, and everything that was blue becomes transparent. So this is something that, you know, everybody sort of is familiar with from TV and so on. And so what we've added in Tiger is a simple way to create an image and specify a key color, the color you want to make transparent, and end up with an image that works just like every other image, which has those parts transparent. The function of CG image create with masking colors, and it's a very easy way to get chroma key masking. for your images, if you will.

In addition, this is something that falls in the category of making simple things simple. It's a very obvious idea. It fits into the previous example we had with clipping. CG image create with mask lets you take an image and apply a mask to it, a soft mask, any depth that's currently supported with image masks, and what you get out is a masked image.

This is something that maybe some people are familiar with from QuickDraw. It's definitely something that took a lot more code, too much code to do in Panther. So for Tiger, we've added a single function, one shot, CG image create with mask, and you get exactly this effect. Very powerful, but very simple as well.

Good. We're adding it for you. Another API to help work with images in certain cases, particular cases, is the idea of a sub-image of an image. This is a pretty common idea if you maybe have cached some complicated drawing in a single bitmap, and perhaps you want to use that different regions of that bitmap for different drawing as you're doing your work.

And here we imagine an example where someone's drawing a font, maybe it's for their cache, and they have nine letters, and what they want to do is sort of pick out each one independently. So there's a now new function, cgImageCreate with image in rect. You start out with an image, and you choose a rectangle, and what you end up with is an image that works just like every other image in Quartz.

It's just a piece of that parent image. The nice thing is that all of these sub-images sort of share the same parent, so you're not making copies of all the data. You can keep it all in one place. So that's a very powerful API for those cases. Thank you.

And then finally with images, again this is something that we really want to make much easier. Bitmap context, we've told people in Quartz 2D, if you want to do off screen drawing, if you want to do some sort of caching like that, use a bitmap context. Later we'll tell you about something new that's even better than bitmap context, but there are still cases where it's appropriate to use a bitmap context. And you do your drawing in the bitmap context and we just sort of, you know, blithely say, "Oh, and then create an image from it and draw with that image." And it turns out that's a lot of code. It's not so simple.

And particularly it's not so simple to sort of make sure the color spaces match your destination and all that sort of stuff. So what we've done for Tiger is added one function that does all that work for you, CG bitmap context create image. You pass in a bitmap context and out pops an image from it. So it's very simple now to sort of do what we've always just waved our hands about in the past. This is a very powerful function, but the thing you need to remember is that it is a copy operation.

Now that's not as bad as it sounds, but it's a copy operation because CG images must be immutable. We expect that an image will be reused -- or rather, excuse me, we expect that a CG image will be uploaded to video RAM once. And then every time that image is drawn multiple times, we don't reupload it. That saves a huge amount of traffic through the bus to the video card. If your image is changed, then we can't do that.

We have to reupload every time. So we're enforcing -- it's always been true in the past, but we're enforcing now the concept that a CG image must be immutable. So that's something that's very important for you to check out in your application. Make sure that you're not doing something where you're changing the context of an image. Because if you start doing that, then your hardware acceleration will really suffer.

Now, when I say it's a copy operation, it's not really a copy. It's a copy and write. It uses the, you know, VM magic of Mach to copy and write. So you don't actually really make a copy. Instead, you have two things that point to the same in-memory destination. And if anybody writes on one of them, then a copy happens. But until something is written, then nothing -- you don't get a copy. You both share the same underlying data structures.

So that's really useful because you can sort of create an image from a bitmap context, draw it somewhere, release it, and you actually have never made a true copy. Another thing that we're adding for images, this is again sort of more of in the advanced category, is what we call deep pixel support.

This is deep in the sense that it's now floating point pixels for both image content and for bitmap context. So we already support in Panther the standard integer formats, you know, one to, well actually up to 32, one to 32 bits per pixel, sorry, bits per component. For Panther, we're now letting you specify a 32-bit floating point value, and the floating point value can be, you know, you can have red, green, blue, alpha as your floating point values, you can see in YK floating point. Pretty much anything that we currently support in integer, we also support in floating point. Same thing for output. For bitmap contexts, you can create a floating point bitmap context that you might use. Now the--excuse me.

The--the The advantage to this is that now for image formats that are starting to come online where you're actually bringing in, you know, more of these higher resolution images, we now can support those natively. Now, the other thing that's important to note is that this is not sort of high dynamic range imaging.

That's a very specialized thing. It's used in, it's sort of more a color sync, color science type thing to sort of figure out how to map your big floating point range to sort of visible. That's a problem that's best left to the color sync guys. There's a special session on that.

Where they talk about that more in depth. What we do is something very simple. We just take the values between zero and one, and that becomes the image. Anything less than zero, we clamp to zero. Anything greater than one, we clamp to one. So we're not actually mucking around with the data ourselves. We let you do that or color sync do that, other parts of the system that are more capable.

Excuse me. Now another thing that we have had sort of missing in the past is image I/O, input and output of image data. As you know, some of you we've been able to support as input, JPEG natively, PNG natively, into Quartz 2D. Raw image data is sort of a big buffer of red, green, blue alpha data, not encoded in any particular file format. For output, we haven't had a way to do any image outputs. Of course, we've been able to output PDF files, which are tremendously rich and complicated. But for something like a simple JPEG output, nothing. So for Tiger, we want to remedy that.

In Tiger, we're going to add support for JPEG, PNG, TIFF, GIF, JPEG 2000, a bunch of other formats supported by QuickTime and other formats as we need. Of course, obviously as well, the floating point types, excuse me, the floating point formats such as OpenEXR and the floating point version of TIFF for output. In the WWDC build, we support JPEG and TIFF output, and we're going to certainly add more of those to come for the final Tiger release.

There's going to be a session that talks about some of this and combines it with the previous topic of high dynamic range on, looks like on Wednesday. So if you're interested in more details about that, certainly go to that session. I'm going to go a little bit into sort of an overview of the APIs and cover them briefly.

So there are two types. There's a way to get images in, a way to get images out. The way to get images in is a CG image source. That brings data into the system and in some particular file format we decode it for you and draw it. So you can bring images, obviously files, if you have a JPEG file, a URL, a raw data buffer similar to what we already support, just a big wad of JPEG or TIFF or whatever that you might have mapped into memory. And of course we support the standard CG data provider, which is the way you get information into Quartz 2D typically.

What an image source does is it creates CG images. So you create an image source from a file and out pops a CG image. Now not all image file formats are just single images. Some are multiple like JIF files have animations. We'll have multiple images for that. JIF files may have different images for different resolutions.

So we support, from an image source you can also sort of get all of the images in the image itself, in the file format, the file data itself. Plus you can get access to the metadata. So if your camera, were to embed its make in the file, then we would be able to return, oh, this is a Canon camera. Or if it had longitude and latitude, we could return that.

So you can get access to the metadata very easily as well. But the most important point about the CG image source and what it creates is that this will be the best and fastest and the primary way we're going to be supporting to draw images in Tiger. That means all parts of the system are going to go through this. We're putting lots and lots of emphasis on optimizing, doing all the vectrics and so on.

And so it's very important that this be the primary way we get images drawn and that we have the best and fastest. And so it's worth looking at from that perspective. Of course the higher level frameworks will also adopt this and sort of use it indirectly. But if you're sort of working at the core 2D level, it's definitely worth looking into. For output, sort of the opposite. Basically the same. You can output to a file, to a URL, to raw data buffer memory. You can output to CG data consumer, which is a standard way that you can output. You can output to things in Quartz 2D.

And similarly to the input, when you create an image destination, you supply CG images and you can supply metadata. And of course, if your image format supports multiple images, you can supply multiple images and write out the result. So then you can write out a TIFF file from CG or you can write out -- or Quartz 2D, you can write out a JPEG and so on. So just to give you a quick demo, I'm going to I have to say up front that this demo, like all types of demos like this, is a little bit trivial because we're supporting a new file format.

Hard to demo that tremendously well, but I'll do what I can. We have a simple app here. And notice that here we have a TIFF file. And uh-oh, this is Panther. We can't drop it in because Quartz 2D doesn't support TIFF. So now we're going to turn on Image.io. And now the TIFF file is accepted.

Hooray! So that's very important. The power of Image.io. Now just as an additional thing, this is a simple example of pulling up the metadata from this file. In this case, it has a bunch of fields, things like the height and pixels and so on, the file size, the color model and so on. There's a lot of information like that that will be embedded in images that you can pull out. Again, a very trivial example of getting the metadata.

Then just to tie it into the previous topic about color spaces, notice that this image doesn't actually have a color space associated with it. We're not using a color space. But when we start using generic RGB, it doesn't change. Why? Because in Tiger, we're automatically making sure that anything that's untagged gets tagged with generic RGB. So you get the same fidelity across all platforms.

And then just as a simple example to prove that there really is something going on, this is a specialized color space which swaps blue and red. And as you can see, now you have a very odd, maybe sick ladybug. Okay. So that's the demo of that. Okay, so what else have we added? Let me get some water and I'll tell you.

Okay, another thing that's very important and is very powerful is, of course, the resolution independence in Quartz 2D. The fact that, you know, any resolution, any output, any scale factor looks just the same, looks just as good as other scale factors. Now, it's not surprising that, of course, you know, down the road we expect monitors to change in resolution.

And we also know that people who are currently disabled use resolution, the scale factors to do, to zoom up their UI so they get some better ability to look at the content and so on. And the thing that we are very, very focused on is making sure that in all those cases it still looks great, so that your app still looks just as beautiful as it does today on the current monitors.

That's an very important thing. Now, that means that you need to actually sort of play with the idea that your application may be scaled, may need to zoom up and so on. What that means sort of on a technical level is that we'll set up the context for you. The CTM will be set up so that the scale factor will be built in to the context. So you don't actually have to worry about that too much. But that means you do have to worry about it in that you can't undo that.

If you undo it, then suddenly you're not drawing at the right scale factor, you're drawing at some other scale. And everything's going to look like, you know, bad things on the screen. So what that means is that we don't want you to sort of undo the transformation by -- this is a little bit technical -- but taking the CTM and inverting it and concatenating it back to pretend like you're an identity. Don't do that, if you understand what I'm saying. If you don't, good. And instead, we really think that it's important to consistently use gsave, grestore.

That's a standard way to save the graphic state, restore the graphic state. We spent a lot of effort in Tiger. We're optimizing the gsave, grestore operation, so it's very cheap. And so we think that that's the right way to do it. And it sort of will save you once we do get down to the road where we are going to start scaling the applications.

All that said, there are still some cases where you really do need to know what pixels am I touching. If you want to draw a thin line, for example, and you don't want it to straddle a pixel boundary, or you need something to exactly abut against something else, those things do happen where you sort of need that. And to help you with those cases when you may have a scaled up user interface, we have sort of a big workhorse function, CGContext.getUserSpace.DueDeviceSpace. Transform.

That is the full transform from the current user space that you're drawing in all the way down to the pixels, to the actual pixels that you're working with, the device space. Now, often you don't need something so heavy weight, so we have some key convenience functions, a function that will take a point in user space, convert it to device space, to pixels, take that point in pixel space, convert it back to user space. And we have similar functions for sizes and rectangles.

As an example, a simple example of how you might use this, here we have somebody who wants to do some sort of specialized rounding in device space, where they really want to round to the middle of a pixel or something like that. So they have a point in user space, they call CGContext.ConvertPointToDeviceSpace, that converts that point through the current transformation matrix all the way down to pixels.

They do their alignment the way they want, and then they take that point, convert it back to user space, and now when they draw with it, they know that what will come out will be precisely aligned. So for those special cases where it's important, this is a very powerful API. In general, though, we think that you'll be just fine using the standard Quartz2D things for scalable UI, but in the certain cases where it's necessary, this will help you out. Now, let me give you a quick demo of scalable UI and how it works with apps.

Some of you may have seen this yesterday in Peter's talk. So we have-- many of you are familiar with Quartz Debug, very powerful tool for development. And we have now a user interface resolution slider. So you can change the slider, and you'll get a different resolution for your application. So for example, here we might-- let's say we'll go up to two times, and let's launch Safari.

You see Safari is twice as big. Now you can do this with your app too. You can take your app, change the slider, and see what happens when you show up, when you bring it up on screen. It should have brought up Apple.com. Maybe they're announcing something else. Who knows? Derek Clegg, John Burkey But the point here is that notice how Safari looks really good. It's still as high quality as it was when it was at a lower resolution. The text looks great. All of the drawing and so on looks great.

So this is something that will help you out to make sure that your application doesn't have any problems with rendering. As you can see, there are some issues, like up in the upper right-hand corner. Clearly I've changed the resolution, but nobody told the clock or the sound bar. And I would show you some other apps where it doesn't look this good at all. But that would be embarrassing, so I won't.

So make sure your app is not one of those. Whoops. Let's see. Safari is not quitting. I'll put it here. And the other thing to remember is to make sure that you quit the OK, we can switch back. Make sure you quit that app, otherwise everything will launch too big and that's difficult to read.

Okay, so that's sort of a lot of the things we've added so far for drawing, for new ways to do drawing, better ways to work with CG context, better ways to work with images and paths and so on. So now we're going to talk about a layer, which is a new thing we're adding for Tiger to help out. It's really designed to sort of help optimize the traditional case of where you want to do off-screen rendering, but you still want it to be hardware accelerated.

In the past, what we've done is said, oh, if you want to do some sort of off-screen rendering, just create a bitmap context, draw into that, and then take the result and composite it to your destination. It's a good way to do caching, and it's a very powerful and very straightforward thing to do.

The one problem, though, is that the data always lives on your side of the bus, essentially, right? It's all sort of in your address space. So any drawing you do there, which you then try to draw onto, say, the video card, will have to be uploaded. That's not efficient. And by design, by the bitmap context design, it's not efficient.

It's sort of limited to that. So CG layers are sort of an optimized version, sort of a better way to work with sort of do this type of bitmap context type caching. Now, the key thing, too, is that in addition to just sort of being a convenient way to sort of do off-screen rendering, when you create it, you specify a reference to another context. And what we'll do is we'll sort of set up the layer that we created.

So that's what we'll do. So we'll create a reference to another context. So we'll create a reference to another context. So that sort of matches the destination context. For example, it will match the color space. So that means you don't really need to know about the color space of the context you're working with, for example, the screen color space or so on.

Instead, we'll sort of create that, set that up for you. So when you draw onto the layer, what will happen is that it will already be matched to that color space. So when it's actually finally used, you don't have to do a separate match. So it makes sure that things stay highly efficient.

And you can sort of think of it, it's really in many ways sort of like, you know, as this example shows, just like a rubber stamp in some sense. You can take the layer, you draw into it, and then you can sort of use that over and over and over. Here, for example, we've created a layer, we've drawn the butterfly into it, and now we can just use that.

And what's great is if the layer is hardware accelerated so that it's already up on the card, when that stamp happens, none of that comes back over to the CPU. Instead, that's all done on the GPU. So that's very, very efficient and very high powered for those types of cases where you need to do sort of off-screen caching and you don't want to have things sort of live on your side of the bus, as it were.

So like I mentioned, we sort of imagine that one common use, probably the majority, will be caching. So you would use a layer instead of a bitmap context for off-screen caching. Another one, which may be a little bit less likely, is sort of a buffering where you might be in the middle of a screen update on this side, so you can go ahead and start drawing into a layer so that the next time you get a chance to composite to the destination, you're ready to go. That might be a little bit less common use. So short demo. We'll go back to my little app.

So here we have our app again. And here's a PDF file. Very nice PDF file, but it's got a lot of data and it's a slow thing to render. Now, if I wanted to animate that, I would be fired because this is not really animation. This is sort of chunky blobbyness.

So instead, of course, obviously what you want to do, in Panther we would have said, oh, well, to solve this problem, create a bitmap context. Very good. Solves the problem for Panther. The problem, though, is of course that's not hardware accelerated. For Tiger, you can create a layer.

And what this does is it will draw the PDF file into the layer, and then the layer is now hardware accelerated. So when we do animation, it's going to be completely smooth. It's all hardware accelerated. Everything looks beautiful. Now the thing to remember, of course, is that it's still in some sense bits.

So if you think about it, it's not the original high resolution PDF file that if you scale up, you'll get perfectly beautiful results at any scale factor. You will start seeing perhaps some sort of image artifacts if you were to scale up this particular layer because it is bits. But in many cases, for something like this, that doesn't make any difference. You don't care about that so much. You just simply want to have high quality, fast, efficient rendering. There's Safari still. OK. So that's that demo.

So that's a lot of the APIs for both drawing content, for high efficiency off-screen rendering, and now we wanted to also add some stuff for PDF support. As you know, PDF is the metadata file format in Mac OS X. It's our high fidelity rendering of drawn content. It's a great format. It works really well for us.

The problem is that sometimes people have trouble sort of both in the creation side they want to do more things and on the sort of input side where they want to really look and introspect more into the PDF itself. So we've added some APIs to help do that. So let's go through those.

The first is links and anchors, very comparable to HTML. Again, it works just with a PDF context that is able to record this information. And you can do sort of the basic stuff you can do with links and anchors. You can specify a URL and say, well, when somebody clicks on that point in the PDF page 17, I want you to open up this URL in Safari. So you can do that.

And then, of course, the anchor type model where you can say, well, if I click here on page 47, go back to page 8 or go forward to page 75. So you can sort of do forward and backward references. And it's a very simple API. It's very easy to use, but it gives you a lot of power to sort of preserve that. And, of course, Safari will be using this in Tiger to preserve links and anchors when they actually go to print from Safari.

In addition, for creation, a lot of times people have wanted to create encrypted PDF. This follows the PDF specification published by Adobe, works with a PDF context, and at creation time, you specify a password, or more than one password, and the permissions associated with the passwords. So you can sort of say, "Well, for this password, don't let them print the file, or don't let them copy anything from the file." For the WWDC world, we support 40-bit encryption, and of course, we expect to extend that up to, I think, 128, which is part of the standard in PDF for Tiger.

Okay, so that's sort of output, so new things you can do with output. For input, what we've added for Panther was something really powerful. It lets you sort of introspect about the document structure. A PDF file, you can think about it as a giant tree of objects. And you can use the APIs that are available in Panther to sort of walk through that tree and look at all the content. The one problem is that the real meat of the file, the actual content stream for the page, which said, you know, draw this circle here and put this text here and draw this circle here and put this text over here.

It was just sort of a big black box. You could look at the content, you could print it out, but you couldn't actually interpret all the pieces of it, unless you wrote a parse yourself. A little bit less, I don't know, a little bit less attractive for most people. And in fact, it's not something you can just sort of read either and sort of figure out exactly what it all meant.

So for Tiger, what we've added is a new way to take the content stream itself and parse through it and call your functions back for each operator you're interested in. So that way you can really take for this page, look at all the pieces of the PDF file and get, essentially sort of blow up in the whole file so you can look at every bit in the file itself.

Oh good, glad you like it. It's very easy to use. The idea here is that for each operator you're interested in, you supply a callback function. You can do it for none, you can do it for all of them, and we'll just run through it through the content stream itself and call you back.

Here's an example. Again, a very simple example. The idea here is you create an operator table where you specify both the callback function and the operator you're interested in. In this case, we're imagining that someone wants to look at the BMC operator, the begin marked content operator. Who knows why, they just want to do that.

You create the content stream, of course, from the page. That's new as well, where you can get the content stream out of the page and create a CG PDF scanner. That takes a content stream that you're interested in, the operator table with your callbacks and an optional info parameter, and you call CG PDF scanner scan. That will parse through all the PDF, do all the mucky work inside, parsing and so on, and call back your function.

So each time it sees a BMC operator, it calls you with the arguments on the stack for the BMC op, for the sort of the BMC op that you're looking at. You can do what you might want to do. Maybe you're counting the number of times they appear or something like that. When that function returns, when CG PDF scanner scan returns, the full content stream has been parsed. So now you're done.

You release the scanner, you clean up after yourself, and you now have the information you want. So it's actually very simple to use, doesn't require a lot of work, and you can do really powerful things with it. For example, this is what we use inside of Quartz 2D to do both all of our PDF rendering and the text extraction and so on that's part of preview. So we use this in two very different ways, but again, it's very powerful with just very simple API.

In addition for Tiger, we now have a new thing called PDF Kit. There's a special session devoted to it. This is a little bit higher level. It's actually what Preview uses now to do all of its PDF rendering. It lets you do things like do two-up drawing, do one-up drawing, do text selection, and so on, all in your own application.

It's a cover for a lot of complicated functions that Preview used to only be able to do, but now you'll be able to do with PDF Kit. That is something you want to do if you want to bring PDF into your own application. A lot of people do text selection, searching, all that type of thing within your own application, and it's definitely worth attending the session right after this session in a different room.

Okay, so this is a lot of stuff that we've added for Tiger. We've added some both easy to use and some advanced path APIs. Of course, some new color space APIs, lots of image support, various ways to work with images, floating point images, of course, as well, and floating point bitmap contexts. A better way to get image data in and out of Quartz 2D.

New layer support for high efficiency, high quality off-screen rendering, and also a bunch of functions for PDFs. So that's sort of the new APIs in Tiger, the new things you can work with. And now John Burkey is going to come up here and talk about Quartz 2D going extreme and what that means for you.

Hi everybody. I'm John Burkey. I'm here to talk to you about Quartz 2D and Quartz Extreme. So before I start, I want to sort of give you an idea of what we're going to be talking about. I want to talk about the architecture enough that you get an idea of what we're doing so you sort of know what to watch for when you're doing your apps.

And then we're going to do some demos so you'll see that it's real in a lot of interesting different ways. And then finally we'll be talking about rules of the road. We're ending with that because there's a lot of very specific things we need you to do so that your apps really, really rock. So let's get to it.

So Quartz Extreme for Tiger. The big change is that 2D is now hardware accelerated. The thing for us that was important was that we needed to maintain Quartz's very, very high quality, but we wanted to speed things up a lot. We used the GPU when we can, we used the CPU when we need to.

The big thing that we did to start with before we began this project was we analyzed a lot of applications. What we did was we made sure that the things that the applications do we worked on first. What you'll find as you get your apps going is that basically all the things that your typical application does are all in hardware and all in GPU, so you'll see a lot of speed. That's where it will be calling out as we talk about specific operations. Then the less common operations, things like stroke lines with dashes, things like that that are less common will be accelerating later, if at all, because those things will be coming up very, very rarely.

Importantly, when I say accelerated, I'm talking about on hardware. The other thing we're doing though, which I think is really exciting, is you'll find that software performance is actually quite a bit higher in Tiger as well. The reason for that is that as we've optimized the pipeline for hardware, we found a lot of opportunities to continue to optimize the software pipeline too. I'm pretty excited about that. And worth noting about that, all the numbers you'll have on screen here will be Tiger software versus hardware. You'll find that even these software numbers, as I'm saying, they're quite a bit higher.

So here's our Quartz Extreme architectural diagram. And I want to just point out a couple things just to sort of set the idea of what we're talking about. We remember that the three big things we do in the Quartz Extreme engine are we deliver 2D video and 3D to the screen. We use the GPU and then we see this word "surfaces," so we use hardware surfaces to deliver video and 3D.

So focusing in on 2D, there's just a few main concepts. So first of all, the application, as you know, renders into the window. We call it the backing store, the bitmap for the window. And then, it says flush. So when we say flush what we mean is we talk to Quartz Extreme, the Quartz compositor, and it flushes the content to the screen.

So what that does is it actually talks to OpenGL and OpenGL executes a flush in behalf of Quartz Extreme. So focusing in then, this is the key point here, is that Quartz 2D up until Tiger was using software, it was running with the CPU. So we were fill rate limited by the CPU's performance. Our CPUs are really, really great today, we all know that, but still compared to the GPUs they're more limited.

Then the great thing about Quartz Extreme was that we used the DMA engine on the graphics cards to do an asynchronous DMA pull of the bits across to the GPU. And that's been great because we get asynchronous behavior and the GPU is really good at this. So we get a lot of speed as you know with Quartz Extreme.

So, that was awesome. But there was more to do. The first thing that Peter alluded to in his presentation is that GPUs have become this other animal. GPUs are now these amazing things that can do these core image effects, you know, blurs, all these different things. And 2D has been isolated from that part of the capability because we've been using it as a blitter to get our content to the video screen.

And the other thing is that window back and stores are very big. So, even if the CPU was capable of writing those pixels into this window back and store very, very quickly, we still have to shovel those bits across the bus to this screen. So, there's still a lot of work to be done.

And then the last thing is that the window actually is a synchronization bottleneck because while the CPU is shoveling the bits into the window, remember we're DMA flushing from that same bucket to the screen. And so, we don't have tearing. They have to work. They have to wait for each other. And that's what we talk about when we talk about over flushing. So, you'll see that in each of these cases we have solutions.

So as I've been saying, it's all about bytes and bandwidth. We're software rendering here, and this is a great number. The G5s really, really rock. But it's five gigabytes per second, and as we know, there are other things for the CPU to do on the system. And also, I'm using these big blocks because we're talking about a lot of data. Window back stores are huge, and when you're using a software renderer, there's a lot to do. And here we are with the hardware DMA flushing. It's 2.1 gigabytes per second, but still we're shoveling big boxes, you know, there's a lot to do here.

and finally the bottleneck in the window back in store. Here's the key point. Look at that, 30 gigabytes per second and climbing. The charts are all pointing the same direction here. You've seen those in previous presentations. So this is our key. We want to focus on that. That's our deliverer.

So, here's the punchline then. This is our plan, right? We want to stop using the CPU and we want to move towards using the GPU. The GPU, as Peter showed in his presentation, has a lot of pipes. It's very, very wide. It's very good at this. And importantly, by using the two together, we can keep the GPU as busy as possible.

The GPU is a single purpose piece of equipment for doing graphics. So, by keeping the CPU as its pipeline, we can keep it really busy. So, what does this mean? It means the backing store is no longer in DRAM. So where does it go? It goes to the graphics chip. OpenGL, we work with OpenGL to deliver things to the hardware.

And this gets pretty interesting here. I'm not good at clicking. OpenGL commands are very small, hence the small purple boxes. That's the key point here for this part is that we're delivering a lot less bytes across the bus, and this is a very optimal stream. The OpenGL guys have worked very hard. As you know, our gaming engines today deliver millions of polygons, so that's what we're talking about here. We're using a very tight stream.

And then uploads, and this is important. We'll be focusing on this issue a lot as we go forward in the presentation. Uploads are very infrequent, and we will be working together to ensure that. They're much smaller than window backing stores, very key point, right? The Aqua artwork, and some would fill a quarter of a screen, but we fill the whole backing store with it. So that's the point. As we upload the artwork across and reuse it, we have less to do there.

Then finally, this is great, back to the flushing bottleneck, with the The window back in store on the GPU, the flushes are occurring all on hardware. We're running two hardware with the small commands and then all the flushes are all on hardware. So that whole flushing bottleneck just goes puff. So that's really, really great. It's a key to our performance.

So, how does it look like? looks pretty good. This is pretty amazing to me. I work with this stuff every day, but I'm always astounded by how much performance you can get from the system. What's amazing is this is still CPU limited. As fast as I can shovel stuff with present-day CPUs, I can still shovel it faster in the future.

And what you'll see then as you test it yourselves is you'll see that performance will scale more in hardware as your CPU gets faster. So we have numbers where the alliance performance scaled between my PowerBook and the G5 number that's here, it scaled by like 3 or 4x when I went to the G5, and it only scaled by 2x, the sort of the CPU normal number. So I think it's really interesting.

I want to call it a couple things. This is about fill rate here. The point here is that with the big rectangle, we're not limited by fill rates still, so we're really, really fast. That's important because this is what we do first when we render a window. We fill it with stuff. So that's important. Text rendering is very important.

There's been a lot of discussion on the net about that, and I just wanted to call it out. We're really working hard on this. We actually spent a lot of man months on making this great, and we're basically almost breaking 5 million here. here, so that's really great. And then lines, this is something we'll be talking about today in detail. We have a new API to really make that easier for you to deliver a lot of line performance on your machine.

So, sort of defocusing here, stepping back one step, here's PDF rendering, which is a user of the core primitives. And what you see here is that there's two documents here and two different ways of rendering the same documents. On the top of the screen is a core graphics benchmark, and we're getting about 2x, 2.2x with that. We're getting the same values below, they're about twice as fast, a little bit less, but quite a bit slower when it's in preview.

And that's okay because preview is an application that has to do annotations and stuff. You know, some of us are speed freaks, but there's a lot of really great value that's delivered by the higher end apps, so it's not always about that. Also, I wanted to point out that, I'll move on, I don't remember.

So anyway, application resizing, this is a very interesting slide for those of you who are focusing in on the numbers. So see the red, this is something that we're using as a reminder for you when we go into the rules of the road section. So we have lots of speed here, but it's sort of varying, and sometimes the blue line is the hardware number. It's actually slower.

So this is the thing, is some of these applications aren't caching their image refs still. We all work together at Apple, but we're all still pitching in. We're not shipping Tiger yet. So we thought it was great to show you some of these cases where our apps are slower too, and to call this out. And I'll have a demo that explains this too.

So again, you see the number on the bottom, text edit is crazy fast. That's frames per second. You actually have to turn off beam sync on your monitors to even take this measurement, 'cause it's just a big flickery craziness. I'll show you that in a minute, it's pretty neat.

So the key is, is as we move forward, you'll start to experience more and more of this performance, because as I showed you, the benchmarks, the core primitives you're actually using are fast. So this is great, I think. This means we can deliver, and this is my experience in bringing apps up and looking around the system and what they're bottlenecked on. We should experience 2X across the board with GUI performance for Tiger.

So, yeah, can I have demo machine? Not that one. Okay, so. For the first one, I'm just going to show you a little finder performance. That's pretty fast. It looks good. And then here's a little-- make sure, yeah, good. Here's a little even faster. So that's all good. There's a lot of light little demos. Just started to give you a flavor for what we're talking about. Here's Safari.

So it's getting pretty fast. I'll show you here with it off. You'll see that it's still fast, but it's a little slower. This number is about 1.5x, and so we're pretty happy with that for where we are. Here's one of the flickery crazy ones. Let me make sure I'm set here. So that number, you can actually-- one of the guys on the team taught me this. You can measure by the amount of tears you see what the frame rate is. But we're looking at around 200 here, so that's really great. This is an interesting one.

Actually, did I do that right? So here's a Java app, which is cool because-- They just told me they were doing this little demo, and so we grabbed it to see what it looked like. It's kind of cute. What it does is it tries to render as many balls and keep the frame rate at 30.

So you can see... With hardware, with this great G5, we can do 432 balls per second. It's a new benchmark number of balls per second. Great. But now when we turn on hardware, This is actually with no code changes from them. They're still-- I'm working with them to crank up their pipe a little bit more.

I still got beam sync off, so that's why it's a little hairy. But anyway, we'll keep going. So it should float up there about 3x of what software was. That's really great. Is there quite a-- there's quite a deep pipeline here to get the code from Java all the way down. That's pretty sweet.

So here's our friend the wireframe. This is an important demo to us because this is something Joseph Marr has been using for quite a few years to talk about performance in Quick Draw and Mac OS 9. And part of our message to you is we take this very seriously when you tell us that you need better performance in these areas so that you can switch over to Quartz.

So this is a big deal for us. So you can see already with Quartz software performance, things are looking great. 1.3 million on a G5, that's pretty damn amazing. And then...

Okay, so this is sort of back to kind of pseudo, this is going to get us into the rules of the road, so pay attention to this one in sort of a different way. Let me make sure I do this right.

Okay, so software, hardware, really, really great. That number's like 20x. And then, oops. Now, so what about this? This is strange behavior. This is hardware with no caching. Look at that frame rate. Woo! So this is our first example of how we need to be compliant with our API. The good news is there's not a new message here, just we actually need you to be compliant now so your app rocks. So here's scaling performance. G5s do great. That's three times higher than my PowerBook. That's nice. So here's hardware. Woo! So-- all right. Back to slides.

Caching. This is the important thing. This is sort of a typical command stream you just saw go by. It contains some commands and contains some resources to be uploaded. We're just showing this cute little picture of a resource being cached. So now we light it up because we're going to reuse the resource and see it was all little blocks. That's the technical term. And what we want to focus on is making lots of little blocks.

So we want our resources to be cached. We want to reuse our resources. And here's another one. So, you know, the point is, this is the kind of behavior you'll see. A lot of the things you'll be doing, there'll be a nice clean command pipe, and then the idea is that rarely you'll be also doing resource uploads.

So using vramdiv room. This is actually, I think, a nice simple story for you. Everything is automatic. All you need to do is use the retain release semantic that's in core foundation. And if you do that, what you're doing is volunteering to be a part of our caching strategy. Importantly, what you're not doing is volunteering to have all of your images put on the card all the time. What we'll do for you behind the scenes is dynamically take care of that to make sure that we're not putting too much up there or too little.

So that takes away that worry that you have of, oh, I've got to retain just the right amount so that my app doesn't freak out. What you need to do is just retain all your image refs, all your pattern refs, all your color refs, all the things you're using. And again, that's how we know, because we can see that. And we can see the hardware you're on. And what we'll do is take care of the rest.

So in a typical picture, here's Finder. We have images and patterns. Text, lines, rectangles, these are the primitives that I gave the numbers on. In a typical case, that will be everything on the screen. We'll take care of everything there. We refer to that as a visual working set. The idea here is that you have this set of resources you'll be using and those will sort of stay cached for you.

So, rules of the road. This is the big one, cache your refs. This isn't exactly how it would look for you, but you get the point, right? Here in the first top part, we have a loop, and every time through the loop we do the cheesy thing, bad thing, which is make an image ref, render it, and then release it. Don't do that.

Instead, some port in your code where you're loading your images, whatever you're doing, create your images, hold onto 'em. That's your cache. Your cache is to hold refs. Your cache is not to hold bits. If you want to hold bits, that's fine. Just hold the rest with them.

And then, as many times you want to render the image, render the image. Just don't do what you did up above there. So layers, great new strategy. They're hardware accelerated bitmaps and bitmap contexts. They're very easy to use. The key point is, again, they're dynamic. We'll make as many of them hardware accelerated as we can based on the card you have, what you're asking us to do otherwise. And they're really easy to use. I really like how the guys did this because they're actually easier to use than the old bitmap context anyway. So these are great. Importantly, right, bitmap contexts always have to go across the bus and layers will be in hardware whenever possible.

So here's one of our new performance APIs. The key point here is you see the top here, good and bad. This looks like nice clean code, just that by iterating a loop like that and giving us one line at a time, we have to create a lot of temporary storage to handle anything you might give us. With the second one here, you tell us how big your storage is and you give it to us all at once. So in actuality, with the hardware renderer, it basically doesn't get touched until we're ready to talk to GL. So that's very powerful.

Here's a Rex case. This API has existed for a while, but we're encouraging to use it more. Same thing, right? Store it all at once, tell us the number, it travels all the way through the system. Don't cache your caches. This is another thing that we found working with the operating system people.

We'll find there's a lot of caches around for different reasons. Rendering has changed a lot in the last few years, so some things are faster than they used to be. This is the key point. Check your caches. We've all got them. And try turning them off and see if they still help.

They might not anymore. So that's what I'm talking about here. Also very important, this is an awesome session. The Shark technology we have is one of the best technologies that's come out of Apple in the last five years. This might sound silly, but really, really, really measure your app before you think you know what it's doing. It will look very different with hardware acceleration, number one.

And number two, as you know, the scale of applications has really grown in the last 10 years. So we need to measure our stuff and then simplify the scenario that you're measuring until you're sure you understand it. Then you know what to do. That way you're working on the right stuff.

So this one's important to me. I used Quickdraw for a long part of my career and I really appreciated Joseph Mara and Mike Brinkovich working hard to make this run great on 10. But here's the deal, Quickdraw, as we're saying here, it's too fragile and undocumented. It's just got a lot of funny stuff about it from the past. Now's the time where we all have to say goodbye. This is the key point for us. We're faced with this. So if Quickdraw is rendered in a window, we're going to turn off hard work celebration. So that's what we ended up with.

So, more good news. It's really easy to turn on. You've seen a bunch of demos here. You have the stuff in your computer. Command D. It's right here and you see that it's listed there. Also, importantly, only some of our products support Quartz Extreme with 2D. These are the ones. The key is this fragment program stuff. We have a very modern new graphics pipeline and we're taking advantage of these new features. so we actually do need them.

So final thoughts. First, core imaging. It's a great new technology. It works with us very well. We support it as a real image. This is the link. It was one of those top secret sessions, so I wanted to make sure you saw where it was when it was.

It's actually very soon now. This is really great, though. You can make an image using their APIs, and it'll be clipped with our clipping and everything. So it's really great. It fits right in. And then here's the rest. So the whole system is now hardware accelerated and This enables us to do a lot of really amazing things. So there's going to be a lot of great things in the future. Here's our mailing list. We love discussion.

And we've got documentation from the Quartz team. Amazing. So we have two different documents. A lot of people are working really hard on these. And this is where you can find them. They're actually on your computers. And I'd like to bring up Travis. He's going to lead Q&A.