Welcome. Good crowd today. So that's me. I work in DTS. and today we're going to be talking about a number of Mac OS X solutions for graphics for games. Rather than go through kind of a dry, boring, line by line overview of the API, we're going to cover some particular topics that are of interest to game developers. Both some traditional problems and some things you might want to think about as you're moving to Mac OS X.

We're going to talk about display management. That's always a big problem. How to change the screen resolution, how to change the bit depth, refresh rate, those kinds of things. We're going to talk about mixing core graphics with Quick Draw. Some of you, maybe full screen apps, maybe not, but some apps may want to start using some of the core graphics APIs, perhaps to draw some anti-alias text, or some other things like that, some of the advanced path features.

We'll talk about how to mix that with some existing Quick Draw code. We're going to cover a few Quick Draw tidbits, some other things about differences with Quick Draw on Mac OS X versus Mac OS 9. Another topic, tear-free blitting. We're going to talk about what you can and can't do on Mac OS X and how to improve that.

And some changes to OpenGL. OpenGL is a little bit different. Mostly we've added a bunch of new features for X. And then we're also going to cover, probably in more detail than some of the other sessions, because this is the first one, we're going to cover the games roadmap for WWDC this year.

With that in mind, we're not going to really be covering much about Quick Draw. I'm assuming you either know what you need to know or you can find it. We're not going to cover QuickTime in detail. There's some other sessions on that. I'm also not going to go through a general introduction to core graphics. There are a number of other sessions at this conference for that. And there are four more OpenGL sessions. So I'm not really going to cover much beyond some real basic Mac OS X stuff for that.

So let's get into the display management. As many of you know, DrawSpocket's been around for a while. Works great on Mac OS 9. We've included it in Mac OS X. It's available for you to use. We've primarily put it in there for ease of porting. A number of you have existing apps, existing games that already use the DrawSpocket API.

And we've got it in Mac OS X so you can bring that code over almost directly. It is callable from CFM and Mac O, so you don't have to worry about going through CFBundle APIs or anything like that to get to it from CFM. One thing that is a bit different, though, for some of you that were used to putting up a window on top of DrawSpocket to just draw into that window to do your graphics, on Mac OS X, you can't bring a window up on top of DrawSpocket anymore.

So if you had been bringing a window up, now you just need to get the front buffer and draw straight into that. It's a pretty standard C graph pointer. Not too hard to do. Another thing is we've removed some APIs like the user select context, which used to let the user select the display. So all of the GUI stuff is pretty much gone out of DrawSpocket.

Now the new API on Mac OS X for doing display management is called CG Direct Display, and that's part of Core Graphics. You'll find it under the Application Services Umbrella Framework, and it lets you do a number of different things that you would expect. You can enumerate displays. You can enumerate the available video modes for a given display. You can capture displays so that you control them completely. You can switch the video modes on those displays. And you can also, if you really need to, get access to screen memory directly. I'll show you some code on how to do all of these things.

So first of all, enumerating displays. You have an array of display IDs of, you know, say you want to cover at least, say, eight displays or so. You get the active display list. That gives you a list of the display IDs for each display that's active on the system. And it also returns you the actual number of displays you found. If you didn't get an error, you can then iterate through that array and look at each display in turn.

Once you get a display ID, what can you do with it? Well, you can find out a few things about a display. You can find out pixels wide, pixels high, those kinds of things, find out the number of bytes per row, always very useful. Once you've captured a display, you can also get the base address.

This is the direct access I was talking about before, but it's important to note, this is only valid for a captured display. Once you've captured it, the Windows server pretty much gives you control over it until you release it. Once you release it, you can't draw directly to its bits anymore. Keep that in mind.

You can also, once you get a given display ID, you can ask it what the available modes are. So when you get ready to switch, you want to find out what modes does it support. CGDisplayAvailableModes will return you a CFArrayRef And then, provided that's not null, you can find out the number of modes that the display supports.

and then loop through each of those modes. That array is actually an array of CFDictionary's. And each dictionary describes one of the available modes. So what can we do with a mode dictionary? Glad you asked. For each mode you can get a number of different values. The CG Direct Display header, cgdirectdisplay.h describes in fair detail what you can get out of that dictionary. Some things you can find out whether that mode is actually usable for Aqua. As you know, Aqua doesn't like to run in 640 by 480, doesn't like to run in 8 bit. This will tell you whether or not that display mode supports Aqua.

and here we just set a string. You can also get pretty much any particular key, the keys are listed in that header and find out the numeric value so you can find out refresh rate, you can find out bit depth, width, height, those kinds of things. Once you've gone through your modes that are available for a given display, presumably you pick one.

Say you need 1024x768x16 bit. You found the mode that identifies the one you want. Now we need to switch to it. Well, in order to be able to switch back easily, you can get the current display mode. And that gives you a dictionary ref that you can use to restore the mode later.

Then you capture the display. And once you've captured it, you've got control over it, as I mentioned. Then you can switch to whatever best mode you've decided you want to use. You can also hide the cursor. Now, the ordering of these three, capture, switch, and hide the cursor, is somewhat important.

If you switch the mode before you capture,

and hiding the cursor is just whether you want the cursor still visible during all this. Some people may want to hide it, then capture, then switch. and once you've gotten all this done, you've got the display to yourself. And your game code here.

If you need to get a CGraph pointer to do quick draw operations, you want to copy bits to this display, whatever, you can use an API create new port for CG display ID. This API is actually in the quick draw header, so don't go looking for it in the core graphics header if you won't find it. And if you do create a port this way, don't forget to call dispose port when you're done, because as create might imply, it does create a new one and you're responsible for releasing it.

Once you're done with your game, you need to shut down, and you pretty much reverse the order of the operations that we did earlier. You put the cursor back. You switch back to the original mode on that display and then you release the display to give it back to the system. Pretty straight forward stuff.

So now I'd like to show you a quick demo. These are actually some little demos written by one of our core graphics engineers, but they illustrate the point pretty well. The first one is Mode. This is already up on our sample code website. And all it does is use some of the code we've described to Go out and simply ask what display is available, what modes does it support.

As you can see here, currently we're running 1024 by 768 by 32 bits, 30 hertz, and it supports a number of different and this is also listing for you whether or not that particular mode supports Aqua or not. This is a good piece of code to look at, especially if you're not too familiar with CFArrayRef and CFDictionaryRef and how to manipulate those. This code has a very nice, simple structure to go through those.

That's one nice little one. And then, to give you a brief example of drawing directly to the screen, this one called Color Bars. shows you how to just draw straight to that base address. Very simple little demo. And here we're just capturing this display, switching it into a mode, drawing some, and then switching back. Nothing too complicated there. So moving back to the slides.

and as I mentioned both of those demos are already up on the sample code side if you'd like to go take a look at them. Along with display management comes palette manipulation and gamma tables. For this you combine the CGDirect palette API and the CGDirect display API. If you want to be able to create palettes and manipulate them, switch palettes and also manipulate the gamma tables on a given display.

So creating palettes has a number of different options. You can create a default one, you can create it for a given display with a given capacity with some samples you already provide. I believe those are float samples and then there's also one for convenience function you can create it directly with byte samples. And then of course you have to release those once you're done with them with CG palette release.

For manipulating them, you can get a particular color, you can get the index for a given color, you can set the color of a particular index, you can compare palettes directly. You can also create a new palette from an old palette blended with a color. So, some nice little facilities there to make palette manipulation a little bit easier. and then of course you can ask a display whether you can set the palette for that display and then you can actually set it if it says you can. Some displays may not support palettes at all.

For Gamma, there's a few APIs here for setting the transfer functions. And there's two ways to do it in this API, either by a formula which just has a few parameters or by a table of values. So if you happen to have your whole Gamma specified in a table, you can very easily call set display transfer by table. We also have a convenience function to do it with bytes. So you can either use bytes or floats. You can also get the transfer back and you can do it by formula as well with these APIs.

We've got a quick little demo here. This one has not been posted yet, but it should be soon after. And this is actually a direct port of an old sample called Mac Gamma. Some of you may have seen that demo before. It just runs through. and starts using those APIs to mess with the gamma.

I have an annoying little app, but it's a good demo. Install it on your friend's machine and then hide it and leave it running. Okay, so more slides. So that's a little bit about display management. The basic capabilities are there. You can get the different displays, find out what modes they support, switch to modes, manipulate those, manipulate palettes and gammas. All of that facility is there for you.

Now, if you already have some quick draw code, you already do a lot of quick draw drawing in your game, but you want to start adding a little bit of CG code to your game. How do you go about doing that? First of all, we're going to talk about if you've already got a quick draw port, say you've used Carbon, create new window, you've got a window, you can get the window port. Now you want to draw CG into that window.

There's some pretty easy code to use to get the core graphics context and start drawing. One thing you need to watch out for, the coordinate systems are a little different. I'll talk about the differences in the coordinate systems and what you need to watch out for. I'll talk about some things that are also a little different with pixel addressing. And then we'll get into covering a little bit of clipping and how clipping in core graphics differs from quick draw.

So to create a CG context for a given port, very simple. There's one call, create CG context for port. You pass it to port, point it to a context ref. Couldn't be simpler. Do your CG drawing. Don't forget to release the context that you created when you're done.

Once you've got that context, how does the coordinate system differ? In Quick Draw, as many of you are probably aware, the coordinate system, the origin is in the upper left-hand corner. Positive X goes right, positive Y goes down. For PostScript, PDF, and of course Core Graphics, which is based on the PDF imaging model, the origin is the bottom left. The positive Y axis points up.

And unlike Quick Draw, where you're pretty much always working with a port, you can always get the port wrecked, find out what the bounds are, width and height and everything, core graphics doesn't really have a concept of a bounds of a context. And that's partly because it's supposed to be very device independent, but also because once you start applying transforms and things, it gets a little weird how do you define a rectangular bounds.

So in particular when you're working with a window context or if you created an off screen context, which I'll talk about in a little bit, you know what the bounds are already. So you pretty much just have to remember those. You can't get them directly from the context itself.

So if the coordinates are different, what can you do about it? Well, the easiest thing is with a few simple calls in core graphics, you can actually change the core graphics context coordinate system to match that to QuickDraw. And for people that are used to thinking in nice, simple QuickDraw coordinates, this is probably the easiest way to go.

And as I mentioned, you need to remember the bounds from the port or the view that you got it from. So the first call, CG sync context with origin, CG sync context origin with port, excuse me. That simply makes sure that if you've called set origin in your port, in your QuickDraw port to move the origin around, this will sync up the CG context origin with that change.

Then you call translate to move the origin from the bottom left to the upper left, and then you call scale to flip the y-axis so the positive y points down. So we've got a little bit of code to do just that. In this first part, we create our context again. We call sync origin with port. So we've got our port and our context ref.

Then we translate by the port height. And it's positive port height because you're positive in terms of core graphics coordinate system right now. And then minus one on the y-axis for the scale. We leave the x-axis alone, so that moves the origin up and flips the y-axis down. One thing to note here, and you'll probably run into this the first time you go through this, I certainly hit it.

This does make the coordinate systems match, and so you can start thinking in terms of quick draw coordinates again. But when you start using CG to draw text, the text draws upside down. Because the text honors that translation matrix that we've been manipulating there. And as soon as we scale it by minus one on the y, core graphics says, "Okay, I'll draw it upside down, no problem." Which is something you never could really do in quick draw. It's one nice feature.

So, and you can also rotate text too, so that's also very nice. I hope you managed to get to some of the core graphics sessions. So the CTM thing, I mentioned it on this previous slide. You see translate CTM, scale CTM. What is that thing? That's the current transform matrix. And you can do a number of fun things with that. You can do basic translations, you can do scales, as we've already seen. You can also do rotations, which is very nice. And you can do skews, which are pretty much just arbitrary affine transforms.

So what do those primitives look like when you actually use them? Again, we get our context. Then scale is simply, you take a scale on the X and a scale on the Y axis, and those are floating point numbers, so you can scale by fractional scales. As you saw before, one pretty much means don't change that axis. Minus one means flip it upside down or left, right.

Translate, much the same. You take a translate on the X and a translate on the Y, moves your origin around. Rotation, again very simple, very straightforward. Angle and radiance, that rotates the CG context around its current origin. And then if you define an affine transform, you can just concatenate that with the current transform. So this is where you start getting into skews and other things.

So what does that all look like? I wrote a simple little app here to show you what you can do

Just calling draw graphic. I have a little routine that draws graphic at the origin. Doesn't do anything exciting. But using scaling, over here, using translation. Oh, and these are all rotated a little bit, so it's a little more interesting. You can do some basic scaling and translation. And then this combines mostly rotation and translation. Good little rocket ship to fly around in a figure eight.

and in each of these cases I'm simply manipulating the transform matrix, tweaking it a little bit, doing a translation, doing a rotation and then calling that one function and that function never changes to draw the graphic. So as far as it's concerned, it's always drawing the rocket at the origin and it's always drawing it on the, you know, straight up the Y axis. It doesn't have to care. The transform matrix and all the CG operations take care of manipulating the pixels, make things a little more interesting. and this sample code will be posting pretty soon after WWDC as well. Alright.

So as I mentioned in the beginning, pixel addressing is a little bit different. Because Core Graphic Contexts tried to abstract out away from the device, Some contexts don't have the concept of a pixel really. If you're printing, you can sometimes get to the pixel, sometimes not, depending on if you're going to a PostScript or a Raster printer.

If you're actually going to a PDF file, for example, you know, they make a big deal out of being able to write out PDF files from just about anything, you can actually have a context that represents a PDF document and you can draw a CG context calls into that context and it goes out into the PDF file. Well, there's no pixels that you can get to behind that context.

So if you want to be able to say do some CG drawing and then get at the bits after that, what do you do? Well, there's a thing called CG bitmap context create. That as you might imagine creates a bitmap context. And you get to specify all of the parameters for this bitmap.

You can say what the data pointer is to initialize it with some data. Or you can pass null and it will allocate the data blank for you. You pass width and height, bits per component, the color space. You define essentially how each pixel is represented. And this gives you a bitmap that you can then use CG calls and draw into. But you can also get to the bits directly.

So this is what it looks like. So you do bitmap context create. Instead of just doing the create for the port, we do a bitmap context. Pass it in your initialization data, a width and a height, bits per component, color space. Then alpha info tells it whether there's an alpha at the beginning, an alpha at the end, no alpha at all, those kinds of things.

And at this point, you can just proceed with your normal core graphics operations, just like it's any other context. or as I mentioned you can access the bits because you still know where the data pointer is. You still know where the bits are. So, you can go both ways there.

Now, I mentioned that clipping was also different. When you use that code that I've showed you before to go from a port to a CG context, There are some things that the context doesn't inherit. In fact, it inherits very little of the Quick Draw state. Clipping is one of them.

If you're using Clip Regions in your Quick Draw port and you create a CG context and you start drawing with CG drawing commands, it's going to draw all over everything. It's going to ignore the Quick Draw Clip Region. So what do we do about that? Well, the straightforward way, you grab the Clip Region from the CGraph port and then you call this new call Clip CG Context to Region. And that simply takes the region you give it from Quick Draw and actually turns that into a CG Clip Path.

This one's a little bit more complicated. So again, we create our port. And now we want to make the clip match. This is an important step, that new region. When I tried to do this demo the first time, I forgot that. It didn't work very well. Let's just leave it at that. So you create a new region, an empty region. You also need the bounds. And then you get the port clip region.

and you clip the CG context to region. So, a couple of simple steps and then you're also, you're done with the region so you can dispose of it at that point. And at this point, your CG context clip path matches the region that you had in Quick Draw. So let's do a quick little demo of that one.

same little rocket ship. It's entered the same demo. Except in this case, before I started, I set a clip, or set the clip region in Quick Draw to a couple of ovals kind of X-ored together. And of course, Quick Draw would honor that by default, but once you create the context and start drawing these CG context images into that, they're not gonna honor it.

So you use this code that I've shown you, pass in this clip region to CG, and it turns it into a clip path and just does the right thing. And incidentally, it also clips, because I clip before the erase of the back buffer to black, so it clips that as well.

So there are a few things about Quick Draw on 10 that are a little different than on 9. One interesting thing you need to be a little bit wary of is create new port, which you use to create a new graph port. At the moment,

Another thing to note is that in the past, especially for things where you were using copy mask or seed fill and things like that, you needed a graph port.

You needed essentially a one bit buffer to hold the mask. You used open port and create port, or new port, excuse me. Well, those have been replaced by create new port, but create new port can't create black and white graph ports.

Another thing that's different about Quick Draw is because of the double buffered windows in Mac OS X, you need this little thing called QD flush port buffer.

and for the most part you may or may not run into this depending on how you do your event loop, but If you're used to not calling wait next event, your game likes to take over the whole machine, and you start doing quick draw drawing in a loop, you may find that your animation never appears on the screen.

And you scratch your head for a while. Well, it turns out, if you're not calling wait next event, which is when an implicit flush of the back buffer occurs, you need to call QD flush port buffer. And that'll actually tell the Windows system to flush the back buffer to the screen.

And so if you're in a tight little loop doing some animation or whatever, you need to call that. Another, I guess, bonus of QD flush port buffer is when it comes to VBL syncing. I'll talk about this in a little bit, but QD flush port buffer is also VBL synced where it's possible. It's not possible in all hardware.

So that leads us directly into tear-free blitting. A good goal. And for most of you out there, we've got you covered. OpenGL itself can sync to the VBL, and in fact, that's one of the best ways to do it, because it's the card itself doing the VBL syncing.

And there's one quick call, AGL set integer on the context, set the swap interval to one, and that will take care of telling OpenGL that you'd really like it to only swap on the vertical blank. So for all of you full screen OpenGL apps out there, there you go.

Now for those of you that are doing windowed apps, You can use the back buffer. This is also an important point. We already provide a back buffer, so if you're doing your own double buffering system,

So use our outback buffer and also when you flush the back buffer for the window to the screen using QD flush port buffer or if you're using the CG context, there's a CG context flush that's essentially the same thing. We flush that sync to the VBL for you when it's actually supported by the hardware on the machine.

So along the lines of tear-free blitting, some other things to watch out for. Since Mac OS X is a fully preemptive OS, you don't always have control of the whole machine. You don't always have control of when you get swapped in and out. So if you try to do direct screen access, which you can do with the CG Direct Display API, it's going to be very hard to sync to the VBO because the scheduler is always trying to balance things and swap things in and out to make sure everybody has time. You may be halfway through drawing to the screen and get swapped out.

That's the way things go. So that makes it very difficult. So if possible, let us do the VBO syncing for you. We've got the best chance of making it happen on the given hardware and we can do it very easily. So use the AGO set integer or use the actual flushing of window buffers. Let us do the syncing for you.

I've got a couple of quick demos of this. Give you a peek at What Novibelle syncing looks like. It actually tears pretty bad on my machine here. That tears every now and then. Let's see if it'll do it for us. That's actually not bad. Figures the demo works better than it's supposed to. Oh wait, I saw it. There it goes. That's better.

No respect. That's what tearing looks like for those of you that have never seen VBL tearing. So now if we actually use the QD flush port buffer, we can sync to the VBL, right? Not exactly. And that's one of the things-- I was initially very disappointed that this demo didn't work when I got here, because I'd had this all planned, gone to the work to make it sync very nicely, made sure that we could get nice, smooth animation, no tearing. Get in the demo room, it doesn't work.

Well, it turns out this brings up a very interesting point. You can't always sync to the VBL, no matter how much you want to.

and be aware that no matter how much you want to sync to the VBL, no matter how much you want the demo to work, it may not.

So let's talk a little bit about a few OpenGL changes. Many of you are using OpenGL. Got some great games out there using OpenGL. OpenGL and Mac OS X rocks, good performance. But there are a few differences that you need to be aware of. And we'll also talk a little bit about how it's different with respect to using Withdraw Sprocket.

So a few important differences. AGL full screen works. That's a big one. In the past, on Mac OS 9, AGL full screen wasn't there. On Mac OS X you can call AGL full screen. I believe you can pass it even the resolution you want, the bit depth, and it does all the work for you. Very good way to get into full screen mode in OpenGL in a hurry. OpenGL does all the heavy lifting for you.

We've also added a few new features. We have support for stencil buffers and aux buffers. I'd like to point out stencil buffers in particular. There's an OpenGL talk coming up. and I will be talking about how to do dynamic shadows with stencil buffers. That's pretty cool. He's also got a few other cool demos in there as well. So that's a good talk to go to. And I'll point you to that once we get to the end of the talk.

and then there's some new extensions. We support packed pixels now and a few other good things. So for more information on those things, go to the OpenGL sessions and they'll give you some more information. Another thing to note is OpenGL is fully supported. You can call it from Carbon and Cocoa.

From Carbon you can use all the AGL APIs. From Cocoa there's actually some very good classes. For NSOpenGL there's NSOpenGL view which makes it very easy to bring an OpenGL view into your window and get all the information set up for OpenGL and then start making OpenGL calls.

A few more differences. AGL update context is more critical. You need to call it when you make changes to the context that OpenGL knows what's going on. I think in some cases you could get away with not calling it everywhere on Mac OS 9, but on 10 it's much more critical, you need to call it. And also, AGL set drawable.

A number of people could get away with calling AGL set drawable just passing in the window pointer. Because on Mac OS 9, window pointer was pretty much the same as a C graph pointer. Well, Mac OS X, they're not the same thing. And if you've got a window ref on Mac OS X from a Carbon window, you need to make sure you call get window port and pass that port to set drawable. Otherwise, it's not going to do what you expect.

So how do we make this play with Drossproget? Well, on Mac OS 9, When you wanted to be able to get hardware acceleration, multiple monitors, be able to support all those things, typically what you would do is you would create a window on top of DrawSprocket, as I mentioned earlier.

Just create a full screen window, it was black, nobody would know the difference. And you'd actually pass that off to set drawable. And that would make sure it took care of multiple monitors, hardware acceleration, all the good things. On Mac OS X you don't need to do that anymore. And in fact you can't.

Because once you've taken over the screen with DrawSprocket, it's the top most window. So you can't put a window up on top of it. So what you need to do now is call this DSPContextGetFrontBuffer. And that will return you a CGraphPointer. And that CGraphPointer you can actually pass to AGLSetDrawable.

So in a quick summary before we get into more discussion of the roadmap, use DrawSpark or the CGDirect display. They're great. The APIs are there for you to manipulate the displays, capture the displays. As I mentioned, CGDirect display takes care of making sure that

Another important thing to take out of here is Mac OS X double buffers so you don't have to.

and David Schmuck. So if you're already doing a bunch of work to double buffer your drawing, we've got that covered for you. Every window is double buffered. When you draw into the window port, you're actually drawing into the back buffer, which also means don't forget to call flush. Make sure you call CG flush port buffer. Or if you're drawing in Quick Draw, make sure you call QD flush port buffer. Also watch out for some of those minor Quick Draw changes that I mentioned before. And OpenGL and Mac OS X. It's there, it works, it rocks.

Use it. So let's take a look at all of the games related sessions that are coming up this week. We're already into Tuesday afternoon, but there's a few other sessions on that 17 session track that I'm not going to call out. I'm just going to hit some of the highlights here. OpenGL. Four very good OpenGL talks coming up tomorrow afternoon and Thursday morning.

You might want to look into those. High Performance 2D is going to show you how to do some multi-texturing of large images and scrolling of images and things for 2D. With the techniques that Geoff's going to present, you can put together a really cool side scroller and actually scroll all over the place in both directions. Right after that, in the same room, we're going to cover geometry and modeling. Todd's going to take you through how to go a little bit beyond that little spinning square demo that we have in a lot of our sample apps to some more advanced modeling and animation.

Then Thursday morning, John Stauffer is going to cover some optimization techniques, how to get more performance out of your OpenGL code, and how to speed things up and also talk about some other interesting aspects of OpenGL performance. And then right after that, Thursday morning, we have advanced rendering.

As I mentioned, we're going to cover some stencil buffer techniques. Also going to cover bump mapping and cubic environment mapping and I forget what the fourth one is, but good stuff. All four of those are good. If you have some feedback for us as far as OpenGL goes and the work we're doing, Thursday at 5, come to the feedback forum. Make sure you come and give us your feedback.

But wait, there's more. So Input for Games. Friday at 10:30 in the morning, Geoff's going to be talking about Input for Games. As many of you know, Input Sprocket is not on Mac OS X, but we're going to be talking about HID Manager, which is the Input Sprocket replacement. And then following that afternoon, we're going to cover Sound and Networking. So we'll be talking a little bit about the Carbon Sound Manager, about QuickTime, also some about Core Audio and some things you can do with it.

And then get into some Open Transport, Open Play, things like that. And another feedback forum Friday at 5:00. If you have some feedback for us as far as games related technologies and things like that, please come to the feedback session Friday at 5:00. And so who to contact. If you have contact information, if you have questions, send it to [email protected]. We'll be glad to help you out.