Good afternoon. Is everybody ready to go home? No? Okay. Well, then make yourselves comfortable. So hello yet again. In case you don't know, I'm Matt Drance. I work in DTS, and I can't stay off the stage. I can't help it. I've been on stage every day this week. I think that might be a record, actually. So I kind of adopted this talk at the last minute. Some of you may have been expecting to see Gerard Ziemski, who's our resident graphics guru. But he had a small incident come up Tuesday night.

So-- So Gerard won't be joining us. I'm going to do the best I can to sound as authoritative as he would on this topic. So what are we going to be talking about? We're going to talk about the changes that we've made in Java 1.4.2 Update 1. You've got Developer Preview 3 available to you as of this week. We're going to talk about the architectural changes we made specifically to on-screen Java 2D drawing.

And we're going to talk about improvements not only in correctness and hopefully eliminating a lot of the bugs that some of you have reported, but as well the performance improvements that we've seen as a result of these changes. We're going to talk very briefly about Quartz 2D and its relationship with Java and how that story has changed with the announcements that have come in with Tiger this week. And we're going to talk a little bit about the third-party APIs that are new to Mac OS X as of the last time we talked here last year, specifically Java Advanced Imaging, Java 3D, and JOGL, which we affectionately know as jogl.

So Java2D on Mac OS X. Like I said earlier, we have a bunch of improvements and some new features that we've made available to you with DP3. Among them is graphics context management, and I'm going to be saying context a lot in this talk, and within the context of this presentation. By context, I mean just a standard Java graphics object, the Java AWT graphics object.

We're going to talk about state management, the state of a Java graphics object, and how we've worked around a lot of problems that we've had with that in the past. We're going to talk about how these changes affect and improve multi-threaded drawing in Update 1. We're going to talk about the intelligent screen update mechanism that we've introduced with DP3. I believe that's new with DP3, right? Okay. And we're going to talk about some improvements to full screen mode drawing and XOR drawing as well.

So let's go right into it and talk about context management. Like I said, it's a new mechanism in Update 1, and it's completely independent of the Cocoa drawing context that we use beneath. As you know, the AWT in 1.4.2 and Java 1.4 is built on top of the Cocoa App Kit. Now, in the past, we used to share our graphics contexts with Cocoa, and it created a number of problems or issues that we needed to be delicate with.

And the most problematic one was that sometimes Cocoa would prevent Java from drawing altogether. So if there was some painting things that needed to be sent through the context, Cocoa would sometimes deny us the shared context, and it would cause us to either skip frames or paint methods altogether, or at least delay the visual appearance of painting on the screen.

So the new system improves performance in addition to correctness and the perceived quickness by the user. Before, not only could the context have been denied by Cocoa, but it could have been modified by Cocoa as well, and we didn't necessarily know what we were getting back. And we needed time to reconcile those states and make sure that the state that you expected your Java Graphics 2D object to be in was the same as the state that we were using behind the scenes. So now, because the contexts are independent... ...we always know the state of your Java context, and we can start drawing that much sooner.

And when I say state changes, I mean things like color, clip, anti-aliasing font, all the set methods that you would call on a Graphics 2D object. All of those things we know at all times now what those states are from what you're setting in your Java code. Before, we had to make sure whatever we got back from Cocoa was synced with what your Java code was expecting. And now that we have an independent context, we no longer have to worry about that. And that, obviously, is less execution time and more time devoted to painting operations.

Here's a visualization of what I'm talking about. It might sound a little confusing. What you see here is, we're having a little fun with it, I guess. You see Java and Cocoa basically fighting over the same context. And that checkered blue and orange there is basically kind of illustrating the unknown state that a given shared context would be in, in the original 1.4.2 release.

And whenever Java got it, we needed to make sure it wasn't in Cocoa's state. We needed to make sure it was in our state. So now in 1.4.2 Update 1, because we have our own contest exclusively available to Java, available to you, we no longer have to worry about any of that reconciliation. Cocoa can do what it wants, and Java can do what it wants.

This particularly is good for multi-threaded drawing. The improved context management that I was just talking about basically allows us to draw correctly from multiple threads. We don't have to worry about locking or contention or anything like that. It allows us to cache graphics 2D objects. Now that we don't have to worry about state validity or anything like that, you can safely cache a graphics 2D object where in the past it was really, really not a good idea. And not only does this make things more correct, but it makes it very, very much faster. And this is a huge benefit, particularly to old applets from the 1.0 and 1.1 days that maybe had been doing a lot of multi-threaded work from inside the browser.

And here's another illustration of that. You can see this huge heavyweight master lock that's basically blocking or barring the entrance into all these contexts from all the threads. And in the past, you had Cocoa, the main Cocoa drawing thread, and all of your Java threads, if you had multi-threaded drawing, trying to go through the same locks, through the same area.

Now with update one, it's just the Java threads. Because we have our own context and because we have our management down, much better. It's a lot less work to go through from multiple threads and draw to the screen. And I'd like to invite Caroline up to show us just what we're talking about.

Thanks, Matt. So this demo is going to show you the huge improvements we've made in multi-threaded drawing support in 1.4.2 Update 1. First, I'll show you a demo of 1.4.2 doing some multi-threaded drawing, or trying to. The first square is actually the normal repainting mechanism, not multi-threaded, where you make your paint method and the events get posted to the AWT thread.

and the second square, let me open this up again so you can see it again. The second square is using a separate thread and calling get graphics on each repaint. The third and fourth squares are each using two drawing threads, but they're not working very well. So if we go to 1.4.2 update 1 on demo 2. On demo 2, please.

You can see that it looks a lot better, and it's considerably faster in all cases. So these bottom two squares, the third one is using two threads. One's drawing in black and one's drawing in yellow, and it's using one cache graphics object to draw into. And the fourth square, the green and black one, there's a green thread and a black thread, basically, each drawing a green or a black line, and they're drawing into two separate cache graphics objects. So if you need to use multi-threaded drawing, it's now a real option for you in using 1.4.2 Update 1. All right, I'll hand it back to Matt. - Thank you, Caroline.

You sure those were both running the same hardware? Same hardware. OK. Same demo. You sure? Okay, so we just showed you that multi-threaded drawing works a lot better in Update 1, but we want to mention a few things just for general advice.

So the next thing we want to talk about is intelligent screen updates.

And this is kind of the marketing term. You may know it more commonly as flushing. So the thing that we were talking about when we were coming up with this mechanism is that hardware is getting faster, software optimizations are always happening, but displays are still fixed at a relatively pretty slow rate.

Your application can usually draw at 100 frames per second or more, and the display is chugging along at 60, 70 hertz. And so it seems like a waste of time trying to push all this stuff up to the screen when it's not going to refresh anyway physically. So what update one does now is it only updates the screen when it's necessary, when we know it's possible. You can see the thing on the screen here. It says 100 Java updates. There's more than 60 screen updates.

There's no point in actually performing that flush operation if you're not going to see it. Now, this might sound a little scary, but we've done a lot of work to make sure it's correct, and we're not skipping any paints or dropping any frames or anything like that. There shouldn't be any visible difference. Well, there should be something of a visible difference, and that's that drawing should actually be faster as a result.

Much to the tune of the state management and the state reconciliation I was talking about earlier, we're spending less time waiting for screen updates. So we're wasting less time doing that. We're wasting less time doing that when it's redundant anyway, and that gives us more time to process your next paint event.

Also, full screen mode. This is a place where we've gotten a lot of, I'll call it, feedback on. We've re-implemented it primarily for correctness, and in doing so, we also did a lot in the speed department. And actually, it wasn't that complicated a re-architecture. We basically changed from a blip buffer strategy, which is basically copying the bits directly from the back buffer into the front buffer, which is a particularly expensive operation.

We changed that into a flip buffer strategy, where you have a front and back buffer, and then just flip those from one to the other back on the screen. The former front buffer is now your back buffer, where you do your next frame worth of drawing. And I think Caroline's going to show us what that looks like.

So I've got one more, another before and after demo. Here we have a full screen application running in 1.4.2. And what this is trying to do is it's trying to maintain 30 frames per second. But since we're only getting 10 frames per second, there's only going to be one ball. The more, it continues to add balls until it reaches 30 frames per second. So one ball. But if we move to 1.4.2 update 1 on demo 2, please.

You can see it keeps on putting in balls until it reaches 30 frames per second. How high are we going to get? This is a pretty fabulous improvement. Full screen is now a really good option for you if you want to use it on Mac OS X. All right, back to Matt.

Thank you very much, Caroline. I don't know if you had seen, had any of you used Puzzle Pirates before when you had seen Scott's deploying demo earlier? Puzzle Pirates did not used to do too well in full screen, and obviously Scott was demoing it in DP3 the other day. But yeah, one ball at 10 frames per second, 600 balls at 30. That's 1800x, I think. And that's all software changes, so Gerard did a good job there.

So we also want to talk about XOR drawing. Last but not least, well, maybe it is least, we did a lot of work to improve the correctness of XOR drawing. And this has been a perpetual thorn in our side. And the main reason being is that Quartz, which is what our Java2D drawing mechanism is built on top of, doesn't support XOR natively. And if you've been to any of the Quartz sessions, you may have heard this from the Quartz engineers themselves.

So we've had to implement it for you on our own. And the correctness should be there now, but obviously there's going to be a cost because we've implemented it all on our own. So the performance may not be what you expect. And it probably never will be because Quartz simply doesn't support it natively.

And so what that means is that you should probably, if you feel like you need to use XOR drawing for some reason, you should probably talk to us to see what you really need to do and see if we can offer some kind of alternatives for you. So we've had to implement it for you on our own.

And so what that means is that you should probably, if you feel like you need to use XOR drawing for some reason, you should probably talk to us to see what you really need to do and see if we can offer some kind of alternatives for you. Things like alpha composites, if you need to do, blending operations.

And the other thing that Barry and Tom mentioned already on Wednesday is that if you're doing XOR to erase things, you're doing a double draw to erase some text, for example, anti-aliased text is not going to be erased by XOR drawing because of the anti-aliasing operations. Those are not taken into account when you do an XOR operation. So even if that was your real reason for using XOR, it's not going to work anyway.

And if you need to do an erase operation, the best thing to do is to keep your static scenes, the before, the before modification scenes in something like an off-screen buffered image and then push that to the screen when you need to do your erase or your undo.

So what are all the performance gains as a result of all these things we're talking about? Well, I mentioned the context management changes. We're no longer waiting for Cocoa's permission to draw. We don't have to do any guesswork on the state from when Cocoa may or may not have modified it.

And we've got safe context caching for multi-threaded drawing if you want to do that. The intelligent update mechanism means we spend less time waiting for the screen and more time actually executing your paint code. And the full-screen drawing change from a blip buffer to a flip buffer, that's almost a tongue twister, that change is really, as you saw, quite an improvement in performance. And here's just a quick graph of some of our internal benchmarks as well as the standard ones like the Java2D demo and the SwingMark benchmark in Aqua. We've seen anywhere between 20% and 50% improvement just with these software changes in DP3.

And I think Caroline's going to give us one more demo of the overall changes. All right. This is the last before and after demo. This is really similar to what you may have seen in the Java State of the Union, except in Java State of the Union, we compared ourselves to Windows. And here, since we've been talking about the graphics changes, we're comparing ourselves to ourselves. So this is the Java 2D demo running in 1.4.2.

And the important point to notice is that we're getting around 115 frames per second. Alright, so in 1.4.2 update 1 on demo 2, getting more than twice that, around 200, 250 sometimes. So these are significant improvements. I think you're really going to like them. All right. Back over to Matt. Matt Draance: Thank you, Caroline.

So I think you can see we've been doing a lot of work to speed things up for you. And also, you know, it's hard to demonstrate correctness. The multi-threaded drawing demo did a good job at that. But hopefully you guys will see this in your applications. And we'd like to hear your feedback if it is or if it is not. Either way, we want to make sure we're doing the right thing before we release this update.

So let's talk a little bit about Tiger. I don't have a demo for you, but as you probably heard, Quartz 2D is going to be hardware accelerated using OpenGL in 10.4. And because we're built on top of Quartz to do our 2D drawing, this is just going to be faster Java graphics for free without any work that you need to do or even much that the Java team needs to do. And I have again on there is because some of you from the 1.3 days may remember that we had an OpenGL pipeline of our own in Java.

And you might wonder why that went away. Well, that's because the gentleman who was working on that for Java is now working on Quartz. And if you're wondering why hardware acceleration went away from Java, it's because it went to the whole system instead. And thank you for waiting, and you should have it very soon now.

And the difference between the Tiger hardware acceleration and what you may have been used to in 1.3 is that it really does look like Quartz. In 1.3, the hardware acceleration looked like OpenGL. And in Tiger, it's going to look like Quartz. It's going to have the anti-aliasing and all the things that you expect from Quartz drawing.

So we've been talking about what we've done. I wanted to talk a little bit about what you guys can do. This is kind of a repeat of things that we said last year, but they're still very important tips, so I wanted to put them on here. The big thing with images in particular is that you need to make sure that you are using an image format that's compatible to the system. And when I say when you're using images, I'm talking about creating new images that you're going to draw into to paint onto the screen later.

Using methods like toolkit create image, graphics configuration create compatible image, that line's actually a lot longer. You need to do about three or four other static getters before you get to a graphics configuration. But static images, meaning things that you've loaded from a JPEG file using imageio.read, new image icons, so on and so forth, those things will be fine. We will optimize those for you once they're decoded. As far as new images that you're going to do drawing operations into, make sure you use these compatible image APIs. Things like particularly the index color formats that are very popular on Windows because they're low memory.

That image format is not compatible on Mac OS X, and we're going to need to do pixel conversion for every operation that you do. And that brings me to that third bullet there that says if you must use a non-native image, please avoid doing direct pixel manipulation. Because every time you have to go get the data buffer and get the raster, we'll need to convert those pixels from whatever the image format is to a new image format. So if you're going to do something that Quartz can actually read, we'll need to do that every single time.

If you use general Java drawing methods like draw string, draw line, we can fake that and not worry about doing the conversion every single time. And obviously that on-the-fly conversion is going to be expensive. So if you can, please use the compatible image formats and try to avoid using something non-native.

So let's talk about some of the new things. It's not exactly new anymore. We announced it, I believe, in December or so. But we haven't talked about it on stage yet. So I wanted to mention that Java AI is now available on Mac OS X. You saw a demo this Monday night with the Mars Rover, the gentleman from the JPL labs in Pasadena.

and you know, it basically brings high performance image processing, a number of new formats that are not available in the standard Java AWT. You can do image processing over a network. We did implement the native media libcode from Sun and brought it over to Mac OS X so you can get the performance that you would expect from JAI. And it's going to be included with Tiger. It's currently a download for you and more importantly for your users if you're writing in JAI for Panther. It's available as a download. In Tiger, it will be pre-installed, and you won't have to worry about it.

And to the same tune, Java3D was made available at the same time. It's a higher level 3D model. It's not really a direct 3D programming API. It's intended for higher level scene graphing and things like that. It is platform independent, so you can write applets and applications in 3D.

We implemented... We implemented Java3D with OpenGL and Core Audio for the audio. And like Java AI, it'll be available in Tiger and currently available for Panther as a separate download. And there's one other thing to talk about, which is JOGL. I'd like to bring Ken Russell up to talk to you about that.

Good afternoon. So my name is Ken Russell. I'm at Sun Microsystems, and I'm one of the authors of the JOGL package that you may or may not have used. And Gerard and I worked a bit on the JOGL port to OS X. So what is this package? It's a binding, a Java binding for the OpenGL 3D API that works on as many platforms as we could make it work on.

In particular, OS X, it works on Linux, it works on Solaris, it works on, yeah, Windows. And the intent here is to let you write portable, fast 3D applications in Java. The binding is open source. It's part of Sun's Java gaming initiative. You can get the source code on java.net. It's one of the core gaming API projects. We've got bindings to JOAL, the open audio library. We've got an input device binding called JInput, which lets you get inputs from, you know, joysticks and keypads and stuff like that.

Some of the features of this OpenGL binding versus some of the others that are out there are first of all that it supports all the stuff that's in OpenGL 1.5. So you use this, you get all the latest stuff, all the latest vendor extensions. You can do vertex programs, fragment programs, shader objects, all this stuff. It's got portable support for P buffers.

So you can do hardware accelerated off-screen rendering in a portable fashion, get the results on screen very quickly. And where necessary, you can actually dive down to the vendor-specific extensions. So it's like if you actually had to use the wiggle APIs for some things, you could do that.

You can sort of test to see what platform you're on and then dive down into those APIs if necessary. We are working on standardizing this binding. And the real reason for this is so that we can get some amount of sort of compatibility guarantee for you, the developer, on all the platforms that you're going to be deploying on. And another intent here is to try to get the mobile device space and the desktop device space where J2ME and SE have a bit of a rift.

We're trying to make the OpenGL bindings for these two platforms as similar as possible so that you'll be able to effectively write a desktop app. And maybe when the cell phone market and hardware acceleration down there catches up, you'll actually be able to deploy basically the same app on a mobile device, which would be really, really cool. So as I mentioned before, JOGL supports multiple platforms.

And what I'd like to point out here are two things. First of all, even if you're not using the AWT and that sort of framework and you're using the Cocoa framework, you can use the Java bindings for Mac OS X. JOGL actually supports that configuration explicitly. So you can instantiate a little Mac OS X GL Impel object and then treat it in exactly the same way as you would in an AWT-based JOGL app. And it works. And we've got -- unfortunately, we don't have a demo today showing this off, but there have been demos developed in the past which have stressed this out.

Another thing to mention is, again, the fact that we're working on a lot of things in the future. So we're going to show you a little bit of the full screen support that was just shown that's working great with JOGL and OS X. So you can write full screen, fully hardware accelerated games and apps. And we're going to actually show one today that's pretty cool.

And I see a point here on the slide that I did not know, which is that this works well with the new OpenGL profiler on -- oh, wait. Is this Shark or is this -- okay. So, well, it's working with the OpenGL profiler on OS X, so it looks like you can get fairly detailed information about what's going on in your apps.

There are also a couple of -- I'm sorry. There are pipelines available for the JOGL binding. And these let you get debugging and tracing information. And this is sort of a Java-specific feature, but it's extremely useful when you're trying to debug something that's going wrong in your app.

It will basically stop at the point of failure at the actual OpenGL call that made the error state in the state machine and give you a backtrace, which is pretty helpful. So what I'd like to show you is a really cool demo done by a couple of crazy guys over in Germany called Bytonic Software.

I assume that this is a play on words of bionic versus bytonic. But so if we could go to the demo 2 machine -- oh, wait. Aha. Wrong machine. Okay. What we're going to see here is a full-screen app. Full screen little game that you guys may have seen before. This is Quake 2.

Now, what are you looking at here? This is not a new engine written in Java. Turn the sound down just a tad. Um, okay, so what you're looking at here is not a new engine. Let me kill some guys here, maybe they'll shut up a bit. Mute. On the keyboard, huh? Ah, nifty.

Okay, thank you. All right, so what we're looking at here is it's not a pure Java new sort of engine that is written to use the Quake 2 file format. This is a port to the Java programming language of the original C source for Quake 2. Whoa. Hold on a sec, I'm going to kill some bad guys.

Yeah, I got to take care of some business first. Okay. All right. Yeah. Okay. So this is a port, a straight transliteration of the C sources for Quake 2 that were available via the GPL on id's website to the Java programming language. All right. They used jogl at the bottom layer to get to the graphics subsystem, and they did whatever data structure conversion was necessary from C to Java to be able to use the pack file format, the original data files from the game in this port. support. This is running at over 85% of the speed of the original C sources.

At this point, there is no reason to not write your next game in the Java language. This binary runs on multiple platforms. It runs on OS X, it runs on Linux, it runs on Windows, it runs on Solaris. Okay, no porting necessary. And now you can do even more cool stuff because you're in Java. You can ship bytecodes around the net and it's all safe.

You can load new stuff in, new functionality, download it to the client. You don't have to worry about, you know, oh gee, did I just open up the door for the next Sasser virus, for example, because you've got a security manager in place and the access is all safe to the underlying hardware features.

So this is actually an open source project. You can go to the Bytonic software website and the demo is called Jake2 for, you know, Quake 2 to Java. So what you can do is just Google for Jake2 and they are, I think, the first hit. There are websites out there.

You can get the binaries, you can get the source code, and I would encourage you to send them feedback because I think that this is just awesome stuff. They did what a lot of us have been talking about doing for a long time, which is, gee, what would happen if we had the huge commercial apps side by side, C and Java? How fast would it be? This is showing that it's pretty darn fast. I think that's pretty much it.

Thanks a lot, Ken. Awesome. Well, thanks, Ken. I kind of hijacked Ken with that OpenGL profiler bullet. At the last minute, we decided he was going to say this slide. So, yeah, he didn't know about that. I was talking to some of the OpenGL guys, as well as Gerard, about the fact that you can use the OpenGL profiler to profile a jogl application. Now, it's not going to give you, I mean, it is a C-based profiler.

But because jogl, or J-O-G-L, is a wrapper, it's pretty straightforward as to what it's talking about. And you get a list of functions, and they may be C functions. But, you know, the names are very similar, and it's usually pretty easy to tell what's going on in your jogl app versus what the OpenGL profiler is reporting. And again, we demoed JAG 2. That's the bitonic.de, as in Germany, link if you want to check it out. And yeah, it's pretty cool. We've been playing it quite a bit.

So in conclusion, we're going to let you guys out early today because it's Friday, but we're working on it. We've heard a lot of your feedback, and we know that we've had a lot of work to do, and we've done a lot of it. We've done a lot to improve correctness in drawing. We've done a lot to improve drawing performance.

We've done a lot with system integration, especially we've been working very closely with the Quartz team to make sure that everything in Tiger is working smoothly, not only for Quartz, but for Java as well. And we've got these new APIs available to you, Java3D, JavaAI, and of course, JOGL. Thank you.

We want to make sure that you do use your images wisely. Use the Java APIs to create a compatible format. And try not to manipulate pixels directly if you don't know what kind of format you're using or what you need to use. And check out jogl. And write your games in Java. You really can do it now. There's really nothing stopping you.

So for more information, obviously we gave you the Bitonic link earlier. We've got a DMG, of course, with every session. The reference library is available to you for all the Java items. We should have a tech note coming out for image performance tips and the kind of things that I talked very much in brief today.

And there's a great article, which is not new at all, about painting in AWT and Swing and what the expected behavior should be on a standard Java2 platform. And these are the kind of rules or guidelines that you should keep in mind when you start to do a lot of heavy drawing in Java.

This is a really priceless article, and it's very similar to some of the stuff I talked about today. I took a lot of my information from that article. The JOGL homepage is off java.net. And, of course, you can go to java.sun.com for anything else you're interested in. in that's not listed here.