Good morning. Welcome to the Resolution Independence on Mac OS X session. I like to call this feature Resolution Independence, but you'll probably catch me calling it High DPI occasionally. So if you hear High DPI, I'm talking Resolution Independence. It just often gets confused in my head as I say it. So the purpose of this session is to tell you everything you need to do to make your application resolution independent on Mac OS X. But before I go into those details, I want to tell you why this is important.

So lots of the technology industries that we're used to constantly make products that are bigger and better. Your TVs are moving towards HDTV, DVDs moving towards Blu-ray, mass storage went from megabytes to gigabytes and terabytes and beyond. And displays really aren't any different. Every couple of years, Apple ships another machine with a higher resolution display than it has in the past. And we expect this trend to continue. And therefore, we are telling you this year that we need your applications to be resolution independent by 2008.

So let's talk a little bit about what resolution independence means. So this is a screenshot from a 23-inch display. I don't remember the actual pixel resolution on each dimension. But when you use this 23-inch display today, it's running at 100 dots per inch. Looks pretty darn good, very usable.

If Apple decided to ship a 23-inch display, however, that used 200 dots per inch, you'd get something like this. And that may look like a lot of awesome screen real estate to you, but in reality, it's not very usable. Right? The text is way too small. The graphics are way too small. It's just impossible to get done what you need to get done.

So what we really want to do is make this text on the display and the images on the display, instead of being smaller, we want to use those added pixels to make them crisper. So let's zoom in on this top left corner. of the display and take a look at how it looks today. So this is a screenshot of text edit. This was an actual screenshot of a very small portion of the display.

And as you can see, it's kind of grainy, right? At a normal viewing distance, it looks pretty good. But when you scrutinize the text, you can see that it's anti-aliased. You can tell we're using bitmap artwork for the various system controls and so forth. And if we actually applied more pixels to this image, we can make a much crisper and sharper looking user interface, something like this. Right? So you can see we've got crisper text, we've got crisper system controls, and it looks just generally really, really good. So again, this is now. This is how it is today, and this is how it could be in the future.

So let me show you resolution independence in action. Can we switch over to the Intel demo machine real quick? Okay, so this is the leopard seed with nothing special on it. Now, we don't actually have a high resolution display here. In order to demo resolution independence, you have to deal with a little bit of, I don't want to call it smoke and mirrors, but it's essentially smoke and mirrors.

I'm not going to give you a high resolution display, but I'm going to make the system act as though you've got a high resolution display. In order to make crisper text and graphics in a given window, we need to put more pixels in those windows. So we've got this concept of a scale factor.

The scale factor is the ratio of the pixels across a certain dimension of a window on a high-res display relative to a normal resolution display. So if your display has twice as many pixels horizontally and vertically, we'd call that a 2.0 scale factor. So Leopards allows you to set a scale factor via Quartz Debug. For those of you that have used Quartz Debug before, this is generally a graphics test tool that allows you to detect redundant drawing, flushing behaviors that are strange and so forth.

But they added something specific for resolution independence for us. In the Tools menu, there's this Show User Interface Resolution item. You choose that, and you get another panel. This allows you to set the scale factor for your Leopard boot. So right now we're at 1. 1 is the default scale. That's what Mac OS has traditionally been.

We can increase the scale factor, let's say, to something like 3.0, and immediately you see a change. It might have been kind of hard to see because it was down at the bottom of the screen, so let's do it again. So look at the bottom of the screen. We're at 1.

The dock is normal sized. If we choose scale factor 3.0, you see the dock immediately gets bigger. The dock is one of the few places on the system that actually react live to a scale factor change. The dock knows it's trying to make each of its dock items three times as big in height and width. It just so happens that there's not enough space to actually make them three times as big, so the dock shrinks it down a little bit. So once we've set the user interface resolution, any application we launch after that fact adopts that higher scale factor.

So if I fire up TextEdit, you can see TextEdit gets really big, right? The window takes three times as many pixels in the backing buffer, both horizontally and vertically, that it would have normally. And it's still a fully functional TextEdit. You can move through the menus, choose commands. I can copy and paste text and delete stuff and interact with the various controls and so forth.

Although strangely, it's not actually working. Huh, okay, well, I'll chalk that up to a leopard bug. Um... Anyway, otherwise it is a live text edit. So clearly we're seeing some benefit here. We've got much crisper, sharper system controls. We've got great looking text. Now there are a few bugs.

If you look over on the right hand side of the monitor, you can see that that scroll bar-- well, it may be kind of hard to tell on this display on the projection screen-- but that scroll bar is still using old low res artwork. It's because we're not finished making Leopard resolution independent.

So there are a few rough edges here and there. Another example that is hard to see are these markers in text edit's ruler. A few of them are still a little bit jaggy. Again, that's because we don't have the high res art. OK, so that's text edit. Let me launch another app.

OK, so we're looking at Safari. And Safari is a little mixed here. So you can see that Safari has these really nice looking window widgets. And it's got great looking menu text. And it has great looking window title bar text and so forth. But when you actually start looking at the web content, you can see that it's a little bit jaggy. And that's because it's web content. The system can scale up all the art in the system, but we can't scale up-- or sorry, we can't actually generate higher resolution art for content that's out there on the web someplace.

So Safari is taking that art and literally scaling it up and interpolating it. So it's going to look a little jaggy and grainy. Now, that's just the-- images on the page. The text on the page still looks fantastic, right? Because it's still rendered with Quartz, and it respects the scale factor and looks great.

OK, so let's see. Let's launch another app here. Get rid of TextEdit. So these are both Cocoa apps. Cocoa does a ton of the resolution independence work on your app's behalf for free. But Cocoa was clearly not the only app flavor that needs to be resolution independent. Carbon, Java apps, and various other sorts of apps also need to be, and they can be as well. So Finder is a Carbon app. Let me quit Finder now and relaunch it.

And you can see that Finder also respects the scale factor. And again, Finder's a little bit of a mixed bag, much like Safari. It's got great looking text, fantastic looking icons. But there's a few system controls in Carbon that haven't revved yet to use the high resolution artwork. And you can see that tool tip that came up is also still drawing low res. But we have a Carbon app here that's also become resolution independent.

OK, so one note about Finder. Finder is what we call a modern Carbon app. They have gotten off of Quick Draw. They're using Quartz to do all their drawing. And they use HIV Compositing Mode. That's the only way Finder could get the crisp text and graphics that they have. Applications that have not adopted HIV Compositing Mode and that still use Quick Draw can't be truly resolution independent and truly look great. But we do have a little bit of a fallback solution for them. OK, let me make finder usable here real quick.

Bring it back at a 1.0 scale factor. Let's crank the scale factor up to 3. And we'll launch Appearance Sample. So Appearance Sample is one of our Carbon sample applications that shows off a lot of the Carbon controls and so forth. It's an unmodified Carbon app that continues to use Quick Draw and that does not use HIV compositing mode.

It's still big. We still make it fit in. But we make it fit in with resolution independence by magnifying a normal old-sized window backing buffer up. So your text is a little blurry and your controls and so forth are a little blurry. And again, this is because we've got a normal resolution window backing buffer that's being scaled up and interpolated by the Windows Server.

So at least this allows unmodified Carbon applications to fit in. But clearly, Carbon applications want to rev to be truly resolution independent. There are some Carbon apps whose clients expect to actually see and manipulate the pixels. Every popular graphics app is this way. A lot of the drawing apps are this way. A magnifier modified drawing app isn't really usable. I mean, you can use it. It's functional. But if an HI designer or an artist can't see pixel perfection, it's not going to work.

So those Carbon apps need to either-- they can rewrite for Cocoa, certainly, and then they'll get resolution independence easily. But the other option is to get rid of their Quick Draw drawing and move to HIV compositing mode. And then it's just as easy to adopt resolution independence as it is with Cocoa. OK, so somebody asked about the menus. The menus still look fantastic. The menus, because they're drawn by the system-- oh, I see, the clipping. That's a bug in the toolbox. We'll fix that before we ship.

Actually, So I'm not sure whether you're talking about the truncation or the menu at the top right. So the truncation is just a factor of the fact that we're running at a 3.0 scale factor, and Appearance Samples menus take up a lot of space. So a 3.0 scale factor probably isn't something a user would run at on a display with this physical resolution.

Typically, when I test and so forth, I will run at a 1.5 or a 1.25 scale factor. I can tell the interface is scaled up, but it's still usable. I don't get my menus clipped and so forth. OK, so that's it for the demo. Let's kick back over to slides.

So I mentioned two kinds of windows. There were the nice looking Cocoa windows, the nice looking Carbon windows, and there were those not so nice looking Carbon windows. So let's go through those and one other, what we call a scale mode. Framework scaled mode is how we got those nice looking Cocoa windows and nice looking Carbon windows. This is where your applications need to be for resolution independence.

It allows you to get the great looking user interface. We take your Windows request for its size, multiply that times the scale factor, and we build its backing buffer out of that many pixels in the horizontal and vertical dimensions. So we've got more pixels that we're using to render the content, and that's how we make it look great.

So that's framework scaled mode. That's where you want to be. What Appearance Sample used was what we call magnified mode. And like I said, this was the compatibility mode to make unmodified Carbon apps fit in. The way this works is we take a normal window backing buffer and have the Windows server scale that up interpolating. So it doesn't look great, but it's certainly usable.

We have a third mode which I didn't show off because it's not implemented, and that's what we call small mode. This is our contingency plan. We found that a number of applications simply don't work in resolution independence mode, either in framework scaled mode or magnified mode. They do a number of things that aren't legal or just aren't compatible with resolution independence, and we can't fix that in the system.

Apps are making potentially bad assumptions about the coordinate systems that APIs are using and so forth. We can't break those assumptions. So what we may have to do instead is put this small mode in place. So in our testing as we're implementing resolution independence, if we find an application that just can't work, we'll probably put an app-specific workaround in place for that app so it comes up in this small mode. Small mode uses the same window backing buffer pixel dimensions as it always has, but leaves the image small as well.

Hence the name small mode. So it allows the apps to actually be functional, but it looks pretty dull. It's pretty jarring, right? If you were on a display that used a 2.0 scale factor and you launched an app that's in small mode, the windows are really tiny and potentially unusable, depending on how high your scale factor needs to be.

So not only will we have app-specific workarounds in place for certain ill-behaved apps, but we will probably also need to make small mode a user option, much the same way the launch in Rosetta checkbox is a user option in Finder, right? Certainly Apple can't test every application that's out there.

A user may be running an app and find some problem running in resolution independence mode that they just can't work around. And so they need a way to request to run this application in small mode so they can get their work done. But we don't have that switch in place yet, but we'll have something for Leopard GM.

OK, so let's talk about how Leopard supports resolution independence. So as I showed off-- A lot of the system controls support high res content in Leopard. Not all of them. There's a few rough edges, but we're working hard to replace the artwork for all of our system controls with artwork that can draw at virtually any resolution. It'll look fantastic at any resolution. We're also adding a few system controls and a few variants of system controls to give applications standard versions of things that they were currently coding up themselves.

And we'll make sure to put high resolution artwork to back those pieces of art so that your app can look fantastic. We've also added another size of icon data to icon services. So it now supports up to 512 by 512 icons. I think the previous limit was 256 by 256.

But most importantly, we're revving system applications to be high DPI savvy. We've chosen a few key applications that stress various parts of the system so that we can make sure our resolution independent strategy works and is actually adoptable. So as I showed, Finder and TextEdit and Safari work, but we also have a few other apps that I listed up here, and there will be more to come by the time LepreGM ships.

We're also adding some new standard art that's accessible via an existing mechanism, but-- well, it's an existing API, but it's a new mechanism. NSImage has supported the notion of named images for as long as I can remember. And generally what this is used for is to fetch an image out of your application's bundle.

So you put foo.jpg in your bundle and you can instantiate an NS image with name foo.jpg and then you'll have an NS image that can render that file. So we're extending that to support system artwork. NS image now supports a number of standard pieces of artwork and this will grow over time until Leopard ships.

So the Cocoa framework makes it really easy to get to resolution independence. Cocoa Windows are automatically framework scaled. You can take an unmodified Cocoa app, launch it, the window will get bigger. And it mostly works. All your system text drawing, system controls are going to look fantastic. Your custom drawing, it's going to be a mixed bag. Your custom text drawing will probably look good so long as you're using any modern API, Quartz API, to render that text.

Your graphics, however, well, we're going to scale those up the same way Safari scaled up the art in the web case, right? We only have the low resolution artwork that's in your application bundles to work with. So we scale it up so that it looks bigger, but it's interpolated.

So you have some work to do there to generate higher resolution art if you're a Cocoa app. But the important thing to note is Cocoa apps basically just work in resolution independence mode. There are a few bugs and gotchas, but we'll talk about those in a bit. One important caveat is that NS Quickdraw View doesn't work in Cocoa.

But the important caveat is that NS Quickdraw View doesn't work in resolution independence mode, mainly because Quickdraw doesn't work for resolution independence. So if you're relying on NS Quickdraw View, you have to rethink that strategy and come up with a different implementation mechanism for whatever it is you're doing.

. Carbon applications, they have a little bit more work to do. By default, as I showed, they come up in magnified mode, which allows them to fit in, but they don't look perfect. So in order for a Carbon application to get to framework scaling mode, they have to get off of Quick Draw and get on to HIV compositing mode. Once you do that, you can go into interface builder and click the framework scaling check box. Actually, I think it's in a pop-up. And that Carbon window will be framework scaled.

And once you get to that point, the story is essentially the same as Cocoa. Everything's going to look great except for your custom content and artwork. You'll need to rev your custom art to make it look fantastic. For those of you who tried out the resolution independent support on Tiger, we used to offer a mode called application scaled mode, which was a way that Quick Draw based apps could be resolution independent savvy.

We did some investigation here and we found out that not only was that very challenging for Apple to make work, but it turns out that it forces an application to do more work to react to that than it did to just get rid of Quick Draw drawing and adopt HIV compositing mode. So we're scrapping application scaled mode altogether. It does not work on Leopard.

Java also supports framework scaled mode. So in general, a Java app looks fantastic. Again, it comes down to the custom artwork. Custom artwork gets scaled up as usual. It's important to note that all Java APIs operate in terms of points. Every existing Java API that passes a point or a rectangle or other sort of coordinate interprets that coordinate in the point space and will scale that up if it needs to deal with pixels.

So our Apple's Java team is working with Sun to put new APIs in place in an upcoming version of Java such that it's got direct resolution independent support to allow your Java applications to tweak their UI a little bit to take advantage of resolution independence in various ways.

So a fourth flavor of application is a dashboard widget. Dashboard widgets are kind of web content, JavaScript. It's a little bit of a hybrid. And dashboard widgets are kind of tricky. So ostensibly, they support framework scaling. But we don't have a full story in place for artwork. So if you increase your scale factor, log out, log in, and bring up dashboard widgets, they will be bigger for the most part.

But the artwork will be scaled up and interpolated, and they don't look great. And unfortunately, there's not a really good way for a dashboard widget to put in higher resolution artwork yet. We're working really closely with the dashboard team, the Safari team, and the WebKit team to establish web standards and standards for dashboard widgets so that they can put high-res artwork out there. Because we want to make sure dashboard widgets looks absolutely fantastic in high resolution.

OK. So there are a number of steps you need to take to adopt resolution independence. The first thing you need to do, look in your applications bundle and find all your art. Find all your PNG files, JPEGs, TIFFs, whatever you're using to render the art in your application. Find it.

Make a list of it. Keep it handy. You're going to go through each piece one by one. Ask yourself some questions. If you're using that piece of art to replicate a standard control, get off that art. Adopt the standard control. The standard controls will be rev to support high res art, so you might as well leverage that.

Maybe your piece of standalone art is something that we now provide via NSImages named image mechanism or via icon services or something like that, or maybe even HITheme. If so, try to get off of your custom art and adopt that other API instead. Again, we're going to rev those APIs such that they've got the high res art, and you want to leverage the system wherever you can.

A third category of art is kind of interesting. As we went through our own applications, we found a lot of apps have bitmap art for things like rectangle fills or a frame, just a rectangular frame around something, or a simple gradient. And all of these things can be represented via procedural code.

So in some of these cases, we've decided to just tell those applications, look, start rendering this via procedural code. The procedural code uses quarts, therefore it's inherently scalable and it looks fantastic in resolution independence mode. So you should take a look and see if you can do that as well.

So once you've made all the code changes, everything left in your pile of art is art that needs to be scaled up, is art that you need to generate higher resolution content for. Oh, sorry. Forgot to do this. The other benefit of going through your art is that you get to toss all of that old art aside and make your bundle smaller, which is important, because pretty soon I'm going to ask you to make your bundle a little bit bigger. So everything that's left in this pile is artwork you need to revise for resolution independence. So what we're asking for is your application art to support scale factors up to 4.0.

or for icons, use the 512 by 512 version, which isn't quite 4.0, it's 2.0, but it still leaves you a lot of headroom. So the reason we want 4.0 or 4X is because it gives you a lot of room for the future. If you generate 4X artwork now, you're going to save yourselves a lot of time when the displays actually catch up to that resolution. So you need to ship your application with versions of the art that work at 1.0 and 4.0.

If this ends up being a problem for apps with a lot of art that are downloadable or so forth, what you can do for the short term is just ship your app with the 1X version of the art and some lower resolution that doesn't take up as much space.

So when you're making your high resolution art, the first piece of advice I have-- and this is just advice, this is not any kind of requirement-- is to avoid the use of bitmaps. If I'm telling you to use a 1x version of a bitmap and a 4x version of your bitmap and put that in your bundle, you've just spent 17 or up to 17 times as much space for that one piece of art in your application. If you multiply that times 100 or 200 pieces of art that we see in the big apps-- and in fact, the big apps often have thousands-- that's a huge increase in disk footprint.

So if you can get off of bitmaps, you're going to remove this huge disk footprint increase from your application. The other problem with bitmaps is you need multiple versions to represent both the low res look of your app and the high res look of your app. Both of those versions need to be maintained. In the future, you may need to add an even higher one.

And this is a time expense that the HI designers would love to avoid if they can. If they can give you a single piece of art today that will work at any scale factor, they'd love to do it. So if you can, use vector formats to represent your art.

PDF is a great example. We're using PDF for a lot of the system art below. I've got a little screenshot of six of the different pieces of system art that we're now using PDFs to render. Obviously, they're much smaller. They scale up to any arbitrary resolution. And it works really great for monocolor art, or art that has slight color transformations to it and so forth.

Hopefully this goes without saying, but we've run into a few people that didn't quite get this. A lot of vector formats allow you to embed bitmaps. Don't do that. That completely defeats the purpose of the vector. The whole point of the vector is that it's not a bitmap. So if you store a bitmap in there, it's essentially a bitmap, and you've lost the whole advantage. So don't do that.

Now, we know from experience that we can't represent everything as vectors and have it look as great as we want it to on the system. We can do a lot of stuff, just not everything. So there are a few pieces of art that we know we need to ship as bitmaps.

I'm sure you're going to find yourself in the same sort of situation. And if you do need to create bitmaps, here's what you need to do. Create your 1x version of your bitmap, which you already have. Have your designers create your 4x version of your bitmap. Make sure you turn on compression so that your 4X version is not monstrously huge.

And then merge them into a single file. Multi-image TIFF is what we're using internally in a lot of our applications. We have the user bin TIFF util tool, which I'll show in a little bit, that allows you to take two TIF files that have a single resolution image in them and merge them together into one file, two images, two different resolutions. But your application code, most importantly, gets to interact with this one file as if it's just one image. You don't even need to worry about the fact that there's two different representations in there. It just works.

Another option that's very popular, obviously, is PNG. If you need to use PNG, it does not support multiple representations in a given file. So your application will need to come up with some kind of naming scheme. You might want to have your existing foo.png and a new foo for PNG, perhaps. And you need to rev your application code to load the right one, depending on what scale factor you're using.

So I already mentioned TIFF Util as one of the artwork tools we have on the system. Preview App has also revved so that it can show the multiple representations in a multi-rep TIFF file. And we've also revved the icon composer tool so that it supports the 512 by 512 ICNS data for your file system icons.

On TIFF Util as well, there's a flag you can pass to it that asks it to validate your art to make sure it's going to work properly in resolution independence. And that's called the CAT High DPI check. There is NS Images code that iterates the multiple representations in an image file, relies on certain pieces of metadata being set properly. And CAT High DPI check makes sure that metadata is set properly. And it'll log to the console if it's not. so you can make revs.

So once you have all this new art, it's time to adopt it in your application. Your file system icons, your application icon and your document icons, they're trivial. Just drop them in your bundle. For the most part, adopting your other revised artwork is pretty easy. If you already were an NSImage client or an ImageIO client, you just drop the new file in, replace the old file, and it just works with your existing code.

Some of you may need to rev to adopt NSImage or ImageIO in order to get this high-res artwork. One thing I don't have mentioned here that we're going to mention in the toolbox session next is that we've revved the Leopard Toolbox to allow you to associate artwork by path with various system controls.

So you can have a bevel button that displays an image, and you can just tell that bevel button, "Go find the image at this path." So that's a great way to make it work with a multi-representation TIFF file, for example. You can just give it the path of a TIFF file, and the Carbon bevel button will find the right image to use in there.

For those of you that want really fine-grained control over your artwork, Image.io is a great way to do it. It doesn't give you as much out-of-the-box functionality as NSImage does. If you want to use Image.io, you can do all kinds of cool things, having to deal with caching and so forth, but you need to manually iterate the images in the given file, find the right one for the scale factor, generate a CG image ref from that, and draw that. So it gives you finer-grained control, but you have to go through a little bit more work. to make that work.

So I have this as step four, but this is really an iterative process, the whole testing part. In theory, we test as we're developing. So the whole point of the testing is try a bunch of different scale factors. 1.0 is going to get tried implicitly by thousands and millions of users. But we have other key scale factors we want you to try. Quartz Debug has five scale factors for you.

There's a 1.0, a 1.25, a 1.5, 2, and 3. And we'd like you to test at all those. In particular, the 1.25 and 1.5 scale factors are interesting because they expose a whole new set of bugs in applications and in the system than the integral scale factors do.

And I'll talk about some of those later. Now, even though Tiger had support for resolution independence, the support is not nearly as rich as it is in Leopard, and we've also fixed a bunch of bugs in the Tiger implementation. So don't test on Tiger. It's kind of pointless. Please test on Leopard.

And when you find bugs, either fix them if they're your app bugs. But if they're the system bugs, let us know. We know there's thousands of applications out there and more every day. We can't test them all. So if you run across some system bug, write it up in Radar, and we'll try to take care of it as soon as we can. We really want the resolution independence experience in Leopard to be awesome.

So to fix a lot of these bugs, sometimes you need to find out what the scale factor is. We have a couple different ways to do that. The preferred way is finding out what the scale factor is associated with the window. The reason for this is you can have an application whose windows, it can have multiple windows, and each of those windows can be operating in a different scale mode. The hairiest example is probably a Carbon app that uses some Cocoa windows, say for the color picker or the font panel, because those are Cocoa windows that come up in a Carbon app.

And that Carbon app has Rev to support resolution independence in its main window. So that's framework scaled. But there's some other windows in that application that haven't been revved yet. And so those will come up in magnified mode. And the Cocoa window will also be framework scaled because it's Cocoa.

And so you'll have these three flavors of windows in your application, and each of them could potentially have a different scale factor. In reality, they will only be up to two. The magnified mode windows will be operating at a 1.0 scale factor because that's what the size of the backing buffer is relative to the old size.

And the Carbon framework scaled and Cocoa framework scaled will be running at some higher than 1.0 scale factor. So that's why we prefer for you to ask for the scale factor on a window-by-window basis. If for some reason you need to find out the global screen scale factor, we've got accessors for that as well.

OK, so as you're testing, you're going to run into a bunch of stuff that we've already run into, and so we have some advice for how to deal with. Clearly, there are bugs in the leopard seed. I already showed off that the scroll bar doesn't have high-res artwork yet, and the segmented views don't have high-res art. Another thing you'll probably run into that happens more in Carbon apps than in Cocoa apps is window jitteriness or menu jitteriness. There are some of our APIs that have not yet been revved to support floating point coordinates.

And floating point coordinates are pretty critical to making resolution independence work. So you get various rounding errors that compound, and sometimes windows grow a little bit more than they should during a grow operation, or sometimes when windows animate in and out of showing a toolbar, they'll jitter size-wise and so forth. And you can see similar kinds of things in the menu bar. So these are all things we plan to fix by the time we ship the seed, but just be aware that you may run into these as you're testing.

There's another bug in Carbon Windows where we did not correctly scale up the drag region for Windows, for framework scaled Windows. I don't remember whether it happens in magnified mode Windows as well. So if you have a Carbon app and you flipped on framework scaling for that window and it's not dragging right, just drag from the top left corner someplace and it should work.

Another sort of annoying thing that we haven't quite worked out are a few of our metrics. We have some text positioning problems in high res. The push buttons and check boxes in particular are the troubling ones. I think the text is positioned a pixel too low. That's something we'll fix. Clearly that means as you're laying out your applications, the text will look a little off between various types of controls, but just ignore that. We'll get that fixed when we can.

So the first sort of application bug that you'll run into is a coordinate system mismatch. So before when I was talking about points and pixels and applications generally not needing to know about pixels, just needing about points, well, there are a few APIs that already talked in point space, and your applications need to be aware of that. Unfortunately, on all previous operating systems, points mapped to pixels at a one-to-one ratio. So you could get away with passing points to these APIs, and it worked. Well, when you're running in high res mode, it doesn't work in high res mode.

It doesn't work anymore. So what you will see for some apps or some windows in some apps is that the windows come up too small or clip portions of the text and so forth. So in these cases, you need to find out whether the API in particular you're using to size that window is expecting points or pixels and pass the appropriate coordinates. And we have APIs that you can use to convert between the point space and the pixel space for both Cocoa and Carbon.

Now we know your applications can't deploy on Leopard and later, particularly since we haven't shipped Leopard. So as you're adopting some of the new system controls and artwork, you're going to need probably to do runtime checks for your adoption for some of the brand new things. But I think what you'll find is a lot of the new system controls have actually been around for three or four releases. So you can probably get on board those right away and deploy them with your existing apps, because they're supported even pre-Tiger.

Let's see. Oh, right. So NSImage has had multi-representation TIFF file support for quite a long time. So you can rely on that. You can actually deploy multi-representation TIFF files in your apps today. And it will work with NSImage so long as you make sure that all the sizes of the individual reps are the same.

But the pixel dimensions are what's different. And that's what the Cat High DPI check that you pass into Tiff Util validates. So if that's not true for your two Tiff files that you're trying to merge into one, Tiff Util will log to the console saying, hey, you need to adjust this particular file because it's got the wrong resolution, or so forth.

OK, so another thing we have that I think they mentioned at the State of the Union yesterday is the notion of pixel cracks. This is another fairly common problem. Pixel cracks occur when you have a bunch of smaller bitmaps that you use to assemble some larger control. You want to make sure those smaller bitmaps align exactly together so you don't see gaps in the image.

And generally, that works fantastic at a 1.0 scale factor. But at non-integral scale factors-- and it's kind of hard to see on that screenshot, so I've got a bigger one-- at non-integral scale factors, those integral pixel boundaries don't necessarily map-- sorry, those integral point boundaries don't necessarily map to pixel boundaries.

And therefore, you get a little bit of anti-aliasing on the edges of your image draw, which cause these lines-- I don't know how visible it is from the back of the room, but there's a couple vertical lines that cut down the screenshot here because those images didn't quite line up right.

So the way you fix this is actually kind of challenging. I wanted to show this in the demo, but it's so hard that we couldn't show it in the demo. The way you fix this is going to be adopting some new APIs that we put out after the leopard seed.

But essentially, conceptually, what you do is when you're about to draw these two images and you want to draw them aligned, you already have your point-based rectangles that you're using to draw them. You need to convert those point-based rectangles to pixel-based rectangles. Round the coordinates appropriately so they fall on integral pixel boundaries.

And then convert that rectangle back to point space and draw into that rectangle. And that's how you fix these cracks. We've got a little bit of sample code in the documentation that works for trivial cases. But as we found, it doesn't work for a lot of the common cases, so we'll get that revved when we can.

Apps that use OpenGL have a unique problem to deal with. OpenGL is one of those API sets that really wants to operate in terms of pixels. That's its API contract. So if you are taking your view, and it's potentially an OpenGL view subclass, or maybe you're a CarbonGL client, and you're taking your point coordinates and saying, OpenGL, please make me a surface this big. Well, if those point coordinates map to more pixels, OpenGL doesn't know that. OpenGL just saw a request to make a 100 by 100 rectangle. So it's going to make a 100 pixel by 100 pixel rectangle, which is going to look too small.

So generally what you need to do is, again, take the size of your rectangle, translate it from the point space to the pixel space before calling the OpenGL APIs. And so this is a little bit of sample code that would work in an NSOpenGL view subclasses reshape method to do that.

In a few rare cases, we've found that our bitmap art does not look as good as we want it to when it's displayed at any arbitrary scale factor. So we're shipping bitmap art in an application that's got a 1x version and a 4x version. When we're running at a scale factor higher than 1.0, we take that 4x version and we scale it down. So Core Graphics, of course, does some interpolation on the image, and occasionally you get some jaggy effects.

To fix this, you need to switch on, well, you can do two things. The easy way is to switch on higher quality interpolation with Quartz. There's APIs from both the C-side and the Objective C-side to do that. But the thing you need to realize is that there is a performance overhead. Anytime Quartz does something that's higher quality, it's going to take more processing power. So there's a little bit of a tradeoff there.

You may find that if you're using this art in a very performance sensitive area, that you can't just ship a 1x version and a 4x version. You may need to ship other smaller versions that match the particular scale factors you care about. So you might want to ship a 1x, a 2x, and a 4x, for example. And that will allow you to make pixel-perfect art for any scale factor.

applications that use off-screens. Not a whole lot of them these days, but a few do. Often you'll see it for apps that magnify other portions of other applications or things of that nature. Applications that use off-screens need to realize that your generation of an off-screen buffer doesn't have any automatic framework support for the scale factor in it.

It's your code. You're saying, give me a buffer that's exactly this many pixels by this other pixel size. So it's up to your application code to make sure you increase the dimensions you're requesting for your off-screen buffer based on the scale factor. So this is one of the other places on the system where you need to think in terms of pixels because that's what the API contract already is.

We've also found a few applications that draw directly to the screen in various nefarious ways. And we don't like it. So stop doing it. There are lots of different ways you can achieve the same kinds of effects from overlay windows and various other things. The ramification of drawing to the screen is that your application will probably get thrown into that small mode bucket by default.

So that's not where you want to be. You really want to be scaled up like everyone else because it looks good. So please stop drawing directly to the screen if you're doing it. The classic example are applications that do XORing as part of like a window resize operation.

You know, it may be an old Carbon application that isn't using our normal window resize code, but they're still trying to get the gray outline and it's doing screen-based XORing. You might find that when you're running at a higher scale factor, the XOR is generating a much smaller rectangle than you'd expect because that rectangle is being actually interpreted in pixels, not points.

Applications that support plugins are a potential challenge. I don't have any super concrete advice here other than offering a general approach. If you have a plugin API that supports any notion of coordinates, you know, maybe it allows your plugin to find out where the last mouse click was or it tells the plugin to draw on a given rectangle, you need to define to your plugin developers what coordinate space those points and rectangles are in. Are they in points or are they in pixels? It doesn't matter to me, it matters to your plugin developers, so document that, let them know so they can do the right thing in the resolution independence modes.

A harder case is plugin models that support Quick Draw. Quick Draw can't be resolution independent, so you need to rethink that plugin model. Maybe this is a great opportunity to come up with a new plugin model that's more modern and you can get your plugin developers migrated over to that path. If you absolutely have to support these Quick Draw based plugins, that portion of your application probably can't be resolution independent. There may be a few other ways.

If you're really in this boat, I'd love to hear from you if you can come up afterwards and let me know who you are and what your plugin API is. I might have a few other ideas. We do something kind of similar to this in the menu manager. We have some, we support older menu definitions that are entirely Quick Draw based and we can allow them to render into an off screen instead of actually rendering to the window that's on screen.

And then we scale up that off screen image when we blit it to the screen. So this slows down menus. It doesn't get particularly great looking menus and it propagates an older API that we'd love to get developers off of. So you may find yourselves in a similar boat. It may just be time to come up with a new plugin model for your plugin developers. OK, so I want to do two quick demos of how you can adopt resolution independence in some apps. So if we could switch over to the demo machine, that'd be great.

So I have a little application we've been working on here called Airport. Shows you the airport status for a number of different airports. And it is a Cocoa application, and I'm running in 1.0 scale factor, normal leopard, so it looks just fine. This application has two things to pay attention to real quickly. It's got this custom view up at the top that says airport status.

You know, you can think of that as the application's logo or maybe my company's logo or something like that. And it has the list of airports. That list of airports is supposed to automatically size so that you can see all the airports on launch. So let me quit this app. crank up the scale factor.

And rerun. OK, so I can immediately see one problem. The window is not sized properly. The scroll bar is showing-- so I can still get to the content of the airports, but really I wanted that window to show all the airports by default. So this is a classic case of a coordinate mismatch, where my application is sizing a window based on points, but the window sizing API I'm using actually wants to operate in terms of pixels. So let's fix that real quick. Go into my Xcode project.

And OK, so I've got a method here called set ideal window frame where I calculate the appropriate height for the list, figure out a delta, and then apply that delta to the window frame size. Ignore the commented out code for now. That's the fix. So what I was doing is I was getting the window frame, increasing the height of the window frame by the delta that the list needed to grow.

And then I set the Windows frame to that. Well, it just so happens that the set frame API expects the size to be in pixels, not in points, whereas my list height delta was actually in points. So I can do a fairly simple conversion to fix that. The first thing I do after getting the window frame from the window, I convert that frame from window coordinates to my views coordinates. And then I apply the delta.

And then I convert back to window coordinates, which are pixels. So if I run that now, The list comes up sized properly. So this is probably the most common problem you'll see in your applications, is your windows don't show up the right size initially. So those are usually really easy to fix. Now one thing that's harder to see-- are some pixel cracks.

Well, let's do first things first. So on Pixie, I'm zoomed in on the artwork for this image behind the airport status. And I can see two problems. One, I can see I have old artwork that's being scaled up. And so it's a little anti-aliased. It's somewhat hard to tell from both the screen up there in Pixie.

But as I stand here and look at the monitor, I can see it pretty clearly. It's scaling up old art. And I don't really want that. I'd much rather have crisp art. Now in Pixie, one thing you can see is that there is a vertical line cutting down just to the right of the curve on the placard.

And that's a classic example of a pixel crack. It's also happening on the right side of the widget as well. So that's because I'm drawing this-- So, first things first. Let's go ahead and fix the art so we have high res art. I'll crank the scale factor down real quick so the other apps are usable.

So, I go through my applications bundle. I find the art. I either rev it myself or I talk to an H.I. design company or my own H.I. designers and say, please give me higher resolution versions of this art. And let's see. They give me back this. They give me back a set of six pieces of art with One X version for a given piece and the four X version for a given piece for both the left, the middle, So, what I need to do is I need to run UserBin TiffUtil on this to take these multiple Tiffs and merge them into -- sorry, multiple Tiffs for just one piece. So, I'm going to take the two left pieces and merge them into a single file using TiffUtil. I'm going to do the same thing for the middle and the right. So, let me show you how that works.

Oh, you know what? Let's relaunch terminal with a higher scale factor so you can actually read it. Is that readable? OK.

Okay, what am I doing wrong here? Why is that not-- there we go. Okay, so I have these six different pieces. What I'm going to do is I'm going to merge the 1x version of the left slice with the 4x version of the left slice. So call tiffytil with cat high DPI check, pass in the two pieces that I want to merge together.

I just want it to be called slices left TIFF because that's the name of the file I already had in my bundle and I just want to be able to take this multi-ref TIFF file and replace the old file in my bundle. And so I run it and it tells me two images were written to slices left.TIFF and I'm glad it said that. It didn't tell me any errors so I know my HI designers did the artwork right.

And so I would do that for the other two pieces, but I'm going to save the typing because, as you can see, I'm not a great typist when I'm miked. So I already pre-merged the multi-rep TIFF files into single files. I'm going to copy them into my images that's part of the project. Let me make sure I copied all three. This has got-- oops. And we get to use Finder in high res. had problems not actually replacing everything. So let's replace them. Go back into Xcode. Clean out all the objects. Rebuild.

And oh, you know what? It did not replace it. So I had my copy bug. Well, let's ignore that bad looking art on the right side of the status bar. So I've clearly got higher res art that I'm using now, and that's fantastic. Now, there were also those pixel cracks that I showed you. And like I said, fixing some of these pixel cracks are non-trivial. So I'm just going to wait for the new APIs that come out in a later Leopard Seed to fix that.

So that's that app. So it actually is pretty easy to get most application content working in resolution independence mode. Let's look at another interesting case. Close this. There is a piece of existing Cocoa sample code that can run a full screen-- OpenGL animation. Let's run it at 1.0 so you can see what it looks like.

So it's a spinning globe, and you can change the light source and move the globe around a little bit. And it's just showing off some various OpenGL functionality and full screen mode. We're not concerned about the full screen mode. I just want to show you what happens when you run this at a higher scale factor. So we'll quit it, set the scale factor to 2, and run.

And you get that. Your animation did not actually increase in size, and it's in the wrong place in your window. This is the classic OpenGL problem. This tells me I need to change the coordinates I'm passing to the OpenGL APIs. And as it turns out, that's actually really simple.

Let's go to... Okay, so that's not the one I want. I have my OpenGL subclass. I have a reshape method that currently calls setViewportRect, which is a method on another class that I've got that wraps OpenGL. with-- sorry, let's look at this a little bit closely. I get the views bounds, and I pass that directly to setViewportRect.

takes that rectangle and calls GL viewport with that rectangle. Just so happens that GL viewport is one of the OpenGL APIs that expects pixel coordinates for the height and width. So I need to fix that in my call site. So what I'm going to do is I'm going to fetch-- GUY FALLEN: So I'm going to set my view's bounds.

And then I'm going to convert those bounds from my own view's coordinate system to the nil view, or to the Windows coordinate system. And the Windows coordinate system is inherently pixels. Sorry, and I'm actually just converting the size. And then I'm going to pass that converted rectangle to set viewport rect.

So this is pretty straightforward stuff. I mean, yes, it is some effort, but it's going to have some big payoff. Like we said, in the State of the Union, we want your apps to be resolution independent by 2008. So start working on it now. When you're revving your artwork, stick to the vector formats when you can.

If you have to use bitmaps, make sure you've got multiple resolutions of those bitmaps. For Carbon developers, we have the requirement that you get off of Quick Draw, get on to compositing mode, and we also have a few other Carbon sessions dealing with some of those subjects in detail. I don't list them all here.

Well, let's just start from the top. We've got an awesome document that talks in more detail about how your application can adopt resolution independence. It's on the attendee website. It goes into more detail on a lot of the subjects I only glossed over. The Cocoa Session 106 talks about more high DPI adoption stuff for Cocoa apps. The session immediately following this in this room is a Carbon session on, among other things, how to take advantage of resolution independence in a Carbon app.

And for Java, there's another session on performance tuning, your Java application, where they talk about resolution independence. We also have some carbon-based sessions for integrating with Cocoa, and particularly NS Cursor and NS Image. And those will be useful for Carbon apps that want to become resolution independent and take advantage of NS Image and NS Cursor. So make sure you check those out as well if you're a Carbon developer.