Hi everybody. Welcome to Session 111. NSImage in Snow Leopard. My name is Ken Ferry, I’m an engineer in the Cocoa Frameworks Group. So, today we’re going to talk about NSImage. Why is this? Because we’ve done a lot of work on it for Snow Leopard and we think you’ll like what we did. So we’d like to tell you. We tried to make it simpler, more consistent in terms of behavior. We’ve tried to improve its performance.

And we’ve also improved the integration with other frameworks on the system, in particular the integration with Quartz is much tighter and with CGImage in particular, if you’re aware of what that is, and if you’re not, you know, we’ll get to it. Okay. So the first part of the talk is going to be about the changes that we’ve made for 10.6.

In order to do that we’re really going to have, we’re going to need to talk about the Core Architecture of NSImage. So let’s get going on that. One thing I want you to keep in mind, usually when you think of an image, you’re going to think of something like this.

A photograph say, this is a photograph my father took. Pretty nice I think. However, for the purposes of this talk, we also want you to keep in mind images of this sort. These are, these are interface images that the user would not necessarily think of as images. Icons and such. The distinction might be the one, the images that you own, versus the images that your user owns. And it turns out that just these sorts of images have somewhat different issues associated with them.

So keep both in mind. All right. So, so if there’s one thing that an image is good for its drawing. So, so let’s just run down and, and see what happens when you ask an image to draw. What is the sequence of events, okay? So suppose you have something like this Pages icon, all right. The first thing to realize is that even as a user you might be aware that the Pages icon is available in multiple sizes, okay. There’s a 16 x 16, a 32 x 32, 48, all the way up to 512 x 512. Different bitmaps associated with it.

And that’s reflected in the API. So NSImage itself is actually a container for what we call NSImage Reps. And in this case, there’s going to be one Rep for each of these different size bitmaps associated, okay. And we’ll discuss NSImage Reps a little bit more later. Anyway, when you’re asked to draw what happens? Well, you have a location that you’re going to draw in. It’s called NSGraphics Context. There’s a rectangle in the Context.

We look at that rectangle, we select the best representation, and we draw it. Okay. We’ll fill this page, we’ll fill this picture out a little bit more later. But even that is enough to start talking about some differences. In 10.5, this was the name of the method that did the representation selection. In 10.6, it’s a little bit different, it takes more arguments.

So in, there was just one argument in 10.5, Device, this was a dictionary. Okay. It describes something like a window, it would say like “This is 8 bit and it’s RGB,” something like that. In 10.6, we take these three arguments, a Rect, a Context, and Hints, which is a dictionary. Let’s discuss what those are. All right.

Well, the Rect is the rectangle where the image is being asked to draw. The Context is this drawing location, it’s this object that encompasses it. And it includes information such as what are, what is the relationship of the units that the Rect is in to pixels. That’s information that’s in the context.

Okay. And that’s exactly the information that is required to pick the most appropriate size bitmap for that pages icon. Okay. Just for one example. There’s other information that might be in there, such as whether it’s gray. Maybe if you’re printing, or one bit if you’re faxing or something like that. So, and all of these things can play into which representation is most appropriate.

And the main thing to realize is just focus on this, oh, I didn’t tell you what the Hints was. The Hints are sort of extra information that you can provide, it’s optional. And it can also be used to override aspects that we would otherwise infer from the Context.

And, and I want to just point out, just ask you to look at this again because it’s going to keep coming up. This Rect, Context, Hints triple is going to be key in all the new APIs. Okay. Anyway that’s the information we need for Representation Selection. Just the dictionary before was not good enough, what would, so how did it work? Actually, it was using an internal method before in 10.5 for actual representation selection. Okay. So let’s fill out that picture a little bit more.

Something else that NSImage will do when it’s being asked to draw is it will cache. Okay, great. What’s the point? The point is that if you’re asking image to draw repeatedly to the same destination, then we can render this, this hopefully optimized object, which when you, which can be drawn down to context faster, repeatedly.

And the reason I wanted to bring this up is, you know, surprise, there are changes. In this, this caching business was, was a little bit confusing in 10.5 because there are a whole bunch of different methods that, that you can set on NSImage that had to do with caching. And unfortunately, they affected more than just performance. Some of them would affect even things such as where the image would end up drawing. How much of the screen it took up. And, and that’s not really how we want caching to work.

In 10.6, hopefully, it will just be a matter of something you can’t even tell if it’s there unless you’re using instruments to profile it. Or unless it’s so slow that you usually can tell. So, and this is part of what I mean by, by making, making it simpler, and behave more the way you expect it to.

So these are all methods that are defined on NSImage that can help control caching and they’re all deprecated. And their deprecated without replacement. And it’s not that functionality has been removed, it’s more like it’s just not necessary anymore because we are doing a better job of invalidating the cache on our own and we don’t need as much advice about how to do it.

Okay, so there’s still more changes in caching. One change that is mostly an implementation detail, but it leaks out in a few places, is that in 10.5 and earlier we would cache to an offscreen window. Okay. It sort of makes sense in that if you’re drawing to a window, maybe it makes sense for your cache to be some other window. Okay. Seems like it could make sense.

But, but there are a couple ways in which that’s not optimal. It’s kind of heavyweight. Windows turned out to be a scarce resource, sort of like file descriptors and as our apps get bigger and more complicated, the fact that you could run out of windows was a problem. And when you do run out of windows, it’s fairly dire.

The system does not do very well. So, so that’s hopefully not going to happen anymore. And also, it’s just, it’s just rather bigger than it needs to be. The CGImage which is a Quartz type is a very good representation for a Cache for us. So we’re going to do that.

Okay. So there was a class that represented the drawing that was backed by a window NSCachedImageRep. Okay. And you see it has a method WindowinRect, that’s the Rect within the window where the image is cached. So that whole thing has to be deprecated, which is fine. It still works, but we’re not going to use it internally and we would suggest that you not use it in your own apps, you know, going forward.

Okay. Oh and, and the difference between the images is that they’re, you’ll see, but they’re being used all the way throughout the, the AppKit internals now in hopefully an optimal way. And, and part of that means is, is that we’re going to be able to supply one of our most common feature requests for NSImage. Which is I want to take this NSImage and I want to get a CGImage out of it. For those that.

Okay. I was going to explain my, my, why you might want this. But apparently, you guys know. I’m still going to say. So why, why would you want this? Mostly because you’re interacting with other APIs that deal in CGImages. That’s the most common reason.

One of them for example is, but we’re also taking care to try to make more APIs take NSImages, as you’ll see in the talk that’s appropriate. CALayer, for example, in 10.6 actually can directly take an NSImage as its contents. You don’t need to get a CGImage for that.

But, that aside, we’re still making it easier to extract CGImages. And we’re also making it really easy to take a CGImage and represent it as an NSImage. That’s the second image with CGImage Size. That wasn’t superhard in Leopard. But that was certainly much easier to implement. But, nevertheless, it’s extremely easy now and its colorful.

However, usually when people want to get the CGImage they want to say, they want a method called CGImage on NSImage, and this one takes three parameters. So that’s not, that’s not quite what people want. So what. Why did we do that? Well the problem is that a CGImage is just not capable of capturing all the information that’s in an NSImage, okay. And so we need more, we need to know more to give you a CGImage. The CGImage is just a flat bitmap.

Okay. It’s not going to be able to represent those, for example, in this case of the size, these, this pages icon, it’s available in multiple sizes. It’s not going to be able to rep, able to represent all three at the same time. Another example is that an NSImage can be backed by a PDF. Okay. Or a window, the way we were showing before. And, and CGImage is not going to be able to do that to preserve all of that information. So what it can do instead is, I’ve been calling it a snapshot.

Think of, it’s just a flat bitmap, but, but, so I’m three-dimensional. Okay. And you can’t represent me all the visuals that I encompass as a single, flat 2D image. However, if you fix the camera angle, that you’re looking at me in, and fix a box that you’re trying to look at, then you can produce a 2D image, which represents me from that angle, and is an accurate reproduction. And that’s what the CGImage is going to be.

So when you pass a Rect and a Context, and this Hints thing, then you’re going to be able to get a CGImage, which if you draw it in the same Rect as you passed, in that context, would be the same as the NSImage drawn in that Rect, in that Context.

Okay. So it’s sort of like the image from an angle. Okay. So that’s the CGImage. There is one sort of unfortunate little tweak to that that I, which is that the Rect that we pass in is not actually a Rect, it’s a pointer to a Rect. Let’s see why that is.

Okay, suppose that the place, the Rect that you’re passing in is not pixel aligned in the Context, all right. So these lines are supposed to represent where pixel boundaries are. And this is the Rect you’re actually drawing. And suppose the NSImage is actually just backed by a PDF. Like I said, NSImage can be vector based. So if it’s just drawing, it’s just going to draw.

You can render a PDF into a rectangle that has nothing to do with pixels and that’s fine. However, if you want to get a CGImage for this, there’s going to be a bit of a problem, because we can’t give you something that’s, that’s some fractional number of pixels wide.

We would have to give you something that’s, what we can do is we can give you something that covers this larger rectangle in the Context that, that is pixel integral, okay. And we can fill our own drawing into that and hand you back a CGImage, which is an appropriate snapshot of that larger Rect. So, so how does this end up working? It’s, it sounds a little harder to explain I think than it is in practice.

But the deal is that if you pass this Rect to the method, and when it comes back, if you draw the CGImage in the Rect you get out, that’s the same as if you’d drawn the NSImage in the Rect that you passed in. Okay.

Whatever. It works okay in practice I think.

All right. So, okay, so that’s fine except you might say well, well that’s great.

I understand why you need all these parameters, but, but sometimes I don’t have a NSGraphicsContext. I need to make one of these CG, I need to get out one of these CGImages at a time when I’m not drawing. Do I expect you to make an NSGraphicsContext? No. These are acceptable parameters for all three arguments. Null, nil, nil. So that’s perfectly straightforward.

And it’s just that they each have a defined meaning. All right. What do they mean? Well Null is pretty much going to mean that you pass the most natural rectangle you could possibly draw the image in maybe, I don’t know, if you think there is such a thing.

Which is the origin 00 and then the size of the image. And then there’s this tweak business about pixel integral, which I’ll explain in a second.

And nil for context means we’re just going to assume it’s as if you had made a window on the main display in the most natural way.

That’s the kind of context it’s going to be. And, and the reason we’d even say things like main display is that it impacts things like color space, because the different displays can have different color spaces. And hints nil, that’s a little bit more obvious. I mean it means you didn’t provide any extra information.

And, and now if I go back and what’s this business about the pixel integral thing? It just means that the, that the CGImage that we’re going to pass back to you will never have any of that padding on the outside like I was just saying. Instead of getting the padding, what you might get instead is that there might be some anti-aliasing artifacts because we might have had to inflate the Rect a little bit from what you’ve passed us. And, anyway, regardless, that’s what that is. Okay. And, and I really did just want to emphasize this contract. Because it’s the one really important thing about the API, okay.

Drawing the CGImage in the Rect you get out is the same as drawing the NSImage in Rect you pass in. And you know nothing else about the CGImage. So for example, suppose you have an NSImage, which was natively just backed by a single CGImage. Okay. That really is, all of its data is encompassed in just a single CGImage.

Well then, that CGImage is always going to be a valid return from this method, no matter what Rect, no matter what context you pass in. Because it does entirely encompass all of the information of the NSImage. So in particular, you know, even if you pass a 3 x 3, you might get back a 27 x 27 CGImage. You can’t tell.

But this is what you can tell:, that it does capture the drawing of the NSImage. Okay. So all of this talk of snapshoting and caching and rendering PDFs in business. Is this starting to sound expensive? I don’t know, maybe. But, I’d like to say that it’s not. That it’s only going to be doing work in the cases where you would want the work done. And a lot of that is because I haven’t showed you all the API yet.

There’s also the same method has been added down at the Image Rect level, CGImageForProposedRect:context:hints. All right. And what is this going to do? Well to do that let, let’s go ahead and now talk about exactly what NSImageRep is. So NSImageRep, the really distinction in NSImageRep then is image, that NSImageRep is built for subclassing.

Very, very similar to NSView. Okay. In the AppKit, we have a bunch of different classes, and each of them represents the, the way it works is you just override the draw method. And you produce the drawing, whatever it is. So in the case of a BitmapImageRep, it’s going to draw the bitmap. In the case of a PDFImageRep, we override draw, we draw the PDF. In the case of NSCIImageRep, that’s like a CoreImage. We override Draw, and we’ll just draw that.

This CacheImageRep, I told you that that represents drawing back via a window and it’s deprecated. But, but it’s still there. So okay. Yeah and it’s, it’s there to be a subclass. If you think about NSView, most people have a subclass, have NSView, or at least they’re aware it’s pretty easy. The one method you really have to override is DrawRect, and in that method, you’re drawing in to the view’s bounds. That’s the location of you’re feeling, filling. NSImageRep is the same way. You, except you override draw, that’s the only method you really have to.

And you, the rectangular you’re filling is 00 Image Size. That’s it, okay. That’s all you really have to do. And it’s worth pointing out because NSImageRep doesn’t look like it’s that easy to subclass. It has all these other methods that you look at and you say I don’t know how to implement these. Things like colorSpaceName, bitsPerSample, pixelsWide, pixelsHigh. Okay, well, the thing to understand about those is that they’re used for representation selection.

Okay. These are best effort methods. When you say colorSpaceName, what you’re saying is this is a good color space for me. All right. Or with the pixelsWide it’s saying if you’re drawing into something that’s this many pixels wide, please choose me. Okay. Which means, and if you don’t, so that’s interesting both from a subclassing point of view, and also as a caller’s point of view, to realize that despite the names, these aren’t, you don’t, shouldn’t necessarily believe these things. Because they’re only best effort. Though in the case of something like bitmapImageRep, yeah, it’s definitely in our contract that pixelsWide has to be the actual number of pixels in the bitmap. Okay. And, yeah and they’re optional.

So you don’t have to do it. Later in the talk I’m going to show you an example of subclassing in NSImageRep. You’ll see how easy it is. Okay, but the whole point of this is I want to talk about this other new method we added, the CGImageForProposedRect:context:hints. Okay. And, and the deal is that you can override this method, optionally, to return a CGImage that you might have lying around. Okay. And the reason I said it that way is because it is optional to override.

Because if you don’t override it, what’s going to happen is that if somebody calls it, we’ll make a new context to draw in, we’ll call the one method that you did override “draw” to draw into that context, and then we’ll create the CGImage from that drawing. Which is fine. And unless you already have a CGImage just lying around, that’s likely to be the best implementation you could possibly produce for this. But in some cases you will have one lying around, and in particular in this bitmapImageRep does.

And it’s just going to return it without doing any work at all. Okay. So which leads us, let’s bitmapImageRep. So, what is this class? You might think this is easy, but my point’s going to be it’s a little harder than you might think. So a bitmap, a bitmap is a rectangular array of colors. Great. Okay. Just the same, way as a string is a linear array of characters.

And, and the reason I bring that up is because a lot of people are familiar with NSString well enough to know that, actually poking at that linear array of characters is not that easy. Because there are Unicode issues, there are StringCoding issues. Looking directly at the bytes is fraught with peril. And that’s true for in this bitmapImageRep as well. So let’s see how.

So, so when I said it’s a rectangular array of colors, well what, what’s a color? Often times people think of a color as being specified by say four parameters. Red, 0.5, green 0.2, blue 1.0, alpha 1.0. Okay, well that’s not actually a color, that’s a sequence of numbers. The way you go from sequence of numbers to abstract color is a colorspace. And, similarly is the way you go from abstract sequence of bytes in the case of strings to a character, what’s that, that’s a string in coding.

And if you don’t know the colorspace, you don’t know the color. And what I have on the board here is two different colorspaces in that same sequence of numbers. How it’s drawn with respect to both of them. And the most interesting thing to see is it’s very different. So though one of these colorspaces is really weird, that’s why it’s so different.

But, nevertheless. Okay. And, and that’s not, that’s actually just the beginning of the issues. Because the other problem is that when you’re talking about a bitmap, it’s really just this huge array of bytes, right? But what do they mean? How are they laid out? Well, there’s a bunch of parameters that are all pretty simple, but they’re, but there are a lot of them that describe the layout of those bytes, okay. You’ve got a certain number of pixels wide. Each of those pixels itself occupies a certain number of bytes. All right.

There might be some padding out at the end of the row for performance reasons, so the bytes per row might be different than the bytes per pixel times the pixels wide. Okay fine. It might be an 8-bit or a 16-bit image. It might be an RGBA order, or, or ARGB order. Might be pre-multiplied, any of a number of things. Well not any number, this many things. But, and this is where this method comes from that’s often put on the board because it’s the longest method in AppKit. For initializing in this bitmapImageRep.

And really when I say, I want to say about this is two things. First of all, calling it is not that hard because even thought there are a lot of parameters, you can just read them off of the format, provided you know what the format is, which you need to if you’re dealing with the bytes directly. And, but secondly, that there are a lot of parameters.

And if you have an arbitrary bitmapImageRep on your hands, your hands that you created, that you didn’t create with this method, you probably don’t know how to interpret the bytes. It’s very similar to if you did just go take that string and start poking at its bytes without knowing anything about the stringing coding. So don’t do that. All right. Yes, I almost forgot.

We were going to talk about CGImages, that, that’s why we are here. Or that’s why we were talking about this bitmapImageRep. So, so they’re very, very similar objects. CGImage is, is nearly the same as NSImage bitmapImageRep. This is already reflected in the API for 10.5. There was a CGImage method. No need for the other parameters, because they’re just too similar. And there was already, and there was a method, and it was CGImage. Okay, great. 10.6 we don’t have to change the API, but we did change the implementation.

Now, the CGImage is the native representation within the bitmapImageRep. It used to be that bitmapImageRep held the data, and it would sort of wrap it up as a CGImage if necessary. Now it doesn’t do that. It stores just the CGImage, which is important for performance, for performance reasons. Okay. Yeah, so, so what are the implications of this? The implications are that from start to finish, CGImages are always preserved.

Everything flows through the pipeline without messing with it. Which is new. It used to be in 10.5 for example, if you took a JPEG and dragged it into text edit, okay, and then hit Command, and then saved the text edit document as a PDF, the PDF would be roughly the same size as if you took the image, the JPEG image and uncompressed it.

Because the, the internal NSImage stuff had to uncompress the bytes and put it back down as CGImage, whatever, like that, okay. Now, because the CGImage is preserved from start to finish, you’ll actually find the PDF is the size of the original JPEG data because the CGImage, because we ended up using the exact same one, had some extra information associated with it.

And, and that’s exactly the kind of reason we want to hold on to the CGImage. Because sometimes they have caches, sometimes they have extra information, I mean, it may have already uploaded itself to the GPU. You don’t want to be scrapping CGImages and recreating them if you don’t have to.

And, oh yes, and, and anyway and all of this business with the caches and the snapshots and all that business, it just, they’re all the same thing. If we’re making a cache, and we already have an ImageRep which is natively backed by a CGImage, that is the cache.

There’s no copying of memory there. Okay. Which is why it’s so efficient. You’re only going to see actual creation of memory if you have something like a PDF that was not natively CGImage backed, and in that case you want it because it’s going to render a lot faster. Okay. So I’ve been talking about lots of ways in which we are now better aligned to the graphics arts architecture with Mac OS X.

The one way in which we do differ from Quartz still is that our text are mutable, and the Quartz types are, or the Cocoa types, excuse me, are mutable, and the Quartz types are not. So the way that works is in Quartz typically they separate the notion of a destination from drawing from the notion of a source for drawing.

The sources are always immutable, the destinations have to be mutable. And, and that’s not true at the Cocoa level. NSImage and NSImage bitmapImageRep both can be modified. You can draw into either one. Okay, that’s not really a problem for the implementation. But it does have interesting forms considerations.

It basically means that the Cocoa level types are all copy on write in 10.6. If you have an NSImageRep and you copy it, you’ll get a new pointer, but both the new and the old copy are going to be referring to the same immutable CGImage. So copying is extremely cheap. You’re just copying surface structures.

But, on the other hand, then if you mutate one of those two copies of the bitmapImageRep, at that point, it will have to be, it’s backing so it will have to be copied away from the CGImage so that you can, so you can modify it. So it’s, that’s called copy on write. And that’s good to know. Furthermore, even though you can mutate the active level object, it is, you need to be a little bit careful.

Uncontrolled mutation can have a lot of undesirable effects. Remote, it affects remote parts of your app. Okay. So if you have an image that you’re using over here and you call set size on it, it’s part of drawing, well you just affected the image used in this totally different window, because they happen to be the same image.

And that, that’s bad. Okay. You don’t want to do that. It also has implications for thread safety. Here’s another thing we’re doing for 10.6, we’re publishing, for the first time, carefully what exactly the threading contract is for NSImage and also its related classes.

And it’s the same as NSDictionary. And it’s mutable dictionary. Which is this, that you can use it from an arbitrary number of threads as much as you want, as long as you aren’t mutating it.

If you are mutating the image, or in any of the other related objects, you need to know that it’s not being used by any other thread at the time of mutation. Okay? And what that largely means, and mutations are things like, like I said, drawing into it, lockFocus, that’s mutation. Setting size, that’s mutation. Okay. So it just means don’t mutate it. When, the way we expect mutation to work is it’s part of the initial set up. You initialize your image.

You call some setters or whatever, you mess with it. And then before, and then at that point you sort of vend it out to the app at large and that’s fine. You don’t want to be taking images that somebody passes to you and modifying them because you don’t know what’s going on at that point.

Rather what you would do in that case is you would copy the image that’s given to you, which again is very cheap by itself, and then mutate the copy. That’s going to create the new memory and such. And then you would vend out that, that mutated copy. All right, that, that’s how we would suggest you do this sort of thing.

Okay. That’s almost all I have to say about this first part of this talk about the changes for 10.6. However, I wanted to go through now and talk about some methods where I often see code that has problems. So, so these are some of the more misunderstood methods relating to NSImage. So the first one we’re going to talk about is setFlipped. I’m very happy about what I have to say about setFlipped. Which is that you can completely stop using it, it’s deprecated.

Okay. Some people have had problems. So what is, what does it mean for an image to be flipped? It means that 00 is in the top left corner of the image. Okay. However, the way it was, and everything follows from that. And I’m not, I’m not going to explain too much about this, because it is deprecated.

But, typically, the reason why anybody was ever calling it is because they wanted to get their image drawn right side up in a view which was flipped, flipped views are not deprecated. And what we’re doing about that is that we’ve provided a new drawing they put on this image that takes respectFlipped as one of its parameters.

And if you press Yes for this, the image will adjust itself to the context in which it’s drawing and it will appear right side up. Okay. And a little bit more on setFlipped, just to say that two years ago we gave a talk called “Cocoa Drawing Techniques,” I think. And we went through in, in some detail what exactly setFlipped means for real, as opposed to what people wish it meant.

And So you can take a look at that particularly if you still have to support 10.5. But on 10.6, use this instead, okay. And the AppKit release notes also discussed this. Okay. If we’re introducing a new drawing method, that’s a lot of drawing methods now. So probably, we should deprecate some, right.

Yeah. So let’s deprecate 6.

I’m, I’m just kidding. Well, I’m not kidding with the deprecated, but I’m kidding about what.

I’m kidding about they’re deprecated. They’re, they’re deprecated because they’re fairly misunderstood. Again, I have a lot to get through in this talk and I’m not going to discuss exactly what they do.

But take a look at the AppKit release notes for more detail. Generally, for those of you who think you’re calling composite because it was doing something you wanted, probably it was another one of this Flip It disk issues. And if you use the new drawing method, it’ll probably already fix your issues. But, but the AppKit release just will explain exactly what these guys did.

But for now, I, I can mostly just say please stop using them. Okay. So that’s what I have to say about flippedness. Let’s talk about lockFocus. LockFocus is a very convenient method. But I think that people often don’t understand exactly what it does. It’s a way of drawing into the image, or saying images are mutable.

And this is one of the primary ways. So what happens when you call lockFocus with this? The image will create a new context, which is its own size. The size of the image. Okay. Like I mean the return of the size method. And it will draw itself into the context.

Then control returns to the caller, where the caller can do something else, like fill an Orange Rect. Because they have no taste. And they don’t like the pages I copied. And, and then when you call unlockFocus what happens is that drawing is captured toward imageRepresentation, and it replaces the representations that were already there in the image. Okay. Is that what you thought it did? Maybe. So if you think about it that’s what it pretty well has to do.

But, regardless, so there’s some changes here. So first of all, this goes back to what I was saying about Windows before. All right. So one of the ways in which the implementation detail that we were using Windows leaked out is that in the middle of lockFocus, we were using an offscreen window and, and a view in that window. Which meant if you called NSViewfocusView, after lockFocus, you’d actually get something. All right. And I think mostly our older developers are aware of this. The, for the new developers, I don’t know why they’d expect anything to be here.

But in 10.6, it’s not. Okay. So just be careful. So, all right, let’s just, let’s go on from that. I’ll come back here to that. Though to, to replace this, this use, NSViewfocusView is flipped. You can call and state NSGraphics is flipped. That’s great because you didn’t have a graphics context. All right.

So the, so what are our conclusions here? So first of all, you should audit your code. Take a look between lockFocus and unlockFocus. If you see anything that calls NSViewfocusView, you need to change that code. All right. And that’s the changes for 10.6. This is not the changes for 10.6. But to understand that NSImagelockFocus is destructive. It’s going, it’s throwing out whatever representations you already had of your image. And it’s making a new one. All right. Which is often fine.

I mean it’s a nice way to make just sort of a snapshot. A new image. But, you know, be aware, it, it’s not a good way for example if you want to make an NSImage bitmapRep from an image. Which we’re going to talk about later. But, don’t call a lockFocus followed by an NSFocusviewRep because you just corrupted your initial image. And the other thing to realize is that it is still set, it’s good for drawing to the screen.

Okay. If you are going to take your image and then save it out to disk and upload it to a web server, you probably don’t want to be using lockFocus because you’re capturing aspects of the, the computer on which you ran it. Because it’s going to have things like the color space of the main display. And that’s probably, you know, for when you’re persisting images, you don’t really want it to depend on where you ran the thing, even if it’s only a little bit.

Okay. So now, let’s talk about Bitmap Data. You’re probably going to know what I’m going to say about Bitmap Data, because I already talked about it a little bit. Okay. Which is this: That with the changes for 10.6, I told you we’re doing a lot less processing of data, right?

Like we’re pretty much just, somebody hands us a CGImage, it falls all the way through the pipeline, it’s great. Well that means, however, that if you were calling Bitmap Data on NSBitmapPaymentDirect, and poking at the bytes, you may have a problem, unless you use this method actually to specify the pixel format.

Because it may have changed on you. And it was never a part of our contract, what that pixel format was, like I said, it’s just like poking at the bytes of a string. But, nevertheless, we see some people doing it. So this is something you also want to audit your code for.

So look for, look for uses of bitmap, of a Bitmap Data, and make sure they’re safe. Okay. And the way to make it safe, by the way, if you do want, if you do want to mess with the pixel image, the actual bytes, what you need to is create the bitmap in a format you understand, okay and draw in it. And then you can get the bytes from that BitmapImageRep that you actually understand the format of.

And then you can say, okay, well this is the byte that represents the alpha at this spot. That’s something you can do. Okay. Last, I’m talking about TIFF Representation. This is a little bit less, less of a problem than the others. This is mostly just, but it’s a little frustrating. Often, this is a good use for TIFF Representation. Okay. When you’re going to take your image and persist it to disk as a TIFF. Or put it on the pace board as a TIFF, or something like that, okay.

That’s fine. When you’re, when you’re taking your image and making it persistent. This, however, NSbitmapimageRepalloc init with data, image TIFF Representation, this is a little frustrating. Because making that TIFF Representation is, is much more expensive than what you really wanted to do here. Think of the Safari icon okay. And supposing you’re drawing, is a 32 x 32. And you want to, and you’re trying to get a bitmap for some reason. Again, I’m going to argue that you should, might not want a bitmap.

But, but you’re doing it anyway. And, and when you call TIFF Representation, it’s taking, first of all its unpacking all of the representations, including the 512 x 512 representation which otherwise might have been entirely unused and we didn’t even read it off the disk. Well we probably read it off the disk actually. But. We probably didn’t unpack it. It was probably just a sort of a JPEG.

And, and now you’re flattening it all out to a TIFF. And then you’re immediately taking that back up into an NSImageRep. Furthermore, the NSImageRep can’t actually read all five of those representations that you might have had in your image. So which one is it going to get? I don’t know, the biggest one is actually the answer. But, but that’s not great.

So, so that’s not a great way to get a bitmap. So how should you make a bitmap? Well, if you really need it, you should draw in a bitmap the way I showed you. Or you can use these CGImage APIs, because often times what people wanted in a bitmap, or was because they were interacting with some other API that took CGImages.

But if you are doing something like OpenGL for example, where you need the raw byte to upload, you’re probably going to want to take this, this version where you, you draw on a bitmap. Oh, oh, and I meant to say, since Bitmap Data, these are, it doesn’t perform any worse at all. Okay. Bitmap Data itself does not directly access the bytes of a bitmap.

Actually, this might be my next slide. Yes. Okay. So a few of the thoughts on the Bitmap Data itself. Okay. As I was just saying, when you ask for the Bitmap Data from a NSImage Bitmap Data Rep, sometimes people do that because they think it’s really fast. Well it is actually a copy. If you think about the way that I told you about the copy on write business, when you call a Bitmap Data on the BitmapImageRep, that’s something that you’re allowed to mutate.

And it will affect the bitmap itself. And which means we had to make a copy of the byte at that point. Which may not be what you really want to do. And, and furthermore, if you think about exactly what drawing is, drawing is also copying the bytes except it’s paying attention to the bitmap formats.

So if you specified a format that actually matched what the source was, it’s going to be extremely fast when you draw. Especially if you don’t end up looking at the whole thing, which is often the case. And, and furthermore, our whole system is just optimized around drawing. We need that to be fast because we draw constantly. So, so looking at the device is not going to be faster than drawing actually in this case at least.

Besides that I do want to say consider CoreImage, we’re in Quartz and Cocoa both are just really not set up around giving you direct access to pixels. Something that we’re fast at. CoreImage is really nice. It has the right level of abstraction to let you play with the pixels in an efficient way. And we’re going to talk about a little tiny bit more about other frameworks like that at the end of the talk.

But yeah, if you do really need to do need the thing to draw. Okay. That is all of what I have to say about these, well, we’ll see it come up. But that’s most of what I have to say about the 10.6 changes. Okay. So now, we’re going to switch over and talk about performance, which is largely going to be in demos.

But, first, I just want to tell you what the principal, there’s one overriding concern, which is this. If you want to perform well with NSImage, than when you have the same drawing it should be represented by the same image. And to tell you what that means I am going to go ahead and switch over to the demo machine. Okay, great.

So here’s the application that I wrote for, for this talk. And what you see in this window is this photograph that my father took of the sunset in upstate New York. And, and as I drag this around, there’s a text field. What the text field is telling you is how long the thing took to draw.

Okay. Now, what did I mean when I said same drawing, same image? Well, what I mean is that from our point of view right now, you can’t tell if every time that image drew, if it was using the same NSImage instance, or a different instance. And performance reasons, if, you want it to be the same instance. It happens that as is currently set up in this demo up, it’s using a different instance every time.

And it’s taking 21 milliseconds to draw. About. And if I took, click this checkbox, it’s going to go ahead and use the same image each time. And it drops to something like 2.0, even a little less, okay. So it’s something like a, 10x improvement in speed if you can manage to use the same image for your drawing instead of different images. And this is not even due to the applicant level of caching I was talking about before. This is more things like, um, like color matching. The color matching only has to be done once.

Or if you’re uploading bytes and GPU that only has to be done once. This, this is the kind of reason why we don’t want people accessing bytes directly because we wouldn’t be able to tell if those, those sorts of caches were still valid. Okay. So that’s that. So let’s, let’s move on. That is the central concern and the rest of the examples are pretty much just going to be expanding on that topic.

All right. So here’s the Safari icon drawing and it’s taking 80 milliseconds to draw. Which based on the last slides seems like it’s slow. But, but you wouldn’t necessarily know that from scratch. I mean if you see just in a trace that you see a drawing taking a long time, how are you going to know that it’s, I mean sometimes a drawing takes a long time, right? How are you going to know that it’s a problem? Or that it could be faster if you’re doing something better.

And that’s what I’d like to show you. So this is an instruments template I had set up. It contains three instruments. Two of them are standard. One of them is custom. Okay. The standard instruments are the File ActivityVviewer, which is going to fire every time I touch the file system.

The Shark-like time profiler, which lets me know what my apps doing. And this is the, this is the one I really want to show you. Which is a custom detrace instrument. And it has to do with this function here, which is new in 10.6. And it’s NewBitmapBackingStore, okay. This is a private function. It’s internal to the AppKit.

It’s our funnel point for making new bitmap backing stores for images. And the point is that every time you see us do that, it means that we used a new image. Which means that you’re not getting benefits of using the same image. So by seeing where this fires, you can, not that word, you can, you can see when there’s a performance problem maybe in a little bit more obvious way. So let me just show you how I made this instrument.

So it’s a nice place to put break points. It’s just a function. So this is essentially just, it’s just de-trace instrument, okay. You could say it hits the framework AppKit. The function and this NewBitmapBackingStore just pasted it in. That’s all it would take to do this yourself. All right. Now I’m going to go ahead and delete it just because I don’t want any surprises when I set up the, the graphing back here. But nevertheless, that’s all it is. Okay. And let’s get this thing running here.

Our profiler.

It’s going to crash. Looks like it wants to crash.

Well I’m going to head it off. Let’s try that again.

Well this is fancy.

I really don’t know what that could be.

Okay. Well.

Well while that thinks about launching. Maybe I will go on and tell you what it’s going to find. In this particular case, what it was going to find is that we’re hitting the file system. And I really hope I can get that back up later. It’s going to find that every single time we do this guy, it’s actually hitting the file system, all right.

And to figure out why that is, we’re going to have to look at the code. Let’s do that. So this is the view, this drawing. Here’s the drawRecMethod. All right. Basically what I was doing is it’s calling its method that we defined above. ExtendedImageNamed, compass and bundle. And I’m passing the safari bundle. And then it’s just drawing it. Okay. So the problem is going to be in this method, ExtendedImageNamed. Which is very similar to the AppKit method, NSImageNamed.

Which does a couple different things, but what’s relevant for this one here is that if you were to pass ImageNamed, a name like Compass, what it would do is it would go look in your main, your applications main bundle in the resources directory and see if there was an image file named something like Compass.TIFF, or Compass.PDF. And if there was, it would load the thing up as an NSImage and return it to you. So just, it’s just a nice little convenience function. The issue though, the reason, sometimes people extend this because it’s only going to search the main bundle of the application.

The application itself. So if you’re writing a framework, or if you’re writing a plug in, it’s not going to search your framework, it’s going to search the executable, which doesn’t do a lot of good. So, this person, this supposedly hypothetical person, who’s actually me, wrote a method called extendedImageNamed, and unfortunately has some sort of problem. Okay. So let’s see what it does.

So it, it gets at your, it has the bundle for the URL for the image, that’s fine. Calls it by referencing URL. Here I actually want to take a quick tangent and actually ask you people. There are two very similar methods on NSImage, okay. InitByContents of URL, InitByReferencingURL.

Quick show of hands, how many people know the difference? Okay. A smattering. Not too bad, it could be worse. Anyway. This may or may not have some to do with the fact that I answered the question on the mailing list about a week ago. But, in any case, and the rest of you aren’t in the mailing list. Shame. Okay, the what, the difference is that if you’re referencing a URL on disk, the AppKit will assume that the image is going to stay where it originally was. It is not going to be modified. All right.

So the in memory representation is just a cache for the representation on disk. And this method right here, InitByReferencingURL, actually does no file system IO whatsoever. It’s lazy. The, and it might, the contents of URL, on the other hand, will suck the data in immediately and disassociate from the file. So one of them is sort of for user owned images, and one of them is for developer owned images. And image names, because it’s searching out of your bundle, your main bundle, it’s going to use the one that references the image.

And be nice and lazy because I mean if the users are poking around inside of your bundle, they probably deserve what they get. So, all right. So what is it doing? So it’s getting this thing. Well actually, sorry, first, it’s checking if ImageNamed is already, returns it directly. Then it’s creating the image, and then it’s setting it as an, and it’s giving it a name. And that’s it, all right.

And the problem’s going to be this: that just giving an image a name does not keep it alive. So the code, as written is reading a new image off disk every single time it draws. Which is probably not what you want to do. So there are a couple different solutions for this. I’m going to show you the more straightforward, you might think, solution first. But I’m actually going to recommend you not do it. That would be the cache.

Okay. We’ll make this NSCacheObject up here in Initialize. And then, and then in addition to just setting your name, we’ll also just toss the thing in the cache. And this is kind of nice, and because what it’s going to mean is, means is the image will not go away right away unless the system’s under memory pressure or something.

And then it might flush the cache and all those images can be destroyed. And that makes a lot of sense. And that’s going to make things nice and fast. However, the reason I don’t recommend you do it is because if you do turn that on, and let’s see how it goes.

So we went from 88 milliseconds to, you know, one or two. So it’s a lot faster and can avoid hitting the disk. But the, the reason I don’t, don’t recommend you do that necessarily is because then if you close the window, well the image data will still be there. It wouldn’t get destroyed. And that seems kind of, kind of bad. So does this permanent cache thing.

So something else you could do is do something more like what NSImageView does. Where rather than calling your ImageNameMethod every single time, you could take the image and put it in an iBar, and then just draw from the iBar each time. And then when the view is destroyed it’ll, you know, clear it’s iBar and everything will get torn down nicely. So that’s a, that’s another option.

And if I click this check box, that’s what it’s going to do. And we’re going to see just exactly its performance, maybe a little bit tiny less, but we don’t have to consult the name table. Okay that’s that. I’m actually going to try instruments again here, because I really would like that to work. I wonder if I want to restart the machine or something. Oh, that’s really a pain. That’s a fun demo too. Oh fine, I’ll reboot it.

Do I have time for this? Well I think while I have it reboot, I’m going to go ahead and tell you a little bit about what’s going on in that case, the case that I was going to show you.

off. Okay. Hard reboot.

I’m not going to take the data on stage.

All right. I don’t think I disabled automatic log in, but oh well.

The, all right. So what I’d like to say, the, the next case I’ll end up just demonstrating it at the very end. But it’s this, okay. We have a particular image, which happens to be backed by a PDF. This is derived from a real, a real problem in an AppLab by the way. And it’s being drawn on the screen twice, in two different image views, one here and small, one next to it and large.

Okay. And what we find is we look at the drawing performance of that and we notice that every single time we redraw the screen we see that bitmap buffer function firing. It’s making new images. We’re not reusing images. And the issue in that case is that, well, just think about what happens. So the PDF draws to the screen in the small image view. It does a check, it checks. Is my cache appropriate? No. Okay, it makes a cache. Now it draws in the larger version, actually I’m just going to go ahead and, oh good.

Then it draws in the larger version. And, and it checks again. Is my cache appropriate? Well, no, the cache is appropriate to the smaller version of the drawing, so the, um so the cache is invalid, it’s thrown out and it makes a new cache. All right. And then when you come around to the next drawing pass they both draw again, and you’re just never getting your cache. So what’s the solution on this? The solution is that (um, Okay I’m distracted. Let’s get this going again.) The solution is to use a copy in both cases. Oh, actually sorry. Please work. Yeah. Okay.

All right, let’s at least see that going. All right, so these are these two images I was saying. One large, one small, taking 33 second, milliseconds to draw the big image. Whenever I resize this window, you can see this thing firing up there in instruments right. You see those black blips up there? That means it’s creating those new, those new images like I said. And the problem is these caches.

So what you can do instead is just, instead of sending image directly on to both image views, set a copy of the image on to both image views. Again, everything’s copied on the right. It won’t copy the backing PDF data, but it will separate the caches. So that now when you resize, everything’s nice and smooth. Okay. It’s taking .19 milliseconds to draw the big one instead of .33. And, and you don’t see those blips up there. So, so the dual points are, different drawings.

Now I’ve already told you, the same drawing, you want to use the same image. You have different drawing, it can be useful to be using different images. Okay. Different NSImages. And so that seems nice. Okay. And the other thing to note is how, is how you scroll the profiling ones. Because it pointed out this problem to you. Okay, let’s look at a slightly harder problem. Let’s look at this, this view here. This is drawing a track that’s actually derived from instruments. Instruments had a performance problem at some point, where this guy was not drawing very fast.

And the deal is that when you, when you resize this, see this is, this is basically a PDF image and it was stretching to fill that rectangle where, there are two images there. There’s one in the back, and stretching to fill that rectangle and one in the front, it’s stretching a PDF to fill that.

And, and that’s, that’s kind of slow. It’s taking something like 104 milliseconds to do that drawing. And, and how are we supposed to deal with this, okay? I mean we can’t just, how are we, how are we supposed to manage the set up so we can hit our cache? Well, what we can do is instead of stretching the image, we can tile it over the region.

If the image is say five points wide, we can just replicate copies of the image to fill up that whole region of the track. And if I do that, I click the file track check box. Much, much, much faster, down to .6 milliseconds. And again, we don’t see this firing up here. It’s supposed to track along the side.

We don’t see it firing. So that’s a nice fix. And then I just want to find out how easy that is, okay. So this is the version here that draws stretching. It just calls draw for the 2 different images. Here’s the version that draws with tiling. And it’s, it’s just as easy. It just uses this function we provide, NSDrawer 3 part image. What that takes is a left cap, something like a button end cap and it might have some round corners.

A write cap, and then the, the stuff in the center that gets capped. All right. And the benefit of tiling in this case is that it means the cache is going to apply. Okay. So, so that’s our track. Let’s look at a harder case yet. All right. This is our same, very, very expensive PDF. But now it seems like we’re in trouble. The issue is, look what happens when I try t do resize. And is it following my cursor? No. It’s way too slow. This is very bad responsiveness.

Okay. And, and the problem here is that the image is being, is being drawn in such a way that there’s really just no way the cache is going to apply. This is a PDF and if you were to use the cache, it would end up pixilated right? Well, maybe that’s a good thing.

In this case, it seems like the responsiveness is more important than the exact drawing, at least during live resize. So again, we set it up so that it does actually use its cache no matter what and it draws pixilated during lab resize. Yes, we can. What a surprise. So I do that and everything gets much faster. We see before here in instruments it was creating these, you know, weren’t good in cache.

Now what it’s doing is we hit, we missed the cache right at the beginning when it renders our one image that we’re going to use during lab resize. And when we let go, that’s it, rendering correctly at the very end. You probably can’t see from the audience, but it’s pixilated during the live resize, a little tiny bit. So see what that looks like.

Okay. What we do, the way we’re doing that is we’re passing off finally this Hints parameter to NSImage that’s in the new drawing method. Okay. And, and the way that we’re doing this is that one of the things in the Hints parameter is this NS AffineTransform matrix, which tells, which specifies what is the relationship of the units of the rectangle to actual pixels. That’s information that’s in the context.

Okay. And I don’t want to go over the mathematics, but I want you to know that it’s here. So that if you are in a situation like that, this is something that you can use the Hints for. You can, you can advise the image, I want you to do representation selection and the caching as if what we’re doing here is as if the image was drawing to a 512 x 512 pixel location.

And that’s a nice optimization. Okay. Just two more of these guys. So here’s, this one’s about BitmapImageReps. And it’s about Hint testing. Okay. So I’ve got this thing set up so that when I click on a portion of this image, that has locate pixels, it, it draws this thing in the back.

And if I click where there are no locate pixels, it does not. Okay. And if you look back there, we see this, this thing firing to tell us that we’re making new images and stuff. All right. So let’s take a look at what that looks like, okay. What’s going on is that we are grabbing, we have a bitmap, somehow, I hope we generated it in a good way because it’s all due for many times, it’s not so good ways. And we’re grabbing the Bitmap Data.

We’re trying to offset to the point where, where the mouse was clicked, and we’re checking if that point has alpha or not. Okay. And the issue is, like I was saying, that, that calling Bitmap Data that’s a copy. And it’s making, it’s forcing the image to, to copy out to a new Bitmap buffer.

Because the underlying core types are mutable. And so this is not so fast. People often, like I said, I mean they think calling Bitmap Data is going to be faster than drawing and it’s not. So, what you should do instead is draw the, draw the image into a 1 x 1 contents.

Think about what’s the difference is. Suppose you made just a one by one pixel, just a single pixel drawing context, and drew your image such that to the point you’re interested in hit that 1 pixel. And that pixel was in a format you already understood. At that point, you could just check if it had alpha.

And that’s much easier and it’s much faster. Because it’s just copying a single, say four bytes in one pixel. Instead of copying out the entire backing store of the image. So it’s faster, and it’s easier. The, now we don’t actually have to do that drawing because there’s a new method in 10.6 for hit testing. So HiTTestRect with ImageDestinationRect, you know, content, bunch of arguments. You can look at it, I don’t have to explain it.

It’s easy enough to use. It’s actually much easier to use. If you are thinking, suppose that you are taking that, that Bitmap Data approach okay, and you want to know if a whole rectangle. Somebody, the user has a selection rectangle like in finder, and you wanted to know if it should, if it hit an icon to select it. Okay. If you were to take the Bitmap Data approach, you’d have to loop through all those bytes and check them, right.

If instead you take this drawing approach, it’s much simpler. You can just draw the region where the rectangle was, to a single pixel, and if there’s any capacity there at all then you’ll see something not opaque in that pixel. Does that make sense? Then that’s what this will do.

So this method allows you to HitTestRect, it’s not just points. Okay. One more. I want to talk about badging. So suppose I have this text at an icon, and I want to put my fancy 10.6 badge down at the bottom right corner of it, because it actually took me a long time to make that thing and I need to get more use out of it. So there are a number of different approaches you could take to doing this. One of them would be to call lockFocus on the image. And draw it.

I think that’s an approach a lot of people would take. However, in this case, the way I happen to get this image it would produce this drawing. Why? Well it’s like I was telling you. That NSImage, the lockFocus is destructive to the original image. That the size of the context it creates is going to be the size of the image, to determine from the size method. And in this case, it happens that we got this image from NSWorkSpace and the natural size of the image was 32 x 32. But it was able to draw nicely all the way up to 512 x 512.

And that information was lost when we called lockFocus on it. So what can we do instead? And how can we get these two together? Well there are a number of possibilities. One nice possibility would be if we can just use two different views. Okay. To draw one image and then the other image on top of it. That’d be great, but it may not really be feasible in the API, however you’re trying to use it.

It could be that you’re calling something else and you just really need to pass up one image. And what we could do instead is we could use an NSImage subfiles. I was claiming that it’s real easy to do. So let’s just see it. Okay. And that’s what it looks like if we do that.

We’ll just resize and it looks real nice. So suppose we do that. Let’s just, the point of this demo is to see how easy this is to make this subclass, all right. We had one method and it was base image, overlay image, overlay frame, takes the background image in the foreground image. If we switch the implementation, here’s the init method it just stores everything, the dealloc method releases it. And the draw method it just draws the background.

And it draws the overlay. And that’s the entire class. So you can see, it was just as easy to make this custom, now you’ve got an image that can do any kind of drawing you want. And it was really easy. And it’s going to perform the same as all the rest of them. So there you go. Now here’s, here’s a caveat. I was hiding this a little bit before. Not really intentionally.

But, as we see this thing growing, look at that. It’s hitting our, we’re creating new, new bitmap backing source. Why is that? Well, by default, the image is configured to try to cache itself. And we didn’t override that method that returns the CGImage from the ImageRep. And in fact, it would be pretty hard to override that method, because we cannot take these two bitmaps and represent them easily as a single bitmap. For any arbitrary size, right.

Like the thing is being drawn at several different sizes. So the solution in this case is, is just that the cache is not actually doing us a lot of good. It, because both of those images are themselves cached, right? We’re seeing something like a 10x speed-up for the cache. That’s okay. We can just draw both images. So I’ll just turn off caching.

The method as NSImage, set cache mode, and NSImage cache network. And if we do that, we’re going to find that it draws just as fast. Oh, I have to regenerate the thing, there we go. It draws just as fast, but we, we stop seeing these, we’re not seeing any more generation of, of new images. Okay. Terrific, that’s all, that’s everything I have in the demo.

So hope that was useful.

Okay. Since all of that was in demo, and nothing was on the screen, I do want to just briefly go through and say exactly what we got. So what we’re, what was in this? The, the real basic lesson. The one thing you want to take away is if you have the same drawing, you want to see it being represented by the same NSImageInstance if at all possible.

Okay. And sort of a, a tweak to that then is if you have different drawing, maybe you want different images that might be faster. I want you to realize that you can profile this. And in particular, in 10.6, you can use this NSNewBitmapBackingStore function. You can make a DTrace instrument like I did.

Or you can use breakpoints or whatever. It’s a really nice way to investigate the performance of your image, imaging. Then there’s the cache is important. Those were all real performance problems that I was debugging over there. They were mostly derived from, from real apps,, internally to AppKit. And lastly that, that though you should use the cache, in some circumstances, you may want to disable it.

The primary circumstances would be, if you know your image is, is short lived, it’s transient, there’s no point in caching it, it’s not going to draw again. Or if you know that there’s already caching going on, either above or below the level in which it would be happening here.

And then there were also just some sort of tips and tricks, because I felt I could squeeze them in. The main table does not retain images. Oh, but one thing I did not say, when you do call image named, and it reads the thing off the disk for you, it’s going to perform caching very similar to the NSCache you saw there.

So when you use our image name, if you didn’t use the extended image named, you would not see that particular performance problem. Okay. We have hit this API. We have this three-part image showing API; we also have a nine part version, it takes four corners, four edges and a center part, and tiles them all out so that it sort of acts like a box. We have this initByReferencing URL method.

I’m glad that some of you knew what it meant, but it does go IO. It’s very lazy. And then lastly, NSImage of subclassing is just really easy. And if it happens to fit your needs then go ahead and do it. You can do like sort of shaders and things this way too.

You can have fancy images that, that draw you know, do some complex rendering right at draw time. Okay. The last part of my talk is going to be very breezy. We’re just going to run down all of the graphics architecture on Mac OS X. And how and what it all does. Because there are a lot of them.

And, and most of the other, if, you know if you’re going to talk about CoreImage, the CoreImage guys are just not going to talk about AppKit, because why should they? But since we’re up at the top, we’re in the middle level class, we sort of can encompass all of the other kinds of drawings. So maybe it falls upon us to actually say what they all are. Okay. So, so the first framework is AppKit.

And if you’re writing a Cocoa app, this is your default choice. AppKit integrates everything else. CoreGraphics. Quartz, excuse me Quartz. Quartz is the foundation for all of the drawing on Mac OS X, well at least all the drawing in Cocoa. And it’s, it’s, yeah, it’s basically interchangeable with CocoaDraw. It’s easy to work in there. And, oh and but, but CGImage, the interaction between CGImage and NSImage has been substantially improved in Snow Leopard and NSImage is working in terms of CGImages.

CoreImage, I sort of alluded to this before. But, several times I was asking, but looking directly at the Bitmap Data and the BitmapImageRep is not a great idea. It’s hard to interpret. CoreImage, on the other hand, is nicely set up for this. You can create this kernel that says here’s what I want to do each pixel, pass it off. It runs it over the image for you.

It uses the GPU. Good stuff. It’s for doing things like Photoshop builders. That’s when you want to look at CoreImage. Okay. It doesn’t do just normal image drawings or filters. ImageIO. The things you realize here is that ImageIO is about reading and writing BitmapFileFormat. With Metadata. And, okay.

And it’s used internally by AppKit and NSImageRep, so you may never need to use it. If you do, but under what circumstances would you want to investigate? If that’s what you’re doing. If you want to do something with metadata and writing bitmaps, for whatever reason the AppKit guys aren’t doing it for you.

ImageKit, this is a framework, where (part of what I’m doing is trying to tell you when you don’t need to look at them by the way). So, so ImageKit is a framework that maybe sounds a little bit more general than it really is. It’s really about providing standard user interface for working with images that are user data.

For example, when you’re using the accounts prep pane, it allows you to take a snapshot of yourself to become your user picture, that’s ImageKit. Or if you want to let the user browse through a bunch of different images, that’s also ImageKit. That’s when you want to look at it. CoreAnimation. This is about sort of declaring an entire scene graph of images that can move past each other into all sorts of different things. And it’s, it’s high performance because it is very declarative.

It leaves a lot up to the system. And the system knows a lot about what’s going on. There’s integration with AppKit in the sense that it integrates with the view hierarchy similar to, the view hierarchy is also about laying out drawing. And, and it’s, we also have improved integration with NSImage in Snow Leopard as I was saying, because each CALayer has a contents property, and you can directly set the NSImage as the contents of a CALayer in templates. Okay. PDFKit. I’ve said, I’ve told you a couple times that a PDF can be represented as an NSImage, if you, very easily.

So why would you use PDFKit versus NSImagePDFImageRep? Well PDFKit’s for, for PDF’s that are documents, okay. If you’re concerned with the text in the PDF, you probably want to be using PDFKit. If you’re just using it as a vector drawing framework, you probably want to be using NSImage. CourseComposer. This is a, this is a very interesting framework. It’s, it’s actually an entire programming environment. And it’s, it’s very visual.

What you see up here are these boxes, which each represents some kind of doing something, and they’re being connected by these yellow lines, which pipe data from one thing to another. And it’s a full-featured environment. It’s a complete integration layer. It’s really a peer of the AppKit. They do also integrate with each other. When would you look at it? Basically if it looks like fun.

So, but it’s also useful for, for sort of quick prototyping of image related tasks. OpenGL. Most of what to say about OpenGL is that it seems to be a little, people went to it a little early. It’s a cross platform API. It’s great for that. It’s also 3D. It’s great for that.

If you have 2D, if you’re working with 2D, it’s, you probably don’t want to be using OpenGL. Okay. QTKit, mostly what I want to say here is that these days QTKit is for movies. It’s not for still images so much. Okay. And then there’s Icon Services. Icon Services is a legacy framework in 10.6, meaning you should switch to something else. It is now, the functionality that is covered there is covered by NSImage and its work space and its URL.

For example, this kit testing thing that we had in NSImage. One of the reasons we did that was because Icon Services could do that thing. And we want to give people a good way to do that as they switch out of Icon Services. All right. And last, because I feel like there’s no way you could have possibly understood what I just said. I’m providing a cheat sheet. And this is not for reading right now. This is for when you download the slides later. This just runs down all those frameworks and tells you what they’re for and when you might want to look at them.

Okay.

All right. So what did we learn? So if, what do I want you to get out of this from NSImage? Mostly I wanted to get, I wanted you to realize that we changed a bunch, that hopefully we made it a lot better. That it’s seen a lot of work in Snow Leopard. So it’s simpler. And it integrates very nicely with Quartz and other frameworks when, when that’s necessary. It performs well.

We hope. Performs better. And the, oh and performance, and for performance please look at the, please look at that profiling function I showed you in this new bitmap backing store. And also remember, oh I totally lost track of this. Remember that the central point of that is if you, the same drawing should be represented by the same image.

Okay. And, and then we did like these graphic frameworks and for that you should take a look at the cheat sheet. That’s about it. If you want more information, the primary thing I would say to look at is the AppKit Release Notes. It’s is, it covers more than I covered in this talk, though of course some of the aspects are a lot easier to communicate to you speaking.

But it has I think everything I said here is in the Release Notes in some form. And then, if you want to email somebody, Matt Drance, the Cocoa Frameworks Evangelist. And we have our documentation, which is, you know, which will eventually cover everything, which is the AppKit Release Notes and covers all the basics quite nicely.