Welcome to session 125, VoiceOver and Making Cocoa Applications Accessible. My name is Rick Fabrick, and I'm an engineer on the VoiceOver team. And in case you missed the State of the Union address on Monday, or our session yesterday, I'm going to be going over a few things at the beginning of this session.

I'm going to start out by explaining what VoiceOver is, and then I'm going to talk about the importance of accessibility. And finally, I'm going to have a demonstration of VoiceOver so that you can see some of the features that you'll be learning in order to test your applications.

Now that's going to take about 10 or 15 minutes, and when I'm done, Mike Engber from the Cocoa team is going to come up and talk to you about the details on accessorizing your applications. So, what is VoiceOver? Well, simply put, VoiceOver is screen reader technology that comes installed with every single Tiger system. And screen reader technology allows those users who have visual impairments to fully access their computer. So everything that goes along, that happens on their computer, is conveyed to them in a way that they can perceive, and they're allowed to ask the computer what's going on.

Now, the main input device for VoiceOver is the keyboard. So every single feature of a fully accessible application needs to be available through the keyboard. The main output device is the speaker. So as I said before, everything that goes on the computer, that happens on the computer, is spoken to you, and we also have some audio cues.

So VoiceOver is built into the operating system. It's not bolted on. Now, what do I mean by this? What I mean is anybody can walk up to any system running Tiger, hit Command + F5, and if you're on a laptop, you may need to hit the Fn key, the function key, and turn VoiceOver on. Being built in also means that it's available during installation time. So even before the OS has been installed, a user can turn VoiceOver on and install it independently.

Once the user has installed the OS and they're setting their computer up, VoiceOver is available as well. The Setup Assistant application also provides a tutorial that walks them through the setup procedure. And you'll notice at the bottom of this Setup Assistant application window, it says in text how to launch that tutorial. But if you can't see the screen, that's not going to help you very much.

But after the introductory music finishes playing, the application will speak instructions. And this is what that would sound like. Mac OS X Tiger includes a built-in screen reader called VoiceOver. If you know how to use VoiceOver, press Command-F5 now to turn it on and set up your Mac. If you would like to learn how to use VoiceOver to set up your Mac, press the Escape key.

Being built-in also means it's available at login. So anybody can access their computer independently without the help of a sighted individual. So it's fully accessible. And of course it's available throughout the OS itself. So all the menus are accessible. You can go through the dock, the finder, the desktop. And many of the applications that come bundled with Tiger have been accessorized already. So from the get-go, users will be able to access their email, send email, access the web. And we're hoping that your applications will be here as well.

As I said, it's available anywhere Tiger is running. Now what this means is, with the latest release of our OS, Mac OS X has become the largest install base for any screen reader software. That's huge. Okay, so now who's going to benefit from you accessorizing your application? Well, obviously those who have visual impairment or the learning disabled, but also those who collaborate with the visually impaired. So what I mean by collaborate is when a couple of people are working at the same computer, a visually impaired person, a sighted person, they're collaborating, they're working together. VoiceOver provides some both visual and audio cues to help this collaboration, to help them work closely together.

And as time goes on, I don't know about you, but I have a lot of my personal information on my computer, right? All my financial information, music, pictures, I access the internet to get my news and whatnot. If I lost access to my computer, it would be a really bad thing.

And unfortunately, the possibility of my eyesight diminishing, it's there. So as time goes on, more and more of us who are computer savvy, we may start to rely more and more on screen reader technology like VoiceOver. Rick Fabrick, Mike Engber Let's look at a couple of other markets that you may want to sell into.

The US government has instituted Section 508, which requires, among other things, that software purchased by government agencies must be accessible to everyone. That includes the visually impaired. If you plan on selling to a government agency, one of the questions they're going to be asking you is, "Is your application accessible?" If your answer is no, they're not even going to be able to consider purchasing it. Other governments around the world are starting to adopt similar legislation as well.

So simply by accessorizing your application, your market share can increase. And when your market share increases and there are more accessible applications available on the Mac platform, the more visible Apple becomes and the larger market share Apple gets, which means your market share can increase as well. So it's a win-win situation there. All right. So let me go over a little demo here. If I can have the demo. Yeah.

Alright, so the first thing that I want to make sure that you understand is everything that's going on is through the keyboard. I'm not using the mouse at all, so let me move the arrow up here. That shouldn't move. Okay, so I'm going to turn on VoiceOver with Command F5. VoiceOver on. Finder. Macintosh HD. Selected. So, VoiceOver, when it launches, when it starts up, it will automatically tell you what application you're in, what window you're in, what element you're on.

So, let me check my email. I'm going to launch mail from the dock, so with the keyboard I'm going to... Doc. Finder. Running. Go to the dock, move over to the mail application and launch it. System preferences. Mail. Running. Mail inbox. Ten messages. Window. Delete button. Alright, so you may be able to see a couple of those visual cues that I mentioned earlier. The first one is at the bottom. It's what we call the caption panel. And basically it just shows you in text and print. everything that voiceover is saying.

The second visual cue is around the delete button at the top. And that's what we call the VoiceOver cursor. It's a black rectangle. And it focuses, it shows the person who's collaborating or somebody who has low visual sight what they're focusing on and what element they're interacting with. And I can move that around with the keyboard. Junk button. Reply button. Reply all button.

And VoiceOver explains to me what elements I'm on and their state. If it's dimmed, it'll say dimmed. Now, if I have limited vision, I'm having a hard time seeing that, VoiceOver lets me increase the size of the VoiceOver cursor. Let me show you that. VoiceOver cursor size 3, 4, 5. And I can continue navigating. Reply button. Junk button. Delete button. Alright, let me bring that back down to normal. VoiceOver 3, 2, 1.

Alright, so I'm ready to read my email. So I'm going to go down to the table of emails and navigate around. Messages table. Interact with. Interact with messages table on. And I can move around inside of this. Image map no AX. May 26, 2005. Alright, so this is the email I want to read so I can have VoiceOver quickly jump down to the body of the email and start reading it to me automatically. Stop message. Interact. If it's working. So. Text. I recompiled it and updated my public folder. New line. Can you verify this one works? For you.

Alright, so this, let's say I want to reply to this email but I don't know what command R is. I don't know the key combination. So, but I do know that there is a menu item up there that says reply. So I'm going to go up to the menu. Menu bar. On. Apple. Go over to the messages menu. Mail. File. Edit. View. Mailbox. Message. Message menu. And go down to the reply. Send again. Can reply. Command R. Reply.

Re. Image map no AX. And now I'm going to navigate to the bottom of the window so that I can reply to this email. To last item A interacting with item HTML content. Correct. And type my reply. So. A. E. S. A. A. K. E. K. New line. New line.

Now I've typed my reply but I want to make sure that what I've typed is correct. So with VoiceOver I'm going to go over and move and check to make sure what I've typed is what I wanted to say. New line. Coolness. And VoiceOver lets me do this letter by letter. C. O. O. L. Also word by word. Ness. Thanks. On. May. Twenty-six. I could do this line by line, sentence by sentence, paragraph by paragraph. We let the user choose how they want to navigate their text. Okay so now I want to send this email.

And let's pretend again I don't know the key combination, but I do know because I've navigated around here that there's a send button. But I don't want to have to navigate out and around and find it. I want to go directly to it. So another thing that VoiceOver provides is the item chooser. And what that does is it will list all of the visible items in the window. So let me bring that out. Bill, item chooser menu, 21 items. So there are 21 items here. I can use type ahead to jump quickly to send. So let me type send.

One item, send button. Send button. So now I'm at the send button. I can simply press it. Press. Text. I recompiled it and updated my public folder. Okay. Well that's the end of the demonstration. Text. I recompiled. Let me turn VoiceOver off. VoiceOver off. Cool. If I could have the slides back up.

All right, so you may be asking yourself now, what should I do now? Well, the first thing that you need to do is you need to support the Accessibility APIs. So everything, all the information that you provide to the Accessibility APIs is the information that VoiceOver gives to the user.

So the more you support these APIs, the more full experience that your users are going to have with your application. Now, VoiceOver may be the main client of these APIs, but there are other ones. Eric Seymour, our manager, demoed one at the State of the Union on Monday.

It's very important to support these APIs, but also full keyboard navigation so you know when you hit tab, focus moves from control to control. Now, since this is Cocoa, a lot of this is already done automatically for you. That's why Cocoa is the preferred method of starting new applications. But if you have custom controls, you want to make sure that those are in the keyboard loop. So that's something that you need to think about as well.

Also, there are going to be some, there may be some features in your application that require the mouse. And you're going to need to identify these and find alternative ways of accessing those features using the keyboard. Now, for example, our favorite example is drag and drop. So in the Finder, if you want to copy a file, you drag it from one place to another.

Now, the Finder has solved this by using copy and paste. So you can select the item, hit copy, open the destination, and hit paste. You may be able to use this in some of the areas that you identified, or you may want to come up with some of your own. Now, once you've done all of this work accessorizing your application... You're going to want to test to make sure that your users are going to be able to take advantage of all that.

So Apple provides a couple of tools that are in the developer package. The first one is the Accessibility Verifier. Now this is a good application to get a general sense of your application. You basically point this to your running application. It will go through all of the open windows and check all of the controls and make sure that you've supported all the APIs properly. It may flag things you've forgotten to do or some things that you've done incorrectly.

The Accessibility Inspector is great to focus in on a particular element so you can get all the information for one element. When Mike comes up on the stage in a couple of seconds, he'll show you how that application would be used in his demos. Most importantly, you need to test your application with VoiceOver. You need to unplug the keyboard and turn off the display.

Run your applications like the user would who is using VoiceOver. Make sure that all of the features that you provide can be accessed. When you can access all those features, you know you have a fully accessible application. Now what you see up on the screen is a key mapping.

This is available for download. You can see the URL there at the bottom. There are several of these. Now, one of the things that I mentioned was dimming your screen. It may be difficult to do that on your laptops, but we offer a feature that lets you do that.

I challenge you, go ahead and find, there's a couple of these maps. Download them and try and find out how to do that. So with that, let me ask Mike to come up on stage and show you exactly how to make your Cocoa applications fully accessible. Thank you.

Hello, my name is Mike Engber, and I'm gonna talk to you today about how to make your Cocoa apps work with the Accessibility APIs. When we talk about the accessibility architecture in OS X, there's really two halves to the story. The first are the APIs that assistive apps like VoiceOver use in order to inspect the user interfaces of other applications. And in this diagram, that's represented by the green box. The other half of the story is what the applications themselves have to do to expose their user interfaces. And those are represented by the red boxes.

And today we're going to mostly talk just about the red boxes, specifically what Cocoa apps need to do to work with the accessibility APIs. Now, before we get started, it helps to review one of the fundamental problems that's addressed by the accessibility APIs. And that is, how does an application represent its user interface? Now, applications have a natural way of representing their user interface. Cocoa apps use NSWindows or NSControls.

and Carbon apps use and I are working on the Windows Refs and Control Refs. But the problem with using these natural representations is that it requires the assistive app to understand Cocoa and Carbon and any other frameworks we add to the mix. And another problem with these natural representations is they include a lot of extraneous detail, clipping views and border views, things that aren't really part of the user interface and are of no interest to assistive applications.

So the solution that we've come up with is to use this abstract object called a UI element to represent everything in the user interface. Windows are UI elements, menus are UI elements, the application itself is a UI element. Your application is a hierarchy of UI elements. UI elements have attributes like a title or a value or a children attribute, which is how you traverse the hierarchy.

And then some UI elements also have actions. A button might have a press action or a slider might have an increment action. And rather than go into detail about UI elements in slides, I'm going to use a tool, Accessibility Inspector, which will let us look at some live UI elements in a running application.

So I'm going to use TextEdit's Preferences pane, and I'm going to launch Accessibility Inspector. I'll increase the font size. What Accessibility Inspector does is as I move the mouse, it's reporting information about the UI element under the mouse. Now, I want to use the mouse for some other purposes, so I'm going to lock our attention on the current UI element, and that's why all the text turns red. So we're locked now, and we're locked on this radio button. Here's the role of the radio button, and the whole hierarchy is shown above it.

So it's a radio button inside a radio group, in a tab group, in a window, in the application text edit. And then if we look further down the window, we can see a list of attributes that this radio button has. It has a role, and you can think of that like the radio button's class.

It has a role description, and so that's a version of the class that's suitable to report to the user. It's a string that would be localized. It has help. It has a value. This radio button isn't checked, so the value's zero. And here's the position. A size. All these other attributes. And it also has an action, a press action. And it also has a press action.

And now that we're locked on this particular UI element, another panel is available to us that we can interact with it. And so one of the things we could do is actually we can pick one of the actions, like the press action, and we can perform it. So when I hit this perform button, we'd expect that the same thing would happen as if I clicked on that radio button.

And we can see it changed state. Another feature of this that we can use is we can highlight the bounds of the UI element we're interacting with, and sometimes that's helpful to give us context as to what we're working on. I'm going to now pick another UI element to operate on.

I'm going to pick one of these text fields and I'll lock on it. And another thing this panel lets us do is pick an attribute. The attributes we can modify have a W for writable after them. We can pick the value of this text field, which is now 75, and we can change it to something else. We enter 42, and then we hit the set value button, and it changes as if we'd entered that value.

Back to the slides. Now this pyramid is meant to give you an idea of what efforts are required to support accessibility in your applications. Like the food pyramid, the stuff towards the bottom are the things that you need to do the most of. Fortunately, those are the easiest items to do. As you move up the pyramid, things get a little more challenging, but they're less frequently needed. Before we go further into this, I'm going to demonstrate the application that we're going to be accessorizing.

The project has disappeared on me. Nothing in it. I do not have an easy way to fix that right now.

Well, there's a CD in my bag. That's the only place I know to get the source. So, if someone can retrieve me my bag, in the meantime, I can run the application.

and show you what the finished product looks like and I'll get a copy of the source up here in a second. So anyway, this is the application we're going to be accessorizing. And the central part of it is a custom control that works like a client-side image map in HTML. So this is an image and superimposed on it are various hotspots you can click on. So each food group is a hotspot. You can see I can click on them and they highlight and they cause something to take effect.

Another part of this application is this little segmented control, and this controls the sort of mode of operation of the image map. So if you click the middle segment, it'll highlight all the hotspots, and you can imagine tying that to, say, holding down the Option key. And in this third mode, the image map will highlight as you mouse over the various hotspots. So these are different modes that this thing can operate in. Fresh copy of the image map example. This time with everything in it.

Okay, so one thing I want to show you is how that image map control works. Even though this doesn't have much to do with accessibility, it will give you some idea of how the control operates. Here's the header file for this custom control. We'll look at some of the methods.

There's methods for configuring it. This sets the image. Here you can manually add hotspots, rectangles, circles, or polygons. Probably a more convenient way to set up those hotspots is the same way you'd do it if you were creating a web page. There's all these tools out there for creating client-side image maps. They spit out HTML that looks something like this. There are methods that will let you specify an HTML file like this one, and it'll initialize the hotspots from that.

And then there's some target and action methods. So it operates like other NS controls. And there's various other methods for configuring its mode of operation. So this control is meant to be sort of non-trivial, something that you could use as a starting point for something in your own applications.

All right, now the next thing I want to do before we proceed is I want to bring Rick back up here to demonstrate how this application works with VoiceOver before it's been accessorized and after it's been accessorized. Okay, so this should be really quick. What I'm gonna do is launch both versions of the application.

All right, so this one on the left is the non-accessorized application before any work is done on it. So I'm going to launch VoiceOver. VoiceOver on image map example, image map example, window blank. Okay, so first you notice that it skipped, the first item is not near the top, so it completely skipped the image map. What I'm going to do is I'm going to navigate through some of the controls and we'll hear what it sounds like before we accessorize.

Radio button one of three. Radio button two of three. So even though I can see. Radio button three of three. The icons, if I can't see the screen, that doesn't tell me much. So I'd be scared to press any of these things. And as I move around. Image map mode selector. Blank.

So it completely skipped the whole image map at the top. So now let me move over to the accessorize application. Image map example, image map example, image map. So you'll notice that we did land at the top of the screen. So I can go to the window. And I can ask what element I'm on.

Fats and sweets button is in the voiceover cursor. So now I know I'm at fats and sweets and I can move around. Dairy button. Meat and eggs button. Vegetables button. And interact with these buttons, I can press them. Press vegetables button. And let me go down so that you can see the segmented control.

To last item. Rollover version highlighting radio button three of three. So rollover highlighting. Visible hot spots. Radio button two of three. So now I know what the controls will do. And I can interact and fully access, use this application. That's it. Let me turn voiceover back off. Voiceover off. Okay. Back to the slides.

Can we get the slides back? Okay, so now that we've seen the application we're going to accessorize, we can flesh out the pyramid a bit more. First, we're going to add some attributes. We're going to add one using Interface Builder, and then we're going to add one programmatically, and then we're going to handle a tricky case involving the segments of the segmented control. Then we're going to see what it took to make the image map view itself work with accessibility, and then finally we're going to look at what it took to make the hotspots themselves work with accessibility, and that will require accessorizing a class from scratch.

So we're going to start off with adding an attribute using Interface Builder. And we're going to add an attribute to the image map itself. It represents itself as a group. So it's a group containing buttons, the hotspots. But that's all VoiceOver could detect now, that it's a group. So we want VoiceOver to know that it's really a food pyramid group. And we're going to do this using Interface Builder. So back to the demo machine.

So here's our nib with our window, and we're going to select the image map, and then we're going to bring up the inspector for it. I apologize the fonts are a little small here. You'll notice that one of the options in this menu is Accessibility. So we bring that up and you'll see that there's a field here where we can type in the description.

So we can type in food, Remember I said we want this to be a food pyramid group. The fact that it's a group, that comes from its role. So assistive apps can already determine that it's a group. All we provide here is the words food pyramid. If we typed in food pyramid group here, not only would it be redundant, it could result in some awkward behavior because assistive apps often combine the description and the role and present that to the user.

So if they did that, it would be food pyramid group group. So when you're filling in descriptions, make sure to just put in the description and not the group. And that's pretty much all we need to do. So using Interface Builder to add an attribute is very straightforward. So back to the slides. Slides, please.

The next thing we're going to do is we're going to add an attribute programmatically. We can visually see that the static text Image Map Mode Selector is acting like the title for that segmented control above it. But they're two entirely separate NS controls. This relationship is apparent to us, but it's not exposed through accessibility. In order to make that relationship available to accessibility, we have to add an attribute, a UIElementTitle attribute.

So, the first thing to note is, if you were really doing this, you could do this in IB, just like adding an outlet. There's a similar interface for adding title UI element attributes, and that would be the easiest way to do it. But for educational purposes, I'm going to show you how to do it programmatically.

The next thing to note is when you see the code, you're going to see that I don't use the outlets directly. And that's because the outlet, which contains an NS control, like the NS segmented control, that may or may not be the right thing to use as the UI element. Remember, there's a segmented control, and then inside it's a segmented cell. So there's this whole issue of which one's really the UI element.

Well, we use a utility function, NSAccessibility, unignored descendant, to answer that question for us. And I'm actually going to return to this issue later in the presentation and elaborate on it a bit more. And then finally, once we get the right object, there's a new method that was added in Tiger, AccessibilitySetOverrideValueForAttribute , that's going to let us override the value of an existing attribute, or if that attribute doesn't exist, it's going to add that attribute to the object. So I need to go back to the demo machine. And if you're following along, the code for this is in ImageMapController.m. And this work is done in the AwakeFromNib method.

And you'll see there's a comment that says, "Set up segmented control accessibility." So the first thing we do is we create a couple local variables to hold the UI elements of interest, and we're using that utility function on the outlets, as I mentioned before. And then we're gonna use the method accessibility set override value.

The receiver is the segmented control. That's the UI element we want to get the attribute. The name of the attribute is the NSAccessibilityTitleUIElement attribute, and the value is another UI element. It's the segmented control title UI element. And so basically this line of code does all the work of interest. Okay, back to the slides please.

Okay, now the third case we're going to handle are descriptions for the individual segments in the segmented control. So as Rick demonstrated, if we don't do anything, these things, as far as VoiceOver is concerned, are just radio button one, two, and three, not very descriptive to the user. So we want to do a better job and elaborate on what those things are.

Now the tricky thing here is knowing what object are we going to set the attribute on. We have a segmented control, and the segmented control has a segmented cell, but there are no NS segments. There are no objects representing the individual segments. So that makes it a little bit tricky.

Now, the solution to this problem is we can ask the segmented control for its children from an accessibility point of view, and it will return them. And we don't really, we don't have to know what kind of objects they are. We just operate on those. So the basic approach we're going to take is ask the segmented control for its children, loop through the children, and then use accessibility set override value, just like we did before, on each of those children.

So back to the demo machine. And the line of code that does that is right here. The actual work's been factored out in a function, which we're going to take a look at. But basically, I specify the segmented control, and then I give it a list of the descriptions to be used on the individual segments.

And I factored it out into a nice little function, so you could just copy and paste this into your own projects if you like. And here we'll take a quick look at that function. The first thing it does is it, once again, it takes the control and uses this utility function to make sure we get to the right UI element.

and then here we're going to message it and we're going to ask for the value of the children attribute. We're going to get back an array of objects. And again, we don't know what kind of objects they are exactly, but we know they are UI elements.

Setting the description attribute and the value is the description. So that's all there is to that. And I think at this point, I will demonstrate the result of all this work using UI Element Inspector.

Here's our app. Let's get some of the other stuff out of the way. So here's our running application. I'll run Accessibility Inspector. So we did three things. We added a description to the image map. So let's lock onto the image map. And again, I'm going to put the highlight on so you can see I've locked onto the right thing. And if we look in here, we see here there's an AX description, food pyramid. So we know that one worked.

Another thing we did was give descriptions to the individual segments. So here I can lock on a segment, and we can see it has an AX description rollover highlighting. I can lock on another segment. and I could see it has a description, visible hotspots. So that worked also. And then finally, the next thing, the last thing that we did, or the thing we haven't examined yet, is we gave to the segmented control as a whole a title UI element attribute.

Now you'll notice whenever I try to mouse over the control as a whole, I keep hitting the buttons. So it seems like I'm kind of stuck. I want to get to their parent. How am I going to do it? Here's a little trick you can use. You lock onto one of the buttons.

And then in this little panel that comes up while you're locked, you can go to any of the attributes whose value is another UI element. So here I'm going to go to the parent. And if you watch the highlight, when I pick parent, you can see I've moved up the hierarchy.

This window's changed or reflected. You can see I'm in the radio group now. And we can make sure that we have an attribute, AX title UI element, that's a piece of static text. And then if you want to make sure, well, is that the right piece of static text? We can look and we can actually see that in this go-to menu, the title UI element shows up. So we can now change the focus to be that thing. And we can see we got the right piece of static text. So back to the slides, please.

Okay, so now before I proceed, one thing I want to point out is we've covered now about 90% of what you need to know to make your application accessible. The two remaining things I'm going to illustrate are going to get more challenging, but I don't want anyone to be daunted by it.

I don't want you to think that accessibility is too intimidating, right? Because if you just stick to what we've done so far, you can make a huge difference in your applications working with VoiceOver. And you can think of the next two topics we're going to cover as extra credit for all the overachievers in the audience.

And the first one we're going to talk about is how did we make this image map itself work with accessibility? The good news is it inherited from NSView. So most of its accessorization it just inherits because it's a view and NSView works properly with accessibility. And because of that, all we have to do is override the specific methods that we want to behave differently. So I'm going to switch to the demo machine. And we'll look at the source.

Okay, all of the work for accessorizing the Image Map is actually done in categories in a separate file, ImageMapAccessibility.m. And that's a nice way to organize your code if you want to keep the accessibility code all together. and here's the categories on Image Map. So you can see the first method that we had to override is accessibility is ignored.

It turns out NSViews by default are ignored. Most views aren't actually part of the user interface. They contain things that are part of the user interface, but the views in and of themselves are not that interesting. So by default, they're ignored. So the first thing we wanna do is say, we're not ignored.

The next method we override is Accessibility Attribute Value, and there's a few attributes that we want to change the value of. One of them is the role. We want to represent ourself as a group. The role description. We use a helper function to give us the standard description for our role. So there's not much to figure out there. And then the final attribute we override is the children attribute. These hotspots are not subviews.

So we don't get them by default. So instead, we have to create children. And that's going to actually be the topic of the... Next part of my presentation, so I'm not going to go into detail here, but you can see that we're creating an array, and for each hotspot, we're adding an element to the array, and exactly what that element is, I'm going to cover in a second.

And then we also overrode the hit test method. And this is because we want to, if the mouse is over one of our children, we want to return that child as the thing of interest. So we get, we're passed in a point and we convert it to window coordinates from global coordinates. Then we convert it to local coordinates and then we use a helper method to give us the hotspot that's over that point. And then we return Our child.

And again, the nature of what this object is, I'm going to cover in a second. But that's what we do when we override hit testing. So back to the slides, please. So now we're going to do, we're going to handle the children of the image map. And as I said before, this is going to involve accessorizing a class from scratch.

So the tricky part here is that there are no objects representing those individual hotspots. Now, if you were designing this control from scratch and you had accessibility in mind, maybe you would create an object for each hotspot and that would simplify the accessibility problem. But again, for purposes of illustration, it's better here to show the hard case. What do you do when you don't have any objects? And to handle it, we're going to invent a class called Faux UI Elements. And this is a pretty generic class.

And it turns out that we're actually going to end up having to subclass it to solve our problem. But it's useful to solve it in a generic way because that way you can take this Faux UI Element class out of here and use it in your own applications.

One thing I want to note is these foe UI elements that we're going to use for our children, we create them on the fly as needed. So when we're asked for our children, we just allocate them, create an array, let it get auto-released, and hand it back. So we're not going to try to cache them and try to keep it in sync with the current state of the control. It's much easier just to create them on the fly as they're needed.

Before we get into it, something that sometimes twists people around is, well, how can I use these foe UI elements? They're not NSViews. How are they going to work as UI elements? A UI element is anything that implements the accessibility protocol. That's it. There's ten methods you implement. If you implement them all, you can be used as a UI element.

So, our foe UI elements, they're just plain NS objects, and they're going to implement the whole accessibility protocol. And when we look at the code, you're going to see that they only keep track of their role in their parent, which then leads to the question, well, if that's all they know, how are they going to answer questions like their position or size? Well, the answer to that is, they're not really going to answer that question. They're going to pass the buck up to their parent.

So, we have this foe UI element protocol that the parent is expected to implement so that the children can ask it to answer, you know, where am I? How big am I? Do I have the keyboard focus? Things like that. And then a final note is that we have to override as equal. As I said before, we're going to create these foe UI elements on the fly.

So, if hit testing returns us, you know, one of these hotspots, and then we go searching for it in the parent array, we want that search to succeed. And by default, is equal is going to do a pointer comparison. And that's going to fail because these are going to be different objects. So, we override is equal. and Mike Engber are working on a new application called the VoiceOver.

So when we look at the code, you're going to see we've implemented 10 methods, 10 methods of the Accessibility Protocol. Four of them have to do with the attributes. There's three action methods, and then there's three other methods. There's the isIgnored, which we've talked about, hit testing, and focus testing. So I need the demo machine back.

So this code's located in another folder, Faux UI Element. And I'll start by looking briefly at the header file. And you'll see here's a, there's two IVARs, the role and the parent. Like I said, that's all it keeps track of. And there's an init method and a factory method for cranking them out.

Here is the definition of the Faux UI Element Protocol. The parent, anyone who hands these things out as their children, is expected to implement two methods. One to determine if this child has the focus and then to give them the focus if the child requests the focus. That's keyboard focus we're talking about. Then there's two methods having to do with the position and size. Again, given a child, what's my position, what's my size? The parent is expected to answer those questions.

Here we'll look at the code that implements it. This is the implementation of the init method and the factory method in DL. As I mentioned before, we have to override isEqual. The first thing we do when we're comparing is make sure the other object is a foe UI element. If it is, then we can compare the roles and we compare the parents. And then if the override is equal, the proper thing to do is also override hash.

All right, so here we start into the accessibility protocol. The first thing we do is return a list of all the attributes we support. Here they all are. And then in the next method, we'll actually return values for those attributes. So the first one is the role attribute. And since we keep track of our role, all we do is hand back the IVAR we have for our role. If we're asked for the role description, there's a helper function that'll give us a standard description for that role. So nothing tricky there.

There's an attribute that answers the question, are we the focused UI element? Sometimes people try to get tricky here. They check, oh, well, are we in the main window? Am I the first responder or do I belong in the first responder? It's easy to get that wrong. A much simpler thing to do is just ask the app, using the accessibility protocol, what it thinks the focused UI element is. And if it's you, say yes. And that way you can never be out of sync with that attribute on the application. Thank you. Next is the parent attribute. Again, we keep track of our parents, so we just have to hand back that IVAR.

We use a little trick when we implement the window attribute. We know we're in the same window as our parent, so all we do is ask the parent, "What do you think our window attribute is?" and we return the exact same thing. We do the same trick when it comes to the top level UI element.

And then the last two attributes are our position attribute. And here we ask our parent for our position, and we turn it into an NS value and return that. And then here we ask our parent for... We use the Faux UI Element protocol to find out the size of our application.

The next methods of the Accessibility Protocol we implement have to do with, is an attribute settable? The only thing that could be settable is the focus. And again, we defer to our parent to answer this question. And then if they actually decide to set the value, once again, we defer to our parent to actually do the work.

There's three methods having to do with actions. Now, we are not implementing any actions, so we return the empty array of actions, and what we do in the other two methods really isn't important. And you might notice, well, these are going to be buttons, these hotspots. How are we going to press them? Well, we're going to handle that in the subclass.

And then finally, accessibility is ignored. We don't want these things to be ignored, so we return nil. And then there's hit testing and focus testing. One thing I'd like to point out about hit testing is when your accessibility hit test method is called, your ancestors have already determined that the mouse is over you. And if you have nothing further to say on the subject, you just return yourself. Just agree with that assessment. But if you have children, you might want to return something more specific.

But in our case, we don't, so we basically just return ourself. And the same thing goes for focus. Our ancestors already determined we have keyboard focus, and unless we have some children inside us that might have it, we just return ourself. The thing you don't want to do is return nil. If you return nil, you're basically disagreeing with your ancestors, and you're saying, you know, the mouse isn't over anything, or no one has the keyboard focus. So that would be an incorrect thing to do. All right. Back to the slides.

So the faux UI elements don't quite do what we want for the hotspots. So we're going to have to subclass. So we're going to create a subclass that inherits from faux UI element. We're going to add one IVAR, which is going to be the index of the hotspot. And we're going to have to override isEqual to take that IVAR into account. So not only does another hotspot, you know, another hotspot will have the same role, and it's going to have the same parent, but it's not the same as us unless the index matches.

So we have to do that. And we're going to add a description. If the data for these hotspots came from an HTML file, we can use the alt tag to provide a description. And then finally, we're going to implement the press action. So back to the demo machine.

So the code for this is back in ImageMapAccessibility.m, and it's towards the top of it. So here, since this is a private class, we only use it in this file. It's declared and implemented in here. And here's its declaration. You can see we added our one IVAR, the index, and we have our own init and factory methods. And in our factory methods, we already know the role.

These things are going to be buttons. So you just pass in the index and the parent, which will be the image map. So here's the implementation. Here's the init method and the factory method. Here's our override of isEqual. The first thing we do is make sure we're comparing ourselves to another hotspot.

And then we do the default thing, make sure the other hotspot is the same role, make sure it has the same parent, and then if it passes that, we want to make sure the indexes match. And once again, we want hash to be consistent with is equal, so we overrode hash. Here's an accessor method for the index, right here.

Okay, then most of the accessorization was done in the faux UI elements, so we've only had to override a few methods. Attribute names, if this image map came from HTML data, what we're going to do is we have one extra attribute we're going to support, and that's going to be the description attribute, and we keep this cached in a local static, and we initialize that static by getting the, we call super accessibility attribute names, and then we add one object, the description attribute.

When it comes to values, most of the time we just defer to our superclass, but if they're asking for the description, we can ask the Image Map to give us the information for this hotspot, given our index, and that information is a dictionary. Out of there we can extract the value of the Alt key, and if there is no Alt key, we can use the Title key. That way we can support a description attribute for the hotspots. When it comes to the actions, instead of returning the empty array, now we return an array containing our press action.

The action description, given an action, this helper function will return a standard description for it. And then when we want to actually perform the action, We just asked the ImageMap to perform the action for a hotspot at a given R index. So again, this method, perform action for a hotspot at an index, is implemented by the ImageMap.

Okay, now here I want to return to the spot where I sort of waved my hands earlier. When we're returning the children for the image map, here's where we use that factory method on Hotspot UI element to create a hotspot. Oh, excuse me, right here. To create a hotspot given an index and we specify ourself as the parent. So that's where we take advantage of this.

And then in hit testing, we also do it. Once we've determined the index of the hotspot we're interested in, again, we use the factory method. We create an element with the index and we use ourself as the parent. And then here's our support for the Faux UI element protocol. Since we're handing out Faux UI elements, we have to support these things.

Now, currently, this control does not support keyboard focus. But if you decide to add keyboard focus, this is where you have to do the work. All right. In these two methods, but we don't. So basically, we say no, no, you never have the keyboard focus. And we don't have to do anything for setting it up.

Here's where we provide the position. If someone hands us one of these children of ours, we determine its index, and then we get its position, and then we have to convert those coordinates to global screen coordinates, and we hand them out. And the same thing for the size. Given a faux UI element, we determine its size, and we convert the bounds and hand them back. So back to the slides.

Okay, so that's it for accessorizing our custom class. And in the time I have left, I'm going to go quickly through some common problems people have when they accessorize their apps. The first thing I want to mention is that there's some debugging information we spew out to the console if you set a certain flag. And that's the NS Accessibility Debug Log Level. If you launch your application from the command line, you can specify -NS Accessibility Debug Log Level 1. And for certain kinds of errors that we detect, we're going to print an informative error message for you.

Could you put me back on the demo machine? I'm going to show you how to do this in case you want to launch from IB. It's useful to know that you go to your... You go to your executables and you can pick your executable and get an inspector on it. And you'll notice the second tab here is Arguments. So you can add an argument here. You can type in NS Accessibility.

Level 1. So if you're not launching from the command line, if you prefer to run all from within IB, that's how you can achieve this. Back to the slides. So, my first tip if you're trying to debug your apps is turn that flag on. That's just going to make your life a lot easier. And then check the console for messages.

A lot of the errors that you're going to run into have to do with ignored UI elements, and I kind of hinted at this before. Accessibility squeezes out a lot of extraneous information from the hierarchy. So in reality, you might have a window containing a frame view, containing a content view, containing a button, containing a button cell. From an accessibility point of view, you've got a window with a button in it. And ignored UI elements is sort of how we achieve that simplicity. All the stuff in between, those are ignored UI elements.

So one of the problems you can have is accidentally having an ignored UI element. So our image map was an example of that. NS views are ignored by default. So you need to make sure that your class isn't ignored. Another problem you can have in this area is using an ignored UI element.

You'll remember we didn't use those outlets directly. First, we passed them to a helper function, NSAccessibilityUnignoredDescended. So if we tried to set the attribute on the, say, the NSSegmentedControl, we would actually have been setting it on something that's ignored. It would have been a complete waste of time.

So that's one mistake you can make. And if you actually, if you try to set an attribute on an ignored UI element, that's one of the cases that the accessibility debug log level is going to catch. It's going to warn you that you're trying to set an attribute on an ignored element. And that's generally a waste of time.

Now, sometimes you need to traverse the hierarchy in the other direction. For instance, you might be implementing your parent attribute. And when you report who your parent is, if your parent's ignored, you really want to report your grandparent and so on. So we have a utility function to help do that search and return the first unignored thing, and that's NS Accessibility unignored ancestor. And then when you're returning your children, if you've got a child that's ignored, you want to replace it with his children and so on. And again, that's another sort of tedious thing to calculate. So we have a helper function for you.

NS Accessibility unignored children. You pass it an array of children. We'll take care of squeezing out any children that are ignored and replacing them with their children. And then there's a convenience function for the case if you only got one child. You just use that function. NS Accessibility unignored children for only child.

Another mistake you can make is forgetting to implement hit testing. Depending on how you test your code, this may be obvious right away or it might not be. If you use Accessibility Inspector, which uses hit testing, when you move the mouse over UI elements, you're going to know right away that something's wrong. Your children aren't showing up. VoiceOver often navigates the hierarchy using the children attributes. If you use VoiceOver to test it, it might not be obvious right away that you forgot to implement hit testing. That's another easy thing to forget.

Another thing you can do is have asymmetry in the parent-child relationship. If a child reports some other UI element as its parent, when you ask that parent for its list of children, that child better be in the list. And if you don't do that, it's likely that that child is going to appear not to work. Once again, if you look in the console and you have NS Accessibility debug log level on, you'll see some error message about failure to find such and such child and such and such a parent.

If you go ahead and start using this faux UI element class I've given you, remember to override is equal because if you hand out a bunch of children and they all have the same role and they have you as your parent, they're all going to be equal. So if hit testing returns some child, you know, and I go looking for him in the children array, it's going to say, oh, he matches the first one. Every single child is going to match the first element in the children array. And so that's going to be a problem. So if you see this sort of bug that every child seems to be the first thing, this is how it happens.

And then the final thing I'd mention is sometimes people forget to take into account is flipped. Some NS controls or NS views have this sort of flipped coordinate system. And from an accessibility point of view, when you're reporting your position, you want to return the lower left corner of the view. And a flipped view, 00 is the upper left corner.

And so this is not really a coordinate conversion issue. If you use the upper left corner, you're just passing the wrong point back. So you have to calculate the bounds of the lower left corner, basically just add the height of the view in, and that's the value you should convert to global coordinates and return.

So, we have various documentation resources. Here's a URL. There are some related sessions tomorrow morning. We have a feedback session. We'll have some Q&A today. And then Friday we'll be here for a few hours for a hands-on lab. You can bring your projects in and try them out. and here's a contact list. So the people that wanna participate in Q&A, wanna come on up?