Good afternoon, everyone. Welcome to session 108, VoiceOver and Making Your Carbon Application Accessible . My name is Patti Yeh, and I'm a software engineer with the VoiceOver team. Let me go ahead and give you a sense what we're going to be talking about. If I could find where my clicker is. Here we go.

I think I'm missing a slide. Oh, well. Let me just go ahead and tell you what we're going to cover in the session today. In the session today, in the first 15 minutes or so, I'll tell you about VoiceOver. I'll tell you why you should make your application accessible. Later on, I'll give you a demonstration of some of the basic features of VoiceOver. After that, we'll have Guy Fullerton up on stage to show you how to make your application accessible.

So, for those of you who don't know what VoiceOver is, it's Apple Screen Reader technology. Some of you may have heard it at yesterday's State of Union presentation. But if you haven't, VoiceOver, what it does is it speaks all the activities on the screen for you. And for the Tiger release, we're supporting English. ScreenReader is also an alternative interface for users to interact with the computer without ever having to use the mouse or the screen display.

There's the session overview if you didn't catch that earlier. VoiceOver is fully integrated into the operating system. It is built in. What does that mean? It means it is part of the installation process. And if you're ready to set up your computer, it is there to help you, to guide you through it. If you wait for the MacBuddy music to stop, this is what you'll hear.

The volume, volume please. All right, well it looks like we don't have the audio here. But if you go and install your DVD, I encourage you to try it out and see what you hear from it. So, VoiceOver is available for you during login time too. And, By saying that it is built-in means that it's part of the Finder, Desktop, Menu Bar, Dock, etc. Also, many of the applications shipped with Tiger are already accessorized to work with VoiceOver, and we sincerely hope that your application will be accessible soon, too.

Like I said earlier, VoiceOver is fully integrated into the operating system. What's more than just that is installed by default onto every Tiger system. That means a user can now walk up to any computer running Tiger. Could be the computer lab, Apple Store, library, you name it. Press Command F5 to turn VoiceOver on, and bang, the computer is now accessible.

So who will benefit from using VoiceOver? Well, obviously those who are visually impaired, blind, or learning disabled. It's more than just that. It's everyone who collaborates with them. That could be the parent, the teacher, the co-worker, sighted folks like you and I who are working together with a screen reader user on that same computer. Later in my demonstration, I'll show you some of those visual cues that help those who are collaborating to work together with the screen reader user so they can follow along when they're using the computer.

So who else will benefit from using VoiceOver? Well, I for one will. See, computer is very important to me. On a daily basis, I probably use it for email, surfing the web, doing work, all these things, right? And also, I store a lot of stuff on my computer, like digital photos, MP3 songs, my financial documents, et cetera. So I don't know what I would do now if I can't use my computer.

See, the other thing is, as I get older, I might lose my vision. I might have a harder time to use the mouse. So when you combine those together, it may be harder to use the computer, and it could be at one point where I won't be able to access my computer. The good news is Apple has already done a lot of things to make sure that there's an alternative interface for users to use that computer via VoiceOver. So all of us can potentially benefit from this.

So as a developer that's representing your company, you may be wondering, what is the business benefit of making your application accessible? The short answer is Section 508. Section 508 is a US law that states all government agencies must provide access to member of the public and the disabled employees. And similar laws and trends are happening around the world too.

So by simply making your application accessible, You have a much higher chance to increase your market share. And as an example, the education system now could begin to evaluate your product when they're making that next round of software purchases. So, of course, what's good for you is good for Apple. Apple's market share will increase, which will then come back and boost your market even further. It's good for all of us.

So at this time, I would like to go ahead and go to the demo to show you some of the basic features of VoiceOver. So, first thing we have to do is turn VoiceOver on. For those of you who might have attended yesterday's session or heard me earlier today, remember what the key is to turn it on.

Command F5. VoiceOver on Finder 108. Demo selected folder. Okay, so the task I'm going to show you today is we're just going to go and check our email. So to do that, I'll go ahead and go to the doc and launch Mail.app. Doc Finder Runny System Preferences Mail Runny Mail Inbox 26 Messages Window Delete Button Remember earlier I promised you I'll show you some of those visual cues? Well, here on the screen you see two of those. One is the caption panel toward the bottom of the window. Everything you hear, we provide a textual representation of that, too. The other one is the black box surrounding the Delete button in the Mail.app. And that is what we call the VoiceOver cursor.

So as I navigate around, you, the audience, which is, you know, as the collaborator, can follow along while I'm using this computer. So let's go ahead and move around. Change button. Reply button. Reply all button. And of course, if you have low vision, here's something that will help you. Thank you.

VoiceOver cursor size 3, 4. All right, and we can still move around. Reply button, check button, delete button. Let's go ahead and turn Zoom off for now. VoiceOver 2, 1. All right, we're ready to go and check what is the selected message in my inbox. Messages table. Interact with messages table on Star Wars Mary Beth James.

Okay, well let's see what Mary Beth has to say about Star Wars. Clicked item text: "Patti, have you seen Star Wars yet? I saw it in San Francisco and it was great. Have a great weekend. Mary Beth." Well, how many of you have seen Star Wars? Hello, most of you.

Well, how many of you like it? Well, I love it. I mean, it's great. So I'm going to go ahead and tell Mary Beth that. There's a couple ways I can reply to this message. One is I could go up to the reply button by the toolbar and press that.

Or if I know the keyboard shortcut, I could do that. But since I can't remember what the shortcut is, I'm going to go up to the menu. Menu bar on Apple. Mail message. Message menu. Send again. Reply. Command R. Okay, we found what we're looking for. Reply. Read Star Wars window. 18 items. Send button. And if I forgot to mention, all this time I'm not using the mouse.

Okay? So what we want to do is we want to go ahead and reply to this message. So I'm going to go down to where that blinking cursor is. Menu HTM interacting. And notice I canceled some of the speech. So anytime when you, for example right now I'm talking to you or if you want to do something else and you don't want to hear something, you could at any time cancel or pause the speech. So what I am going to do right now is reply to her and say I spell these A I E S O O exclamation mark.

Um, first what you just heard is it's echoing every character back to you as I'm typing this. So I want to verify that this is what, you know, I want to reply back to her. So let's go ahead and read this again. I love it too. So this is just reading word by word. We have many other text commands and many other things you can play with. And since VoiceOver is on every Tiger system, I encourage you to give it a try. Later on, we'll show you where to get some of those VoiceOver commands.

So at this point, though, I want to go ahead and send this message off. So for most of us who are sighted, what we usually do is we just glance up and see, oh, there's a send button. Let's go ahead and take a mouse and click on it, right? Well, for people who are visually impaired, it's a little bit harder because they have to go and navigate through the entire window and find where the send button is. So what we have is this thing called ItemChooser to help them out.

Build ItemChooser menu, 25 items. Okay, it's telling me there's 25 items on this window alone. That's still going to take a little bit of time. So I'm going to do a little shortcut called the type. Type ahead and type S-E-N-D. One item, send button. So it found what I'm looking for. Send button. And notice that the VoiceOver cursor is now on the send button. So let's go ahead and press that, assuming we have network here.

Prep text. Patty, have you seen-- Good. Okay, we just sent it off. So let's go ahead and turn VoiceOver off. VoiceOver off. So what I just shown you is something we do on a daily basis. You know, it's kind of like waking up in the morning. Brushing your teeth, checking your email. Something we do and we take for granted.

But keep in mind, there's people out there who haven't been able to use Mac OS X because they couldn't even do something as simple as checking their email. Now they could with VoiceOver. The VoiceOver team has done a lot of work in TigerTime Friend to provide this alternative interface for users to use the computer. Plus, the accessibility team has worked really hard in the last couple of years to build up this great foundation of accessibility APIs so that your application could work together and communicate with the computer.

So... In a nutshell, most of the work has already been done for you. But you still need to do a little bit of work to complete that accessibility solution. And that's why you're here today. So, would you mind putting back the presentation slide? Presentation slide, please. Thank you.

So what do you need to do to make your application accessible? Well, first of all, support the Accessibility APIs. And Guy Fullerton will come up to show you how to do that shortly. Secondly, support full keyboard navigation. By that, I mean you should be able to take using the keyboard only.

Try to navigate around every widget in your application. If you could do that, that is full keyboard navigation. And of course, if you have any operation that's done by the mouse, please provide keyboard alternative for that. As an example, you know how in Finder you could take the mouse and drag and drop files? Well, what Finder has done is provide a keyboard alternative to do that exact same thing using copy and paste.

So, as you continue to develop your software, I want you to keep these three points in mind. If you leave the session not remembering anything else, at least remember these three points. Okay? So, of course, we have some tools to help you out. In the developer package, you're going to find these following tools: accessibility verifier and accessibility inspector. Later on in the presentation, we'll show you how to use the inspector.

But most importantly, please test with VoiceOver. Okay? The key diagram you see up above, it contains some of the VoiceOver commands. You can download that on the website listed below. So download this PDF file. Take about 15 minutes. Learn some of the commands. And at the end of the day, I want you to unplug your mouse, turn off your screen display. And of course, if you have a laptop, you might not be able to do that. So we have a VoiceOver command to allow you to temporarily disable your screen. So find out what that command is from the file.

I want you to use the VoiceOver commands you learned and without the mouse or the computer display, if you could get to every spot in your application and you could use your application without any problem, then you know you've done the right thing. Okay? So with that, I would like to go ahead and turn to Guy Fullerton.

No? Oh, there we go. That's much better. All right. So I'm going to explain everything you need to know to make your Carbon application accessible. And I'm going to start with a quick overview of the Accessibility architecture. Now, if you've been to these kind of sessions in the past, you've probably seen a diagram that looks a lot like this.

This represents the fact that the Accessibility architecture allows an assistive app, which is that app over on the left, to communicate with a wide variety of other apps running on Mac OS X, regardless of what framework was used to implement those applications. So assistive apps can talk to Carbon apps, and they can talk to Cocoa apps, and they can talk to Java apps, even though those three flavors of applications use fairly different APIs to generate their windows and views and menus and so forth.

So let's take a look at how the Accessibility architecture allows an assistive app like VoiceOver to communicate with your Carbon app. So VoiceOver might need to query your app about something. It might need to know what user interface element is at a particular point. So it's going to send a message across process, which is going to be caught by the green box, which is sort of the Carbon Accessibility implementation. And we're going to translate that into a Carbon event, this getChildAtPoint Carbon event. And we're going to route it to the right place in your app.

Now, part of your app is going to catch that Carbon event, and it may decide that that mouse point is over a push button. So that code will build a UI element, which is a framework-agnostic way of representing a piece of your interface. It's going to build that UI element, which represents the button, and will put it in that Carbon event as an output parameter. That UI element then is shuttled back across process to VoiceOver. And VoiceOver might think about it a little bit and decide that it wants to know more about that element. So it sends another message. message.

The Carbon Accessibility implementation catches that message and translates it into a Carbon event. In this case, the assistive app VoiceOver wanted to know what the role of a given UI element is, or in other words, what its purpose is. So we translate that into a get named attribute Carbon event. And that Carbon event gets sent to the UI element.

You'll see how that's possible in a little bit. The Carbon event comes with two pieces of information. It comes with the attribute that VoiceOver was interested in finding out about, in this case the role, and it comes with the UI element. You're going to need to extract some information about it.

So it's a button that you were asked about. You knew it was a button. So you put that role, a standard system role, which is K-A-X button role, into the Carbon event as an output parameter, which then gets shuttled back over to VoiceOver. VoiceOver might present some information to the user, and the user might decide that they want to press the button.

So VoiceOver issues another command to press that UI element, which again gets translated into yet another Carbon event, which goes to the UI element. And that Carbon event comes with the UI element, as well as the action that the user wanted to invoke, in this case press. And then it's up to your application to carry out the appropriate response. You might do something like call HIVU simulate click, which causes the button to flash and send out the appropriate Carbon events for control hit and commands and things like that.

This all works because we have a framework-agnostic way to describe a piece of the interface. What does that mean? That means you can make a UI element to represent virtually any part of this window. You can make a UI element that represents that segmented view there. You can make a UI element that represents just a part of that segmented view, that second segment in the segmented view. You can make one that represents some static text. You can make one that represents a custom view or even a window or even a menu or even the HI application object supported by Carbon.

So how is this possible? Well, on Carbon, a UI element is composed of two separate pieces. The first is a reference to any arbitrary HI object. Now, views, the Carbon views, HIV ref or control refs, are HI objects. Window refs are HI objects. Menu refs are HI objects. And our application object is also an HI object. So this reference can be to any one of those types of constructs. This reference can also be to a custom HI object subclass that you create.

Let me show you a quick example of that. So I was talking before that you can have an element that represents the entire segmented view. So in this case, the HI object in the element will be the segmented view's HI view ref, and the identifier will be zero. Now, zero has a very special meaning in terms of the identifier space. It always means the entire HI object ref. In this case, because we're trying to create an element that represents the entire segmented view, the identifier is zero.

Now, in contrast, if we want to represent segment two of the segmented view, the HI object ref is going to be the same. It's still going to be the segmented view. But the identifier is going to be two. Now, how do you know what the right identifier is? Well, the answer is it depends on your view implementation. Like I said on the earlier slide, the identifier space is completely private to the view implementation. But in a few cases for the system views, we publicize the identifier space. We know, for example, that your application may want to generate an element.

So we tell you that identifier one means the first segment, identifier two means the second segment, and so on. Most views, however, keep the identifier space private. The only other one I can think of where we publicize the identifier space is the tabs view. Because again, there are times when you might need to provide a description for tab two in a tab view.

Obviously, managing elements is very important in your accessibility implementation. You create them with the AXUI element create with HI object and identifier API. It's a very straightforward API. It takes the HI object ref and an identifier and it gives you back the element. The element is a CF type ref, so you can put it in dictionaries and arrays if you need to.

So, of course, to release it, you call CF release. Now, once you have an element, and you're going to get elements in most of your accessibility Carbon events, once you have one, you often need to decompose it to look and see what the HI object and the identifier are. So, we have the two APIs listed there to extract those two pieces of information.

Okay, so let's take a look at the effort required to accessorize a Carbon application. So you can think of the effort, the total effort involved, as a big pyramid of effort. Now, thankfully, The system has already built most of that pyramid for you. We already provide accessibility information for all the standard system controls, windows, menus, and so on. So your application is only left with a much smaller amount of effort. Let's go ahead and zoom in on that tip of the pyramid and talk about each of the steps you'll need to take.

The first step to becoming accessible is doing a bunch of basic work that isn't even necessarily accessibility work. The second step is to provide context about certain system controls when the system isn't able to give all of the information to the user. The third step is to remove extraneous UI elements from your accessibility hierarchy.

Next, you need to provide accessibility information for your custom views. And finally, some of you may need to provide accessibility information for the completely custom parts of your interface. These are things that aren't even necessarily implemented within HIView. So let's go ahead and look in detail at the basics. So like I said, these are things that aren't necessarily accessibility features, but they're features that accessibility can leverage and give a better overall experience because you implemented. A great example of that, like Patti said, is implementing full keyboard navigation.

That's going to make you a better overall application in terms of a user that doesn't want to use the mouse, but it's not strictly an accessibility feature. And a lot of other users, particularly Matt right there, really wants your application to be keyboard navigable. So you're going to make a wider user base happy when they use your application. Another great example is using window proxies for your windows.

If you're a document-based application, generally you want to have the little proxy icon in your window title bar so that the user can click and drag a document away. It'll also support a path pop-up, so you can find the path to your document and reveal it in the finder.

And that's very cool functionality, but it also happens to allow the window manager to add an AX document as well. So you can add a document to that window list for free. Similarly, support help tags and the accessibility subsystem can provide all of that information through the AX help attributes.

So once you've done that, you've taken the first step of building your accessibility pyramid, and it's time to move on to the second, which is providing context. Now, what do I mean by providing context? Well, let's take a look at an example interface here. This is a snapshot of the toolbar up at the top left-hand part of the Finder window. And you can see a number of buttons here.

And as a sighted user, I know what the purpose of all those buttons are for. I know that the leftmost one is a back button, and there's a right button, and there's a view switcher, and one takes me to column view and icon view and so forth. But to a non-sighted user, those buttons are just anonymous. There's no kind of textual description that VoiceOver can speak. So it's up to your application to provide something called an AX description that represents those buttons' purposes.

Another type of context you need to provide is for something like this. What you're looking at here is a screenshot from an application that takes shipping information from somebody. Maybe it's a web-based form for a shipping program. I have a static text off to the left which says "Shipping Address," and I have four edit text fields to the right that allow the user to type in their address.

Each of those text fields by themselves is kind of this anonymous entity. A non-cited user doesn't know what the purpose of each of those text fields are unless your application gives us some clue. It just so happens that your app already gave a cited user a clue by virtue of putting that static text piece on there. What your application needs to do is link those two sets of views together.

Each of those text fields needs to have an attribute added to it called the AXTitleUIElement attribute. The value of that attribute will be a UI element that refers to the static text. In a way, you're saying that this edit text right here, its title can be found over in this other element.

The reverse relationship also needs to be established. As you can see from the screenshot, it's actually a one-to-many relationship. You need to set up an axServesAsTitleForUIElements attribute on the static text field. Build an array of UI elements, one representing each one of those edit text fields. Use that array as the value of the text field. So what you're saying is this static text field serves as the title for several other elements.

So in general, you're going to want to take a look at your interface and find places where you either have views that don't have titles or you have views that have titles represented by some other view and link those together or provide descriptions. And I generally refer to these as auxiliary attributes because that's what the API is named.

So I'd like to bring up David McCloud, who is a card-carrying member of the Accessibility Task Force, as you can see by his T-shirt. And he's going to drive the machine as we add some auxiliary attributes to an interface. Okay. So the first thing we want to do is run our sample app.

Building, building, building. No errors, please. There we go. All right. I think I showed the screenshot earlier in the presentation. What we have here is a food pyramid window. Go ahead and click on one of the buttons real quick so we get that other static text. So this window is primarily that custom view that's drawing the food pyramid, and that's implemented as an Image Map view where we're reading in an Image Map XML file and determining where each of those button parts are and so on.

Now beneath it, there is some text that tells the button you clicked on. And beneath that, there's a separator line and a segmented view and a static text beneath it. So as we examine this interface, we can see several kinds of problems that we need to solve, several places where we need to provide context.

The most obvious is probably that big Image Map view. There's no text in the Image Map view. The user, an unsighted user, isn't going to know at all what the purpose of that is. So we need to provide some kind of description. In our case, it's a food pyramid group. It's a group around a bunch of buttons. So we will need to provide a description that calls it a food pyramid.

Similarly, each of the buttons inside that food pyramid has a text icon, or sorry, an icon representing its purpose. And again, a non-sighted user isn't going to know what that means. So we need to give descriptions to each of those button parts. Then if we look at the segmented view, it suffers from the same problem.

There's a bunch of pictures as the title for each of those button parts, but a picture isn't good at communicating information to a non-sighted user. So we need to give descriptions to each of those buttons. And similarly, the segmented view as a whole doesn't have a real title as part of the view, but the static text beneath it does provide some indication as to its purpose. So we need to link up the segmented view with the static text beneath it. So that's what we're going to tackle. We're going to tackle most of that in the first batch of source code.

All right. So David's going to add some code here, and I'll talk about what each bit does. So the first thing we're going to do is give a description to the image map view, which is the outer food pyramid concept. What we do is we call hiObject setAuxiliaryAccessibilityAttribute, and we pass in the image map view, which is the hiObject that we're interested in providing a description to. The next parameter is the identifier, which is zero, because we want to provide information about the whole image map.

The next parameter is the description attribute, which is a standard system attribute name. We pass that in because we're going to be providing a description. And the last parameter is the value that we want to give that attribute. In this case, we need to give it the name Food Pyramid.

Now, it just so happens that our Image Map View already has ways of querying the XML and pulling any text out of that XML that might be present. So we're calling one of our utility APIs to do that. So once we've made that API call, the Image Map View for the Food Pyramid has a description.

So the second thing we're going to do is we are going to associate the static text beneath the segmented view with the segmented view. First, we're going to look up the static text itself. We're going to call find control by ID. We happen to give it an ID in the nib so we can look up that static text. Next, we're going to build an element out of the segmented view.

So to build an element, we pass -- oh, I'm sorry, no, we're not building it out of the segmented view. Here we're building it out of the static text. We pass the static text view in, and we pass the identifier zero. So now we have a UI element that represents the static text view as a whole. Now what we want to do is we want to say that the segmented view's title UI element is this static text element we just created.

So again, we call hiObject, setAuxiliaryAttribute, pass in the segmented view, pass in the identifier zero. So we're saying associate this attribute with the whole segmented view. We pass in the attribute we want to give it, which is the title UI element, and we pass in the value we want to give that attribute, which is the element of the static text view. So now let's go ahead and implement the inverse relationship. Looks very similar. The only trick here is, like I said, the -- An element can only have a single title UI element attribute, but an element might have a services title for multiple other elements.

So what we do here is we are going to create an array of all of the things that the static text serves as a title for. In this case, there's only one thing in our array. It's the segmented view. So we create an element representing the segmented view. We build an array. We add the element to the array. And then we call HIObjectSetAuxiliaryAccessibilityAttrib ute to link the two up. And we pass in the services title for UIElementAttribute.

Now we're going to give descriptions to each of the segments of the segmented view. This is going to quickly reinforce the notion of identifiers other than zero. So we call hiObjectSetAuxiliaryAttribute. We pass in the hiviewref of the segmented view. And now instead of passing zero in to represent the whole segmented view, we pass either one, two, or three in to represent one of the three parts of the segmented view. Then we pass in the description attribute, which is the attribute we want to give, and then an appropriate value for those attributes. Okay, so let's go ahead and rebuild and rerun and take a look at that with the Accessibility Inspector application.

Make the font really, really big. Ah, yes. Good. Okay, so now if we hover over the food pyramid view, You can see that we've given it a description called Food Pyramid. Now there's some other problems with it. You see that it's an AX unknown, and we're gonna solve those problems later. But for now, you can see that we've actually given a description called AX Food Pyramid. We can look at the individual parts of the segmented view.

Oh, look at one of the, like the second segment. Yeah. So you can see those also have appropriate descriptions. In this case, Rollover hotspots for one of the segments. And now if we point to the static text beneath the segmented view, We're going to go ahead and log on. You can see that it has a top -- sorry, it serves as title for UI elements, which is an array of size one.

And if we lock on to that, we can use this little go to pop up to navigate to it. See that it serves as a title for an AX group, and if we go to that AX group, we will see that that is the segmented view group, and we can see that it also has the inverse relationship, which is the title UI element attribute.

Thank you, David. And let's go back to slides for a few minutes. Yeah, slides please. Thank you. All right. So that is the second step of the pyramid. The next step is to remove unnecessary elements from your accessibility hierarchy. Ooh, that's not right. Hey, they cut out a whole part of my presentation. That's not good. All right, so we lose the screenshots I had in the slide. Well, let's just go to a demo real quick.

I can show it off. Yeah, they did. All right, so what we're going to do, we're going to go ahead and rerun this application. And as he's bringing it up, let me talk a little bit about Why we need to do what we need to do. So a lot of you, when you implement your Carbon applications, you might use a user pane to group multiple elements, or sorry, multiple other views. You might do this just for convenience because you want to hide and show multiple views together.

Generally, that's why you do it. It's just for convenience or some kind of logical grouping. Now, unfortunately, the Carbon Accessibility implementation that's built in doesn't know what the purpose of any of these arbitrary views are. So we have to represent them in the Accessibility hierarchy. We can't make an assumption that there's no functionality there because we just don't know. You might have installed Carbon Event Handlers on it that either do something or do nothing. We can't make the right decision.

So we put those out there in the interface. Sorry, in the Accessibility interface. Now, unfortunately, that adds a lot of noise, literally, to VoiceOver. A user of VoiceOver that hits one of these user panes is going to find an unknown element, and they're going to have to navigate into it, and that's just a big pain. And so you can see that here. If we focus on the dairy item, if you look at the very top of the window, this shows the hierarchy that that static text item lives in. You can see that above the static text is an AXUnknown element.

That's a user pane that happens to group both the static text and the Image Map view of the food pyramid. So what we want to do is get rid of that unknown element. And we can do that by calling a very straightforward API. So let's jump into the code.

So the first thing we do is we find that user pane. We happen to give it an ID in our nib so we can look it up by control ID. Then we call hiObject setAccessibilityIgnored and pass true. And this marks that individual view as ignored. Now the cool thing that the Carbon Accessibility implementation does is it automatically promotes children of an ignored element to be children of that ignored element's parent. So the best way I can explain this is by re-running the application.

and showing you what changed. So if we look at the static text again, you'll see that that unknown element was removed. The static text is now a child of the window. And we can show this again by looking at the Go To pop up on the lock window. And if we go up to its parent, we'll see that we're at the window. So we do all this promotion automatically, which is kind of convenient. Okay, now I'd like to go back to slides.

Slides, please. Thank you. Okay. So we've now done three pieces of the pyramid and with far less slides than it took originally. So now we're gonna go ahead and we're gonna do the fourth piece, which is accessorizing any custom views or custom CDEFs you might have. The good news is that the HIV subsystem already provides a bunch of free information, a bunch of free accessibility information about your custom views.

We already know that a view has a parent and that a view lives in a window and that a view might have subviews. So we can already do things like provide their parent attribute, their window. We know their size and position so we can provide those through accessibility. But for a custom control, we don't know what your control's purpose is.

So it's up to you to provide a certain amount of other information. You need to tell us, or you need to provide the role for that view. You need to provide a value if your view shows some kind of value. You might need to supply a minimum or a maximum or various other things depending on exactly what your view does.

If your view is clickable or pressable, you probably also want to support Accessibility Actions. Probably is the wrong word. You must support Accessibility Actions to allow a user of VoiceOver to trigger that functionality. Some views, a small subset of views, need to also provide notifications, which I don't go into detail on here, but we've got documentation that explains how to do that.

So let's go ahead and go back to the demo machine and I can show you how to accessorize a custom view. So if we run the sample app again, actually on one of the earlier runs we already saw this too, if we look at the image map view that draws the food pyramid, we can see that it's unknown.

It has some information provided for it automatically by the HIV subsystem. It has a role and it has a role description, but it's a completely meaningless role and role description. It's unknown. It has a parent. That's because we know what its parent view is. It has a top level UI element, which is also the window. It's just an interesting way to refer to sheets and certain other things. And we know its size and position and so on.

We want this outer food pyramid to be a group around other elements. We need that guy's role to be AX group. And we need its role description to be something like group as well. And finally, we need that pyramid to have a set of child elements, one each representing each of those buttons inside of it. So we're going to go into the code and do all that.

Okay, so this custom view was implemented with HIFramework. So this is a little bit of a quick plug here. HIFramework deals with some of the grunt work having to do with HI objects subclassing and creating custom HIVs. We automatically take the Carbon events, we handle them, we call method calls on your objects instead.

So you can easily do, if you're a C++ user, you can do a C++ subclass of our TView class, just override the methods that you're interested in, and we're going to do the rest of the registering grunt work and things like that. So just for illustrative purposes, I want to take you into HIFramework and show you how some of the details on how the Accessibility Carbon events are handled. So this is the event handler in HIFramework that handles those Carbon events.

It's a Carbon event handler like any other. The first thing it does is it looks at the event, the incoming event class. If it sees that it's an Accessibility event, I'm actually going to walk over here because I can see David's monitor better than that one. If it's an Accessibility event, the first thing we're going to do is we're going to extract the UI element out of that event. Now, don't be scared by this code. It's just a templatized function. HIFramework's just doing some magic behind the scenes. Really, this is just a call to get event parameter.

Once we have the UI element, we extract its identifier as a convenience. You'll see that we end up passing this identifier to the various C++ methods in our custom view implementation. Then we look at the kind of the event that's coming in. If, for example, if it's an Accessibility get child at point two, we extract the parameter from the event. This time we're extracting the global mouse location that's coming in in the event that the assistive app was interested in using.

And finally, we're going to call a method on your object, your custom view object, called copyAccessibleChildAtPoint. Now, if your custom object had a child object at that point, we're going to go ahead and stick that back into the event as a return parameter. So you can see that if you just derive from HIFramework, we're going to do a lot of the parameter shuffling convenience things for you. And you can just sit back and be a C++ client and have an easy life. The rest of the code is very, very similar. We're basically pulling some parameters out, maybe massaging them a little bit, calling a method, and then stuffing the response back in the event.

So now, what we need to do is we need to change the values that our custom view is returning-- sorry, change the default values that are getting returned for us for our custom view's role and role description. And we also need to provide the value of its child attribute.

Now, to do that, we're going to need to override certain accessibility Carbon events. Now, HIFramework, by default, does not install the Carbon event handlers on views for efficiency's sake. So the first thing we're going to do is call activate interface, which is just a shorthand around installing the accessibility event handlers on our view object.

Now, once we've done that, we can put some prototypes in for the methods we're interested in. The first one is the getAccessibleAttributeNames method. This method is going to be called when an assistive app causes a Carbon event to be sent that's requesting all of the attributes that a given view supports. The next method we're going to do is getAccessibleNamedAttribute. This method is called when it needs your view to supply a value for a particular named attribute. All right, so let's go to the implementations of those.

So here's the first one, get accessible attribute names. So this method receives a couple different things. It receives the Carbon event call ref and the actual event itself. It also receives the UI element to which the event pertains, and it receives the 64-bit identifier that's been pulled out of the event, just as a convenience. Now, it's also receiving a CF mutable array ref. This is the array. It's a mutable array that you're supposed to fill out with all of the attributes that your view supports.

So what we're going to do is we're going to look at the incoming identifier for this call. And if it's zero, that means it's a request for all of the accessibility attributes for the view as a whole, meaning that food pyramid group. So this is exactly the case we want to handle.

Then we call next event handler. This allows the default HIV implementation to pre-populate that array with all of the attribute names it supports. In this case, size, position, and so forth. Then we add the single attribute that we are providing, which is the children attribute. And then we return no error and we're done.

So just a note as he's pasting this other code in, the HIV base class already added the, The role and role description attribute strings to that array. So that's why we didn't have them there. When we called callNextEventHandler, those attributes were added to the array. Okay, so this is the getAccessibleNamedAttribute method. This is called when we need to generate a value for a particular attribute. Again, we get the call ref, the event, the element, and the identifier, but the new thing we get here is the attribute name that we need to supply a value for.

Again, we look for identifier zero 'cause we're supplying the information for the whole view. Then we look at the incoming attribute name and compare it to the ones we're gonna supply values for. In this case, if we get the children attribute coming in, we know we need to supply the array of children that we have.

Now, like I said before, our custom view is an ImageMap view that has a number of buttons in it. We need, our children are all of those button elements. So what we're gonna do is we're going to iterate, sorry, first thing we're gonna do is we're gonna create an array.

Then we're gonna iterate over all the button parts in the ImageMap, and we're gonna build up an element to represent one of those button parts, add it to the array, move on to the next button part, create an element, add it to the array, and so forth. And then finally, we're going to call setEventParameter to stick that array into the event as an output parameter so that it can get sent across the wire to the assistive application.

Actually, go back to the create with HI object identifier. I wanna talk about that very, very briefly. So we're passing our own HI object ref, which is our own HIV ref of the ImageMap view in here, and we're passing non-zero identifiers. So we've just set ourselves up. We've just defined our own 64-bit identifier space. We're saying that identifier zero represents the whole view, identifier one represents the first button part, two is the second button part, and so on.

Okay, so the next attribute we're going to supply a value for is the role. Now by default we got the unknown role, but we wanted to have the group role. And in fact we want to use the system standard group role. So what we do is we go and look up in one of the AHA services headers, axroles.h, and we find the group role. And we add that to the event as an output parameter using set event parameter. And that's all we have to do.

Now whenever we provide a role, whenever your view implementation provides a role, it is your responsibility to provide a role description. And a role description is a human readable or a speakable string describing that element's purpose. Now anytime you use a standard system role, you can access the standard system role description by using the HICopy Accessibility Role Description API. So here we're just generating the standard role description string for group, stuffing it in the event as an output parameter and returning no error. Yeah, so that's it. So let's go ahead and recompile and show what we added.

Okay, so now if we hover over the food pyramid view, we can see that its role and role description have changed to something suitable, in this case group. We can also see that it has an array of children. - "Six children, one for each of those button parts." And if we go to that pop-up, we can see that they're all unknown. So this is something we're going to have to fix later. So with that, let's go ahead and go back to slides and I can talk a little bit about the next step of the pyramid.

So we've built the fourth step. The fifth step is accessorizing your completely custom interface in your application. Now, what does completely custom interface means? Well, the honest answer is that the line between the fifth step in the pyramid and the fourth step of the pyramid blurs a little bit. What we just showed, we accessorized the outer part of a view, but we didn't accessorize the inner part of a view.

Now, the inner parts of that view, the actual part codes, count as completely custom interface because the HIV subsystem can't do a whole lot with parts. We don't know what rectangle corresponds to a certain part, what its functionality is, and things like that. So if you're accessorizing certain parts of a custom HIV, that counts as completely custom interface.

Another example of completely custom interface is when your application is using your in-house view system or a third-party view system like Power Plant. The toolbox doesn't know anything about Power Plant. We can't make assumptions about Power Plant classes, and we certainly can't provide accessibility information for it. So it's up to your application to provide the accessibility information for those custom views.

Now, finally, the most-- Probably the hardest thing to accessorize is a completely custom interface that's procedurally driven. Maybe you've got old school wait next event code. You're watching events come off the queue. When you see a click come in, you do some special stuff to handle clicking and so forth. But you don't really have an object model that defines your interface. So that's also a completely custom interface.

The next thing you need to do is make sure it's inserted into the accessibility containment hierarchy properly. By default, the toolbox will build an accessibility hierarchy that looks just like the Windows view hierarchy, essentially. A window has child views, those views have children, and so forth. You need to make sure that your HI objects and the elements you build from those HI objects are inserted into that containment hierarchy properly so that they can be found during hit testing and so forth.

So in particular, we have events that get sent out to find a child at point. You're probably going to need to install Event Handler on your parent so that you can intercept its Get Child at Point functionality and then provide your child, or provide the element you're interested in serving up.

Of course, your element must have that parent as an attribute. And we also have an event that is sent out to follow the focus chain, so you want to hook into that as well. And finally, your view might also need to support notifications. So let's go into the code and let's accessorize the completely custom part of our sample app. So we can switch monitors over to the demo.

Okay, so let's go ahead and rerun. Oh, prototypes first. All right, so David is just going to paste in a few prototypes to do what we want to do, and I'm going to discuss these when we actually get to the implementation. Oh, we're going to do, I want to go do a run first. So actually take out the prototypes. Or just do a run without building.

Okay, so now if we look at the Food Pyramid group and lock on and then tunnel into one of its children. This is what you get. This is not good. A couple problems here. First of all, it's got a role and a role description of unknown, and it's got a parent of null.

But even worse, its hierarchy is empty except for itself. Now, all this is just not good, so we need to fix all this. Each of these button parts is basically just like a push button. You can click it, and when it's clicked, some functionality happens. So we need it to support the press action for one. Of course, it's visible, so it's got to have a size and position.

It's in a window, so we need to provide that. We need it to have an appropriate parent. And then finally, we need to make sure that the parent, the Image Map View, has each of those button parts as children. And we already did part of that, but there's a little bit more we're going to do in the code. So let's jump over to the source.

So this is the Get Accessible Attribute Names method that we already implemented to provide the attribute names for the group. Now what we're going to do is we're going to provide attributes for each of those button parts. Now here's where the identifier gets cool. We already kind of defined implicitly that while identifier 0 means the whole group, identifier 1 means the first button part.

So we look at the incoming identifier, and if it's in the range of 1 to the number of button parts we have, we're going to take action. We're going to call the next event handler. That's going to allow the HI object mechanism to add three attributes to the array. It's going to add the role, the role description, and the parent attributes to the array. So we're just going to leverage that free functionality.

And then we are going to add a bunch of other attributes to the array manually. We're going to give it a description. Now, here I want to reverse to something we talked about earlier. Earlier on when we looked at the interface and found places where we needed to provide context, we determined that we could provide context sort of as a client in the application. We had a static text that we knew we put next to a segmented view, so we associated them. And you'll find that in a lot of your applications interface, you're going to link things together and give things context as a client.

Now, in this case, we're going to give it context as an implementation. We know because we're an XML based image map view that there's text in our image map XML that describes what each of those buttons do. So our implementation here is going to be able to provide the description attribute. Then we're going to give things like size and position in the window and so forth. So now let's go ahead and provide some values for those attributes.

Paste in a little bit of code. Yeah, so from the peanut gallery, it looks like a lot of code. No, this is a good point. It looks like a lot of code, but it's really not a whole lot of code. It's basically a big switch statement. Put a lot of comments and a lot of white space in the sample code so you can understand what we're doing. But really, each of these is just comparing a string and then stuffing an event parameter in an event. So it's really simple stuff. So again, we're providing attribute values for button parts of our view as a whole. So we look for an appropriate identifier.

Next, we look at the incoming attribute string. So in this case, we've got the description string. We're gonna give them an appropriate description. Again, we have a utility routine inside our Image Map view that can look up the appropriate description from the XML. So we generate that, stick into the event as an output parameter, and we're done.

Another example is the parent attribute. Here's something a little bit interesting. We know the parent of each of those button parts is the Image Map View as a whole. So we create an element representing the Image Map View as a whole. We pass in the view and we use identifier zero, and the resulting element is stuffed into the event as an output parameter. Now we know each of our button parts is basically a push button, so we need to give it the standard system button role.

We look in the AX Services header in ax-roles.h, find the standard button role, add that to the event as an output parameter, and we're done. And we're going to give it a role description using the HICopy Accessibility Role Description API to look up the standard system role description and so on. The only other one I want to talk about is the ax-position attribute. Now, when you provide a value for the ax-position attribute, it must be the global screen location of your element.

Because we're running in a compositing window, and because we've written this view to be a compositing view, we can very easily generate the bounds of one of these button parts in view local coordinates. So we do that, but now we need to translate those view local coordinates into global coordinates. We've added a convenience API in Tiger called HIPointConvert, which can do exactly that. So once we've translated that point, we add it to the event as an output parameter and return no error.

Okay, so now this code that David selected here actually has a bug in it in the version you can download. This is the correct code. The version you can download today I think always says that a button part is not focused. This code is the right code. We actually provide, we tell you a correct indication of whether a button is focused or not. We're going to try to get our new code out there on the web as soon as we can after the show.

Yeah, so now let's start doing some other interesting methods to support Various other functionality. All right. So this is the isAccessibleNamedAttributeSettable method. This method is called when your view receives a Carbon event when an assistive app wants to know whether a given attribute is settable. Now, whenever you have a focusable element, focus is implicitly settable. You should be able to set its focus to true, but you should not be able to set its focus to false. You can only focus something. You can't unfocus it via the Accessibility API.

So we need to announce the fact that our AXFocused attribute is settable, and we do that with this method. So again, we look at the identifier space. We make sure we're getting an identifier that represents one of these button parts. If so, if we're being asked about the focused attribute, we return a settable value of true, and HIFrameworks is going to stuff that into the event as an output parameter. There was one other note I want to mention about this.

Um... Right, so we only handle one attribute in this function, and that's because HIToolbox, by default, assumes all attributes are unsettable unless you explicitly override that via the Carbon event or the method if you're using HIFramework. So you only need to handle this method or implement this method for settable attributes.

Okay, so we know focus is settable, so let's go ahead and make it settable. So we implement the setAccessibleNamedAttribute method. This corresponds to a Carbon event that comes in from an assistive app when an assistive app says, "Please set this attribute's value to foo." So we look for the appropriate identifier.

We look for the appropriate attribute. We extract the value out of the Carbon event. Now, the incoming value could be potentially anything. You never know what an assistive app is going to send you. VoiceOver is really good, but who knows what other app might be trying to stuff values in with.

You can call getEventParameter and just ask for the type you like. getEventParameter will automatically coerce the data if it can. Otherwise, it'll return an error. So if we get the appropriate Boolean value, and if it's true, we go ahead and call setKeyboardFocus on ourselves, passing in the view and passing in the identifier, which is a control part code.

And that just happens to be how our view is implemented. We know how to focus control part codes. But if focused is false, we return a special error, which is K-A-X, error illegal argument. You return this error any time you get a set request with a value you can't handle.

Okay, before, we already gave the Image Map View an array of children elements, but that's not the only way. We need to do more to completely splice it into the hierarchy. We need to make sure that that Image Map View can report a child at a given point of itself. So here we're going to implement the Copy Accessible Child at Point API, which corresponds to the Get Child at Point Carbon event that I showed in earlier some of the slides.

The important parameter here is the In Where parameter. That is a global screen location that is being queried. So it's your views or your element's responsibility to return the first order child of itself at the given point. Don't tunnel down into grandparents. Just return the first order child of yourself. Now, in this case, because we are the Image Map View here, we're going to need to return child of ourselves. So we're looking at identifier zero coming in.

We first take that global screen location that came in, we translate it to view local coordinates because we just so happen to have our own internal hit test function that can generate a child part and a view local coordinate. We generate the child part, and if it is indeed in one of our child button parts, we build an element out of that by calling the long API there. We pass in our HIV ref and we pass in the appropriate identifier, and that child gets returned in an output parameter and stuffed in the event by HIV framework.

Hold questions to the end, unless you found a bug. I have a question about that. We'll come back. We can show this during Q&A. So the next method is get accessible action names. Now, each of our button parts is a button. It needs to be pressable. And you do this by supporting actions.

Now, just like supporting attributes, your element needs to be able to provide all of the actions that it supports. In this case, there's only one action we need to support. So we look for an appropriate identifier coming in, and then we add the press action to the output array. And it just says, "We support the press action," and we're done there.

Anytime you support an action, you need to be able to provide a human-readable or speakable description of what that action's purpose is. So there's this Copy Accessible Named Action Description API, which corresponds to a Carbon event that HIFrameworks is handling for us. So we look for the appropriate identifier.

And since we support the Press action, and the Press is a standard system action, you can generate the standard system action description with the HICopy Accessibility Action Description API. Now when the press actually happens, when VoiceOver actually issues the press, "Please press this button, Command, to you," we send out another Carbon event, the Perform Named Action Carbon event. HI Framework in turn calls this method on your view.

Oh, that's right. So this is another example. The sample code that you can download today-- Oh, it is the focus one. All right. Ignore that. So what we do is we look for the appropriate identifier, and we know we only support one action, so we don't even bother looking at the incoming action name. And then we call HIViewSimulateClick, which will cause the part to flash, causes all the Carbon events like KEVENT_CONTROL_HIT and KEVENT_COMMAND_PROCESS to get sent out.

Maybe talk about that. Well, okay, so since we're rebuilding here, most, maybe about half of the focused code in the sample app you can download is wrong. I already talked about some of it, but specifically the code in SetAccessibleNamedAttribute, if I remember correctly, it... either didn't exist or it allowed unfocusing.

I forget which. Anyway, we're going to correct that. You'll be able to download new code soon. All right, so if we launch UILM or Accessibility Browser, and hover over one of the button parts. We can now see that it makes a lot more sense. Its role is AX button.

It's in the AX group, which represents the food pyramid itself. It's got the appropriate parent. It's got an appropriate description, size and position and so forth. So that was about it. That is how you accessorize the completely custom parts of your application. So let's go ahead and go back to slides.

So now that you've implemented the final tier of the pyramid, you've achieved full accessibility power. Not only is this going to allow you to work with VoiceOver, but it's going to allow a whole suite of other assistive applications to work with your application. It can offer improved scriptability, and if you saw the State of the Union, you saw some of the other types of applications that can leverage accessibility features.

Now, you've got all the tools you need to accessorize your application, but it's time to make a decision. If you have a lot of completely custom stuff in your app, it's quite a bit of work to do all the accessibility gunk for it. I mean, I have to be honest here, older code takes quite a while to accessorize.

However, we can offer you free accessibility support through HIView. Now, it's going to take some effort to adopt HIView as well, but if you're already planning on spending a certain amount of time implementing accessibility, why don't you spend some of that time implementing HIView, get a bunch of accessibility work for free, and then also set the groundwork for doing things like compositing mode and generally making your application a lot more modern.

So, think about those tradeoffs a little bit, and if it makes sense, adopt HIView as part of your accessorization process. Finally, we've got HIFramework, which is our C++ wrapper around HI object instantiation, HIV subclassing, and so forth. It's what the sample code uses, and it makes accessibility a heck of a lot easier. So you might consider using that as well.

So we have just a few minutes left. I'd like to bring Pattie back up, and she can run VoiceOver with our finished Image Map view and show that everything is working well. Russell, go ahead and-- yeah, there we go. Which app? This one? No. So go into here.

So that's the inaccessible one. Alright, so I'm going to launch the two of them side by side. First, I'm going to run VoiceOver on the accessorized version. Of course, we have to press Command + F5. VoiceOver on Finder Accessible Window Back Button. Image Map View Food Pyramid Window Window Fasten Sweeps Button. Okay, that's the accessorized version, which I'll show you shortly. Image Map View Food Pyramid Window Window Unknown.

Okay, so we're on the application on the left, which right here, which is the unaccessorized version. And I'm taking my VoiceOver on the application right now. And let's listen again to hear what the VoiceOver cursor is on. Unknown is in the VoiceOver cursor. That's not very useful, is it? Well, let's keep going and see what else we find out about this application.

Select Radio Button. Radio Button. Radio Button. All right, so far, I know -- the only thing I know about this application is there's an unknown and there's some radio buttons. I don't even know anything else about this application. So for me right now, this application is not very useful.

So let's go ahead and see the version that has been accessorized, and let's see the difference. Image Map View Food Pyramid Window Window Fasten Sweeps Button. Okay, so again, if you listen again. Fasten Sweeps Button is in the VoiceOver cursor. All right, so at least right now I have a sense of where my VoiceOver cursor is. So let's go ahead and navigate around. Dairy Button. Meat and Eggs Button.

Good, so now I have a better sense that there are lots of buttons in this food pyramid, and each of them belongs to a food group. So let's go ahead and interact with that. Press Meat and Eggs Button. Good, now we can interact with the buttons. Let's go ahead and see what else is in this application. Vegetables But Fruits But Grains Button. Meat and Eggs.

Invisible Hot Spots. Select Radio Button. Invisible Hot Spots Radio Button. Oh, so those radio buttons we heard earlier, now I have a better sense what they are. For example, the one we're on right now is the Visible Hot Spot Radio Button. And of course, I can also press it.

Press Visible Hot Spots Radio Button. So suddenly this application is much, much more useful. So just by making the application accessible, now VoiceOver can run with it. VoiceOver off. Go ahead and have Guy back up. Thank you, Patty. All right, now if we can go back to slides real quickly. Try to get to the last few.

Okay, so some other views, or some other, I said view far too many times this session, some other sessions that might be of interest. Tomorrow we talk about how to build a custom HIV from the ground up. So if you're one of those applications that has old code that's thinking of modernizing, in part to get accessibility support, I highly recommend you attend this session.

We're covering a few topics that we haven't talked about in the past in our HIV sessions. We have an accessibility feedback forum Thursday morning, and we're running out of time here, so I think you're going to have to save your questions for then or catch us after the show, or sorry, catch us after the session.

And finally, on Friday, starting at 2 p.m., we have an accessibility lab. So if you're having troubles accessorizing your app, or if you want a little bit of help or pointers, feel free to drop by then. We're going to have VoiceOver engineers here, some engineers from my team here to help out with the HIV aspects. So please drop by. by and bring your questions, bring any problems you've found.

If you have further questions, you can contact Mary Beth Janes, who's our Senior Segment Manager. And we also have an Accessibility Dev mailing list, which is monitored by several of the accessibility engineers and the VoiceOver engineers. If you have questions after the show, feel free to post onto this mailing list, and we'll do our best to help you there. Right now on the developer website, we have documentation, conceptual documentation and API level documentation for accessorizing Carbon applications. So I urge you to check that stuff out. The sample code that we showed today is also up there.