My name is James Dempsey. I'm a software engineer on the Cocoa Frameworks team, and I focus on application accessibility. So what are we going to be talking about today? Well, we're going to do a brief accessibility overview. Then we're going to take a look at some of the major use cases of application accessibility, both for users with disabilities, as well as how application accessibility is very useful to power users of your application and how you yourself can use application accessibility as part of your development process. Then we'll walk through the accessibility architecture on Mac OS X and spend the bulk of the time talking about how to make your application accessible. So let's get started.

So let's consider for a moment the case of a person with a visual disability using a graphical user interface. Now, there needs to be some alternate way for a user to interact with their computer. So they need to have the interface described. They need some way to navigate and interact without using a mouse. And that user also needs to find out when things happen, like when windows and sheets come and go.

Now, software that provides this alternate interface to a computer is commonly called a screen reader. Mac OS X, starting with Tiger, has included a built-in system-level screen reader called VoiceOver. Now, the reason why I'm focusing on VoiceOver is that the way VoiceOver works to provide this alternate interface, it's really a collaborative effort between VoiceOver and all of the other applications on the system. Those applications provide accessibility information about themselves, and VoiceOver takes that information and provides a standard spoken interface to all of the applications on the system, or even a refreshable Braille display interface, in fact.

So another reason why VoiceOver, we're talking about VoiceOver, is that VoiceOver ships with Mac OS X. Every version of Tiger, every version of Leopard, in the future every version of Snow Leopard has VoiceOver built in. And so 16, 17 million copies of this software are already out there and there's an ever increasing number of users of your applications whose only way of interacting with their Mac is through VoiceOver. Now, we have one such user here with us today, and I'm very happy to introduce Dean Hudson, who is a QA engineer on the VoiceOver team. And Dean is going to give us a little walkthrough and tour of VoiceOver. Take it, Dean.

Next Dim button. Idle image. Mute button. 100% volume horizontal slider. Full volume button. Text list button. Album list button. Cover flow button. Edit text blank. Now, as I was navigating, you were watching the rectangle and you were seeing what was spoken in text. And you might think, boy, that's a lot of information.

I mean, geez, you know, can't we just say button? But as a blind user, that information is very important to me. If I land on something that just says button, I don't know if that's a mute button, a play button. It just says button. I land on something that just says edit. It could be a secure edit text field. It could be a regular text field. That information, description, gives me a bit more information about what's going on and helps me use the application.

More effectively. So I'm going to try and do something interesting and play a song, I think. Sources Table 1 Interact Audiobooks Live Radio Store iTunes Store Now I was just going to play a song, but Karl said that's too boring. So let's try and search for something on the store. Stop interacting. Edit text blank. See if I can.

SIG AED DEED AND D space D N A E A L E D sign D wield clear search button sign D sign D wield D D E L A E A space D E N G G A E D space A N D space S E A E D sign and seal clear vertical split. I'm your host, James Dempsey, and I'm back and not ashamed to cry.

James? Oh, I'm sorry. Shoot, I forgot. Mail. I was getting too excited. Just having too much fun. Finder. Mail. Okay, so let's do something serious. Horizontal split messages table. Mail, you all know, it's a very, very popular application. So I'm just going to use this to send a message, something that users typically do.

And again, pay attention to some of the feedback that I get. I'm also going to get out of this slow boat mode. We're going to speed it up the way a blind user would use it. 60%, 78%. Window, new message. To edit text, K, C, scram, m at apple.com.

Subject, edit text, customize header, menu button, subject, edit text, insertion at beginning of text, L, U, N, C, H, customize header, Gmail, pop-up button, text.

Mac account off. Menu check. Gmail. Gmail. Let's go. Text. Messages table. No selection. Messages table. That probably won't send. Horizontal splitter. Ah, it did. Sweet. All right, James.

Thank you very much, Dean. So Dean just showed us using VoiceOver. And you may be wondering, I had mentioned that it's a collaborative process between VoiceOver and in this case, the iTunes application, the mail application. Let's talk a little bit about that interaction between the two before we jump into what you need to do in your application to provide accessibility information.

What's going on is it's a fairly straightforward client-server relationship. VoiceOver isn't doing any odd things within your process. It lives in its own process space. And Mac OS X has a very well-defined public accessibility client API. That's shown in this diagram in green. The prefix we use on those APIs is AX. You'll see that a lot. AX for accessibility.

I kid you not. Those APIs allow VoiceOver to get information from your application, all the applications on the system. So all the apps, in essence, are their own mini little accessibility information server. And the red sections up there represent the fact that that server infrastructure, the piece that can respond to accessibility requests, is built in to the Cocoa framework, built into the Carbon framework.

So your applications, if you have a Cocoa app that you've written, it's already doing some accessibility, even if you didn't do a darn thing. Now let's narrow it down a little bit to one application, the back and forth between two. You'll notice that VoiceOver has changed to assistive application. That AX API, that client API, is a public API. Therefore, not only does VoiceOver use it, but in fact there are other applications that can use it.

And we call that general class of applications assistive applications. And so an assistive application might want to talk to your Cocoa application. And to start off, one piece of that client API takes the process identifier of some app and calls into that and says, give me back the application UI element for that item.

And on the client side, the UI of your application is represented in a very lightweight but rich way. It's represented by these little UI elements ref

We also have a checkbox that has no children underneath it. It's kind of a leaf in the tree. But it does have a parent. Also, what window it happens to be in.

And this thing called a top-level UI element. And this is, sometimes we'll have things in a sheet or a drawer or a window. And the top-level UI element is which one of those three things it happens to live in. Those items happen to be children of the window that they're attached to. We also have geometry.

So the screen position and the screen size. This is what VoiceOver was using to draw that VoiceOver cursor that was following around what it was speaking. And then information about this control itself. Like its value. Is it checked or not? Is it enabled? Is it enabled or not? Is it the focused element? Does it have the keyboard focus? And finally, very important, a title. So that a VoiceOver user knows what the heck this checkbox is for.

And so with these UI element refs, attributes, and the other functionality, there's a great deal that an assistive app can do interacting with your application. And of course, VoiceOver is just one of many assistive applications. In fact, on Mac OS X, we ship a number of features and technologies that use this client API.

Now, another one is Accessibility Tools. We'll take a look at Accessibility Inspector in a bit, which uses it to give you a more raw, unprocessed look at the accessibility information that's being returned by your application. But also, user-level features. GUI scripting, which allows somebody to write Apple scripts or using the new scripting bridge, actually a number of scripting languages, to basically implement the accessibility of your application.

So, for example, we can actually control the user interface. Now, advanced users who are building automated workflows love this feature. But for developers, this is extraordinarily useful for creating automated test suites that drive your UI the way a user would. So you can, and we're actually going to have Karl, he's going to show you that in a little bit.

In addition, as a user-level feature and a developer-level feature, Automator has a feature called WatchMap. WatchMap is a feature that allows a power user to record and play back certain UI actions as part of an automated workflow. That same technology underneath, based on accessibility, is also used in instruments, which you heard about a lot yesterday. The performance tool, not only can you just drive it yourself, in instruments you can record a series of UI actions, and then perform it yourself.

But you can play them back again and again and again, using various profiling tools, to be certain that you are testing the exact same thing over and over again. Which is especially handy when you have that one case that you need to do like these 17 steps in an exact, and then you try to debug it and you always forget step 13, and so you have to restart over. Very handy with instruments to be able to catch that thing by recording and playing back your actions.

All of these rely upon the accessibility in your application, that it gives out complete information and valid information. Now, to show you a little bit about how you can, your application can take advantage of this using audit, or excuse me, using GUI scripting in instruments, I'd like to introduce Karl Schramm. He's an engineer on the VoiceOver team to walk us through automated testing in instruments. Thank you, James.

Good morning everybody. So in Dean's demo, we got to see how VoiceOver can interact with rich applications like iTunes and Mail because they're accessible. However, VoiceOver isn't the only assistive application. I'm gonna show you two such tools that you can use in your development process as soon as you make your application accessible. So the first thing I'm gonna show you, let's get rid of that. Am I live? Okay. Technical difficulties.

Great. And we're back. Thanks. Okay. So the first thing I'm going to show you is how you can make automated use cases using AppleScript GUI scripting. And the application we're going to be automating is Dicey. Dicey is available in the developer examples directory of your Snow Leopard installation, and it's this great single-player dice game.

You play Dicey by rolling the dice three times and selecting the dice to maximize your score. So I'll just pick the threes, and we'll try for a full house. Oh, we're short. So we'll just mark this up to chance. So that's Dicey. Now for our script, all we're going to do is click on the roll button and select the last die here.

Now, in order to do this, we're going to have to learn a little bit about Dicey. So we're going to take the Accessibility Inspector and the Accessibility Inspector works by hit testing whatever's directly underneath the mouse. So we can see as we move over these dock items, we can see the information changing in the Accessibility Inspector window. And we can lock onto an element. I'm going to lock onto the roll button right here.

This top part of the Accessibility Inspector window shows the accessibility hierarchy. We can see that this is an AX button, its parent is the AX window, and of course its parent is the Dicey application. Now below this we have all the various attributes for this button. We can see that its title is role, it has a size and position, and a bunch of other stuff.

Now below this we have all the actions we can perform and this button only has one action and that's AX Press. We can actually perform it in the interaction window here. So we'll select AX Press, perform it, and we see the dice roll. We can also traverse the hierarchy by going up to its parent and the highlight's going to change to show that we're now on the window and we can return to the roll button by going to the children of the window and selecting the roll button.

So now we know enough about the roll button, let's look at these dice over here. And as I move over these and lock on to the last die, now the highlight's red and the die is red so it's kind of hard to see that we have this highlighted.

But if I perform an AX Press, we can change the selection state so we know we have the right die. So the die is accessorized as an AX checkbox. and its parent, The Playing Field, is an AX group. It's important to note that this is a custom NS view and we wouldn't know anything about this had it not been for the fact that Dicey's been made accessible.

So let's go up to the parent, the playing field. And we'll return again, look at its children. Now, the children are returned in the opposite order in which you see them. And that's just the way Dicey returns them. So the first child is the first checkbox, which is the last die. And we know we have it selected. Okay, so now we know enough information to make our script. And I have that script right here.

So the first thing we're going to do is tell the Dicey application to activate. And then we're going to talk to the system events application. And this will enable us to communicate to Dicey through the Accessibility API. So we're going to talk to the Dicey process. We're going to talk to window one, which is Dicey's only window. And we'll click on the button with the title of role.

and we're going to tell group one, which is the only group, to click on checkbox one, which is the last die. So let's clear the playing field before we start the script. Now you also may have noticed that I made a couple calls here to the say command because we're going to use the Alex synthesizer to annotate our demo. So here we go.

First, let's click on the roll button. Now let's click on the last die. Okay, we're off to a good start. So using this, we can make a script that will play an entire game of Dicey. And I have this script right here and we're going to run it.

Now this script just plays the game randomly, an entire game of Dicey, and you can use this as a basis for other scripts. For instance, you can make it into a regression test by adding validation checks for the game state or you can make it into a fuzz test by trying to perform some illegal actions or set some illegal values. And then once you've made this suite of tests, you can run them at the end of your build process on your daily builds or use it as a validation before a major milestone shift. So that's how you can make automated use cases using Apple Script GUI scripting.

Next thing I'll show you is Instruments. Now as James mentioned, Instruments UI recording feature utilizes the Accessibility API to record and play back all your user actions just like Automator's Watch Me Do feature. Now, because of this, the more accessible your application, the more robust your recordings can become.

So here I've launched the Dicey Xcode project. Now, I could launch straight into Instruments. However, Instruments is just so nicely integrated into Xcode that I can launch it while I'm coding. I go up to the Run menu of Xcode, select Start with Performance Tool, and we're going to go to the UI Recorder Instrument Template.

And as soon as I click on this, Instruments is going to launch, and it's going to launch a debugged version of Dicey, and it'll begin recording all our actions. So here we go. Here comes Instruments. There's Dicey and we're off and recording. Just going to do a few simple actions.

So we can see in the track view all our actions have been recorded. Now, Instruments has a whole wealth of instruments that you can tune and tweak all sorts of aspects of your application. You can look at graphics performance, I.O. usage, garbage collection. We're just going to look at memory allocation and leaks. So we'll grab these two instruments, drop them in over here, and we'll kick off leaks at about 10 seconds.

And as soon as I hit the drive and record, it's going to play back everything that we've recorded, and the object alloc and leaks instruments will begin recording trace data. So Instruments is taking control of the mouse, performing all our actions. We can see the object allocations in the detail view of Instruments. Now leaks is prompting me for my password.

And we'll stop it. And we can see our object allocations are okay. There's no spikes. However, we do have some leaks. So if we go to the detail view here and get some more information, we can get the stack trace for where this object was allocated. And it's in the Dicey project. And as soon as I double click on this, Dicey's going to bring us back into Xcode where the object was allocated. And it says counted set right here. If we search for it, it's only used twice and it's never released. So we do have a leak.

So if we go back to Instruments, we can save this file and pass it back to the developer and they can use this file to reproduce the issue and fix the issue. Later on, you can use the same file to verify that the bug's been fixed. So as you can see, the Accessibility API serves as a foundation not only for great user technologies like voiceover, but great developer technologies as well. Back to you, James.

Thanks a lot, Karl. So Karl has just showed us two ways that you can take advantage of the accessibility information that you put into your application. You can use it in your own development process. Now, we've taken a look at sort of the client side of things, but by now you've got to be wondering, well, what do I, tell me what I need to do in my application. So, for starters, every one of those UI elements that an assistive application uses gets from that client API, maps back to some Cocoa object in your application.

It's a Cocoa object that implements a 14-method protocol called NS Accessibility. And in fact, all of the heavy hitters in the Cocoa framework, NS application, window, views, controls, cells, they already implement this. And so there's already a pretty good default level of accessibility already built into the frameworks.

Now, What happens when an assistive app makes this call? How does this all work? Well, the assistive application calls those client APIs. That comes into our app kit's accessibility infrastructure. And based on that UIElement ref that was asked about, we map that back to the right Cocoa object and we send it a method in the accessibility, in the NS accessibility protocol.

We get back the response, we package it up and send it on back. So again, fairly standard client server sort of thing. It's just that in your application, you're always dealing with Cocoa objects. You're never dealing with those UIElement refs that assistive apps deal with. You're just treating windows and views like you always do.

So a little more graphical representation of application accessibility. It really is a very lightweight representation of your UI. So we have your applications running. Maybe there's a window with a very boring button in it with no title. Shame on that button. But anyway, and that hierarchy is exposed in accessibility as an application with a window, and that window has a button in it. It's very clean and simple. And in fact, from the window on down, you'll see that it kind of maps to the view hierarchy.

However, in Cocoa, we know that the view hierarchy is a little more complicated than that. The application has some windows. The window has a frame view, which has a content view. That view has a button in it. And then that button has a button cell, which is really doing most of the work of a button.

And so it's that button cell that is reporting itself as a button, but there's all this extra stuff in the way. We have a concept in accessibility called ignored elements. In this case, we don't report those items in between because to an accessibility client, they're not terribly important. They're just noise. We only need what the user is seeing, which is an app with a window and a button in it.

Now, let's take a brief look at this Accessibility Protocol. For all of the things that an assistive app can ask for, as you might imagine, there's a method to return that information. So for attributes, we can send back a list of the names. We can send back the value for a particular attribute, whether it's settable and setting a value.

You'll also encounter something called parameterized attributes, and this is often a mystery to folks. For complicated elements like text elements, sometimes we need more than just give me your value. Sometimes to navigate through text, we need to find out things like the bounds of a particular range of text. Parameterized attributes are things that allow you to get more specific information, pass in a parameter when you get back a value.

Typically, you will not have to implement the Accessibility Protocol. You can implement these methods, though. Actions, again, things like pressing something, incrementing or decrementing a slider, An object can return its list of action names, so things that it supports, a description that can be presented to the user, again localized, and then finally be told to perform a particular action.

And then hit testing. As we saw in accessibility inspector moving the mouse around, accessibility inspector is able to get what UI element was under the mouse. This is done by a hit testing method. We haven't seen it in a demo, but we can also -- or an assistive app can also find out what item has the UI focus. And this is done by implementing the accessibility focused UI element to return the focused UI element. And finally, we saw that things might be ignored.

That's what this last item is. If it's yes, I am ignored, that doesn't mean it's hidden and all of its children are hidden. It's not like an NS view being hidden. It just means that it is skipped in the -- as the hierarchy is reported. Its children are reported as if they were the children of the parent of the ignored item.

And finally, notifications. So these are mostly automatic. NSWindow already sends them out when the windows come and go. Sheets do the same thing. But if you write your own custom control, especially if you are not going through object value, if you override that, you need to let accessibility know your value has changed.

And if you have a custom control that within the control you're updating the UI focus, you need to let accessibility know that the focused UI element has changed. And so those are simple strings that you send out. And it's one function call in AppKit. The element is the thing reporting the notification, and the string is the string constant.

Now that all said, let's talk a little bit about default accessibility. So one of the things that I spend a great deal of time doing is making sure that the built-in classes to AppKit, so all the Cocoa widgets, have great default accessibility built in. So that if you're using standard controls, there's a lot less work that you need to do to make your application accessible. And so we definitely recommend using standard controls whenever possible. However, we are unfortunately unable to read minds. We're working on it, but we have nothing to announce today.

So things like I have an image in the user interface, the framework doesn't have any idea what that image is for. Or you've written a custom view or a custom control, the framework has no idea what its purpose is, what its value should be reported as. And therefore, in those situations, we need your help.

So in past WWDCs, past conferences, what we've typically done is we've written a sample application, Dicey would be one example, and we just do it without doing any accessibility. Then we walk you through how you would make it accessible, and then we release the source code. And in fact, those pieces of source code, I highly recommend them. They're available in your Snow Leopard Seed in the Microsoft Office.

The developer examples accessibility, so please take a look at them. But the one problem with sample code is that, well, it's sample code. Whoever wrote the sample code had a particular thing they wanted to show, and they wrote the code specifically to demonstrate something in particular. And so this year we wanted to show a real-world example of making an application accessible. And so we thought, what if we went and grabbed a Mac application out of the wild? And took a look at it and voiceover, and then figured out what we might need to do to change it.

And then hopefully at the end of the session, show that same user interface again, but with its accessibility issues corrected. Now, we had a hunch that possibly a hot topic this week might be telephony. Just a slight hunch. And so a great Mac application for making phone calls is, of course, Skype. Skype.

So we called up the Skype folks a little over a week ago. And we've been working with them. And what we'd like to do is show you, one, the Skype that you can download today, and show you just one little piece of the UI, then talk you through how you would go about auditing just a application out of the wild, your own app, auditing its accessibility, and then addressing the issues that you find. And then at the end, we'll show you the results. So I'd like to bring up Dean again. is going to walk us through just a small piece of Skype's API, or UI.

All right, so while I'm getting sound back on my machine. So visually impaired Mac users are no different than sighted users. We don't buy a program or a computer just to use iTunes or Mail. We really do want to use other applications. Just like sighted users, you hear about things, you see things on the Internet, and especially when they've got the 30-day free trial.

And in the case of Skype, it's free. It's very exciting. You're like, man, let's download this. Let's check it out. I'm going to be able to use this program, make some free phone calls, and it's all going to be good. So I've downloaded Skype. I suppose I should turn voiceover. Cover on.

Welcome to Macintosh. VoiceOver is running. System Preferences. Window. Universal Access. Toolbar. Skype. Skype. Welcome to Skype. All right. So here's what I hear. Skype. Welcome to Skype. Image. Image. Wow, that was loud. Welcome to Skype. Skype. Not much there for me. Now you can see, of course, what's going on, that there is a login screen and password.

But for our voiceover users, there's no way for me to know, unless I have someone sighted come along and say, ah, you're actually in the login screen and maybe try doing this. For a user who just downloaded it, that's all they would get. Now, James is going to magically make this work. James?

For your Cocoa applications, you've shipped them, they're out there. VoiceOver users are downloading them, they're looking at them, they're interacting with them.

If you've not thought about accessibility, it's not that you've necessarily, you've not done anything wrong, you just haven't thought about accessibility and taken any steps to address accessibility issues. So what we're going to talk about now is, all right, let's take this login screen. Let's see what will we do to get it to be more accessible so that Dean would be incredibly happy with it.

So an accessibility audit. Sounds very fancy, but it's kind of simple. What you want to do is first use the accessibility tools we provide. They're located in your developer tools, developer applications. In the utilities, there's a folder of accessibility tools, accessibility inspector and verifier. You can also turn on voiceover as you saw me do on Dean's machine with an impromptu demo.

To see what voiceover reports when you put it over your application. So using those tools, the other thing to do is to focus on areas that commonly need additional accessibility. We noticed images was one area. We had that Skype logo that just said image. Image. Kind of useless. Also, custom controls and views tend to be hot spots where you need to do some accessibility work. It's good to start with the most commonly used areas of your application.

Those often tend to be the places where you've done the most custom work as well, but they're also the meat and potatoes of your app, so that's a good place to start. And proceed window by window. The thing to note about accessibility is certainly we would love every application and every just part of every application to be wonderfully accessible, but accessibility is a very, very important part of the application.

And so we're going to focus on that. Adding descriptions to images, that's a great start. Every release, make more and more of your application accessible if you can't do it all at once. Any work that you do is really, really a fantastic extension of your application. So now to walk us through an accessibility audit is Karl.

Here's our Skype window. It exhibits three of the most common issues you're gonna encounter when accessorizing your application. And the first issue is that images need descriptions. So let's look at the Skype logo over here, and we'll lock onto it using Command + F7. We can see that it's an AX image. Let's roll this AX image.

However, when we look at the description, it's empty. Now, to an assistive application like VoiceOver, it really needs this description so it can describe it to users like Dean. So the next issue I'm going to show you is related to the first. So just like images need descriptions, controls need title elements. So let's look at this combo box over here and we'll lock onto it.

Now there's nothing in any of these attributes that tells us that this is a username field. However, to sighted users, we can see that this Skype name label directly above it should apply. However, to assistive applications, it could be the Skype name element, or it could be this don't have a Skype name button that's right below the combo box. So to eliminate any ambiguity, you should provide a title element, an AX title UI element, for all of your controls. And you can do this in Interface Builder while you're building your application.

So the final issue I'm going to show you is that you need to make sure that your accessibility hierarchy is traversable. So let's go back to the Skype logo and we'll lock onto it. So it's an AX image and its parent is the Skype window. So let's go up to the window and look at its children.

And it only has seven children. The first three are the close, minimize, and zoom buttons. The next one is the welcome to Skype label. The one below it is an AX image, however, it's the white background image and not the logo. And then we have the Skype window title.

And finally, we have the grow area, which is the control in the lower right-hand corner of the window that used to resize the window. So the window knows nothing about that logo, and therefore, VoiceOver will know nothing about that logo. Because it traverses the hierarchy from the root down. We can also see this issue in the controls over here. So we'll lock back onto our combo box.

And we'll go up to the parent, which is an AX group. And as we recall from looking at the children of the window, there is no AX group as a child. So you can use Accessibility Inspector to find all these issues like I just did, or you can use VoiceOver. And when you do find these issues and remedy these issues, your application will be much more usable by a VoiceOver user and much more scriptable to boot. Back to you, James.

Oh, give him a big round of applause. Great. Dean showed us kind of some of the ramifications of accessibility from his perspective. It's also interesting to note that if you were trying to write say an automated GUI script, you would not be able to in that log in panel because the hierarchy problem would prevent you from actually accessing the text fields which you probably would want to be using in your automated script.

So how do we deal with these issues? So the found issues are we came up with images with no descriptions, elements like those text fields, the combo box with no title, and we had a hierarchy issue, some parent child issues. And who doesn't have parent child issues? So solutions.

Two of the three actually have relatively easy solutions, which is excellent. The third one, it's not a lot of code. Just conceptually, you need to know why it's a problem and then what to do to fix it, which we're going to tell you right here. So adding instance-specific accessibility information.

So certainly buttons and text fields have a set of attributes that they always have, like their role and their value and what have you. But we can also attach additional attributes specific to each instance, and that's what we'll do to give images descriptions. It's also what we'll do to associate controls with their titles. And then for correcting parent-child reporting issues.

There's two things that we might do there. We might override the NS Accessibility Protocol. So when we're asked for the value for our children, we might return a different set of children than the default. Or, as might be the case here, we might just refactor our view hierarchy in a different way such that the right information is reported automatically. So let's look at the first set first. So descriptions.

The first thing I want to talk about is the convention for what those strings should be. They should be localized because a user is able to switch VoiceOver into other languages. They should not include the role itself. Don't say this is the login button. Just call it login because VoiceOver is going to use that role description we saw before and tack that on to the end and then it will just sound stupid. Login button button.

The way I think about these is if this weren't an image for this button, if this were just a button with a word on it, what would that word say? That's usually about the length that you want for your description. And don't confuse this with the role description. One of them is just telling the role description is describing what kind of UI element this is, whereas the description is more information specific to this particular item.

Now, those titles we want to know, especially when a voiceover user is tabbing through the fields. They don't want to tab to a field. They don't know what's around. They have to go to the left. Is there a title over here? No. Let me go back over. Is it on top? They just want to land on the field when you tab to it and have that described. And it's very easy to do that. We just connect the control to its title in Interface Builder.

And then one that we didn't see come up in the login page, or the login window, but is very important also, are linked UI elements. So, in fact, you saw Dean use this in interface builder, excuse me, in iTunes. When he was in the source list, he got to the iTunes store and then he was able to jump to another part of the UI.

By linking related UI elements, it allows a voiceover user to move to related spots. Examples would be from the search field down to the search results and then back up. Or from a source list to the stuff on the -- that's being displayed for the selection as we see here.

So how do we do all this magic? I'll actually go in reverse order, because you can set this up in Interface Builder. You select a button, select some UI element. In the Inspector, you want to go to the Identity tab, and under the Accessibility Identity section, you just type in the description. That's very difficult.

Now, for attaching things or relating things, it's a control drag, just like any other connection that you make in Interface Builder, except that this time we drag from the text field to its title, and instead of picking an action or an outlet in the very same pop-up menu, we just pick how we want it related for accessibility. In this case, this is a title, so we would pick title from that list to make the connection. If we were linking, we'd pick link.

Now, sometimes you're not building your UI in Interface Builder. You have some things that happen to be programmatic. Sometimes there's things you can't access in Interface Builder, like the segments of a control, of a segmented control, in which case we can also set these attributes in code using Accessibility set override value for attribute.

Now I wanted to talk a little bit about these overridden values, because sometimes they cause confusion. People see that method and they think, oh, I can just override everything with this. Now, we saw before how an attribute tends or is how a request for an attribute's value is responded to by a Cocoa app. In comes the request. We map it to your Cocoa object.

That accessibility protocol method is called. The return value is sent back. When you add an overridden attribute, either in Interface Builder or using that API, it doesn't happen quite the same way. The request comes in and AppKit's accessibility infrastructure is keeping track of that UI element ref that somebody has a hold of in their assistive app and all of the overridden attributes associated with it and when the request comes in we say, "Oh, we have that overridden. Let me just send that back." And your Cocoa object never finds out about it. We don't want to bother you. We already know how to deal with it. Now, this leads to just a couple of caveats.

The first is you should only use overridden attributes for those instance attributes that I've talked about. Don't use, don't try to change the role of your user interface element using an overridden attribute, because your object doesn't know about that change. And the Cocoa frameworks often are doing different things depending on when it asks itself, what's my role, it'll do different things.

If you're changing things with overridden attributes that are so central to that Cocoa object, but it doesn't know about it, just bad things ensue. The other thing to note about overwritten attributes is that they are static. We set a value and that value is what is returned until another value happens to be set.

So that static portion can make dynamic image descriptions a little bit tougher. So this is just the example of some chat client. It has a status image. It's away, then you're idle, then you're online. You don't want to, for this dynamic information providing image, you don't want to put a description on it that's called status because that's not very helpful. Okay, it's the status, but I still don't know what it is. You want to provide the description of what the status currently is. Now, there's a few ways you could do that.

You could Use an overridden attribute and every time that you change the image, you could re-override the value with the new description. And that's not so bad. However, if that happens to be in a table or an outline view, that gets very difficult because in tables and outline views, accessibility is keeping track of overridden attributes based on what row things are in. So if somebody resorts everything, then you have to redo all your descriptions. It's kind of a mess.

So we're thinking, well, what can we do to make this a little bit easier, or actually a lot easier? And we're thinking, well, strings don't have this problem, because a string essentially carries its own accessibility information with it. Its value is its value. You set a string and we just pull the value out of the string because it is a description.

So we thought, why don't we do the same thing with images? So what we've added in Snow Leopard and is in your seed is an accessibility description on NSImage. And the way it works is pretty simple. You Take a localized string and you set the accessibility description on an NS image. And now, any time you set that image in any kit class that displays images, so image cells, button cells, the like, that description that you set on the image just comes on through. That makes it a great deal easier.

As I said earlier, we can't read your mind, but sometimes we can do a little bit better than we have been. And one thing that was introduced in Leopard were named system images. So in NSImage, you'll see a long list of common images that you can use in your applications and that we use in our system applications. We thought, well, now that we have the ability to put a description on an image, why don't we provide a default description? And so if you use these named system images in your application, you will get a default description.

And then thinking a little bit more, okay, so image description, we have it down to one line of code. You can call on an image. Could we get it down to zero lines of code? Does anybody want any guesses on whether we could? I think yes would be a yes. We added string file support.

And because we noticed that what people do is they have a bunch of images in their app package and they use image name to bring that in and then they stick it in a button or stick it in some spot in the user interface. So, okay, let's allow folks to add a specially named strings file, accessibility image descriptions. The keys in that strings file are the names of the images. The values are the descriptions you want to use. And if you pull in an image, set it in a user interface element, we'll automatically pick up that description.

So what's the search path here? We'll always use a programmatically set string first. Then our first call to accessibility description. We're going to see if you've provided one in the strings file. If not, we'll use the default framework description if it's present.

And then the last thing that we saw were parent-child relationship issues.

And in an accessibility hierarchy in the tree, a parent can have multiple children and each child has one and only one parent and the children, they have to match. You can't have one thing thinking somebody is its parent and this other thing thinks it's its child. It gets, it's like a, yeah, it's like a bad episode of the Jerry Springer show at that case.

So why do we get into these parent-child issues? Not very Dr. Phil here, more accessibility related. If we take a look at what was going on in that Skype login panel, It was this issue, is that there is a, sometimes you have a control, an NS control, and NS controls often use a cell to do their drawing and event handling or tracking. We call those single-celled NS controls. And those controls are expecting to report their cell as their child. The control itself is ignored, and it's the child that reports its accessibility information. The control isn't reporting its subviews at all.

But in this case, we had an NS image view. The image view itself is ignored as far as accessibility is concerned. It reports its image cell as a child, but that image cell doesn't know about its views, subviews. It doesn't know it should report them. And in fact, it really shouldn't report them because it's a little odd to have subviews in an image view.

And so there's a few solutions that we could -- or approaches we could take for a solution. The first would be to subclass NSImageView and return both the cell but also our subviews as children, in which case they would all show up. We'd put them all into one big happy array and send it back.

Or the other thing we might do is refactor our UI a little bit. So we might make a custom view that displays an image. We're going to see in a moment maybe instead of using an image view that displays a white background, maybe just a custom view that draws whites. It's ignored itself.

It already reports its subviews. Possibly refactoring is a better way. So we went through and saw the issues. Here are the ways we can solve those issues. Now let's take a look at... The results. Now what we're about to show is just a very, very early peek at the accessibility directions of Skype for the Mac.

VoiceOver on Skype, Window, Skype, Skype Name, Combo Box. All right, so we've got the Skype up here and take two on this for me as the new Skype user. Welcome to Skype. Oh, that's nice. Skype Name. Skype Name, Combo Box. Don't have a Skype Name? Button. Password. Password, secure edit text.

Forgot your password? Button. Remember password, uncheck checkbox. So you can hear it's a lot more accessible, said a lot more things. I as a user getting a lot more information out of it than it was before. So great job. Also, I don't know if you've noticed, through all of my demos, I've been using the keyboard exclusively.

So when you make your apps accessible or usable, I know it's easy to think about making a rich graphical interface and mouse driven, but think about accessibility and those who may not be able to use a mouse may have to use a keyboard. Maybe you'd have to use some alternative input devices to control your application. The last thing that I wanted to say is, as James mentioned, VoiceOver is included on each computer shipping Leopard and upcoming Snow Leopard.

So for you to check your accessibility really fast, it's just a matter of turning VoiceOver on and just navigating the UI. And that will give you a pretty good sense of what someone visually impaired would experience with your application. Thank you. Thanks. Thanks, Dean. Actually, let's give Dean and Karl a big hand for the demos today. Thank you, guys.

So to summarize, application accessibility affects your users. It also can affect you and your development process. So absolutely encourage you, exhort you to audit your application for common accessibility needs. Images without descriptions. Take a little time. Go through the images on your app. You can drop in one of these accessibility image description strings files.

Don't even need to touch a line of code or even a nib file to get that done. Connecting controls to their labels. Again, a simple control drag with an interface builder. Looking at hierarchy issues, which can be a little more complex, as well as custom views and controls. And start making your applications accessible.

Now, for more information, our evangelist is Derek Horn. I highly recommend if you are looking at making your application accessible, which I very much hope that you are, is to join the Accessibility Dev Mailing List. You can go to lists.apple.com. You'll find it on the list there. But certainly all of us working on accessibility at Apple, we read that list, we respond on that list. It's a great place to get your questions answered and your answers questioned.

The documentation, there's the accessibility documentation. I would also recommend taking a look at the App Kit release notes. There are a number of minor changes we've made in App Kit that are in your Snow Leopard Seed that we didn't talk about in the presentation. Finally, related sessions. On Thursday, we'll be doing a chunk of accessibility as part of polishing your Cocoa application. There you'll see if I have a custom view, what do I need to do to make it accessible. And then a few other sessions where some accessibility will come up, but I'm not allowed to say how.

Some labs. Today, 3:30 to 6:15 in Mac Lab A is the Accessibility Lab. Please bring your applications, your accessibility questions. I'll be there. Dean and Karl will be there. Members of the VoiceOver team will be there, all ready to answer your accessibility questions. I also mentioned at the Cocoa Open Lab on Thursday at noon, I certainly will be there to answer other Cocoa questions, but also accessibility questions.