Technology. My name is Eric Seymour and I manage the VoiceOver engineering team at Apple. And I want to thank everyone for coming tonight. We know that there are other things you could be doing, other labs, other sessions. It's been a long exciting week, perhaps a long day. The fact that you're here tells us how much you care about accessibility. We certainly care too and we definitely appreciate it. So thank you for coming. So, tonight you're going to learn all about how to make your applications accessible with our accessibility APIs in Mac OS X.

Before we get into the technical meat of the presentation, I'm going to spend a few minutes up front talking about accessibility from the product perspective. And this is important because this is how users who are using accessible technology with your application are going to view Mac OS X in your app. They're going to view it through the lens of accessibility if you will.

So Mac OS X shipped several targeted accessibility technologies. Perhaps you're familiar with them in our Universal Access preferences pane. You've got seeing and hearing and keyboard and mouse and track pad, things that focus on specific special needs. And some of these things have migrated over into just regular mainstream use.

For example, like zoom, I know plenty of people who are sighted with no visual impairment and use zoom all the time. Also, some of these features work without any additional work in your applications. And some of these features work with cooperation with your applications, like VoiceOver and of course, that's why you're here.

Your implementations of the NS accessibility APIs help us provide accessibility for end users. Now, the other thing that you might notice, especially if you've got experience with Mac OS X is sprinkled throughout the operating system are great universal access features and designs. So for example, the magnified doc, or magnifying a font for a phone number or other things are useful for everybody, not just somebody with a specific visual impairment for example in this case.

And so these are things that have sprinkled throughout other products, not just Mac OS X. Let me just name a few. In the last year, we've added closed caption support. We are the first to offer this for digital media. You can go to the iTunes Store, purchase closed caption content and run it on any of our players. We've built it into QuickTime.

So it's a really great feature. And again, good universal design. This is good for folks who have a specific need for this but it's also just useful for everybody. And then another one is Talking iPods. I'm sure you've seen or heard or perhaps even own a Talking iPod. Another great example of universal design where certainly somebody with a visual impairment could benefit from one of the devices. But also, it's just a cool device, right? I mean, who doesn't want to have their track names read and their menus spoken.

And then there's VoiceOver. So VoiceOver is screen access technology for Mac OS X. And basically, it speaks items that are on the screen, it speaks as a result of user action, it uses sound effects to embellish the experience when you can't see the screen. It's basically a spoken interface or an audible interface in fact. And VoiceOver has believe it or not been shipping now for five years.

For five years we've been here talking about VoiceOver. And it's pretty exciting. We've done a lot of cool innovations, but perhaps the most important thing that's happened is VoiceOver now runs on and is installed on well over 30 million compu0ters worldwide. Which is really a phenomenal number. This is orders of magnitude more than any shipping screen reader on the planet. And what this means for you as developers, when you make the decision to add accessibility or to improve accessibility in your application for an end user, the barrier to entry for that end user is completely eliminated.

All that end user needs to use your application is a Mac, whether it's theirs or a friend's or the library or in a school. Every single Macintosh is ready for your app. Whereas on some other systems, an end user might have to pay up to 1000 dollars just to equip the computer to work with your application. So this is a really big deal and this really is the cornerstone of universal design.

It's got to be available everywhere on every system. So we've also and this started about two years ago with Leopard, we also pioneered really fantastic plug-and-play Braille. In fact, I read a quote the other day on one of the discussion boards where somebody not only said Apple has clearly the easiest to configure Braille but it's the most fun.

Right? And so how great is that when we actually produce, you know take a relatively technical thing that's a big headache to configure and we make it so easy that users are starting to say it's fun. That's really, what we want to go for. And so because of all of this, we're pretty proud to have received the Access Award from the American Foundation for the Blind. And this is basically for pushing accessibility into our products, for pushing accessibility into the iTunes ecosystem.

So really, we're quite proud of this. And so where, that brings us to now. Right? So everything I've talked about is Leopard. We've been shipping Leopard for a while now. And in the meantime, behind the scenes, our team has been working feverishly to bring the greatest Snow Leopard VoiceOver and Accessibility features ever. So we've significantly improved VoiceOver for features, for performance, for enhancements, bug fixes. We've really, really worked hard and we're going to bring we think a fantastic release. There's far too much to show here.

However, there's one feature that's worth showing. And the reason we want to show it is because we think that it marks perhaps the turning point in what it means to be a screen reader. So let me explain. So if you take a look at this screen, if you can see the screen, it's a website with a bunch of nicely laid out content. There's so much visual information on this screen, that if I'm sighted, I can look at and very, very quickly and efficiently go to the thing I'm interested in and interact with it right away.

Unfortunately, if I can't see the screen so much visual information is lost. Efficiency is lost. Basically even with modern screen reading technology, the elements on the screen typically are boiled down into a list for the end user because it's all coming through a speech engine. You know things are just spoken.

So I lose. How big is that thing? Where was it placed? Was the image really big taking the center of the page? Or not? How do I know? I don't really know. And it's really hard to use. I've got to you know remember keystrokes and filter my way through. And so screen reader users actually become quite adept perhaps far more advanced than nonscreen reader users because they have to do more steps to get the job done.

And we really want to solve that. We want users to be able to use their computer or their device without having to read a manual and without having a steep learning curve. And we want to give back, actually, we want to give for the first time spatial layout information. And so we thought what if we take this information and we lay it into the track pad, the Multi-Touch track pad that every user today purchases and for the most part hasn't really been focusing on as a screen reader user. And so this is what we've done.

We've taken content from the screen and we lay it directly into the track pad so that all you have to do is touch the pad and it's as if you're touching the screen. And you hear sound effects and speech and you can move your finger around the pad and know what you're touching. So I can tell how big that image is in the middle.

I can tell where the title is, I know where the menu bar. I can very quickly rip across the page and go from point A to point B without having to filter through dozens of elements. So the spatial layout piece is totally there and most importantly ease of use.

Because the learning curve is what it should be. I should be able to touch the track pad and it should do what I thought it should do. And so we really think we've got a profound advancement here. This is going to be coming to VoiceOver for Snow Leopard. And really the best way to experience this is just to show you. And so to do that, my colleague here, David Tseng who works for VoiceOver on Quality is going to give you a demonstration.

Thanks everyone. Hi everyone. My name is David Tseng and I am a QA Engineer on the VoiceOver team. And I'm really pleased today to be able to show you VoiceOver and some of the new features in Snow Leopard that we have coming up. So I will go ahead and first start VoiceOver using the Start command, which is Command-F5.

VoiceOver on, mail, window, inbox, 5 messages, 5 unread, toolbar.

Alright, great. So, the first thing I want to point your attention to is a few of the visual features of VoiceOver, which allows for great collaboration between the VoiceOver user and a sighted user. And it also allows you to follow along as I move around the user interface. So the first thing is the caption panel, which shows everything in print that VoiceOver speaks. And there's also a VoiceOver cursor, which currently is on the toolbar.

And it's a rectangular highlight. And it allows you to follow me as I move around in the user interface and highlights the focus. So today I wanted to go through something that we all pretty much do every day, which is read and send email. And I'm just going to go ahead and use the keyboard and navigate around and give you a feel of how VoiceOver reads.

Mailboxes, table, one row selected.

Column 1 of 1. Inbox, mailbox, 5 unread.

So I just want to stop and point out that you heard VoiceOver reading quite a bit of information just there. The mailboxes table, column 1 of 1 and it also says the selection. So you'll, for a VoiceOver user, all that information is really actually really essential. The name of the item, the role of the item and what has selection. So I will continue and go and select a mailbox.

Interact with mailboxes table, row 2 of 13 selected, column 1 of 1. Inbox, mailbox, 5 unread, edit text, send. Mailbox level 1. Trash, mailbox 1 unread, level 1, send, mailbox, inbox, mailbox, 5 unread, level 1. Interact with messages table, row 1 of 5. Status from Jean LeTeam, Jean LeTeam, subject import.

You meet for lunch, meet for lunch.

Alright, so I'm just going to read that message using a keyboard command, VoiceOver J.

Text, Hi David. How is demo going? Hope everything is doing fine. Don't forget to show the cool gesture supporting VoiceOver. How about grabbing lunch before the Accessibility lab in the afternoon tomorrow? Patti.

Alright, great. So you'll notice that again VoiceOver is reading a lot of text and there's quite a lot of information being conveyed.

And also there's a lot of keyboard commands involved. I think I must have pressed around 10 keyboard hotkeys in total to achieve the task of reading mail. So there's always, there's something missing from this whole exercise. I really have no understanding at all of the user interface currently on the screen. And in order to deal with that as Eric mentioned, we put in this feature that allows me to touch, really touch the UI using my track pad. So instead of trying to explain it further, I'm going to just show you how that works.

Stop interact.

And I will turn it on using another command.

Track pad command 1, Close toolbar, Close button, Zoom button, Minimize button, Zoom button, inbox 5 messages, 1 unread.

And as you can see, there's a highlight that shows you exactly where I am and we've turned that on for demo purposes.

Toolbar, Messages, Toolbar button. Messages, table, 1 row selected, status, blank from Patti Hoa. Subject Meet for lunch, date received, June 5th 2009, 5 mailboxes, table, vertical splitter. Mailboxes, table, one row selected.

Mail Activity. Horizontal splitter. Mail Activity, Hide Mail Activity Button. Mailbox Actions, Menu button, vertical splitter. Message content, scrolled area.

Alright, so was that quick? Around 10 seconds sweep of my finger across the track pad, I've actually gotten more information from this screen reader than I think I've ever had with any other screen reader.

I really have a good understanding of where things are. For example, I know that you have your usual Close, Minimize, and Zoom buttons on the top left. You have a toolbar spanning the width of the window. There's a mailboxes table, separated by a vertical splitter from the messages table and a horizontal splitter separating that from the messages content area. So that type of information was really unavailable to screen reader users before this.

And now we think that it's, it's great that we have it and it really allows me to understand what's going on on the screen, what's prominent on the screen. And it allows me to go back to a piece of UI if I know where it is. So I'm going to reply to Patti and tell her that I would love to do lunch.

Interact with message content, scroll area, two items. Message headers text from text. Hi David, interact with text. Word, tomorrow, reply, now in window, RE meet for lunch. Toolbar, sure, semi space, let's do it exclaim. Subject. Close button. Subject edit text. Subject minimize, Zoom button. To zoom button.

Toolbar, characters, interact with toolbar. Send button.

And I'm just going to double-tap this and go ahead and send the message.

Press, now in window, interact with dialogue for window RE meet for lunch, cannot send message using the server SMTP.me. Sending a message content to, servers, tape edit, SMTP server list.

Alright, so looks like we have some connectivity issues, but that's how Gestures work and I'll go ahead and pass it off back to Eric. Thanks.

Good job.

Thanks.

OK, so the support that you saw, Gesture Support, is built into VoiceOver. It uses the standard accessibility APIs which you're just about to learn about.

So it's not like this is some newfangled thing where it's going to require tons of support by a third party apps. It just works by taking advantage of existing APIs. So to talk about those APIs I'd like to invite another colleague, Patti Hoa on stage to get started.

OK. Well thank you Eric. That was a great demo of the VoiceOver. Now that you've seen a demo of VoiceOver, you're probably wondering well how does VoiceOver interact with the application? Well to do that I'm going to give you an overview of the accessibility technology that we have on the Mac OS X platform.

So, after I give you an overview of the accessibility technology, then I'm going to have Greg come up and show you how to determine how accessible your application with some of the tools we have. And offer you some solutions on how to accessorize your application. So in order to tell you about the accessibility technology I'll first tell you what are some of the available assistive applications we have for the Mac OS X CD that you have right in your hand.

And I'll explain to you what the accessibility hierarchy is, what are some of the accessibility information you can obtain from UI elements. Then later on I'll show you or take you through how VoiceOver interacts with the objects in your application. And lastly I'll explain to you what the NSAccessibility Protocol is. So VoiceOver. VoiceOver is considered an assistive application.

By that I mean application that takes advantage of the accessibility API we have on the Mac OS X platform. Now there are many other assistive applications that comes with the Mac OS X CD. One of them is the Accessibility Inspector which is a tool that helps you to analyze how accessible your application is.

And later in a demo we'll show you how to use this tool. In addition, we have many other tools that help, that takes advantage of the accessibility API for things like UI interaction recording and UI automation testing. And some of those tools are like the instruments, the Watch Me Do feature of Automater and GUI scripting.

For today's purpose, I'm going to talk about VoiceOver and focus on that. So one of the questions you may have about VoiceOver is well how does it access the information about the interface in your application? Well to do that I'll give you a simple graphical user interface as an example.

And you'll hear me often abbreviate it as GUI. So in this example, we have just a very simple window with a button in there. And this button just have an image with a little triangle in as the image. Well, we can interpret as maybe a Forward button or a Play button.

For today's purpose, we're going to just say this is a Play button. So visually, we see these three objects. We see your application, we see the window and we see a button with an image. Well, for assistive applications, they need an alternative interface to access this very graphical user interface.

And to do that, we take advantage of this proxy object called AXUIElementRef. It's a very light object that reference the actual object that represent the GUI that you see, object you see. So in this example, an application, we know internally it's implemented with an NSApplication object. We have an AXUIElementRef that access these Cocoa objects. So the AXUIElementRef can access many accessibility information about object. One of which is the role of the object, which tells you what type of UI it is. In this case, an application object will have a role of AXApplication.

And that will be the same for like a window. A window will have an AXUIElementRef with a role of AXWindow and the same with a button too. And the way these AXUIElementRef are represented is in a tree hierarchy with a parent-child hierarchy level. So for example, in this particular GUI will see that the window is a child of the that application and the button is a child of the window.

And vice versa, the window is the parent of the button. Now as a Cocoa developer, you know the way this GUI is implemented is slightly different. Right? You have an NSApplication object with a NSWindow object and inside of it there's the NSButton object which contains the NSButtonCell. And you may also know that the window also have a frame layer and also a content layer.

So obviously this is many layers, more layers than that's relevant for the accessibility hierarchy. So what Cocoa has is a concept called ignoring elements that allows to you in essence not expose certain layers for the accessibility hierarchy. And so in this case, NSView and NSButton are by default ignored.

And by, when I say ignored, it doesn't mean that all the children are hidden you know in any sense. It just means that we pass right through those layers. So in this case the window, NSWindow layer because the NSView and the NSButton layers are ignored by default, it pass right through to the NSButtonCell layer.

So with that, we're able to do a 1 to 1 mapping back to the alternative user interface that contains these AXUIElementRef. So let's take a look at the button as an example. And let's see what kind of information can VoiceOver obtain about this piece of UI element. Well by default you can always get the role information which is, which tells you what type of UI it is.

In this case it's AXButton. And then the role description gives you a user-friendly description for that type of UI. In this case it's button. And then we have attributes like parent and earlier you saw the parent-child hierarchy for this GUI or graphical user interface and you know that the parent of that button is the window. So that's what we return here.

And then we have attribute like the AXTitle, the AX top level UI element and by default most of your UI will have the window as the top level UI element, but if your UI sits inside a sheet or a drawer, then you want to return that sheet or the drawer as the UI element. Then we have attributes such as enabled that tells you the enabled state of that UI. Now obviously if it's not enabled, then it's probably grayed out or dimmed out in the interface. Then you have focus that tells you whether or not it has the keyboard focus.

Now, many of the buttons have labels associated with them. Now in this, so if you do then you provide the AXTitle. But in this case we actually have an image for this button. So what you want to do is actually provide the AXDescription for it and you know because you as a developer know what this button will do. So you have to provide description. In this case, you know, we know this is a Play button, so you return the string play.

Now we also have attributes for the position and size that tells you where this UI sits on the screen. Now these attributes allows VoiceOver to draw its VoiceOver cursor around the UI, which is that black rectangular box you see around the UI. And then for some UIs they have an action associated with it. By action I don't mean actions for what this button can do, but I mean actions that more of a user input action. So we know that for a button usually you can mouse click to press it.

So in this case, there's a AXPress action for this button. Now, when VoiceOver land on this button, it'll probably say something like "play button". And that's because VoiceOver used two attributes in specific, one is the AXDescription which has the string play and the role description which has the string button.

So with those two alone VoiceOver is able to speak "play button". Now you see that whole list of attributes that's available for the Button UI element. Well, if you just use a standard Cocoa widget like an NSButton or such, you'll get all of this for free. AppKit will provide accessibility information for that button. The only thing you will need to provide is the AXDescription. And that's because AppKit has no idea what your button would do. So, now you know how VoiceOver access the information from your interface.

The next question you may well have how does VoiceOver interact with the objects in your application? So to know that I want to first tell you that VoiceOver sits on a separate process than your application. And the way VoiceOver access accessibility information from your application is through the accessibility API which we often abbreviate as AX API.

And then the AX API will take that request and you know sends it off to the application that's in interest. So to show you, give you a more detailed explanation on how that interaction works, I'm going to use the example of the AXRoleDescription when VoiceOver asks for that particular attribute. So what happens is VoiceOver first gets that button AXUIElementRef.

And as you know goes through the AX API and asks for the role description, that request goes over to the AppKit side or your application and since your application is a Cocoa application, that AppKit accessibility infrastructure would handle that request first. Alright? So the AppKit sees that button AXUIElementRef and says oh, I can map this to the actual Cocoa object which in this case is the NSButtonCell. By default, you should know that it's the cell of the control that handles the accessibility information. OK? So in this case, AppKit will make a request to the NSButtonCell.

And it does that through the NSAccessibility protocol which are a bunch of methods that allows you to access accessibility information for a Cocoa object. And one of those methods is called accessibilityAttributeValue that takes the attribute as a parameter. So in this case, the one we're interested in is the role description attribute. But note that I have a different constant there for the attribute. It's called NSAccessibilityRoleDescription. That's equivalent to the AXRoleDescription.

This is the AppKit version of the constant. So the AXRoleDescription is equivalent to NSAccessibilityRoleDescription. For your purpose when you're coding, you'll be using the NSAccessibility version of the constant. Alright, so AppKit sends this request to the button cell and the button cell says oh I got a request for the role description. Well I know my role description is button. So I'll return that as the string. So now AppKit gets that answer back. It bundles the information back into the AXUIElementRef and sends it across to VoiceOver.

And that's how VoiceOver is able to speak button. So now you see how the interaction work and I want to talk a little bit about the NSAccessibility protocol. By the way, the NSAccessibility protocol is not the same as the Accessibility API that I mentioned earlier. The Accessibility API or AX API is used for assistive applications. You as a developer of Cocoa application should focus on the NSAccessibility API that I'm showing here. Now you've seen one of them which is the accessibility attribute value.

It's probably one of the ones you'll be using most often to provide additional accessibility information or tweak some information. Now there are other methods like the one that returns the list of attributes or there's a method that tells you if an attribute is settable and sets it if you need it to.

Also, there's the parameterized version of the method that takes, basically takes an additional parameter. And also if your UI has some user interaction for, then you need to provide the action that this hears, you know like for example the AXPress action I mentioned. And here's a list of some of the actions that are supported like the AXPress action, AXShowMenu action, etcetera. And also we have some other methods.

For example, the accessibility hit test, which takes a point on the screen and tells you what is the UIElement underneath. It's used in applications like the Accessibility Inspector. And the Gesture support in VoiceOver. And also you have a focus UI element which tells you what has the keyboard focus. And also the accessibilityIsIgnored.

This is the method that allows you to determine if your UI should be exposed in the accessibility hierarchy or not. So by default we know for example that NSView is ignored. Well if you have a custom NSView and you want to expose it in the accessibility hierarchy, basically you know allow VoiceOver to see it, then you may want to consider taking a look at this method and override that. And also we have some notification support. Now this is very useful because it tells you what are the changes that's occurring on the screen. So you know things like keyboard focus changes, selection changes, etcetera.

By default, if you use Cocoa with just these all done automatically for you. But if you have some custom NSView and you want to be, you know you might want to take a look at this and see if you need to provide some of these notifications yourself. And if you do, we have API called NSAccessibilityPostNotification that allows you to basically provide those notifications. So in a nutshell, this is NSAccessibility protocol. I know this is a lot of information, but the good news is that if you use standard Cocoa widgets, you wouldn't have to do much at all because AppKit will do most of this work for you.

OK? And with that, I'm going to now turn over to Greg Hughes to show you how to determine how accessible your application is and offer you some of the solutions on how to accessorize your application.

So as Patti said, I'm going to show you how to audit your application.

And by audit I mean how you can examine your application for the areas where accessibility could be improved. And that includes areas where accessibility is lacking completely and areas where the accessibility information just isn't quite enough or you could add more information and it would be really helpful to a VoiceOver user.

And so to demo this, I'm going to use the old Apple example called Sketch. It has been a developer example back in Leopard and its also available in the Snow Leopard Developer Tools.

So here we have Sketch. We have a main, pretty simple window where we can draw shapes.

We have a Tool Palette. And also a Tool Inspector. Whenever looking at accessorizing an application, the first thing I always use is VoiceOver. I find it gives me a pretty quick way to look at the overview of the application. There's also another tool, Accessibility Inspector which I'll show you after.

So I'm going to start VoiceOver.

And it looks like we don't have sound.

Log on, building line output. Sketch window, graphics, now in window, sample, 100%, pop-up button.

OK, so now we have VoiceOver on and we just heard that we're currently on a pop-up button. So when looking at an application, I always try to start at the top level. I look at the window level of the application.

So I'm going to bring up VoiceOver's window Chooser which shows me a list of current windows.

VoiceOver Help menu, six items.

Since we'll assume I forgot the command, we have a really nice help system where I can look up any command line voice.

Control option question.

Where I can look up any command with VoiceOver.

Commands Help menu, Commands Help menu, 13, 10 items, Window Chooser menu, control option F2, F2. Window Chooser menu, 3 items.

So here we have the Window Chooser. And we see the first problem with this application.

One of our windows is just called untitled. And that doesn't really give a VoiceOver user much information for what that window contains. So if I arrow down to that window, we'll see the VoiceOver cursor, the small black bar highlight around the toolbar.

Preview current, untitled.

And so we see now in this tool palette, instead of untitled, this should say Tool Palette or something more, or something appropriate. So if we continue now and look at the Tool Palette and at the items inside of it.

Selected radio button one of five.

Radio button two of five. Radio button three of five.

We see that the radio buttons, the buttons on this Tool Palette, don't provide a description. And arguably, or most of you, many developers might argue that radio button really isn't accurate here because they don't look like a radio button. But to a VoiceOver user, it behaves exactly like a radio button.

Only one of these can be selected at a time and when you select another one, the one that was selected becomes deselected. So from an accessibility standpoint, these are radio buttons but we just don't have a description, we don't know what they do. So if we continue on and look at another window.

Window, Window Chooser menu, three items, preview current graphics.

Window 3.32, edit text content selected.

So we see here that we just heard 3.32 edit text content selected. We don't really know what this value means. And if we move along to another edit box in this window.

131.00 edit text content selected.

So we have a lot of information and the information is accurate.

But there's more information that could be told to a VoiceOver user. There's more information that could be output. And that's a label for these fields. There's really no contextual information. I could move around and infer here that the label to the left width is what is supposed to go along with this edit text.

But it'd be much better if the developer here could provide a label for this field programmatically so that we know from a VoiceOver standpoint what we should output as the label.

3.32 edit text content, 18.4% slider.

And here for the slider, we just hear a percentage.

So again, the output isn't wrong, it's just not the best output that could be provided to a VoiceOver user. The percentage represents where the slider is, but it doesn't tell me that it's really going to be a pixel value. It doesn't tell me that 3.32 value that's displayed next to the slider.

And if I jump now back to the main window.

Window, Window Chooser menu, three items, preview current sample. 100%, pop-up button.

Here when auditing your application, you'd go through and make sure you can do the entire workflow using just VoiceOver. Alternatively the entire workflow using just the keyboard. And with Sketch, we quickly find out that we can't add any shapes without using the mouse. This program requires you to move the mouse, click and drag in order to add a shape. And so this is a part of accessibility that's really important.

And that's making sure that your application can be used by users who might not have enough mobility to use the mouse or users who might be using just VoiceOver. And therefore not have access to the mouse. And the last problem we'll see here is that this drawing area is not accessible with VoiceOver.

If I try to move to it now.

I just get an error sound, meaning that there are no more elements to move to. And this is because this is a custom view and by default NSViews are ignored by accessibility. Because they could be representing anything and it's up to the developer to implement accessibility for that view. So now I'd like to show you Accessibility Inspector.

VoiceOver off.

And Accessibility Inspector is another great tool for auditing your application. But I personally find it much more handy when I'm programming. To use it, you just hover your mouse over any component. Here I'll hover over one of the buttons. And we see all of the accessibility information that's returned. So when I'm auditing an application, I find this a bit overwhelming. I don't really know what's missing, I don't know what really should be there. For example here, it's tough to notice quickly by looking at this that there's no title or there's no description.

Whereas with VoiceOver we saw that in a matter of seconds. When I switch and I'm programming, it's really nice here to make sure that the proper title or the proper description is returned or that any of these other values are actually what I'm sending back from my application. One nice thing about Accessibility Inspector is that you can lock on to an element using Command-F7. And you see here it highlighted the button in a light shade of red. And with Accessibility Inspector, I can navigate now up and down the accessibility tree with the pop-up button here.

So I can go up to the container and then all the way up to the window and look at the accessibility information there. And this is really helpful when you're trying to find problems with the accessibility hierarchy because in order for VoiceOver and other assistive applications to work properly, the accessibility hierarchy needs to be intact both going up the tree and coming back down the tree.

So this is a nice way to be able to find those problems and navigate through your application. So now we've looked at our application. We've seen the problems with it and we've seen areas that were lacking accessibility information completely and also areas where the accessibility information could be improved.

Specifically in Sketch, we saw six problems. The first problem was the window, the Tool Palette didn't have a description and we didn't know what was in that. We just heard untitled from the VoiceOver window Chooser. Next we saw the buttons in the Tool Palette didn't have descriptions, landing on them we heard radio button 1 of 5, radio button 4 of 5. And the radio button part was accurate, but we really need a description. We need to know that this is the Circle tool or this is the Selection tool.

Next we saw that in the Inspector, the edit fields and other controls in the Inspector didn't have labels and so an assistive technology user would have to try to navigate around and infer what value they were changing. And in most applications, there would be a fair amount of success, but depending on the layout, this can be really tricky for a VoiceOver user. And it ends up requiring a lot of guessing.

So here's where we could improve accessibility. We also saw that VoiceOver didn't really give us informative output for the slider. We heard a percentage instead of the value that the slider is actually representing. And this is just kind of the inherent nature of sliders. They provide a value back to the programmer of the application, which then has to interpret that value.

Well, for accessibility reasons it's good for that interpreted value to also be returned over the accessibility API. We then saw that the application didn't have, provide full keyboard access. The application required the user to use the mouse. And this is a really big part of accessibility as I mentioned a minute ago. And it's making sure that your application has multiple methods of navigation, multiple methods of input.

Specifically the keyboard and mouse. Last we saw that the canvas area was inaccessible. This is because it's in NSView and there's no accessibility information added here and AppKit could really only guess. Unfortunately we're not mind readers yet and we don't know what a custom view is going to contain.

So here's also another area where there is no accessibility information in this application. During this talk, I'm not going to have a chance to get to accessorizing this custom view. But there's going to be a really great talk on Friday about advanced accessibility. And it'll pretty much pick up where we left off and talk about customizing or sorry, accessorizing your custom views.

And it'll use Sketch as the example view in accessorizing that So now we have our list of problems. So what you would generally do with this list is just like any other bug or enhancement you're working on, prioritize them based on what's the most important area of your application to be accessible.

Sometimes it's tempting to grab the low hanging fruits but generally it's more important to do the area that's used more often by users. So here we're just going to go through them in the order that we found. Generally here we probably want to start with the canvas, but unfortunately since that's the most difficult, we will start just as I said in the order that we found them. So the first one was that the window title, that we didn't have a description for the Tool Palette.

For argument sake, let's assume here that the developer didn't want to have a title. Generally or sorry, all the time if there was a title, this title would be provided over the Accessibility API for us. So assuming that for whatever reason the developer didn't want to have a title as in this case, we're going to have to provide a description. And specifically we're going to use the NSAccessibilityDescriptionAttribute to provide that description.

This is a human readable, localized string that should be in all lower case and you want to make sure not to confuse this with the role description. This description is going to tell us what the button does. For example, the word print. It's not going to tell us that it's a button. And you want to also make sure not to include the role in the description.

Otherwise you'll get output like Print button button. So how do you provide this? Well the easiest way is to use Interface Builder. And in Interface Builder all you do is bring up the Inspector for any object you want to add this accessibility description to and in the Inspector there's an Accessibility tab.

And you can add the description or the help right in there. And just like the rest of your project, you would want to localize this with the rest of your nib. However this isn't always an option. Specifically in this case, you can't add a description in Interface Builder on Windows because the assumption is that there will always be a title. But in some cases you don't always have an object in Interface Builder that you can add this to or maybe you add something to your nib programmatically.

So we also have ways to do this programmatically. On the class level, you can override the methods accessibilityAttributesNames which returns a list of all of the attributes that that object supports. And accessibilityAttributeValue, which given a specific accessibility value returns the value for that attribute. So these have a few downfalls. They need to be used in the right place.

Specifically on the class level, now these are going to be used for all instances of that class. So if you had a button class that was used all over your application it might be difficult to do this on the class level because all of your buttons don't mean print. So here we can also use accessibilitySetOverride forAttribute.

And this allows us, given a single object, to set the accessibility override. So this takes a value in the attribute that we want to override and it sets it on that object. The two important things to remember when using accessibilitySetOverride is first that this shouldn't be used for dynamic values. This should be used for a value that stays the same. For example, we saw with the description of this window. That's not going to change. If something is going to change a lot, then we don't want to use this.

But also we want to use this specifically only for description, title, sorry, it's description, title UI element, linked UI element and help. And there are quite a few reasons for this. You can come and talk to any one of us later if you're really interested. But for now let's just take this on the assumption there's only these four this should be used for. So we've solved the problem now or we've seen a few solutions on how we can add a description to the window. How about the buttons? Well we could use any of those existing, any of those three methods that we just saw.

But we found that a lot of you have a lot of images all over your application. And so for Snow Leopard we have a great new API and some great new ways to add accessibility information on these images. So given an object that is an NSImage, you can do setAccessibilityDescription and provide a description for that image. Now, automatically this description will be picked up by some Cocoa objects, specifically in our case the NSButtonCell. And that description will be used automatically. But now we have an even easier way to provide accessibility descriptions on images.

And that's that we've built in strings file support. So as an example, let's assume we have our circle image here that's loaded as an NSImage or even used in Interface Builder as the content of a button cell. If in our project we created a file called AccessibilityImageDescriptions.strings. We could add a line that says circle equals circle tool. Now automatically AppKit is going to see this, attach this as the, link this as the image description and then the NSButtonCell will use that automatically. And this is really, really helpful because now it allows me to isolate where I'm adding all these descriptions.

So rather than go through my code and find where all these buttons are or where all the images are loaded, I can just add this one strings file in one place and add all the image descriptions right there. The one slight downside is that this image description now is going to be used every time that image shows up in your UI. So if you have something like an Add button, and in one location in your UI it meant add person, in another location it meant add mailbox, adding in a line entry here isn't really the best solution because you want to have a different entry for both buttons.

So you would have to choose another programmatic solution. So now we've seen how to add descriptions and we can move along to how we can improve the interface, how we can add labels for some of these UI elements. So this is really, really simple. And like description, we can do it in Interface Builder.

We can just control-drag from the object we want to label to the label and select title. And the easiest way to remember this is just like anything else in Interface Builder. You always start at the object that needs to know about the other object. In this case, the text field needs to know what its label is.

And then you just select title. So again like any of the other accessibility attributes, we can also do this in code. We can override the accessibility attribute names to make sure that we're returning NSAccessibility title attribute. Sorry, title UI attribute in this array of support attributes. And then we'd return the appropriate element when asked for the title UI attribute.

As we saw earlier, this is also one of the appropriate attributes or one of the safe attributes to use with set accessibility override. So we could do that on the object level as well. Now with NSAccessibility TitleUI element, you'd almost always want to do this on the object level. There would be very, very few cases where you would want all objects of a single class to have the same TitleUI element.

So if we move along to the slider. The slider gave us a percentage value, it didn't give us the actual value of the slider. So here again we want to provide more informative output. And for the slider, the way that we're going to do that is through the NSAccessibilityValueDescriptionAttribute. Unfortunately since we saw that this is both a dynamic value and not one of the safe values to use with set accessibility override, we're going to have to do this on the class level.

And here we're going to override these same two methods again. accessibilityAttributeValue and accessibilityAttributeNames. And this is the third time I've shown this and it's because it's really important that you think of these two methods together. Anything that you support in accessibilityAttributeValues has to be one of the values returned by accessibilityAttributeNames. I can say I personally have countless times added all the code to accessibilityAttributeValues but not added that I added the value, the attribute to accessibilityAttributeNames.

So then that information was never used because any client didn't know that my object was supporting for example NSAccessibility value description. So it's really important here to make sure that these two methods are in synch. That they both support the same attributes. And you also want to try to remember not to add an object to the array twice. As you'll see in the demo, we're going to append NSAccessibilityValueDescriptionAttribute onto the array of attributes that super supports.

So the last problem we saw that we're going to be able to address here is the full keyboard navigation. How can we add keyboard navigation to this application? Unfortunately that alone could take up an entire presentation because there are tons of different techniques and tons of different ways.

And it's really application specific. Generally the best course of action is to add menu items and keyboard shortcuts. But maybe your application is more graphical and might require different technique or more keystrokes or arrow navigation to allow full keyboard access to a user who might not use the mouse.

So for accessibility purposes here where we can look at examples of the accessibility API, we really now have four issues that we can deal with. First again was the title of the window. And I'm going to show you how we can solve this now with all of our accessibility APIs that we've talked about.

We're going to choose some and implement them. Here to fix this, I'm going to choose to use the accessibilitySetOverride forAttribute method on the NSAccessibilityDescriptionAttribute. For the buttons in the toolbar, I'm going to use the AccessibilityImageDescriptions strings file. And there we can have one isolated place where we add all of our image descriptions for all of our buttons and it will automatically be used by the Cocoa buttons.

Then we saw the Title UI elements. And the easiest way to do this is just in Interface Builder, we're going to link up the controls to the element that we want as the label.

And lastly for the slider that gave us a percentage rather than a value, this is probably the hardest one we're going to do.

But as you'll see in a moment, it's still really, really simple and it doesn't take much time at all. And this is that we're going to have to subclass NSSliderCell. And override the two methods accessibilityAttributesNames and accessibility attribute description. Sorry, accessibilityAttributeValue is the other method.

So here we have our game plan for how we can go ahead and begin accessorizing Sketch.

And we can get right into it.

So here we have our Sketch project. And as we said the first thing we wanted to look at and fix was the title of that Tool Palette.

So here I have the Tool Palette controller. And I can find the windowDidLoad method.

And in this method, I'm just going to add two very simple lines of code.

And so here the first line of code is just generating a localized string. This is important to remember with all of the accessibility API. You want to make sure that these strings, just like your UI are localized appropriately for the interface.

So here we're going to generate a localized string for Tool Palette. And then we're going to use set accessibility override for the window and we're going to set the override for NSAccessibilityDescriptionAttribute. Here one other little tidbit that's really useful, I know that window in this case supports accessibility description. But as Patti said earlier, buttons don't. So if for example I had an IB outlet to a button, this wouldn't work because the button isn't what I set the accessibility override on. It's the button cell.

And so there it's a really nice way to approach that is that there's a method called NSAccessibility unignore descendent. And so you can use NSAccessibility unignore descendent on self, that will return the button cell and then you can set the override on the button cell. And so that's a really common mistake that a lot of developers make that you need to make sure that you're setting the value on the appropriate controller, the controller that handles the accessibility information.

So now we saw, the next thing we saw was that we wanted to add a description to the buttons. And the way to do this as I said, the solution I chose here was to use the new Snow Leopard API of AccessibilityImageDescriptions strings file. I have already written so I don't have to type it over again.

And here we have really simple entries like circle equals circle tool. And you'll notice that circle is the exact name of the image. And that's obviously the important part here. That's what links up the image description to the image that your application is going to load.

So that's all I need to do here and this is what makes the strings file really nice. I don't have to go through my code and find where these images are used. I just add this one file. Next we saw that the items in the Inspector didn't have title UI elements. They didn't provide that contextual information. So we're going to open up the Inspector.

And as I said, this is really simple. All I do is Control-drag from the object I want to label, to the label, and then select title. And so I've already done this for the rest of the controls in this window. Here's also a good thing to point out. The description is really easy to set right here if I wanted to or the help. So that's all there is to adding those labels. I'll just save that.

And back to Xcode. Now the last thing I said we were going to do is subclass NSSliderCell. And this is the most difficult thing we're going to do for accessorizing this application only because we have to subclass and a lot of developers we found are afraid of sub classing. But it's nothing you need to be afraid of for the NSSliderCell. It's really not that much work to add this accessibility.

So here I'm going to just add a new file and it's an Objective-C class.

And I'm going to name it accordingly, according to the naming convention of this project. Now here's a good point to note when doing accessibility of your project. Sometimes it's really nice to keep accessibility completely out of the rest of your code. And by that I mean create an accessibility files for each of your classes.

And you can create protocols for your class. In the class of this project, our accessibility code is interweaved within the rest of the code, which has its advantages and its disadvantages. So for your project you generally want to choose one of those two and stick to it. Like the rest of coding conventions.

So here we have our pixel slider. And this isn't going to inherit from NSObject, it's just going to be, it's going to be an NSSliderCell.

And the implementation again is pretty simple. We just have these two methods. The first one, accessibilityAttributeNames. And this method as I said earlier is going to return an array of all of the attributes that our object supports. Well our object is going to support all of the attribute names that super supports along with NSAccessibilityValueDescriptionAttribute. Then we have, sorry, then we have accessibilityAttributeValue. And as I said, all this method does is given a specific value it needs to return the attribute.

So given NSAccessibilityValueDescriptionAttribute, we're going to return a localized string representing the number. And so here it looks a bit complex, but it's just localizing the number format so that if your locale is set to use a comma instead of a period, we'll properly use a comma instead of a period here. And then importantly, we don't want to have to deal with everything else because super already knows how to deal with it.

So if we're asked for something like our frame or our position on screen, we just pass that call off to NSSliderCell because the NSSliderCell knows where we are.

So the last important thing here is to tell that slider to use our custom slider cell.

And that's it. Here we have our accessorized Sketch application.

And if I start VoiceOver we can verify that all of our accessibility information has shown up in this application.

VoiceOver on, Sketch window sample, 100%, pop-up button, window, Window Chooser menu, three items, graphics, Tool Palette.

Here we see the Tool Palette has the appropriate name in the Window Chooser.

Selection tool, selected radio button, 1 of 5, Rectangle tool, radio button 2 of 5, Circle tool, radio button 3 of 5.

We see that all of the tools in the Tool Palette have the appropriate descriptions that were brought in from the strings file automatically with very little work on our part.

Window Tool palette. Window Chooser menu. Pre current graphics.

Window fit, yellow at 3.32, line width edit text content selected.

So here we now hear the label. We heard 3.32 line width. Because line width was the label for that field.

131.00, width edit text content selected.

And again we heard that width, we heard width after the value.

So that's telling us that width is the label for this field. If I now go up to the slider.

3.3.32, line width slider.

Now we hear 3.32 line width slider. Which is much more informative to a VoiceOver user than 18.4% as we heard earlier when we did our demo.

VoiceOver off.

So just as we saw before, we can now use Accessibility Inspector.

And we can see now for this button, we properly have an AX description which tells us what this button is representing. So we go through it again, just we did with VoiceOver and use Accessibility Inspector to look at the rest of this UI to verify that everything that we've done works.

The one last thing that I've gone ahead and done is that I've added full keyboard navigation. So I can use hot keys or VoiceOver commands to go to these menu options. And the object is drawn for me and I can then use the Inspector here to move it anywhere on the screen I want.

So now we saw that it was really simple to add all of the accessibility information. It probably took me about three or four minutes to add the information to our application and it makes a huge difference. We still have that canvas area which again I encourage you to go to the talk on Friday on Advanced Accessibility.

But the rest of it we have a vast improvement. Before we couldn't get to the button, we didn't know what windows were called and there was no contextual information in the Tool Inspector. So we've made a huge difference in a very short amount of time on this application. And really it doesn't take much effort.

The biggest challenge is to just go in and start auditing your applications. And so we have some final thoughts. And throughout this talk, the first thing we saw was that accessibility provides a lot of great benefits for both you and your end users. For you it allows you to do a lot of GUI scripting, it allows you to use instruments in a more powerful way.

It also allows you to use Automater Watch Me Do. And these are all important tools when trying to reproduce bugs or script certain unit tests or script just general tests for your application. Without accessibility information, all of these things are much more difficult. For your end users, they get all that, all those same benefits.

They can use scripting to automate different tasks. But it also extends your marketability to the VoiceOver audience, to users who are using assistive technology. And there are just two, there are two main steps that you need to remember about your application on how to accessorize. It can be a bit overwhelming to look at an application, wonder where to start. And then the first is just either you use Accessibility Inspector or my personal favorite, VoiceOver to audit your application, to look at where do you need accessibility information? And generally you'll be a bit surprised at how much information is already there.

Then once you've analyzed your application, look at the APIs and figure out what API is appropriate for your problem. And most often, it's just a simple Interface Builder solution to either add a description or add a title UI element. There's a lot of resources available. There is a contact at Apple, Eric Hope, but we also have a really great public developer list for accessibility. And Apple engineers read this list and generally respond in a really timely manner.

But also the community responds with a lot of helpful information if you have any questions. I'd strongly recommend using this list. There's also the accessibility documentation and we have a great new website which has a lot of accessibility information both developer related and end user related. So if you're just generally interested in accessibility at Apple, this is a great resource as well.