Okay. Good morning everybody. My name is Eric Seymour. And today along with my colleague, James Dempsey, we're going to talk to you about adding accessibility to your application. And actually, specifically, I'm going to spend a little bit of time this morning, just a little bit, talking about some of the higher level issues related to accessibility, give you a little feature peak into VoiceOver for Leopard.

And then for those who are new to accessibility on Macintosh, either from an end-user or a programming perspective, just touch on what our accessibility APIs mean so that when James comes up on stage and actually does some real detailed work, everybody is kind of on the same playing field, understands what's going on. So let's get started with the first question that always happens at the beginning of accessibility talks and that's why make your application accessible? So there are several really good reasons.

And one of them starts with your user so a really good reason to access enable your application is because there's this community of users out there that's getting bigger all the time, especially now that VoiceOver has been shipping for two years. And these users don't use a Macintosh in a conventional way. These users use assistive technology. So they're not using the graphical interface all the time.

They're using VoiceOver. They're using some other, some other solutions that I know some folks in the room create. And when you access enable your application, what you're effectively doing is making your app, whether it's the most popular in its class or not, you are making it the best of read application for that user.

And that becomes a highly leveragable thing when folks or entities who are purchasing on behalf of that user, for example, a school system or the federal government or a state government or anybody who's contracting on behalf of one those entities, when they're looking for a solution to purchase, your application becomes best of read. And it makes it easier for us when we're trying to do a large volume sell to reference something. For example, like your application as a solution.

But it opens up a big opportunity to not just get one or two new seats, but to actually get perhaps many, many more. So we think there's a lot of value add here. And I'm going to mention a statistic in a couple of minutes about VoiceOver, which has made this even more profound.

So the second reason really has less to do with accessibility and it has more to do with enabling other technologies on the system. So obviously system applications work when you enable your, when you enable your application. But also internal are tools that we ship. For example, Automator, a new tool that we're shipping this year called X-ray. Both of those leverage the accessibility APIs.

You can think of this as user interface automation. Internally to Apple, we take advantage of the accessibility APIs and some internal tools to do testing so that we can make our products better and more robust. In fact, I know that some of the folks, perhaps even in this room, some developers use accessibility APIs not for assistive devices, but specifically because it helps them test their application.

And I know this because I get bugs in our bug system that have to do with accessibility and the driver's not an assistive application. The driver is we're trying to automate our testing process and this particular bug is preventing us from doing this. And so we fix it. And that's a really good thing, see. At the end of the day, accessibility is more about user, more about user interface automation.

The fact that it helps a significant group of users is really, really important. So on to VoiceOver. So give you a little VoiceOver update. So the first thing is VoiceOver for those of you who haven't used it before or are unfamiliar with it, VoiceOver is screen reader technology for Mac OS X.

So a screen reader is typically software that allows you to get to every place on the screen, even painted text, regions that wouldn't ordinarily let you get there from a keyboard, it lets you have full keyboard access to those locations. So sometimes it's called screen access software. It speaks everything on the screen so clearly it text the speech capability in VoiceOver. Indeed is very, very strong and it's getting stronger as I'm going to demo in a couple of minutes.

So if you have seen VoiceOver or any of our marketing materials, VoiceOver for Tiger, which has been shipping for two years now. One of the things that's been really resonating with our customers is our collaborative experience. And that is when you're running VoiceOver as a user who's visually impaired for example, there's some things that happen on the screen that you might not be able to see.

Or if you have low vision, there's some things on the screen which might help you because you can make the fonts really big. But what's cool about this is if you're a sighted user and you happen to be working with somebody who's visually impaired, you can understand what they're doing on that computer and they can understand and hear what you're doing on your computer. And we think, and feedback has told us this instinct was correct.

After a couple of years of shipping, we think that this is resonating with our customers. You know, a child in school who happens to be visually impaired has the right to work side-by-side with their sighted peer or their sighted parent or vice versa. So really, really nice message. So what's new for VoiceOver and Leopard.

Well, there's lots of new features. We've taken a lot of feedback. There's two things I want to focus briefly on today. The first, some of you may have heard already and that's this great new voice named Alex. Alex just sounds really, really good. I'm going to demo Alex in a couple of minutes. It's basically a new approach to concatenative synthesis.

It's really better just to demo it. One of the things that it provides and it's something that's very important to screen reader users is incredibly fast talking while still sounding good and still being understandable. And I'm going to show you that in a second. And the other thing is Alex is such a nice voice, you can actually with, with good quality audio, you can actually hear Alex breathe. Take breaths before speaking something.

Then say something. And it turns out that that's a very important que for users who are listening. In fact, us, not just users, anybody who's sighted or non-sighted who's listening to speaking, we actually take a lot of ques from things that we wouldn't think about when we're listening to speaking. And so Alex takes advantage of those and offers some of those things. So big feature number two is Braille support.

So VoiceOver will be adding refreshable Braille support to Leopard. And for those of you who, do you have a display I can show, I can show? For those of you who don't know what Braille is about, I'm going to give a little Braille primer here. But basically, they're these devices. This is one. There are dozens of them.

And they have a row of Braille cells. I'll talk about Braille in just a second. And a user who's visually impaired, he might have one of these displays, might prefer to read in Braille what's happening on their screen rather than hear it or have a combination of it.

And so they would take this and they'd take their USB cable or their or what have you, they plug into their system and then you would sit it down in front and they would be feeling what's going on in their computer. And so this is a refreshable Braille display.

So I'm going to demonstrate this with VoiceOver in a second. But before I do that, it's important to have a little Braille primer so that everybody is on the same page. So Braille, if you've ever, if you're sighted and you're not familiar with Braille, you may have noticed some Braille on elevators and placards and things. And you typically see it in six dots. And these are, these dots are very standardized, one through six.

And in computer Braille, we actually add two more dots because we can and it gives us more options of what to display. And basically, these dots when used in different combinations are symbols for letters of the alphabet or numbers or punctuation or whatnot. So there's an A. There's a B. There's a C. Sometimes you can combine multiple cells to mean something so for example capitalizing a letter could, with six dot Braille, could mean add dot six preceding the letter.

Adding those cells that you could see there, dots four, five, three, four, five and six preceding something converts a letter to a number. In computer Braille you can accomplish the same thing by using dot seven, instead of the prefix cell. So it gives us some more options. So anyway, Braille cells are put together to create Braille words.

And Braille words are put together to create Braille phrases. Now, you could imagine, one thing about Braille is, Braille is a standard size. Visually, its size is about 18-point font. 18-point font is pretty big. And Braille is also fixed width. So you can imagine that if I had a dictionary or an encyclopedia or a long book, you know, big volume of Harry Potter or what have you, it would take up a very, very large amount of space.

Sometimes book shelve space. So one of the things that's done to counteract that is the concept of contracted Braille, which means sometimes phonetic sounds are of common word groups are turned into symbols so that we can shorten the amount of Braille that's taken. And it turns out that Braille readers are generally really, really fast at reading Braille.

Shockingly, faster than those of us who are sighted might, we would be surprised. Just as surprised as we are to know that somebody who's listening to a voice at an incredibly fast rate can actually understand what's going on. And it's important when we support Braille for us to support both contracted and non-contracted and to know when to contract and when not to contract because depending on whether you're editing text or you're reading buttons, we have to do the right thing and provide the right options. And so that's what I'm going to demonstrate. In fact, the best way to that is actually to go to demo. So, oh, actually, no. Before I go to demo, one more thing.

So now here's our updated screen shot with voice-over running a Leopard and, for those of you who are looking at the screen, there are a couple of visual aspects to the screen that are the most common hallmarks of voice-over. One of them is a rectangle, a black rectangle around one of my icons.

That's the voice-over cursor and that helps somebody whose sighted see what voice-over is currently focusing on. Then there's also this caption panel that you can turn on and this always prints what's being spoken. So, somebody who can't understand the fast speech that somebody might be using can just read it right there and they know exactly what was spoken.

Well, Braille produces this other challenge. Now there's this physical device, somebody's not even getting output, they're feeling dots on a page. Well, not it could be on a page, but in this case, it's on a refreshable Braille display and so how do we add collaboration to that? So, we're introducing the concept of a Braille panel and the Braille panel will always display what is on that Braille display.

It will also translate in real time whether your in grade one, grade two, that's contracted or non-contracted Braille, everything that is going to that Braille display. So, a sighted user, a teacher , a parent whose watching their child learn Braille and use this Braille display and knows exactly what they are reading when they're reading it. So, the best way to continue here is to show a demo.

Ok, so, this demo's not going to be standard voice-over demo where I chain myself to the keyboard and I get rid of the mouse and all that. Voice-over is kind of tried and true in that respect. This is sighted guy giving demo of a couple of key technologies. I'm going to trade back and forth with the mouse just to keep things moving. First thing I want to do is talk about the voice. Here's just a standard text document and I'm going to turn on voice-over.

So what I'm going to do is I'm going to speed up the voice to an acceptable speed. In fact, I'm going to jack it up to 100%. I'm just going to go into voice-over utility.

Windowdemo dot rtf. Voice-over is intended to provide those who are blind, have low vision, or learning difficulties the ability to use a Macintosh. It's perfect for those who have never used a computer or a screen-reader before, for those who are new to Macintosh, and for those using screen-readers on Mac OS 9 and want to ?

Ok, so what's important here is why you might not just say Here, highlight this entire document and start reading it to me at 100% speed. What you want is the ability to navigate exceptionally quickly and have your response be really, really fast and know what's going on.

So, as I arrow around? ( Computer speaking ) >> if I go word by word. ( Computer speaking ) >> In fact if I just lean on the cursor? ( Computer speaking ) >> I want to be able to move around and hear things really, really fast. So, that's what we think the new voice provides.

So, the next thing I want to talk about is Braille. So, I am now going to enable the Braille caption panel, so pretend, if you will, that I plug this in, but this is kind of a hard thing to demo to a crowd in the dark who's not right on top of it.

So, I'm going to turn on the Braille caption panel. What would happens here when I plug this in, voice-over will automatically detect it and it will bring up the Braille caption panel by default so that a sighted user can also see what is going on.

Voice over utility window. Table one row selected. Row nine (inaudible) panel.

Text added Windowdemo dot rtf.

- All right, here's the caption panel and let's do another thing here.

- Let's go to speech?

What we're doing is straddling dots eight and seven of the previous and the following cell to mean where the cursor is, which maps exactly to the way Mac OS X actually does cursor positioning with voice-over and you can just move around.

So, that basically means when I'm in text and I'm on something it selected, you know that when you're feeling that thing that's selected, you're getting letter for letter which is exactly what you want, but everything that's surrounding it that the selection is not touching is contracted and, of course, you can turn contractions off, so if I Shift plus highlight something?

Let's do this?

Ok, so now I want to talk about some accessibility basics and this is really just the primer before James comes up and goes a little deeper about some things. Accessibility basics? Mac OS X accessibility really comes in three forms. The first form is just universal access. We've had really good universal access support with all sorts of features that don't necessarily relate directly to screen readers or assistive technology, but there's zooming, making the screen contrasty, and different speaking features and things like that and most apps just work with them.

Then there's also the concept of full keyboard navigation, again all the plumbing is there in Cocoa and in Carbon to do keyboard navigation and to support that through the accessibility hierarchy in all the right places, but you still need to do a little bit to ensure that your application can be driven without a mouse. That's something that's really important. Part of a QA cycle should be Hmm, let me unplug my mouse and see if I can actually use this application and make good use of it and get to every feature.

And, then finally, there are assistive applications like voice-over that want to talk to your application and drive it via remote control. So, that's what we're talking about today. So, accessibility basics, you've got an assistive application out there or perhaps its Automator, perhaps its x-ray, or perhaps it's your testing tool, right.

But, let's say it's an assistive application and in voice-overs case it's just this process that's running, that's fairly tightly coupled with the rest of the operating system and it has an assistive user interface. It's a different user interface, it's not one built with Cocoa widgets and buttons. It happens to speak and now it happens to Braille and what it needs to do is it needs to communicate with your application and it wants to do it using this accessibility API that we've been shipping for several years.

The real question comes down to is the application we want to talk to, your application, is it accessible? And, so, what does this mean? Perhaps, more explicitly, how is your application coded and should an assistive application have care about that, in fact, we don't want to care about that. So, one of the primary things that the accessibility APIs do, is they abstract the notion of how your coding into a common language and that common language is the UI element. It's the unit of access that one uses when their accessing your application.

So, everything is a user interface element to an assistive application. There are windows, every window is a user interface element, every button, text field, even larger canvases and things nested with it inside canvases and elements live in a hierarchy. Elements have parents or elements have a parent chain and elements can zero or more children.

Your application is also an element, so everything uses this element as its unit of communication. We contact your application element and we say Give me your window elements and then we ask your window element Give me your widget elements and so on and so forth. Then what we do with those elements is we deal with their attributes, sometimes we ask for them, sometimes we set them, sometimes we call actions on them, like we say press for a button, and sometimes we receive notifications. For example, if somebody slides a slider, we need to be able to know when that slider is slid so that we can speak something to the user or Braille its value or what have you.

So, attributes really describe an element. For example, I might have a UI element that's a button in an application, for example this address button in mail, and there are all these attributes that I can ask about. Some of these are attributes that you get for free by implementing Cocoa. In fact, most of them you get for free by implementing Cocoa and even Carbon.

Some of them you should never change just because the role and the role description are very standard things that voice-over relies upon and it'll just do the right thing. Then, sometimes, there are contextual attributes that only you can predict. Like if you have an image with a picture on it, we don't know what that is until you add an AX description that says This picture means stop and so, then we can speak Stop when land on that button or whatever it means. This picture means delete all files, maybe the user would want to avoid that, so they should know what that is.

So, attributes are, at the basic level, presented in your code by some methods and fairly easy to implement. James has several examples, some of which he's going to reference that we've shown in years past, and then he's going to build on a new one today. But, basically you add attributes by just implementing or sub-classing methods in the NS accessibility protocol and returning Here's my list of attributes or I'll take a value for this attribute and set it. It's fairly to implement.

Actions, same thing, there are some standard defined actions that are very basic. Actions aren't actions like print this panel or save this document or something that's more workflow based, they are gestures. Press open menu and so you just put actions in all the right places and it means the user with a special device or assisitive application can do those gestures. The API for those is very similar, there're methods that you would override for adding actions and returning them and for invoking actions when you're asked to.

Then there are notifications. Notifications, a little more complicated, you get them for free for the most part with Cocoa, but basically if you've got a widget, for example, that changes the value, you would need to post the value change notification. Notifications are really important aspect of assistive applications because we need to know what your application is doing so that we can say it in a timely fashion to the user.

There's also the notion of hidden elements. This is a really important point. James, I think, has some really good slides coming up which talk about this, but when you code your application most of the time you're coding, sometimes for convenience with the kits, lots of grouped views. These views, sometimes, aren't apparent to the user. What the user might see is two views rather than four or five and it's important that the accessibility hierarchy that you create, that's represented, is what the user sees, not what you see in interface builder or what you see in your view hierarchy.

So, there's an easy method to implement on your views where you can say Even though this view's in my hierarchy, accessibility shouldn't show it and, in fact, if you just take advantage of NSview, the whole set hidden mechanism hides this for you and you never need to deal with it. Then, a really important aspect is hit testing.

One of our accessibility tools, Accessibility Inspector, relies on this and that's, basically, when I move my mouse around the screen, anywhere that mouse is hovering over, I should be able to ask your application What's the UI element under that point? It's fairly easy to implement. Its usually, again, you get it all for free with Cocoa and Carbon for the most part, but when you've got a custom view with stuff inside it, you need to make those things available. Ok, so, now I going to ask James to come up and talk to you in more detail.

Thanks, Eric. Hi. My name's James Dempsey. I'm an engineer on the Cocoa frameworks team and one of the main things that I do on the team is work on the accessibility portions of Cocoa. So, first we're going to do a brief Leopard update, talk about some of the new features and functionality that we've added in the frameworks to accessibility in Leopard. Then we're going to talk a little bit about implementing accessibility in terms of kind of methodology that you can use in terms of doing so and we're going to use, as a specific example, a new technology introduced in Leopard which is CoreAnimation.

So, Leopard update, we're going to go through these one by one. The first thing we have done in Leopard, Bertrands, I think, in his state of the union was talking about CoverFlow and the Finder in mentioning that we're now able to put Cocoa views in the midst of Carbon applications. In fact, in the Finder, CoverFlow is a Cocoa view in a Carbon app and there's a new HI toolbox, Class HI Cocoa view.

What we've done is done a good deal of work to do Carbon and Cocoa integration to make sure that, first, Cocoa windows in Carbon applications, all of their accessibility will come through and, also, if you're using HI Cocoa view that as long as that Cocoa view is accessible that all of the accessibility will just work for you no matter whether you use that in a Carbon app or a Cocoa app.

So, toolbars, in Tiger and earlier. Well, actually, as long we've had toolbars there have been two ways that you can make a toolbar item in Cocoa. You could give us an image or you could give us a view. If you gave us an image we would take that image and the title you gave us and make one nice big button out of that image. If you gave us a view we would use that view as a group and then all of the elements in it would be sub-elements, which makes for some, in some cases, very complicated toolbar items, especially if you gave us an NS button.

We would have a group with a button and a label when really what you probably wanted was just one big button. We, also, in the case of things like mail where people are using segmented controls and multiple labels, we would end up with two sets of information. So, in Tiger we've done a lot of work for some NSToolbar enhancements where, for things like segmented controls with multiple labels or NSButtons, we will treat those items as one giant button rather than a complex set of UI elements.

So, we've also taken a look at things like QuickTime Player and these ideas of media timelines and we've introduced a new sub-role of a slider AX Timeline. And, so, if you're building an application and you are implementing your own media slider, this would be the sub-role to use.

Note that, as opposed to a typical slider which has one slider value indicator, in QuickTime, if you have QuickTime Pro and you're editing the movie, there are three places you can edit. There's the current time, as well as an end point and an out point for the edit. We represent these in an AX timeline as having value indicator children.

It's also nice to note that a value indicator now actually has an AX value in Leopard. If there are multiple value indicators, you should identify them with a separate AX description for each so that the user can tell them apart. If you're writing an assistive app also be on the lookout for apps like QuickTime reporting this new sub-role.

As we were developing Leopard we noticed that, over time, we were seeing a lot of grids showing up. So, this is image kits, its image browser view is in the form of a grid. This is PDF kit has a new thumbnail view which again, very grid-like, and even the DOC with its new stacks has a grid orientation. A grid is not quite a table, which might be one way you might think of accessorizing it and it's not quite a list.

In fact, it's kind of more like a list because it's an ordered group of things, however, there's no information about tables and rows. So, we have added a new role in Leopard, AXGrid, for these types of interfaces. The children of that UI element are ordered, but we also give you information about how many rows and how many columns and whether it's row major or not. So, ordered by row is a Boolean.

We also, as we've been going through in Tiger have noticed that occasionally you'll have a slider where it's providing a very complex value. For example, in the energy saver panel, you can pick one minute, one hour, or never, and it's very hard to let a user know that a hundred means never.

It's hard to map never to a value and so we've added a new attribute for these sorts of situations, AxValue Description. It's an optional attribute and you should use it if you're accessorizing whenever AX Value does not provide enough information for the user. It should be very similar to an AX description, a localized lowercase string and you should dynamically change it so that it matches whatever the appropriate value would be for, in this case, the slider.

So, for energy saver, it might be anything like, they tend to concatenate things, so somewhere in the middle there you might get an hour and fifteen minutes, it might be never, it might be a minute. AX timelines would also be a good candidate for adding an AX value description since, again, you are specifically talking about a start time and an end time, so often giving that description in some sort of time code of minutes and seconds would be very useful.

In Tiger, I believe it was Tiger, we added in the Cocoa text system the ability to be able to select multiple pieces of text. However, we now have the ability to get those selected text ranges via accessibility. It's an optional attribute, however, all Cocoa text objects implement it and it returns the full array of selected text ranges of the already existing attributes selected text range will still behave as it did before, which is returning that first selected range.

And then the last thing I'd like to mention in the Leopard update is Interface Builder 3 point 0. Interface Builder has had a very nice makeover. I've been playing with it a great deal over the last few months in Leopard and one thing they have done is they've moved some things around.

So, I wanted to point out things that you might be used to in previous versions of Interface Builder, where they've moved to in the new version. So, for things like adding AX descriptions, which are incredibly useful for images or buttons with images, for instance, or AX help settings and the Identity Inspector we can put those on.

And, then if we're connecting to or dragging to connect different UI elements for a link UI element or to say that this particular text field, its title UI element is this piece of static text. We, just like before, can control drag from one element to another in Interface Builder, but nicely enough, all of the accessibility connections show on this pop-up and it's handled exactly like all other connections. We don't have to go to a separate panel anymore, so that's very nice.

I haven't done technical training for a while, I always feel like saying Any questions??, but not this time around. We'll have that at the end. So, implementing accessibility. As I've been working on accessibility, I've been thinking about it. We often think in terms of voice-over and kind of the result that we get in terms of the speakable text, but when we think about it also in terms of things like Automator's watch me do feature or Xray or using it as a testability tool.

I was kind of thinking about accessibility and what it really is at the heart of it in terms of the implementation and, at least in my mind, it's an out of process experience for your user interface and, if we think about it, it's a little bit like an out of body experience where kind of all your pixels are left behind, you're not really drawing the particular images, and its only the very core and the very essence of your user interface that's kind of over here in this other process while the apps running.

And, so, it's a very powerful notion or this wire frame of your user interface, the very essence of it, being able to be projected into another process and controlled. It's actually a very functional out of process experience. So, we can find out notifications that things have changed, so as things update in your application, of course, notifications cause us to be able to get information from that other process.

We can drive the application from there. If we're just pressing a button in one place and it takes effect on the other, dismiss the sheet and we can get a lot of information about each piece of the user interface. It's an extraordinarily lightweight representation of your user interface and, at least for me, just from a conceptual point of view that's a very, very cool thing.

But, also I think that mental model can help a great deal as you go about making your application accessible because although, yes, the end result is voice-over gets to use it and interact with it and assistive devices and assistive applications and tools also get to do it. In a sense, what you are doing when you make your application accessible is you are giving the best possible, or are trying to give the best possible, representation of your user interface to these other applications and mapping what's going on in your app to what's going on in those applications.

So, how do we implement accessibility? We'll talk through these, a number of these topics we have definitely covered in the past, so there are a few that we will spend a little less time on. They're very well documented on our site as well, but we'll certainly mention them. First, test using accessibility clients.

We have two accessibility specific utilities. A verifier, that if you've never really touched accessibility on your app, you can fire up this verifier, set it to your running app and let it loose, kind of hit test and it will run through and do a verification to see kind of what kind of shape your application is in.

Do you have buttons that have images on them, no descriptions so nobody can tell what they are? Do you have big blank spots in your application? The Accessibility Inspector, so you can interactively look at various pieces of your applications accessibility and then voice-over certainly an end-user application, but we've found that it's also incredibly useful to use for testing.

So, once you do that, probably the first thing that you'll need to do and certainly the item to do that has the most bang for the buck, as they say, is to add instance-specific information. So, in this case, we have two items in our user interface and then on the left we have kind of that wire frame representation in the other process. In the first one, add playlist.

What will come across automatically from Cocoa, because we have the text already, is that it has an AX title of add playlist. So, in that other process, it is pretty obvious what this button is going to do. However, at the bottom, we have this button with an image in it and its title, it doesn't have a string for a title.

There's nothing being displayed in the user interface, so it's very difficult to tell from that assistive application what the heck is this button for Is this the Don't press this button button? Hopefully not because I'm going to press it. So, how do we address that in Interface Builder? The easiest way is you select that button and you add an AX description and, actually, I think they fixed my capitalization, those should be lowercase add playlist. But, it should be a localized lowercase string and it should not include the name of the user interface element. So, add playlist.

The other thing you can do is, especially in Interface Builder, is relate user interface elements. Somebody looking at the top one, a sighted user, it's pretty obvious that that name label goes with that text field, however, it's a little less clear when we bring that over into an assistive app and, so, we can add an attribute, the title UI element attribute, where we connect the text field and have it refer to its static text label. Again, this we can do easily in Interface Builder. So, with just those two things, adding descriptions and connecting relationships between the two user interface elements, all in your NIB file, all in Interface Builder, small changes have massive usability results on the other side for assistive applications.

Another thing you made need to do are make some sub-class specific tweaks. So, what we can do in Interface Builder, we can add static pieces of information. This is related to that or this is an add playlist button, but, for example, that slider where the value I think your description will change depending on the value and obviously ahead of time we as the framework designers can't know what value descriptions you're going to need.

You need to write a bit of code. In this case you would subclass the slider and then you would implement these two methods. You'd override accessibility attribute names because you're going to be returning a new attribute. In this case accessibility value description attribute, and then you would over accessibility attribute value and if the string or the attribute being looked for happened to be the value description you would return the appropriate string.

Now note it's very likely if you're already, as the energy saver panel is, using that string in a particular tool tip it's possible or probable that you already have a method that returns the right string, so you would just call that. If not you would obviously write a new method. And then if it's not that particular item you just get the super classes value and off you go.

So after we get through those items, kind of tweaking some subclasses, adding descriptions, adding connections like title UI elements, we might end up with seeing some trouble areas that get reported, and this is ripped from the headlines. This is, I was working with a developer and we had this problem.

They had an inspector with a bunch of buttons across the top, and they'd use that to choose that to choose between different inspector paints. But then they would look at it in voice over, or they'd use voice over with it, they would look at it in accessibility inspector and what they'd up seeing there would look more like what's on the left, where they'd have these two ends hanging out and we're looking at that.

Why was that, well they were misusing a segmented control, like really misusing a segmented control. They didn't like the look of the end caps but they liked the look of what was in the middle so they made it too big, clipped it to the windows so that it looked right.

They also noticed that as they used full keyboard navigation that it would hit the first one which was off screen tabbed through the middle ones and hit the last one. So they turned off full keyboard navigation. They went out of their way to turn it off. So although it looked right, it wasn't right.

And I think sometimes when we're developing applications, the Macintosh, very graphical user interface, we often do get into this mindset that if it looks right that it must be right. But obviously that is not the case. In this particular case the developer had first gotten rid of full keyboard navigation on their path here, and then we're trying to figure out some way to lie to accessibility that these two items weren't around. Now in this case the solution was fairly simple, just to make a framework friendly change.

They changed over to using a matrix of button cells in radio mode and suddenly full keyboard navigation came back for free. Suddenly accessibility worked exactly the way it was supposed to for free. So often although what we see on the screen is on the left, a lot of what accessibility is, kind of conceptually is getting that out of process experience, getting that representation of your user interface to be extraordinarily clean, extraordinarily functional and usually one of the best ways to do that is to not fight the framework, is to make framework friendly changes.

Some other tips as you're thinking about making framework friendly changes. The first is that sometimes you're possibly not sure if your making a custom UI element whether you should subclass NS View or NS Control, and one thing that might help you decide in terms of just how much accessibility you get for free is NS Views are very much about returning their sub views as children, whereas controls assume that by default that a control is going to have a cell. So the control's cell is used as its child. So if you happen to be doing something with a lot of sub views it's most likely that NS View is your best bet. That way you'll be able to use a lot more of the accessibility for free.

Another thing to note is that in coco it's the cells inside of the control that do the chunk of the accessibility work. That allows us to take that cell and use it not only in say the slider control itself or the text field itself, but also put it in matrices or put it in table views or outline views. And so typically NS Controls implement accessibility is ignored and returned yes, I'm ignored and it's the child, the cell of the control that is really the thing that is communicating or talked to to get its accessibility information.

So sometimes you'll be trying to do something programmatically and you'll say oh, things aren't quite working. Why isn't this taking effect? And it turns out you've been talking to a control when you should have been talking to the cell. We have some nice convenience methods for getting to un-ignored descendants and children in the NS accessibility dot H header. So those are handy to use.

And finally, I call it being actively lazy, going out of your way to take advantage as much as possible as to what the framework provides. Sometimes as we saw with those segmented controls, an accessibility issue is not really an accessibility issue. It's a well we're in a rush, we did it this way, it looked right and we haven't gotten back around to make it better yet issue.

And so sometimes when you're dealing with an accessibility issue the first thing to look at is, is this the cleanest way we can do this in coco anyway? Because if it's not possibly just moving to that makes it better in many ways, not just accessibility. But also, certainly, we really do appreciate your feedback. So if you're working with coco, working with accessibility, filing issues and enhancement requests either to the accessibility dev list or through bug reporter is definitely greatly appreciated.

We very much value that feedback, both about accessibility issues but also about things that you want to do in the user interface that possibly we're not providing. Because if you have to create it yourself you have to make it accessible yourself and if it's something that a lot of people are doing a lot of we'd like to present that or provide that for folks for free where possible.

Okay, and now the biggest piece. We have trouble areas, but access enabling unknown areas, and this is a screenshot of a basic CoreAnimation application. Its one big view and everything inside of it are CoreAnimation layers. We'll talk about that in a moment. And sometimes you'll take a look at your application and this is what it looks like on the screen but across the wire in that other process what they see is something like this.

Just a big blank area, big empty box called AX unknown or maybe just a window and nothing in it. Well that's not a very rich representation of your user interface. It's a pretty useless representation of you r user interface. This is the case that usually happens when you're doing the most custom work, so we can't just figure out what you're doing for you. It's also the one that tends to take the most amount of time. However it doesn't usually end up taking tons of code.

It's not usually like tens of thousands of lines. It's usually a relatively small amount of code. It's usually the conceptual what would be the right thing to do here, how should I go about doing this, that tends to take the time. Once that is kind of clear then implementing is usually the smallest part of it. There's not a great deal of code there.

In fact, there are four scenarios we are going to take a look at when you hit one of these unknown areas, like a custom control, something like a CoreAnimation animation. But really there is one basic methodology that I think really helps in making something accessible from scratch. And the first thing is mapping structures.

So first deciding what that wire frame should look like once you are fully accessible, what you are exposing on the other side of the, kid of the other side of the transom from your app to that other process. And then once you figure out what it should look like based on just the appearance of your app, comparing that to what is the object structure of your app. Is it a view hierarchy? Well we'll get into a number of cases, and then using the techniques we are going to talk about to map between those two.

Often that first step, that's the one that you should tackle first. It's also the once that once that is done you're well, very well on your way to being fully accessible. That's the one where you got that done, it's time to go have a beverage of your choice or two.

Because that's a very nice feeling, once you see that structure coming across on the other side just as you had expected. After you do that then implementing the necessary attributes, so for a text field oh we need to return the value and notify when the value changes and we're going to see a list in a moment, that menu. We need to know what child in that list is selected. Those are all things that once you've got that basic structure usually it's kind of very much downhill from there in terms of the amount of effort required.

Adding actions required for a role would be the next thing you do. Actions are relatively rare though. We have the buttons to press but most often in accessibility we make changes in the application, the target application by changing the value of a property as opposed to sending actions. So if we wanted, as we'll see in this list if we want to change the selected child we set the selected child as a new value and that changes the selection.

We don't press the item for instance. And then adding notifications as necessary, and two of the notifications that are most likely to come up are they you've changed the focus UI element or that a value has changed. So let's take a look at these scenarios are about mapping structures, and first of all look at the default case.

So here's a window that's very similar to what Eric showed before in terms of a single button in a window, and so we look at that and take a look at what's going on in our app in terms of the object graph. And so it's a window, it has a sub view, excuse me this is the, yeah, the window has a sub view, the frame view then a content view. Within that there's a button and the button has a cell. But really what we want to come across to the user, I highlighted it and then I'll fade it out, is just that we want a window with a button in it.

And so the default accessibility we would end up with something like this. The window ends up reporting itself and its accessibility information. The button cell ends up reporting its accessibility information. Really everything in between is ignored as far as accessibility is concerned. It's important to point out that for everything on that wire frame diagram, every user interface element that we expose, that in your application there is an object that implements that accessibility protocol that is the thing that's answering the questions being asked from the assistive app. And so in this case NS Window and NS Button Cell both implement that accessibility protocol. So when questions come in like what are your accessibility attributes, what's your value for this accessibility attribute, those can be answered.

Now default accessibility, this is the ideal case. Wherever possible this is the one where how much work needed to be done here by you the developer? None. That's nice. If that button happened to have an image in it you'd have to go into interface builder and add a string. That's a good amount of work. Now you may have custom elements. So in this case, least year we did a little program called Dicey, which was a little kind of dice game where the main view showing the dice was one view that drew five dice.

However those dice were not views, they weren't cells. They looked something like this. So the dice view was in NS View and then we have these other classes. And so if you find yourself in a situation where your custom view has objects representing sub elements, but they don't happen to be cells, they don't haven to be views, then typically you will use that existing structure and implement the accessibility protocol, those methods that Eric had displayed earlier, for that class so that those items can then answer the appropriate questions. And so we want to expose in this case that outer area as well as those five dice.

And so for this scenario we did a set of sample code last year called Dicey, and so if you find yourself in that situation when you do that mapping, that okay I have structure. I'm good there. It's just those items don't implement the accessibility protocol, then taking a look at the Dicey example as some very nice prototypical sample code of how you would go about doing this is an extraordinarily valuable resource.

Now we might also have a situation where in this case this is an image map where the different parts of this, in this case this food pyramid are clickable. However, if we look at our object graph there is one view. It's doing all the work. So there are no sub views, there are no sub element objects on which we can just hang the accessibility protocol. So in a case like this where there's more structure we want to expose, actually let's expose it.

We want to go from that image map single item to all of these sub elements, well then we need to do a little bit of work. We essentially make up a faux, a fake UI element and its only job in life really is to implement the accessibility protocol and when questions come in about say the top element in that pyramid, it answers the questions correctly about its size and position and role, responds to actions correctly. And usually these faux UI elements are very tightly integrated with that super view because obviously the view knows where everything draws, etcetera.

Now when you hit this situation, because it does come up, a very good thing to do is to hit the image map example. This sample code we did a couple of years ago and it has a reusable faux UI element class that is invaluable. Whenever I'm talking with developers and we hit this situation you can literally pull it right out of this sample project. You subclass it and it does 80, 80 percent of the work for you. So when you hit that, that sample code is very useful.

Now scenario four is we have this new framework in Leopard called Core Animation. Wonderful framework for doing all sorts of compositing, rendering, and we see certain pieces of user interface being directly in Core Animation. A good example would be cover flow in the Finder. So I thought we'd talk today about how we go about doing this and kind of stepping through that process with CoreAnimation as an example, and then we will be showing kind of a before and after at the end, and then finally we'll be, is this what I'm looking for? Finally we will be releasing sample code that should make it very easy for you to make Core Animation layers accessible.

So first let's talk just a bit about what these things are, Core Animation layers. They're very analogous to views. They have a rectangular area. They, a layer can have sub layers. They have a super layer. They're very much about drawing performance so Core Animation is very highly tied into Open GL below and it's very optimized to cache drawing and be able to move it around very quickly on another thread. We can do all sorts of visual effects with it, and we can also do combinations of media and drawing typeset really have never been done before on Mac OS10. So certainly I think we're going to see a lot of people in Leopard using Core Animation.

In Core Animation, what those animations are built out of are called layers. These are the building blocks. A CA layer essentially holds drawing or an image, and then we have different types of layers that hold different kinds of content, a text layer, something for Open GL drawing, a QuickTime layer for QuickTime movies, Quartz compositions. And one thing you'll note is that this is a graphics framework. We don't have CA button and CA slider. So a layer could end up being just about anything in your user interface.

So Core Animation Layers, question mark. It may be if you're building a user interface using, and you want animation in it, that using layer back views may very well be a much better solution. We had talked about this on Tuesday in the building animated coco user interfaces. So as you see Core Animation is a graphics framework, so it does not have built in accessibility and it really is more focused on content types right, QuickTime, Quartz, text. So we couldn't just take a CA layer and say oh that's a slider because it could be used for just about anything. It would be like taking a rectangle and saying oh that rectangle it's a toolbar.

AppKit views could very easily use Core Animation using set once layer, which allows you to build user interfaces using AppKit classes and controls and still access Core Animation to get both the animation, the visual effects and what have you. So I believe in most cases for building user interfaces that animate using layer backed views is probably the best choice.

However, we know that some folks will need to use layers directly, so let's talk about that. In this particular example we have a menu. It's sort of like you might see on an Apple TV. It's got a Quartz composition in the background. It's got five items. It has a selections. It looks something like this in the object graph. We have a host view. It's layer is a Quartz composition layer. The selection is a CA layer. Just drawing an oval, and then all of the items are text layers.

And we would want to expose this. Currently it's just a big unknown square. What we would want to come across to an assistive application is something that looks something like that, the outer list and all of the items. Now in this case notice that the selection is something that is ignored, because really if we have a list the idea of selection, whatever the selected child is, that's an attribute of the list. We don't need to have the window treatment of the visual of selection be a separate UI element, and we would want that outer item to be a list, to be it's role, and we would want each inner element to be a static text.

So some general notes as we go about taking that hierarchy and turning it into that representation and accessibility. Some extraordinarily core attributes are the hierarchy attributes. So AX parent and AX children, those really are the core of defining the accessibility hierarchy. In addition, finding out the window and top level UI element, which would be the sheet or drawer or window that something happens to be on, are certainly important.

But usually the initial focus is on hooking up your parent, hooking up your children and making sure that these happen in both directions. Occasionally you'll hook something up and forget the other direction and then you'll find out a little later when either voice over won't work because it tends to come down the hierarchy, or accessibility and inspector won't work because it tends to go up the hierarchy.

Now for Cote Animation the typical case is pretty simple because parents and children, we have super layers that we can report as our parent, we have sub layers we can report as our children and it's a lot like views. The root layer, however, is a specific special case.

It needs to report it's host view as it's parent, however layers don't by default have a reference to their view. So what we do in the sample code is just a simple subclass of your root where we add the ability to point to the view and return that as the parent.

The hosting view obviously is also a special case. It usually wants to talk all about it's sub views, in this case it needs to report it's layer as a child. Now all of those items in the animation can get the window and top level UI element from the host view because it's already implemented there.

In addition, we have geometric attributes and hit testing, so figuring out where each of these are. Once we know where we are in the hierarchy, that's kind of step one of making that wire frame show up. Step two is size and position. What's nice about layers is its already got them. It's a rectangle.

It reports a size and a position. However, when we report it, in accessibility we're always talking in terms of screen coordinates. In fact with the origin in the upper left hand corner. So we just need to do that translation and then we're pretty much fine with layers and it's geometry. Actually I think I've just said everything on this slide one slide too early. That's fine.

Hit testing, there we go. C layer has a hit test method that we could use to find the appropriate point. It may be the case though, for example with our selection, in Z order it's above all of our items, however it is ignored. So if I try mousing over the selection it's going to report that ignored item as the hit tested layer. So depending on how much of your animation is ignored, you may need to implement I'll go back one accessibility per layer to filter out those ignored elements.

And then the last thing to talk about in terms of getting that basic structure going are the roles. These are extraordinary important. It's how an assistive application knows what kind of thing it's dealing with, and in fact is a three attribute set. There's a role, a modifying sub role and then a kind of localized description.

Don't make up your own roles and sub roles because assistive applications won't know what the heck you're talking about. AX My Slider, I don't know what that is, and it will probably just call you unknown or a group or kind of fudged in some way. For the role description don't make up your own. Use the function and it's accessibility role description to get a localized string automatically.

In Core Animation, whereas all the geometry and positioning is very simple, again a layer could be just about anything. We could make a default layer an AX unknown, sort of like what we do in NS View and in fact in our sample code that's what we've done. Typically you would subclass to add the role and in fact if you are building some user interface out of Core Animation, most likely you're going to create some sub class that encapsulates the behavior of say a button or a slider or something of that effect.

That would be the right place to add the role. And so once we have those all complete at that point that major step one that I was talking about is done and we end up having those objects map. What I'd like to do now is have Eric come on up and show a little before and after of accessibility before we did any work and then accessibility afterwards for that Core Animation.

Look at that. What not to do before demo. Okay, so here we are. I'm going to go ahead and kill our text here, and we've got a couple of applications. First one is James? Handy Application here so ooh, Core Animation. Isn't this cool? I can go up and down. It's nice and animated, really great experience for cited user. So now let's actually run voice over on this, turn on voice over. Okay, and let's complete the experience here. Ah this is what most users experience with voice over, and so let's try to move around this app.

Oh bummer, what window were we on. It's an elevator app. I'm on window before. Let me try to drill in. That's a bummer. I don't think I'm going to buy this software. So let's be collaborative with our inaccessible app and let's turn off voice over and let's quit this one, and now let's go to our other app accessorized.

Same app, moves around, looks really nice. Now let's turn on voice over. Ah that's what I'm talking about. Right, now several things are happening here. It's reading each item and also I'm just using the arrow keys. I'm not even using the voice over cursor and we're getting the notifications at the right place so that we move the voice over cursor when the selection changed, really nice and accessible. Thank you.

And so we will be making that available. What is included in this sample code essentially is we've added a category to CA layer and also one to CA text layer, because although a layer could be anything probably it's a good guess that a CA text layer is going to come across as an AX static text, so we've done that and also the couple of little places where we need to subclass to hook it into the view hierarchy. So we will be making that available.

If it doesn't happen today it should happen very, very shortly and that would kind of round out this suite of sample code for hitting all of the non-default cases, having your own items, your own structure, having CoreAnimation layers as structure or a view that does all the work itself where you need false items or faux UI elements to get across the accessibility.

So in summary, certainly everything in Mac OS10 more and more we're seeing things that are relying on accessibility. Not just voice over. Certainly we're seeing a growing community of users of voice over on the Macintosh and so accessibility over time is becoming more and more important, and so getting that out of process experience, getting that clean representation of your user interface is becoming more and more important.

The first thing you could do if you do nothing else, add descriptions and title UI elements. Hook those up, because that makes your application instantly much more accessible. Where possible AppKit views and controls, including using the animation additions and set once layer in app kit, custom views and controls making them accessible. Don't forget about the sample code.

It will save you tons and tons of time because it is designed to be reused and very helpful. And please give feedback and ask questions, which then leads us more information. I highly recommend the accessibility developer's list, documentation and sample code, Deric's our evangelist, and the accessibility lab this afternoon at 2. We'd like to see you all there