Good afternoon. My name is James Dempsey. I'm an engineer on the Cocoa Frameworks team, and this is Session 127, Design Patterns to Simplify Mac Accessibility. This session primarily are just a handful of things to keep in mind while you're building, especially custom views, that make it much, much easier for you to add the accessibility support later or after the fact.

Now, we're also going to talk about just a few other things. We're going to do a brief Lion update. And before we even jump into the Lion update, I just wanted to take a few minutes to talk about really, at a very high level, why we have this session, why we're all in this room today. So at Apple, we really strive to provide an excellent user experience to everyone. Sure.

Now in 2011, a visually impaired user can walk up to any Mac that has shipped in the last six years and turn on VoiceOver. And that can be their Mac, or the Mac of their friend or their relative, or at a lab at school, or at any mall that has an Apple store. Walk up to any computer and just use it. And it's that kind of amazing leap in usability that's really why we're all in this room today.

Now, Job 1 is always providing everyone a great user experience, but making an application accessible does have additional benefits. It's a programmatic interface to your app's UI. So it also lends itself very well to automated user interface scripting, which is great for internal testing, automated UI testing. Power users love it.

And our own development tools take advantage of this to record and play back. actions in the UI for doing things like profiling. And then finally, Section 508 requires that apps that are sold into government and education accounts are accessible. But despite all the benefits, again, we want to provide everyone a great user experience.

So, talked a little bit about VoiceOver. Why do we have a session at WWDC? Well, VoiceOver doesn't work alone. It needs a partner. In order for it to describe what's going on in your application, it uses accessibility client APIs to make requests of your application, and your application needs to respond with the correct information, the appropriate accessibility information.

However, we don't put the entire burden on you. In fact, we provide both app kits and tools to do a lot of the heavy lifting and doing a lot of the accessibility work for you. So before we jump into your application, let's take a look at those framework and tools and what's changed in Lion.

So in Lion, we've had a great number of new user features. We've had a great number of new app kit enhancements. And as we've worked on Lion, we've really, really strived to provide as much accessibility support out of the box. And so what you'll find in Lion is that the new user features are accessible.

So let's just talk about some of these inversions. It's your document surrounded by all the versions in space.

Just as always, you make your document accessible. We've handled all the rest of the chrome around the edges. We have new features for auto-correct pop-ups, little lookup pop-ups. These are all automatically accessible, so just continue using your standard text views or web views, and VoiceOver loves these guys.

Full screen, again, you make your content accessible, and we handle the rest. If you play with full screen in your seed, there's still a few things we need to hook up before we finish up. In the end, the message is the same. You do your content, we handle the rest.

Sandboxing. Has anybody gone to the sandboxing talk? What are the accessibility implications of sandboxing? Well, one of the things that happens when your app is sandboxed is when you bring up an open and save panel, to a user, it looks like there's a new -- there's a save sheet on a document in your application. But, of course, behind the scenes, one application, the one in the sandbox, has its document window, but that sheet is handled by a completely separate privileged process that's able to access all of the file system.

And yet we want to provide to accessibility clients the same seamless appearance that this is all happening in one app. And, in fact, in Lion, if -- in this case, we're going to have a single application, and in this case, I went and looked at text edits, accessibility hierarchy, it is all properly stitched together. It handles accessory views correctly. So all of that works without, again, any additional work on your part.

Overlay scrollers. For accessibility clients, they'll still see that there are scrollers there, but they'll be able to find out whether they're hidden or not, and also be able to flash the scrollers as if the user had scrolled. Popovers. NSPopover is fully accessible. If you are using your own kind of popover-ish mechanism, highly recommend moving over to NSPopover. VoiceOver loves it, loves them. View-based table views.

So it's always been difficult to have complex table cells. You'd have to make your own NSCell subclass, and then the accessibility of such a thing was also very difficult, and in some cases, I think impossible. View-based TableView takes that and completely revolutionizes it from an accessibility perspective. You build a view-based cell, and because it's view-based, we just take advantage of all of the existing accessibility for standard controls, and we provide a much, much richer table view experience with view-based table views.

A few accessibility notes. When you have a table cell view, you want to hook up the text field outlet to the primary text field. VoiceOver will automatically use that text field to find out which piece of text is the most important to be spoken. Similarly, rather than just reloading data, using the new row insert move and remove methods where appropriate is very useful. It allows apps like VoiceOver to track the rows as they move around. It's also more efficient.

We've also added identifiers. It's a new protocol with two fairly simple methods. This protocol is used by table views, view-based table views, for keeping track of which rows and which cells map with which columns. It's also used by AppKit's persistence mechanism for recording pieces of state that need to be reinstated when we resume.

But we also use this identifier through accessibility. So accessibility clients are able to read that identifier and have a persistent way of identifying pieces of your UI between launches of your application. This is particularly handy for folks doing automated UI testing, since you can set the identifier and then use it immediately in your testing scripts.

In addition to all the work we've done to support the various new features in Lion, we've also done a new version of an accessibility inspector. The interface looks refreshed, redesigned. We've kept all the existing functionality, but added some new features, especially navigation features. And I'd like to show that to you right now.

This is Accessibility Inspector. And what Accessibility Inspector does is it uses the same accessibility client APIs that VoiceOver uses, but it gives us a much more detailed look at exactly what each UI element on the screen is reporting to VoiceOver. So for instance, so let's navigate to it.

You can find it in your Developer Tools, Applications, Utilities, Accessibility Tools, Accessibility Inspector. There we go. And what Accessibility Inspector will do is, as I point to items, it will give me information about whatever is under the menu item, or the item under the mouse. So if I take a look, say, here in the Finder, here is a menu button.

If I point to it, I can also lock onto that item using a keyboard shortcut, Command-F7. And when I lock onto that item, I can inspect it a little bit further. First, I can turn on a highlight so I can see which item it is. I can look at its place in the accessibility hierarchy.

It's a menu button in a group in a toolbar in the window in the application. But new, it's easier to move around. If I want to move to that toolbar, I can just either go to Element or double-click. We also remember where you've been so I can navigate back down the hierarchy.

The navigation can be done either through these navigation buttons or through command control and the arrow keys, which is a very handy way to navigate through without using the mouse or perhaps disturbing your key state or what have you. Once I'm focused on an item, I can also perform actions such as popping up the menu.

If you're new to accessibility, Accessibility Inspector is a great way to find out the sorts of information, one, that your application is already providing to accessibility, and also the sorts of things that accessibility needs. Let's do one other thing, which is, let's inspect the window navigating up to it. And let's change the size.

For any attribute that is writable, I can change, for instance, the height, set the value, and it updates. So that's a look at the new Accessibility Inspector. So reviewing Lion technologies, again, user features and framework features, we've done a lot of work to make those items accessible without additional work for the developer, for you. We have a new accessibility inspector, and then I'd encourage you to check the AppKit release notes for more details about the items I've talked about and some other smaller things.

Let's talk about design patterns to simplify Mac accessibility. And kind of the idea of this talk is if you've ever had the experience where maybe you're at the grocery store and you get home and your significant other says, oh, if I knew you were going to the store, I would have had you pick this up.

And if you had that additional requirement in mind when you were at the store, it would have been much, much easier, saved yourself a trip. You could have incorporated that requirement with very little effort on your part if you had just known. What we're going to do is talk about a few of those things in terms of writing views.

So we're going to talk just a bit about implied versus explicit information in your custom classes. We'll walk through a couple of kind of general approaches for building a custom view. Then we're going to see that the accessibility client, what it's expecting, drives the requirements, and those requirements drive a really short list of things we should consider as we decide on approaches for building a custom view.

So all of the information, 80, 90, 100% of the information that accessibility needs, that we saw in Accessibility Inspector, for instance, is already in a view that you've written. It's just sometimes that information is very implicit as opposed to explicitly spelled out. Even what we see up here is a relatively simple user interface.

It's a window with a button in it. If we focus in just on that button and think about all the information being conveyed, Well, first we know it's a button, first of all. We know it's containment. It has a parent. We can see what window it's in. All of that's conveyed very implicitly.

We know it's enabled. We know it doesn't have keyboard focus. We know what its title is. We know where it is on the screen. And since it's a button, we also know we can press it. Now there's a myriad of ways that you could get that particular image to show up on a screen or in your view. Accessibility needs to have explicit and structured information provided because it's a programmatic interface.

And so choices you make in building your view greatly affect how easy or difficult it may be to provide that information. So we're going to start with a really just dirt simple, but ugly, custom view. It's a white box with two colored squares in it. And how does it work? What's the interface? Well, if I click, it selects one. If I click the other, it selects the other and deselects the first one. It also supports full keyboard navigation, so you can see the focus ring is entered and is on that red square and hit tab again, and now it's on the green square. Very simple view.

But even with such a simple view, there's like a gazillion ways you could write this view. We're going to take a look at a few approaches and then see what their accessibility implications are. So in approach one, it seems straightforward and simple. View handles everything. I have one class, simple view. In drawRect, I just draw a background square, and I draw two squares, and I'm done with drawing. Event handling, I figure out which square was hit.

It seems very straightforward. It also seems like we've implied a lot of information in there. We don't know anything about those rectangles and only DrawRect really knows what's going on about those squares. Approach 2, we might add a little substructure. So maybe we have our simple view class, we have some color swatch classes. They each know their bounds, they know how to draw themselves, they can hit test themselves.

In approach three, though, we might say, do we need a custom view at all? What if we composed three custom views? Well, in this case, we actually pick up a lot of functionality for free. We don't have to write the hit testing because the view hierarchy already hit tests. We don't need custom code for handling keyboard focus because the first responder already does that, but only to the view level. And we also get a lot of basic accessibility support.

So when we're thinking about a custom view, one thought is, could I just compose views? Would that be OK? Would that work? Would that make my life easier and make my code more maintainable? But then the fourth approach is, isn't this really just a segmented control? I can click either one.

One is selected at a time. I can tab in and select each item. Why am I reinventing a whole new view when there's a perfectly good framework class I could use? And so, anytime you're like, I need a custom view, stop for a moment, take a breath. Check the headers. See if anything's new in 10.7. Maybe I don't need a custom view. Because if you don't, By far the easiest path to an accessible user interface is reusing the work somebody else has already done.

Now, of these approaches, we're going to focus on two for this talk. Using framework classes, there's not a lot to say there in terms of making it accessible. It already is accessible, so we'll drop that out. Composing three custom views is really just a very specific subversion of approach two. So we're going to focus on the view handling everything or objects used for substructure.

Now that we have talked a little bit about two basic ways of putting views together, Now let's see what an accessibility client needs and how that drives our requirements of what we're going to need to do to these two views. So an app bends a hierarchy of UI elements. In each node in the tree, your app, the window, the button, is represented by a lightweight node in this hierarchy vended to clients.

And each of those nodes supports actions, as we saw in Accessibility Inspector, a list of attributes. We also support notifications to find out things when windows come and go, for instance. And behind every one of those UI element nodes is some object in your application, and it implements the NSAccessibility protocol.

We're going to take just a brief look at this protocol. Essentially, a client wants to know what attributes do you support. Accessibility attribute names gets called on your object and returns the value. Essentially, this is the set of methods that provides the response for the request that an accessibility client has made.

Here are some that deal with attributes. We also have that deal with actions, hit testing, finding the focused element, pruning the tree of extraneous items, and posting notifications. Now how do clients get at this tree? Well, they can start from the top of the tree, and then using the axchildren attribute, navigate their way down.

So based on what the client needs, things to keep in mind. One is, if we have an element in the accessibility tree, it needs some object in your app to implement NSAccessibility and respond to those methods. The other thing that's important is every element needs to know its children in the tree.

So how does this translate to our approaches? Well, when the view handles everything, we certainly do see there's a view and there's two things in it. We should report those things as unique visual elements in the tree, but we only have one object per instance. So in this case, we need to create objects specifically to report the accessibility information. We often call them faux UI elements. In Approach 2, we already have the structure in place. The simple view has two color swatches as children. So we already have objects in our original design that we can hang these accessibility methods off of.

So the design choices. In approach one, we need to come up with

is relying a lot on their parent because the parent is where the smarts are. They have a pointer back to their parent and they ask their parent, "What's my bounds because I don't know myself?" More typically, that sub-element is a little smarter. It handles its own accessibility, and it's a great place to put that logic.

Now, in addition from this top-down approach, clients can also access by hit testing. And in this case, as we saw in Accessibility Inspector, let's say we hit test that point. It hit tests down to the button cell, and so we return the corresponding UI element to the client. And from there, the tree can be navigated asking for the AX parent.

Which adds a few more requirements to our list. Oh, one other thing that the client can do, and often does do, is ask for the size and position. That's how we were able to draw the highlight. That's how VoiceOver draws its cursor. We need to provide screen coordinates.

So, in addition to children, every element needs to know its parent, and every element needs to know its bounds on the screen, which brings up a few requirements. Now at this point, the two approaches merge a little bit. The view handles everything. The foe UI element needs to know who its parent is.

Needs to know for two reasons. One, it needs to report a parent, but also potentially it needs to ask its parent, since the parent is a view, to translate its coordinates from view coordinates to screen coordinates. We end up with the same requirement for approach two. That sub-element, whether it's a faux one or one that's built into the structure, needs to know the parent.

So geometry design choices. In approach one, you will probably want to factor all of your geometry of your sub-elements into some method where you hand in some identifier for the element and get back the correct rectangle. In Approach 2, the sub-elements likely have their own bounds that they carry with them. But whichever way you go with, the same mechanism can be used for view drawing, hit testing, accessibility reporting, and accessibility hit testing. So one factored method handles both cases, or both sets of cases, your standard drawing and event handling and your accessibility case.

One requirement that people don't often think of, or sometimes miss when building a custom view, is that those sub-elements do need a reference up the hierarchy. This is true regardless of approach, and it's something to keep in mind. You need the parent because you need to report it.

But again, we report in screen coordinates to accessibility. We hit test in screen coordinates. So even if you are in a hierarchy and you're two or three sub elements down from the containing view, you need either a direct or some indirect way of having your view translate the coordinates for you.

The next thing a client requires, they really rely on keyboard focus. VoiceOver users tend not to use a mouse, sometimes use a trackpad, but very much navigate using the keyboard. and Full Keyboard Navigation. So finding out, first asking the application which element has the focus, but also registering for notifications so that when the focus changes, a notification is sent to the client that focus has changed.

So that adds one more item to our list, our relatively short list of things to keep in mind. Now, when there's a change in focus, again, if you're dealing with views, the first responder handles that automatically. If you have your own custom view, you're handling focus changes yourself.

Just as you need to handle the focus ring changes in tracking what subelement has focus, you need to also do the appropriate accessibility notification and reporting. It's best to have a single point of control where focus change happens. I've seen it scattered to the four winds in some pieces of code I've looked at over the years where it's just you jam something into an IVAR and there's no control point. That control point is really useful to have both for accessibility purposes but just for general maintaining your code purposes.

And the final thing to keep in mind is that clients can interact with your app. They can perform actions In comes an action, presses that button. They can also set attribute values as we saw in the demo where accessibility inspector changed the size of a window. Now, it's important to keep in mind that not all interaction with your view has an NS event attached to it. I've definitely seen code over the years where it's relying on just the current location of the current event. Well, there might not be a mouse involved at all.

It's typically best to factor things like triggering behaviors, changing the selection into general methods that possibly if you're changing selection by index. And then call those general methods from both the event handling methods that you write and also from accessibility actions and attribute setters. One, it centralizes your code and makes your life easier. But the second thing it does is it ensures that when a VoiceOver user presses that button, that it's going through the same code path as when a user clicks on the button in the user interface. So let's just do a brief review.

We didn't talk about it except for briefly, but again, I stress, when possible, use standard controls and views. And there's a handful, five things to keep in mind when you're building custom views. You need a backing object. So if you're on the fence about just having a Mondo view do everything itself, or having some substructure, making it accessible maybe will make you lean more towards using some substructure. You need to know the child and parent and rectangle. Having that back pointer to the parent and potentially to the containing view is something that often you're not thinking about when you write a view at the start. That's a good one to keep in mind.

bottlenecking or putting a control point for your keyboard focus, and also making your interaction not rely on NS events. Keeping those in mind as you write views will really make it much, much easier to Come in later, because we know when you sit down and write a view, you can only focus on so many things at once, and you're trying to get it to draw right and handle events right. Follow these steps, keep these few things in mind, and then as you add accessibility later, you'll find most of the work's already been done.

So I want to point you to some resources. We have three sets or three pieces of sample code. Image Map really details how you would work with case one. Dicey is approach two. Sketch Plus Accessibility shows how you would implement accessibility into a larger, more complex app. So I highly recommend taking a look at those pieces of sample code.

I also wanted to point you to the relatively new accessibility roles and attributes reference. It's, as far as I know, the most up to date we've ever had in terms of mapping which roles go with which attributes and defining them very well. So I would love you to take a look at those.

And then finally, the accessibility programming guidelines for Cocoa. For more information, The related sessions are in the past, but feel free to check out what's new in Cocoa. And also, we talked a little about table views. And finally, if you would be willing, I'd like to do a little song.

I've been working on accessibility for about five years now, so it's about time to do a song about it. Um, plain and simple. I'd like to, before we start, introduce the breakpoints. On guitar is Darren Minifee. He's an engineer on the VoiceOver team, accessibility team. And on keyboard is Victor Alexander.

Who as you might know is the best slide advanced man in the business. And we're going to try something a little new today. We'll have a little instrument break with an iPad somewhere in there. So watch out. Thanks. Excellent. Are we in tune? I think so. Cool. All right. And with that, the accessibility song.

♪ Application is a tiny little server ♪ ♪ Of information of its user interface ♪ ♪ It dishes out the dirt with such a quiet fervor ♪ ♪ You might not even realize it's taking place ♪ ♪ The client gets a lightweight representation ♪ ♪ Of your app's UI elementary ♪ ♪ Then it's just elementary my dear Watson ♪ ♪ Your app's an open book no more mystery ♪ ♪ Think how much use for bitmaps on screens ♪ ♪ They get the lay of the land through this alternate means ♪ ♪ If you wanna build an app with maximum utility ♪ ♪ Then take a little time and implement accessibility ♪ ♪ Tell the story of your application ♪ ♪ Describe the glory of its interface ♪ ♪ So we can see sight unseen what's going on on the screen ♪ ♪ Yeah, build your app for the whole human race ♪ ♪ There's an object behind each UI element ♪ ♪ Accessibility methods compute ♪ ♪ The response to each request that the client sent ♪ ♪ For any action or attribute ♪ ♪ The client wants to be notified when things are happening blow by blow ♪ ♪ Like when key focus changes where it resides or windows come and go ♪ ♪ For standard controls this is built right in ♪ ♪ For a view of your own we need your help for the win ♪ ♪ This isn't rocket science in some government facilities ♪ ♪ So take a little time and implement accessibility ♪ ♪ And tell the story of your application ♪ ♪ Mind the query of its interface ♪ ♪ So we can easy as pie for you to automate your UI ♪ ♪ Show your app to the whole human race ♪ ♪ A thousand words they say ♪ ♪ I don't believe this ratio anymore ♪ ♪ 'Cause just a word or two goes such a long, long way ♪ ♪ To describe what an icon is for ♪ ♪ You can add descriptions to match your style ♪ ♪ You can put 'em right in IB ♪ ♪ You can add 'em to an image description file ♪ ♪ Or do it programmatically ♪ Let's just go back.

♪ So the way we get things done ♪ ♪ Accessible apps really are more fun ♪ ♪ It's been rumored to augment virility ♪ ♪ So come on everyone implement accessibility ♪ ♪ Describe the glory of its interface ♪ ♪ So we can know right away the things we ought to say ♪ ♪ And share your app with the whole human race ♪