Good morning. My name is Gregory Hughes and I'm the engineering manager for accessibility on OS X. I'm really excited to be able to have the opportunity to speak to you this morning about some of the great new accessibility features in OS X Mavericks. Here at Apple, we care very deeply about accessibility and we work really hard to deliver a lot of great accessibility features. We work on all sorts of products, like VoiceOver and closed captioning, Zoom and Guided Access. And if I were to summarize what our does, I would say that we solve problems and remove barriers for users with disabilities.

The two big problems that we look at everyday are how can we deliver an outstanding operating system that's fully usable by all users regardless of their abilities or disabilities? The second problem we look at is how can we provide you, the developers, with outstanding API and developer support to help you do the same thing, to help you deliver applications that are accessible and usable by everybody? So to start off, I want to talk that first problem.

What features do we deliver as part of the operating system to help users with disabilities, because you're maybe aware one of the flagship products is called VoiceOver. VoiceOver is a screen reader which is a piece of software designed for users who are blind. A screen reader converts all the graphical user inter-- graphical user information-- sorry, screen reader converts the graphical user interface into a spoken interface so that somebody who's blind can navigate OS X and get all the same information that you would get, sighted, looking at the screen.

VoiceOver also has the capability of outputting to Braille. And now, this is really important, both of these features within VoiceOver. Because what this means is someone who is blind and deaf can use all of the features and all the functionality of OS X to its fullest potential. And because VoiceOver ships with the operating system, this means that the computer's operable right out of the box for this class of users.

Now, VoiceOver shipping as part of the OS also has a really significant impact across the board. Now, gone are the days where student who's blind has to work in a separate room, work on a separate computer with assistive technology, because VoiceOver ships standard with OS X, a blind student can use the same computer as everybody else. They just need to turn on VoiceOver.

The same is true in the workplace. Corporations no longer need to buy expensive assistive pieces of technology and colleagues no longer have difficulty collaborating because VoiceOver is standard in OS X. Furthermore, because VoiceOver is standard in OS X, you have a world class screen reader built right in to help test your applications. So, it's incredibly powerful and incredibly enabling for both users with disabilities and developers who are developing to help these users with disabilities.

So that's a brief introduction about VoiceOver. We also look at a lot of other disabilities. We work really hard across the board to deliver closed-- an outstanding closed captioned experience. There are really great closed captions in iTunes, on iOS and even the iPod Classic. So whenever you open up your media that has closed caption content, you can enable a closed caption track right on the device. But we've gone above and beyond that in OS X Mavericks. We're happy to introduce some great new closed caption support for users who are deaf.

In the Accessibility Preference pane, you'll now notice the Captioning tab. One feature that I'd like to draw your attention to is this checkbox here that says "prefer closed captioning in SDH." If you're unaware, SDH stands for "Subtitles for the deaf and hard-of-hearing". What this checkbox does is it provides a single global place for user who's deaf to basically tell the OS that they want captions whenever possible.

Now, under the hood, it's actually pretty complex. In every application across the system, whenever media is played, we're going to look and find the best caption track we can. We're going to cascade through some complex rules, try to find a closed captioning track, try to find the subtitles for deaf and hard-of-heard track, and leaving cascade down to just a regular subtitle track based on the system locale.

So, this is really, really helpful because now a user doesn't have to dig through settings in each application to try to find how to enable closed captioning. They don't have to dig through a track list of 10 to 15 text tracks to figure out which one is best for them. Users now just have to check one checkbox and we're going to do all the work under the hood. Users can still override our choice if they want to. But we found that generally that's not necessary.

As you'll notice here too, we have closed captions with captioning styles. By default, we're going to ship the system with three styles that our [inaudible] team has designed and they look really great. We have the default style, the classic style and the large text style. So just like that checkbox, these are going to apply across the system to all the applications that adopt this API that are displaying captioning. So now, it's really simple to create-- to select the style that suits your needs. You're sitting across the room from your iMac, it might be easier to use large text style. If you're watching a movie on an airplane on your laptop, you might prefer default style.

And, for users with needs to aren't suited by these styles, with-- for users with needs that aren't met by these styles, rather, you can create your own style. And this is really great, for example, for a user with a vision impairment that prefers a font that has all capital letters.

You can also customize the color, the opacity, and really create a great closed caption experience regardless of your needs. And as I said, this is where it all applies across the system whether you're in iTunes, in QuickTime and even on appropriately tagged webpages, so really, really powerful and really enabling for these users.

Now, we're not going to have time today to go into the nuts and bolts specifically of closed captioning. There's a whole talk tomorrow on preparing and presenting media for accessibility. They're going to talk a lot about the APIs and preparing media to be displayed in your application. So if you have an application that displays a lot of video content, I strongly encourage you to attend this talk tomorrow morning at 10:15. Or otherwise, the video, just like all videos will be available.

So the last topic that I want to talk about are users with mobility impairments. So, we've delivered a lot of really great features for users with mobility impairments. We have things like Sticky Keys and Slow Keys which help user that have difficulty typing. We have Mouse Keys which help users who don't have the physical dexterity to use a mouse. And in OS X Mavericks, we're really happy to introduce a great new feature for users with severely limited mobility.

This class of users that we're talking about quite literally only have the capability to operate a single button. In some cases, this maybe a button mounted on the side of a wheelchair that these will activate with their head. It could be also called "sip-and-puff straw" that these will activate by blowing air into or sucking air out of a straw. Some buttons even manifests themselves as a small sensor they might place in their cheek that's activated by clinching their jaw. But in all of these cases, the common denominator is that the user has some physical mobility to operate a single button.

And we thought about this problem and the more we thought about it, the more we realized, this is a really hard problem. How do we take all of the power of OS X and fit it in one button? How do you use an entire operating system, all the functionality and features with just one button? And the problem really boils down to the standard user interface, the standard input available to a user.

The average user has a keyboard with 78 keys, a Trackpad with two dimensions of movement. It also has acceleration between one and five buttons and scrolling capabilities. The case with the Trackpad, you also have Multi-Touch gestures. How do you take all of this input and fit it in one button. All right, it's not an easy task.

So after much deliberation and much kind of intense iteration on how to fit all of the functionality of OS X into a single button, we're really happy to introduce Switch software for OS X. What does this software does, it enables a user who can only operate one button to use all the functionality in OS X.

When you turn on Switch Control, it appears there's a small menu on the top right of your screen. This menu here that you see is what we call the Home Menu. And now, with only one button, you don't have the option to select Next, Previous, Up or Down.

So, we're going to stand through the interface for the user automatically and the user can set the rate of scanning speed that they desire. So as we scan through the interface, you notice each button gets highlighted on screen. You need to press your Switch when the appropriate button is highlighted. In this case, if we want to open up keyboard, we press the Switch, the keyboard would appear.

Now, as you can imagine, navigating through your keyboard button by button would take a really long time. So, we grouped each row. So, we navigate down, grouping rows. And then each row has groups and you press your button to drill in. And eventually, really intuitively, get to the single letter that you want to type.

As you'll notice up at the top, we also offer spelling suggestions. That gets populated with the next most likely letter and next mostly like word that you're trying to type. And we use the incredibly powerful spelling and spell checking engine built right into OS X to help provide us some great spelling suggestions here.

So now, I'm sure you're thinking to yourself that there's lot more to OS X than just a keyboard. Now, we do have access to the other keyboard keys, of course, like the media keys and even the hardware buttons that might appear on your monitor like the brightness controls. But going beyond that, we have full functionality of the mouse.

And this problem alone is really hard. How do you take a trackpad, a mouse with that two dimensions of movement, and boil it down into a single button. Well, I'm fortunate enough to be able to work with some of the brightest people on the planet. And they've came up-- they came up with a really great solution.

When you want to move the mouse, you'd select Move from this menu here. And if we take an example of Calculator, if we want to move the mouse to select the Five button, we're first going to scan the x-axis and the user is going to press their Switch when the appropriate region is selected. You then press your Switch again to select the point within that region and we repeat the process on the y-axis. So, really, really simple and fast. Just using one button, we're able to move the mouse anywhere on the screen.

So this is incredibly powerful because now that we have access to the keyboard and mouse, the full functionality of OS X is usable with just a single switch. At this point, you may note, a user could do a left click, a right click, they can even do click and hold to do Drag and Drop operations.

So, really powerful and all the functionality is available to the user right with their single switch. So, these are just some of the build-in features of Switch Control in OS X. And we've gone above and beyond that. Switch Control and these panels are also fully editable by the user.

We have a fully functional Panel Editor for users to create their own panels, to create panels of their own shortcuts. And this is incredibly critical for this class of users. As you may imagine, it can be a little bit tedious to navigate an entire OS with just one button. And just like you or I might create keyboard shortcuts, especially in tools like Xcode or professional tools like Final Cut, Switch users need to be able to create their own shortcuts, their own panels.

We've seen a lot of users-- a lot Switch users do video editing and really enjoy video editing. And this class of users really like creating panels that have commonly used functionality come in macros, things like insert a Crossfade, insert a Fade to Black, or trim this Clip by Five Seconds.

So with this tool, users have the ability to create their own panels, create their own shortcuts, and really design an interface that works for their workflow. Now, one another that I didn't get into with Switch Control is, just like on iOS, Switch Control has the ability to navigate the user interface. And to do this, it uses the Accessibility API just like VoiceOver. So now, more than ever, it's absolutely critical that your application implements the Accessibility API.

And as I started out this talk, I said that we solve two problems, and the first problem is we try to deliver some of the best features within the operating system for users with disabilities. But the truth of the matter is no matter how great our features are, no matter how many weekends we spend in the office, no matter how many late nights we spend in toiling over these interfaces, users don't buy our products because of the features we build into the OS. Users buy our products because of the great applications you make.

Users buy our products because of the extended functionality and integration that your apps provide. So now, more than ever, it's absolutely critical for your applications to be accessible and work correctly with the Accessibility API. To help show you a little bit more about making your application accessible, I'd like to bring up my colleague, Patti Hoa [phonetic]. [applause]

Okay, my name is Patti Hoa. I'm a software engineer on the accessibility team. Today, I'm going to help you guide you through some of the basics of accessibility technology on OS X. And then I'm going to present to you some of the UI challenges we have and some of the new APIs we've introduced to solve some of those problems. And last, I'm going to also talk about some of the security improvements we've made.

So, let's first talk about accessibility technology on OS X. First, there is your application that presents lots of pretty UIs and fancy cool features. And then there's a few software, what we call, "assistive software" that uses Accessibility APIs to obtain informations from other applications. Some examples of those are VoiceOver and the Switch software that Greg just talked about.

So the way it works is the assistive software will request accessibility information from your app and the application vents those accessibility information back out. To use a specific example, VoiceOver here as the assistive software and then just a generic application that perhaps just has a Play button in the window. So the way they communicate with each other is through, what we call, the NSAccessibility Protocol.

And the way it works is that every UI, like this Play button, is conveying its accessibility information through attributes. So, VoiceOver can simply ask for a specific attribute and get some information back. In this case, VoiceOver to ask for role description and get the type of UI information back which is button. Or, it could ask for AX description which describes what kind of function this UI presents. In this case, you could return the string Play. So, VoiceOver could then take these two keys of attribute information and either speak or output to Braille the words Play button.

So let's take a quick look at the NSAccessibility Protocol. There's methods to determine if any particular element is ignored in accessibility hierarchies. And then there's methods to get attributes in such. And there are methods to perform actions [inaudible] UIs that you could interact with. And then there's hit-testings and focused testing that you could do with accessibility.

And last but not least, you could do-- you could post notification when things change. I'm not going to go into details about how this protocol works. If you're new to the NSAccessibility Protocol, I encourage to you look at the previous WWDC videos or look at our documentation online. What I do want to do right now is to show you a tool we have on OS X to help you going at information about any particular piece of UI.

So this tool is called Accessibility Inspector and the good thing is, now, it's a bundled with Xcode. So you could simply go to Xcode, open Developer Tool and launch the app.

And Xcode won't hide. There you go. So the way Inspector works is it does hit-testing. So whenever UI you're pointing under mouse, it's going to display it in the Inspector. So I'm going to use this app I have called Accessibility UI examples. If you came here last year, you know that we used this demo app. So you've seen some of the examples we had. This is a great app to use to determine how to accessorize various type of UI. So this year, we added four more examples for you here.

Now, what I want to do is use Inspector, take my mouse over, say, for example, this Recycle icon. I'm going to lock the screen so that way I could continue to move my mouse while we're inspecting the content. So this icon has its AX image. This shows you the hierarchy of this image and then underneath all the attributes that it supports. For example, here it tells you, it has a role of AX image, a role description of image, description of recycle, et cetera. I encourage you to take a look at this Inspector and use it while you're implementing your accessibility to support for your app.

So now, I'm going to tell you about all the exciting new APIs we have coming up in OS X Maverick. So we came up with a few UI challenges that we're trying to solve. So in OS 10.9, we introduced some new APIs to solve those UI problems. So I'm going to talk about some of the new subroles we've introduced, some-- a new set a of APIs to support-- excuse me, set of new APIs to support Transient UIs and also notification support to-- for accessibility announcement.

So let's first take a look at some of the UI challenges we had or some of the new behavior we've noticed. So, traditionally, NSSegmentedControl, you can only select one segment at a time. But we noticed that developer have altered the behavior so that you could select multiple segments at a time, like the Xcode for View Controller in this example. So to solve that problem, we introduced a new subrole several called the Toggle Subrole.

Similarly, we also noticed that a lot of developers are creating UIs that look like a switch to do a function that where, traditionally, they might have used a checkbox, like in a time machine perhaps where you could use a switch looking UI to turn something on and off. And to support that, we introduced a new subrole called the Switch Subrole. In the Accessibility UI example demo app, there's example of the Switch, you could take a look at that and see how that's accessorized.

Next, I want to talk about UIs that appears and disappear as a result of user actions. And the most common one we're seeing are mouse hovering. So the example I have up here is the Reminders app. When you take your mouse over of the Reminder item, the Information button appears. And similarly, if you take your mouse away, the Information button disappears. So this is a mouse-only function that, you know, if you're not a mouse-user like VoiceOver user, then there's no way to bring up that Information button.

We could tell you to add keyboard support like we've done for many years, but we know that's not always possible. And even if you did add the keyboard support, shortcut support, it may not be discoverable. So in 10.9, we introduced new APIs to solve this problem. All you have to do are two things. First, when VoiceOver wants to show or hide UIs, you simply need to implement the Show Alternate UI action or the Show Default UI action.

And when VoiceOver wants to know when UI changes on the screen, you simply need to post the notification layout, change notification. This notification takes a user information dictionary, a key-- UI elements key that takes in a rate of changed elements. So if you've ever implemented or posted accessibility notification, you may be wondering right now, "How do I pass in this user information key?" Guess what. In 10.9, we introduced a new API for you to do just that. This post notification was user info. API takes in additional parameter which is the User Information Dictionary. So to see how this all works, I'm going to go ahead and demo it to you.

So in this app here, the Accessibility UI examples, there's the Transient UI example. So note that as I take my mouse over here, the Previous and Next buttons appear. And you could click on this to change the page. And if you take your mouse away from this, the UIs disappear. So obviously, this is not possible for a VoiceOver user to use currently. So let's see how you can add those API support to solve that.

I have the Accessibility UI example project. In that Transient UI folder, there's a Transient UI Trigger view which is just a subclass of the NSView. This is the view that holds the two buttons. So what I want to do is add the NSAccessibility Protocol for this view. So I created a category for that.

So the first thing I want to do is make sure that this view is exposed through the Accessibility hierarchy. NSViews, by default, will not exposed through accessibility. So I have to explicitly return No for Accessibility is Ignored. So once this element is exposed through the hierarchy, the next you want to do is, well, define what type of UI this is.

So in the accessibility attribute value, this is when you ask for the role, I'll just return the group role since the only function this view does is hold other UIs. For any attributes that you don't explicitly support, call Super because [inaudible] underneath will do all the work for you.

So now, we're ready to add some of the new APIs. The first thing is the new actions. Depending on if those buttons are visible or not, you can support the Show Default UI action or the Show Alternate UI action. Then when you're asked to perform those actions, you could go ahead and make those buttons appear or disappear.

Every time those UI appear or disappear, you want to remember to post the notification. And this is where we can use that new API, the post notification with user info API, pass in the Layout Change notification, and the User Information Dictionary with those changed elements. So with that, I want to show you how this works with VoiceOver.

You can launch VoiceOver with command F5 and the very first time you launch that, it's going to ask you if you want to run through a tutorial. If you never used VoiceOver, I really encourage you to try it out, the tutorial. For now, I'm going to skip that.

The next thing I want to talk about is Accessibility Announcement. In your app, there may be times where you want to convey some status to the user. And the example I have here of the Keyboard Preference where when a user assigns a key that conflicts with the existing key, Keyboard Prefs will display a little status text at the bottom of the screen. Now, this is a-- what I call, a "visual announcement" which works great if you're sighted. But if you're not, this is useless. So to help you make the experience the same for every user, we've introduced a new notification called Announcement Requested Notification.

It takes an Announcement key so you could pass on a string of what you want the assistive software to announce. So with that information, once we post that, VoiceOver can then either determine to speak it, output to Braille, or do something about it. Now, not every announcement has the same priority, so we also have a new priority key that you could pass in to determine the priority. We've predefined some priorities for you, Low, Medium and High.

So, so far, I've talked about APIs that help to solve UI challenges. Next, I want to talk about some of the security improvements we've made. Two things, specifically, one is the text content protection and the other one is a more secure usage of Accessibility API. So let's first talk about content protection. If you run apps such as the eBook that display copyrighted content like a book, or if you have a medical app that display valuable sensitive patient informations, what I'm going to say next is for you.

If you have not made your app accessible because of text content protection concern, we now have one more reason for you to make your app accessible. We've heard your feedback and now we're introducing a new attribute that you could implement to make your content more secure. So this attribute is called the Contents Protected Content Attribute. If you pass in Yes for this, then your text content string will no longer be viewable by other applications.

There is one exception and those are the system-- system assistive software apps such as VoiceOver for obvious reasons. To use this new attribute, you will also need to make API call. This is called the Set Make Contents Protected Content API. So to see how these works, I'm going to go ahead and demo to you.

So in the Accessibility UI example, there's the protected text. And I simply have a view that display a paragraph of text. And for demo purpose, I also included a checkbox so we could toggle the protection on and off. Now, let's take a look at this using the Inspector. So I'm going to lock this thing.

What I want you to look at specifically is this AX Value attribute. Note that, right now, you could see their entire string of that content. So if I go and check this Protect Content checkbox and refresh the Inspector, note that you can no longer see the string anymore. And we can do the same thing with VoiceOver and see what happens.

And so, I want to override some of the Accessibility APIs here. So since what I'm doing here is dynamically toggling the text content protection on and off, I'll make the API call in a place where I'm setting the protection turning-- toggling that protection. So I could just call NSAccessibility Set Make Contents Protected Content, turn it yes or no. In your app, you probably want the protection to last throughout the lifespan of your app. So, you could power it like in the app launch time.

Next, I want to add this new attribute, so in Accessibility Attribute Names. In addition to all the attributes that app already supports for the text field cell, we're also going to add this new Contents Protected Content attribute. And then when you're being asked for this attribute, this is where you return yes or no if you want to-- if you want that protection. So that's all you have to do to protect a content.

So, so far, I've been talking about your app and how events various accessibility information to enhance the user experience. Now, I want to switch gear and talk about some changes that affect assistive software. At Apple, we really care about protecting end-user's privacy, basically, all of their information. So in OS X, if any application that wants to use the end-user's location information or context information will have to get the users permission. We're taking that great mechanism to accessibility too in 10.9.

So in 10.9, any application that wants to use Accessibility API to obtain information about other applications will also need gain the user's permission. So the very first time the app uses the Accessibility API, the system will prompt the end-user for permission on your behalf. And until your app is on the approved list, the APIs would be disabled. So if you're an app that heavily depends on the use of the API, you will probably want to remind the end-user more often.

So we created an API for you to do just that. It's called the AX is Trusted, Process Trusted with Options that takes in a dictionary. So in your app launch time, you could simply pass in the Trusted Check Option prompt, it has Boolean for the-- Boolean true for that into the dictionary and set that as the argument for your API. And when that happens, then every time your app launch, it'll prompt the user for permission again. So if you want more information, there's e-mail list for you to join or a contact.

And if you miss the accessibility in iOS session earlier today in this room, you can watch it on video once it's available. And if you want to hear more about the close captioning stuff that Greg talked about, tomorrow morning in Uphill [phonetic], you can listen to that. So to recap, Apple is really committed to ensure that every user have equal access to the rich experience of using all of our Apple products. On our end, we work really hard to continue to improve our existing products like VoiceOver and new products like the Switch software, and to continue to enhance our existing set of APIs.

However, all of that are just tools to help the user to use your application. So, your application is the important thing in this equation. I'm hoping that I've given you some basic information on how to use the Accessibility APIs and the tool to make your application accessible. If you have further questions, contact us, watch videos, previous WWDC videos, or look online for documentations.

Today, I have also presented some UI challenges and how we have new APIs to help solve those problems. If, in your application, you have any of those UIs that I presented, I really encourage you to implement those new APIs. As you've see in the demo, they're really easy to implement.

And I've also talked about some of the security improvements too. So, I'm hoping that you go away with this and start to make your application accessible. If you have further question, again, come to our lab this afternoon at 12:45. And if I don't see you at the lab, have a great day. Thank you.