Welcome to iOS accessibility. My name is Chris Fleizach. I'm on the iOS accessibility team. Today I want to talk about how do you take your app to the next level in terms of accessibility. I'm going to be saying the word accessibility a lot in this presentation. So what do I mean when I say this word? It can have a lot of different meanings. Mainly I'm talking about using technology to overcome challenges. So things like computer controlled wheelchairs, assistive communication devices like ProLogo to go for the iPad, screen readers, so people with low vision and blindness can use computers, and of course there's many, many other examples of this.

On iOS we have a number of accessibility features that are designed to help people use a device in the same way that everyone does. So VoiceOver is one of our flagship products. It allows people who are blind, have low vision to use a touch screen device. It works by changing the gestures that are used so that instead of a touch to activate model, you have a touch to speak model. So this provides a safe environment to explore the phone without worrying about something that will happen.

We also have Zoom. Zoom allows you to use three finger gestures to zoom in and to pan around so you can get in close on text and read things if you have low vision. In iOS 5.0 we added assistive touch. Assistive touch allows access to all the hardware buttons, multi-finger gestures, even creating custom gestures through either a hardware device or if you only have access to a single finger or a stylus, you can also drive these gestures.

We've also added some enhancements that's built upon previous work. So custom vibrations was added in 5.0. It only applied to phone calls and FaceTime, though. In 6.0, it will apply to everything. So if you want a custom vibration for a text message from your wife, for example, you can set that and have it play specifically just for a text message. And that applies to calendar alerts, Facebook items, reminders, and the whole list of notifications.

VoiceOver and Zoom finally worked together. Before, there was conflicts between the gestures being used. We've overcome some of those so that you can use them at the same time. and we made a number of enhancements to SpeakSelection, which I want to show you now. SpeakSelection was introduced in 5.0. It allows you to select some text and have it spoken. We made two enhancements that I want to show. One is much better support for multiple languages. So I have the same sentence in three languages, English, Arabic, and Greek.

And now if I want to speak them, Speak selection tells me that there are English and two other languages available and when I select those. The revolutionary iPhone also includes an equally revolutionary screen reader. iPhone is also included in the revolutionary screen reader. The revolutionary phone also includes a six-way revolutionary program for computer recognition.

So, can automatically detect languages based on script and other identifying information. If there are ambiguous languages, we look around and see what other languages you might want to speak at any time. So, much better smarts for international support. We've also added word-by-word highlighting. So, if I go to... Turn on highlight words in the speak selection menu and come back.

When we speak these items now, the revolutionary iPhone also includes an equally revolutionary screen reader. iPhone Thauria yatadoumenou aydon qari ishashati Thauria ala hadsa wa. To epanastatiko telephono perilambani episis ena eksisu epanastatiko programma anagnosis othonis. So I'm sure you got all of that, but just some of the enhancements that we've added for speak selection.

So let's talk about what the meat of this presentation is about, and that's really using UI accessibility to make your app more accessible for voiceover. Now, why is this important? VoiceOver is a special application. VoiceOver is used to overcome a very challenging difficulty, blindness, which made more so by the touchscreen in and of itself. So, making your app accessible means that you implement some API and a VoiceOver user can get more information out of your app and know how to use it effectively.

So, we're going to be talking about the basic API, how do you do the most important things to make your app work well. We're going to be talking about the new API for the things that we've added in 6.0. And then I want to have a deeper dive into some things that you may not have known. This is the fourth year that we've done this. Some of you might be very familiar with the API, and I think there's some things that you'll be able to learn.

So how does this work? VoiceOver is running on iOS when you turn it on. And it intercepts all gestures. So when I touch somewhere, for example, I touch somewhere in notes, VoiceOver intercepts that and says what's the element at this point? And it does that by going through UI accessibility and then asking the app.

The app gathers some information about what's at a specific point and then bundles that up and sends it back over to VoiceOver. VoiceOver is then able to take that and transform it into synthesized speech or maybe Braille output or some other alternative output that's appropriate for the user.

So how do we add accessibility to your app? The good news is a lot of it comes for free. So if you're using UI kit controls to do things, most of it is just going to work. So your job in that case is mostly to add labels. Things that are images or buttons that have images inside of them. All you need to do is set a label most of the time and things will start to work.

So most of this information is conveyed through attributes. Attributes are the core of an accessibility API. Basically, they allow you to encode certain information about an object so the voiceover knows what to do. So, for example, we have a UI image view here. By default, this has some attributes built in. Specifically, we know the file name.

So here, you know, voiceover might have seen the image view and known that the file name is Apple logo 512 by 512. But obviously, that's not a great user experience if that's what they're hearing. So instead, it's up to you to set the appropriate attribute, in this case, the label for it. And the label might be something like Apple logo.

The two most important attributes in the API address the fundamental questions about is your app accessible. One, can a VoiceOver user reach an element? Can they touch and have it spoken? And then two, what will be spoken? The first question is answered in is accessibility element. You return yes, that means VoiceOver will be able to see it. The second question is answered with accessibility label. Accessibility label is a string that you can return that will be the textual representation of the object. By default, these are filled in for UIKit controls, so a UI label automatically returns its text for the accessibility label.

Some other common attributes, accessibility hint is a good way to provide a little bit more information about what to do if the context is not completely clear. It's optional. And accessibility traits provide you with a way to control what are the behaviors, the roles, other important information about this object. Is it selected? Is it a link or a button? So a little bit more about the traits. Say we have a screen that looks like this. At the top we have an element that might be labeled with a static text trait.

In the middle, we have an element that has the image trait. Towards the bottom, there is a button. And at the bottom, very bottom is a slider which would have an adjustable trait. So there's about 16 or 15 or so traits that allow you to decide how VoiceOver will interact with something.

So how do we go about and add these attributes? Obviously there's some objectives to see methods, but you can also add them through interface builder. So in the inspector pane, you might see a screen that looks a little bit like this, and all the information we talked about is encoded here as well. Is accessibility element corresponds to this checkbox? Accessibility label is the label, hint the hint, and then this array of traits where you can select what's appropriate.

So interface builder is great if you sort of have a static design or a button just is there all the time. But if you have a little bit more dynamic of an interface or you're doing things in code, you may need to set values in code. So one way is to use the setters. Say, for example, we have a my control object. By default, things that are just UI controls don't have any accessibility. So you need to go in there and set is accessibility element yes and set the appropriate accessibility label.

In this case, maybe it's a play button. If you have objects that change their value or change their label depending on your data model, you may need to override. You don't just have to use the setters. You can override and say, yes, this is an accessibility element and my accessibility label is something that's appropriate.

Accessibility attributes, that will take you 85% of the way there. Most of the work you'll do will just be setting attributes here and there and your apps are going to become a lot more accessible. Now, sometimes you also need to tell VoiceOver when things happen. And these are done through accessibility notifications. So when a few things change on the screen, you need to tell VoiceOver to update.

So if we look at this example in Clock, when you press the edit button, what do you get? Well, there's a few new elements on the screen. A delete switch, something disappears and so on. In that case, you need to tell VoiceOver to update itself. Just a few things have happened. To do that, we call UI accessibility post notification with a layout change argument.

The other type of notification that's important is the screen change notification. This is when the whole context switches out, VoiceOver needs to update itself, it will play a sound, it will move to a new element, and so on. And you do that similarly by sending the screen change notification. So, for example, if you select a new tab in the tab controller, that will generate a screen change. And these are sent for all the basic UIKit controls already. It's only if you're doing custom things do you need to worry about these notifications.

So that's going to take you 90% of the way there. Notifications and attributes, are really the most important things you have to worry about when making your app accessible. So let's start in with the demo and some of the code and see what we can do. The demo app that we're using this year is one that I've created. It's called the NSZombie Apocalypse.

Let's see how the app works before we try to make it accessible. Then we can look at it with VoiceOver. So basically the zombies, the NSZombies are running amok in your app. And when things happen, more zombies go and eventually you run out of memory and the app will crash. So your job is to keep deploying these memory release techniques like arc or auto release and you can make these zombies go away. I believe I have some sounds, too.

If you use garbage collection, that will actually add more memory to your program, so don't do that one. We also have this little help screen with a little story that I wrote. So, the question -- so this is how you play the app. The question is: How do we make it accessible? So the first step to doing that is to turn on voiceover and sort of audit it for accessibility. So how do we do that? The easiest way to do that is to go to settings and -- let's see.

Go to accessibility and then you go to triple click home and set VoiceOver to triple click. Once we do that, we can triple click and VoiceOver will turn on and off when you toggle that. Let's go back to our app, turn on VoiceOver. Landscape. Home button to the left. NSZombie Apocalypse.

Let's start to examine what's wrong with the app. The first thing, if I try to touch in the zombie meter... VoiceOver on. Zombie meter. Nothing really happens. An object was retained too many times. Six minutes. I can touch that. So that looks fairly standard. What about the buttons? Self-release. Self-release. So it didn't say it was a button and also the frames are rather small. That one just says button.

: There's a screen change here, but nothing really happened. Those ones also say button. If I start to examine by swiping around so I can move from element to element by doing a swipe gesture. Self-delock, self-release, self-autority, garbage, self-free button. An object was zombiemeter. So I can swipe through and go through all the elements in the screen and find out if something's missing.

I also wasn't able to touch on the zombies at all. That's sort of an empty area. So there's a few things that we can do to make this a better experience. One, we can make sure that the zombiemeter is described. We can make sure the zombies are elements. We can make sure the buttons say that they're buttons.

We can probably give some hints about what to do because it's not immediately clear what you do with those buttons. And then obviously there's the Help Button needs some labels and stuff like that. So let's go to the code and see what we can do. I wrote down a bug list and I also have some screenshots so it will be easy to look at what we want to do.

So here, the buttons at the bottom are not really buttons. That means that, let's see, these buttons are not really buttons. So instead, we need to make sure that : How do we do that? Let's look at here and go to our button view. How come it didn't speak button? Well, it looks like the button view is just a UI control.

Inside of that, there's a label which looks like the thing that VoiceOver was focusing on and then it does some other stuff. How do we do this? Is accessibility element return yes? And then let's return the label. What is the label for this button? The label is likely just the text of the label view. And then finally we need to set the traits to say button.

So that will make sure that the buttons behave like buttons. Now the buttons also need some hints. So how do we do that? Let's override

So hopefully that should give enough context what to do. Now the zombiemeter description sort of needs a label. So all I could touch was on that label of the zombiemeter. Luckily there's a class called zombiemeter. It's just a UI view with a label inside of it. So what we're going to do is make this an accessibility element. So you just have to find one giant zombie meter. We'll make sure it has the right label.

And that will also be the text of the label. And then we want to return the value of the zombie meter. The thing kept going up and up. It has some sort of percentage associated with it. So in that case, we can use something that I haven't mentioned yet, which is called accessibility-value. Value is great for things that change, that have some dynamically changing value to it. So in this case, we can return in a string, a string with format and percent 0f, percent percent.

zombie level times 100. Great. So that would give me accessibility value. Now the question mark buttons need a label. So let's look at our question mark button here. It looks like it takes a symbol and might have been a question mark or an X or something. And then it looks like it does a draw rect right in the middle with that string.

So it draws a rect with the circle and then it looks like it has a label beneath it. So we added a label subview to that button. So we can see why VoiceOver doesn't speak anything. There's a button and there's a label inside of it. So what we have to do is tell VoiceOver to speak the right thing. Now we could say it speak whatever that label has. We could tell it speak question mark or X.

But that's not the semantic information about the button. The button is really about Help or Close or something like that. So let's find out where we set these things. and set the right accessibility label with a setter. We can call this one help and find the next one. This one looks like it should say next or something. And the last one is hiding here. Looks like this is a help one.

When you press the question mark, VoiceOver needs to reset itself. When we're on the screen and we press that X or we press the question mark, we need to reset itself so VoiceOver knows the context has changed. How do we do that? Let's find out where that happens. I'm looking at question pressed. This looks like the right place. I'm going to post an accessibility notification.

screen change notification. So that will tell VoiceOver to reset when the question mark is pressed. Let's also do the same thing when we close the help. : So that will be good. And then our final bug, the zombies should be elements and have labels. So where are those zombies hiding? The ZBE walking dead. It looks like they have a bunch of body parts. Let's just find a body and then we can set accessibility element and the accessibility label will be something appropriate.

So make sure you also localize all your strings. I'm not doing that now for the sake of brevity, but VoiceOver speaks like 30 languages or so. So make sure you use localization if your app is going to be localized. All right. So we tried that. NSZombie Apocalypse. Zombie meter, 0%. So we check the zombie meter. Zombie meter, 15%. So saying the right percentage.

Walking dead in the house. Great. I can touch the zombie if I want to. Self-release button. It says self-release button. Memory technique over zombies and release to deploy. And that was the hint. If we can trust the help button. Help button. When we press the help button, we should reset.

Great. So we reset to the top of the screen. Let's check. That says the right thing. That says the right thing. So let's close. Zombie meter, 35%. Chris Fleizach: So we also went to the first element in the screen, which was the zombie meter in that case. Zombie meter, 42%. Chris Fleizach: So it looks like we have addressed all those bugs.

The app is becoming more and more usable all the time. So let's go back and So those are the basic accessibility attributes. With just a little bit of work, we can start to make our apps more accessible, even something as dynamic and graphically intense as that game is. So let's talk about what's new in 6.

We've already learned that with just a little bit of work we can get a lot of gain. So where do we go from here? We've added new ways to interact with VoiceOver. We've added some new attributes and traits to help make your app more complete. And something I won't demo, but if you have a custom text view based on UI text input, VoiceOver will work with it natively. No extra work required as long as you implement UI text input.

So let's look at the new API. First one, Accessibility Perform Magic Tap. So what is this? So if you've used VoiceOver, you may know that there's a gesture. It's a two-finger double tap. When you do a two-finger double tap, VoiceOver does the magic thing. This means if you're playing music, it will pause the music.

If you're in a phone call, it will answer the phone call or hang up the phone call. If you're in the clock app, it will start or stop the stopwatch. If you're in camera, it will take a picture. So two-finger double tap magically does the right thing in whatever context you're in. You can also add this to your app, too, if you need to. So you implement this method, you do whatever you need to do and return yes, and make your app seem like magic as well. Just like that.

So we also allow you to now move VoiceOver focus if you need to. Sometimes you want the voiceover to be at a specific place in your app to learn and understand what's going on. Before 6.0, that was quite challenging to do. So now what you can do is use the argument for the accessibility notification for either layout change or screen change and pass in an element that you want voiceover to move to. So, for example, if I have a screen change and I want voiceover to move to a specific button, I can do that passing in as an argument. You can also now find out when voiceover finishes speaking something.

Finishes speaking what? There's another notification that I haven't mentioned yet called the announcement notification. That came out in 4.0. And the announcement notification allows you to tell voiceover to speak something. Any string you want. Now, this is sort of nice if you need to have an announcement or the screen changes or something's happening that's not immediately clear. You can use that to inform the voiceover user what's going on.

But if you needed to string multiple instances of those together, for example, you're reading sort of a book in a row or something, you wouldn't know when voiceover is finished speaking. So you would interrupt what voiceover -- what you told voiceover to speak. So now you can listen for the announcement did finish notification. And that's an NS notification center notification. And when that comes in, you know that voiceover is finished speaking your string.

We've added a new API to help you order things when voiceover is navigating through your interface. So what do I mean? Let's look at the clock app, for example. The clock app has all these little alarms and sort of a random grid of array. And you would like voiceover users to navigate through them in a specific order.

How do you navigate? Through a single finger swipe, left or right. So as you're doing this, swiping through left or right, by default, voiceover is going to go top left to bottom right. So it's going to go sort of row by row through things. If you want to order them differently, you should group accessibility children. And that tells voiceover that these things in this view should be together.

We've added a new trait, the UI accessibility trait header. So this screenshot of Game Center, I would identify two things as possibly being the header in this case. So they allow you to identify the VoiceOver that said this thing is sort of like a heading over some sort of content. And that can be useful for locating where you want to be or understanding the layout of the app. So let's go back to our app and see what we can do to use some of this new API to make it even better.

So the second demo bugs, what do we want to do? Let's implement this magic tap and see what we can do. Let's try to move VoiceOver focus when the question mark disappears. So when you press this X button, let's try to move VoiceOver focus to something more appropriate like maybe the status view. So we can try to move VoiceOver

on here to on the NSZombie Apocalypse heading to identify that appropriately. All right. So first up, perform magic tap. So I'm going to go to my app delegate and Override this.

Return yes. And you can implement perform magic tap on an object, an individual object, or a parent in the view hierarchy or the UI application or the UI application delegate. VoiceOver will go up the chain of things, trying out each one. And I think what we should do is toggle pause. So I know this is a fast-moving game. It's very challenging. If you get tired and need a break, you should be able to pause it.

Move VoiceOver focus to the status when the question mark disappears. Let's go to our view controller. Help did close. This looks appropriate. Now let's just pass in status view. Done. Should group children on the button collection view. How do we make these things order together? There's a nice button collection view. And then here... Let's return yes for that. And then the heading trait on the question mark title. So help view. There's a giant label here somewhere. This looks like it. : You can set the accessibility traits to be the right.

is running. NSZombie Apocalypse. So what do we do? So the first thing, the swipe order of this should be together. Self-delock, self-reli-self-watch, garbage, arc, help, button. So if you notice I swiping through I went from... Self-self-self-self-garbage-arc, help, button. I went through all the buttons and then after that I went to that help button. So buttons ordered together. Good. Things are working. We did the magic tap.

paused. Zombies are finally paused. You can go get a break and get yourself an Odwalla. When you're -- can double tap and start the game again if you need to. What else do we want to do when... Help. NSZombie Apocalypse. Heading. Great. That thing says heading just like we asked it to.

And finally when we close this one, we should move back to that status view. Memory leak, three megabytes. Great. So we closed that thing and we told VoiceOver to move to a specific element and it worked just fine. So let's go back and talk about some other cool things.

Let's talk about some of the more deeper topics. If you've made your app accessible already, you may have had a standard app. It might have gone very easily. But chances are that as your apps become more complicated, you need to use some other accessibility techniques to continue to ensure your apps remain accessible. The first thing I want to talk about is what happens if there is no view. So, for example, you're using draw at point with a string. You're using overriding draw rect. Maybe you're using OpenGL. A good example is Maps.

Obviously, there's not a new UI view for every road. So what do we do in this case? So what you want to do is make an array of UI accessibility elements. UI accessibility element is basically just a container for some idea in your program. So for every distinct interface object that you want to expose, make a new element.

How does this work? So, for example, say we have an array of roads. We want to gather all of our roads together and return those as elements so VoiceOver can interact with them. First thing we need to do is make the element. So we do that by initing it.

Notice the accessibility container should be the view that does the drawing. So VoiceOver starts by drilling down through the view hierarchy starting at the window and then all the views all the way down. If your view is the last thing before it starts drawing random things everywhere, then that is the accessibility container.

You need to set the accessibility label for that object so VoiceOver can speak something. And then maybe you want to store that in an array. So it's good to cache these things. VoiceOver will ask for them often. So if you can cache them in some sort of array or something, it will make life easier.

Chris Fleizach So next up, how do we tell VoiceOver what they are? So we want to implement what we call the UI accessibility container protocol. Three methods, pretty much just like NSArray. First one, the count. How many accessibility elements are there? Easy enough if you have an array set there, you can just return the count.

Chris Fleizach Next up, what's the index of an accessibility element? Here you can mirror index of object from NSArray. And finally, what's the element at an index? Again, mirrors NSArray. Chris Fleizach So six or so lines of code. Pretty simple if you have this modeled on an array to start with.

This is what UI accessibility announcements are good for. So for example, if we have a continued tracking method where we're tracking where this object is, we can detect when things happen and tell VoiceOver about it. So for example, if we're near an edge, we can tell VoiceOver you're nearing the left border, you're nearing the top border, nearing the bottom border.

If we are on empty space, we can say on empty space, lift finger to cancel. And if we're over something that's important, we can say on top of artist, lift finger to replace. So notice here I've pound defined post to be the much longer UI accessibility post notification. That's just so it fits on the slide.

The last thing I wanted to talk about is direct touch or direct interaction. This is designed as a way for you to allow VoiceOver to interact with something that is very dynamic. A good example is a keyboard. Keyboard really requires you to press things in real time. You don't want to double tap on every key.

That would be a very tedious way of making a song. VoiceOver allows you to do this by setting the direct interaction trait on an object. When you do that, it means a certain area will allow VoiceOver to interact with it directly. You can also allow VoiceOver to explore elements within the direct touch area.

This keyboard is composed of many keys that have information. For example, you might know that this is a C3 key and you might want to inform the VoiceOver user of that. So how does that work? Well, VoiceOver has a number of gestures to control things and one of the things they control is whether direct touch or direct interaction is on by default or off. So if they turn it off, they can continue exploring an area. When they turn it on, they can interact with it.

So some code to show how you do that. Maybe you have something like a piano view. And in your accessibility traits of piano view, you'll return the appropriate trait, which is this direct interaction trait. Notice that it is an accessibility element because this is what you want VoiceOver to interact with. But you can also set the objects within this view as accessibility elements. So a VoiceOver user can explore things within a direct touch area. So sort of a nice trick to remember if you have an app that uses this kind of paradigm.

So let's take a look at how we can make our app even more accessible using some of these techniques. And I want to try to use the use announcements while dragging technique to provide a little bit more information. So the one thing that might have been hard about this app is where do you drag those buttons, especially if you can't see? So I believe we have some helpful methods here.

Button selected, button dragged, button finished. So I think we can make this a better experience by posting some notifications That tell you what to do. So when the button is first selected, what should we say? My guess is something like drag button over zombies. : That should be good enough.

Now, when you're dragging, there's a few different states. First example, if the button is inside of the iPad view and it's the first time inside the iPad view, We should probably say something appropriate like lift to deploy memory. And if you've left the iPad view for the first time, maybe we can say, "No longer over Zombies. Live to cancel."

So just by providing a few extra announcements, a few lines of code, you can make your app, even if it requires a lot of interaction, make it a lot more usable.

So final topic I want to mention are things that you may not have known about UI accessibility. Some cool things that you can do if you've really gone into the advanced techniques. So one is control the language of the entire app. So how do we do that? You can set the accessibility language on a UI application. So accessibility language is another attribute that tells VoiceOver what language to use when it speaks. If you want it for the entire app or just a specific element, you set it on the right thing. You can also set it for a range within a string.

So for example, let's say we have an attributed string that has some Japanese and some English in it. And you want VoiceOver to speak with using a certain language for a certain range. You can add the attribute accessibility language with the right value for that range. Now the values for these things are some standard Unicode thing. J-A-J-P is Japanese. F-R, F-R is France, French. Easy enough to find a list of them and the ones that VoiceOver supports.

And then once you have that, you can use that as the accessibility label. So accessibility label says it takes a string. You don't have to believe it all the time. You can also pass in an attribute to string if you want to. Now that attributed strings are much more prevalent in iOS 6, How do you deal with controls without views? So there's a few controls in iOS that are challenging to get at.

One of them is the UI segmented control, specifically, you know, something that looks like that where you want to override it to say something different than degree sign F or degree sign C. In this case, in weather, you would like it to say Fahrenheit or Celsius. So how do you get at those things? Normally, you create your UI segmented controls with the NSString or image. So in that case, just set the accessibility label on the NSString itself. or if you have an image, just set the accessibility label directly on the image and then when you insert those things into the segmented controls, VoiceOver will pick that up and speak the right thing.

Another example is UI table view index titles. Here's a thing that you return an array of strings for a table to get the ones that you want to appear. You can change what VoiceOver speaks for each of these things as well by overriding the accessibility label and setting that on NSString.

So, in summary, obviously, we want you to add accessibility to your app. I think you guys are all committed already. That's why you're here. It's an important task. The reasons are numerous. Just a few of them. One is going to increase your user base. Obviously, you want more users using your apps. So, that's a good reason.

I think another great reason is that you get a lot of really good feedback from users. So, users are very excited when they find out that an app is accessible. It's really nice and rewarding to get that kind of feedback. And finally, Apple takes accessibility seriously. It is a partnership with developers. We are working hand in hand to try to make the most accessible platform that's ever existed. And it really requires Apple's commitment. And your commitment to ensure that everything works as well as it can.

So, I think this is a really worthy goal. Something that you should be telling your friends about. It makes for a really compelling use case for iOS to see just everyone being able to use this device. And getting the same satisfaction that all of us do every day.

If you're interested in related sessions, there's an accessibility in iBooks session tomorrow. And if you want to make a custom text view, like I mentioned, keyboard input in iOS is a good one to attend. And lots of more information. There's accessibility programming guideline which will repeat a lot of the stuff that I said here about common attributes, protocol reference, and then VoiceOver user manual will tell you all the gestures that VoiceOver can do. So, I think it's quite useful. Thanks for coming out and attending WWDC 2012 and keep making great accessible apps. Thank you.