Welcome to iOS Accessibility. My name is Chris Fleizach. I'm on the iOS Accessibility team, and we have some really exciting things to talk about today for iOS 5.0. So what is accessibility? Well, at Apple, we think about accessibility as using technology to create possibility. So that means things like adding closed captioning to videos, allowing Braille displays to completely control an interface. allowing use of assisted input, and of course, there's many, many other examples.

Now, in this past year, we have just been so overwhelmed by what has gone on with accessibility on iOS. And it's been really exciting to watch. I'm proud to say that Apple has been a part of that success, but a lot of that success has become because of hardware manufacturers and software developers making great products and great apps. And through that combination, iOS has become such an indispensable tool for so many people.

So, of all the things that we've come across, I wanted to show you one example of something that we thought was especially cool that really gets to the core of what we mean when we say using technology to create possibility. So, I'm going to show you a short video before we start of Joshua A. Mealy, who's a scientist at the Smith-Kettlewell Research Institute, and he's developed something called the Wearer Braille.

Hey, I'm Josh Mealy. I'm a scientist at Smith Kettlewell. We're a research institute in San Francisco, and we do design and development of accessible technology for blind people. One of the things that we've been working on is this thing that we call the Wear Braille. I'm wearing it on my hands right now. This is just a prototype with lots of wires and stuff. But the Wear Braille is a new type of technology that we're calling a virtual wireless Braille keyboard. What are on my fingers are accelerometers.

They detect when I tap on the tabletop. So what we can do is use the accelerometers to detect taps, and then we use a wireless connection to send information to a smartphone that tells the smartphone what Braille characters and commands I've typed. So let me just pause them for a second. What they've done is develop a device. That monitors where your fingers tap on any surface, and they can translate that into Braille cording commands. So not only are you allowed to type on an iPhone, but you can actually completely control the iPhone device.

Basically, we just turned on VoiceOver, the iPhone screen reader, turned that on, and connected this device to it as a wireless Braille device. And we can do all of the commands and text entry that we want to using the Wear Braille. And I'll show you how we do that. So I can navigate around on my screen with Braille commands. I can go to the next. Contacts. Calendar. Dictation. Or I can go back. Calendar, contacts, next bus.

So that's a pretty cool demo of just some of the potential that iOS has that we're really excited to see going forward. So accessibility in iOS is all about equal access through alternative input and output. So that means we have solutions for low vision, blindness, hearing impairments, and motor and physical disabilities.

So a lot of these new settings are where the other settings are. In the Settings app, you go to General Accessibility, and now there's almost three screens full of options. As usual, we still have VoiceOver, the best screen reader for a mobile device. System Zoom is great for magnification purposes, and Large Text is a godsend for those of us who can't stand the tiny text and messages. But we've also added things like speak selection, so that you can go to any area in the system that allows selection and then have it spoken back to you.

We've added custom vibrations so that someone can assign a specific vibration pattern to a contact. LED flash for alerts will flash the torch on the back of the device when an alert comes in if you're on the lock screen. Great way to be notified of something happening. We've added a left/right balance for stereo control. Assistive Touch allows adaptive hardware to be connected to an iOS device to completely control it. And there's an ability to route incoming calls to a specific place. So when you answer a phone call, you can decide should it automatically go to speakerphone or to your Bluetooth headset.

So I wanted to give you a short demo of a few of these features because we think they're really cool, and you may not have seen them yet. So first up is Speak Selection. So I've already enabled Speak Selection in Settings, General, Accessibility. And you can set the speaking rate there as well. And then once I come to something like Notes, for example, I can select some text. And there's now a speak bubble.

Once I speak, press that. Comes with innovative new features that make it easier for people with mobility, hearing, vision, and cognitive disabilities to get the most-- Great. So that just works. But Speak Selection is a little bit smarter than that. Say you speak more than one language, but you run in English all the time. Speak Selection will even be able to detect-- What language it should speak. Formado por Desconcejales del Partido Popieler, nueve de bchicaxt, uno del psoe y uno de actua. Los o... Great.

So... So next up is custom vibrations. And the genesis for custom vibrations is that if you're hearing, you can assign a ringtone to a contact. When that person calls, you hear that ringtone, and you know, I'm not going to pick up this call. But if you're hearing impaired, you don't get that same level of granularity. So we thought, how can we solve this? Well, we can add custom vibration patterns. So once you enable this feature in Accessibility, you can go to Contacts. And when you edit a contact, there's now a section for vibration.

And there's some defaults that you can select, but what's really exciting is that you can create your own pattern. And all you have to do is just start tapping here, and the longer you hold down, the longer the vibration. The shorter that you hold down, the shorter. You can stop it. You can even replay it if you want. And then when you're ready, save it, give it a name. And now that vibration pattern is going to be used whenever that person calls you or FaceTimes you.

All right, so next up, the last demo is AssistiveTouch. And AssistiveTouch is something that we're really excited about, because we think it's going to level the playing field for a whole new group of users and bring them into the iOS community. So what I have here is a Dynamic Controls DX2 control system with iPortal. And this is what it looks like in the field. And so Dynamic Controls is the world's leading manufacturer of electronic controls for motorized wheelchairs and scooters. And they've been kind enough to modify their iPortal system to work with AssistiveTouch.

So let's see if this works. So the first thing you'll notice is that I now have a finger on the screen. And the finger allows me to do everything that I would want to do, which is hover around and wait until I touch something. So for example, I can go to Photos and press that.

with the button that I have, open up photos, and it's sending these exact same touch events that will be generated as if you actually touch the screen. So that's pretty cool. I can get around, I can touch. But obviously that's not going to get me too far, so we need a way to drive the rest of the iOS device. And when we say complete control of iOS, we mean complete control.

So I can bring up a menu and have access to all the things that you can do if you were physically holding this device. So for example, I can go to Device. I can press the volume up or volume down. I can mute or unmute the device. I can even rotate the screen to whatever orientation I want to.

Likewise, you can press the home button to get back out, and the menu disappears automatically, because you know after you press the home button, you're probably going to want to do something else. That covers a lot of interaction. You can access all the features of the device. But we realize iOS is not just about pressing buttons and touching with one finger.

You also need a way to do multi-finger gestures. We have a gestures menu where I can select, for example, three fingers and get the standard layout for three fingers. And, for example, if I do a three-finger double tap, we all know that activates zoom. So I can drive zoom with this if I wanted to.

And so that's going to give me access to sort of standard layout. But we know that we can't possibly predict all the variations that apps come up with for gestures. So there's a way for you to add your own gesture. And then once you do that, it sort of populates this menu. And we've added some ones in automatically by default. So Pinch allows you to obviously pinch in and out. And we'll have some other built-in ones so that you can drive all the standard gestures that you want to do. So that is AssistiveTouch.

We think it's going to be really powerful and enable a lot more people to start using iOS. So for the rest of the talk today, it's mainly focused on making your apps accessible. And when we say make your apps accessible, what we're really talking about is make sure your apps work well with voiceover.

And voiceover is sort of a unique case because it requires the participation of app developers to make it a great experience. There's system-wide accessibility features like custom vibrations and system zoom that work everywhere. They don't really require any other interaction. But to make voiceover work well, we need app developers to make sure their apps work with voiceover.

So we're going to talk about how to use the UI Accessibility API to do this by looking at some of the basic attributes, some of the new things we've added, and some really cool API if your app is of a specific kind. And then finally, we're going to finish with accessibility apps, or apps that are designed for users with specific needs. Thanks.

So what is UI Accessibility? How does it actually work? Well, VoiceOver is running on the system, and it needs to ask your app a question. For example, what is the element at this point? So that VoiceOver can touch somewhere and then find out what is there. So to ask that question, UI Accessibility gathers up the relevant information from your app, like what is the label, what is the frame of this object, and then sends that back to VoiceOver so that VoiceOver can output it to your app.

Chris Fleizach So what is UI Accessibility? How does it work? Well, VoiceOver can also be used to create a text, and it can be used to create text in an alternative manner. And that means something like using synthesized speech to speak it, or maybe send it to a Braille display to have it Brailled. We can even draw a bounding box on the screen so that you can see where this element is.

So all of this really illustrates this alternative input and output. VoiceOver is all about driving the interface in an alternative way so they can output things in an alternative way. So the good news about adding accessibility to your app is that it's easy. A lot of it's going to come for free if you use UIKit standard controls. And most of the time, you just have to add labels.

So the core of the API is about attributes, and attributes convey information to VoiceOver. VoiceOver can then transform that into whatever means it needs to. So for example, say we have an image view, we can set the accessibility label on that image view so that VoiceOver knows to speak Apple logo instead of just speaking the image name or some other random data that may not be appropriate.

So the two most important attributes in Accessibility are answer two fundamental questions. Can VoiceOver touch this element, or can VoiceOver use this element? And how does VoiceOver speak this element? And they're answered with isAccessibilityElement, where you return yes if you want VoiceOver to see this element, and accessibilityLabel, where you return a string that adequately describes what this object is. So with those two attributes, two lines of code most of the time, your app is going to become 80% accessible.

There's some other accessibility attributes. I'm not going to cover them all, but some other useful ones. Accessibility hint, an optional string that allows you to provide a little bit more information about what to do or how to interact with an object. And accessibility traits, which define the role, the state, the behavior of an object. And accessibility traits are a bit mask of integers so you can combine them together.

So if we looked a little bit further at accessibility traits, say we took a screen like this, and You have various things. At the top, you might have static text. Because this is an integer, we can combine this with another trait called the summary element, which tells VoiceOver to speak this automatically as soon as you come into the screen.

For the image in the middle, we can use the image trait. For the button at the bottom, we can say it's a button trait, and we can also tell VoiceOver that this starts a media session. So that means when VoiceOver presses this button, it should stop talking because something's going to happen. And then finally, the slider on the bottom gets an adjustable trait, which informs VoiceOver that they can use this object by either swiping up or down to alter it, and they also know that it interacts in a little bit different way than other objects.

So where do we add accessibility? Accessibility, if you only need to change a few static values and you have a nib, well, it can be done in the Accessibility Inspector panel in Xcode. And you'll see that there's fields to set isAccessibilityElement. You can set the accessibility label. You can set the accessibility hint there. And then you can choose from the smorgasbord of accessibility traits.

But if your app needs a little bit more than Xcode provides through Interface Builder, well, we need to add stuff in code. So if your accessibility values don't really change, they're just sort of static and right there, well, you can use the setters. So for example, say we have a control that's a play button. It's a play button and never changes. In that case, we set the isAccessibility element to yes. We set accessibility label to play, and now it's accessible.

If your classes are a little bit more dynamic, so for example, say we have a product view that changes based on what kind of product it is, well, in that case, we can override the class methods to return the appropriate values. So return yes for isAccessibility element so that VoiceOver can see it, and then return the appropriate accessibility label. For example, depending on what we know inside of the class, we return the right information.

So accessibility attributes are going to get us 90% of the way there to make a great accessible app. Sometimes you need to tell VoiceOver when things happen, though. And this is where notifications come into play. So two ones that I want to point out are used when things change, basically.

So if only a few items change, VoiceOver needs to update itself. So for example, say we have a table view cell, and we press the edit button, and there's a new item in that table view cell. Well, the whole screen hasn't really changed. It's just a few items. We need to tell VoiceOver to update itself. We can do that with a layout change notification. We send that, and VoiceOver will know to reset itself.

Now, when the whole screen changes, that is a different cue for a VoiceOver user. And then when that happens, say we select a different tab, we need to post the screen change notification. And that will tell VoiceOver to play a different sound, reset itself, do enough stuff to let the user know what's going on. So we have an app here, and let's just show what this app looks like. show it in the simulator, get you a little sense of what it is and what we have to do.

So we have an app here. Basically a table view cell, some buttons we can press. When we select something, this comes up, and we go back. All right, great. So we have the app running. The goal of this demo is to show you, you know, you have an app, how do we make it accessible? Well, the first thing is we need to test it with VoiceOver. So how am I going to do that? Well, I'm going to exit this app. I'm going to go to -- well, I could go to settings, and I won't for now. But what I've done is I've turned on the ability to enable VoiceOver with triple click.

VoiceOver on. So I've pressed the Home button three times, and that turns on VoiceOver. It is a great tool if you're developing and you want to test things quickly. So, um... PhotoBove. Let's take a look at this app. Double tap to open page two of two. Accessibility Basic. So there's our app. Double tap to open. And all I've done so far is just touch on the screen. And if you're using VoiceOver, first thing to remember, you want to hear something, you touch it. If you want to activate it, you need to double tap.

Accessibility Basic. So VoiceOver sort of alters all the gestures that are normally used, because we need to provide a safe environment for the user to explore their app without worrying about activating something or doing something bad. All right, so now we're set up, and we're ready to test the accessibility of this app. So let's just start touching around. Button. Button.

Nothing there. If I select this, voiceover is still sort of hanging out in the background. It hasn't updated itself. Empty list. It's saying empty list. I can't touch the icon. Go back. It doesn't tell me it's a button, anything like that. Still hanging out in the wrong place.

We're going to use our knowledge that we have gained in the past five minutes to make this into a very accessible app. So what do we have to do? Well, let's first look at making those buttons on the side say something appropriate. So right now, here we have our interface builder. We'll open up the inspector panel. And over here, we see the Accessibility tab.

And there's no label there because it's just an image, so we'll need to add a label. For example, we can say these are iPad products. Same goes for this one. This one will be iPhone products. All right, so now those have labels, and that sort of covers what we can do with Interface Builder through accessibility. For the rest of it, we need to go into the code.

So let's also see if we can make these table view cells accessible. So conveniently, there's a product table view cell, and if we look at this one, should become clear why there's nothing spoken. It's because we have a bunch of strings, and then we draw at a point those strings. And that's a great way to make your app inaccessible.

But we can overcome it by implementing-- I don't need to type this. I can just copy and paste-- accessibility label. So what I've done with the accessibility label is take the strings that I have that I know about and concatenate them with commas. That tells VoiceOver to pause a little bit. So I'll take string one, string two, and then the product name, and return that as accessibility label. Now VoiceOver is going to speak that.

So what else did we want to do? Well, there was a big product view that had an image of something, and that was not accessible either. So there's a big product view class, and inside of it, we find out that there's an image view. Well, we can make that image view accessible so that VoiceOver can touch it and speak it. So let me do that by saying product image view is accessibility element equals yes.

and set the accessibility label to something like large iPad or large iPhone. So, that will take care of the product view image. Now, how about that go back button? So, all it said was go back. It didn't actually say it was a button. So, making sure it's a button will make sure that VoiceOver user knows how to interact with it. So, we can find where that label is. And if I look in the view controller, I see we have a UI control with a bunch of things and then a UI label inside of it. Well, let's make that control the accessibility element. And we can do that.

by overriding his accessibility element, the accessibility label, and then set the accessibility traits to the trait button. So let's take a look at how this works. Accessibility Basic. Double tap to open. And see what kind of damage we've done with just adding a few simple attributes here and there.

Accessibility Basic iPhone Products, iPad Products, Button. That's speaking. Zero, Product Name, iPad, one, Product Name. So that's working. Now if we go into it. Large iPad. Image. Go back. Button. Great. So with just a few lines of code, we've turned this into something that was completely non-understandable to something that's pretty accessible. So there's a few more things that we want to do to make this into a great accessible app.

One is we couldn't tell which of the products was selected. It just said "button" for both of them. So we can use a trait to define that behavior. So inside of our view controller, there is an updateSelectedTrait method ready, and I'm going to paste in what I want here. And I'm going to say my iPad button and iPhone button have the button trait. And then depending on which is selected, I'm going to add the selected trait to that. So by adding that information in, I'm using the OR operator to combine those things.

We can know what is selected. So finally, we need to make sure that VoiceOver doesn't hang out in the wrong place after it goes to a new screen. And that's where we use the notifications. So, for example, when we did select the row, we should send a screen change notification, because when we select that row, we sort of transition to a new view. It's basically a new screen. So I can post a notification with the screen change.

And then finally, when we go back, we also want to post a screen change notification. And I believe that happens in remove overlay. So I'll post that as well. So let's give this one more run. Accessibility Basic. Double tap to open. And see how we did using just these basic accessibility attributes.

Selected. iPad product. iPhone products. Selected. iPhone products. Great. So now it tells me which one is selected, depending on that. If I select a product name, Large iPhone. Image. Great. So VoiceOver saw that there was a new screen change. It automatically focused on the new element and updated itself. If I go back... Go back. Button. Three, product name. It goes back and it realizes that it came from a certain table view cell, and it will go back to that one automatically.

You're clapping because it's that easy. That's how easy adding accessibility is to your app. But we want to take UI accessibility beyond so that you can use this for greatness. So apps can obviously become accessible with just a few attributes. But if your controls go beyond the basic controls, you realize that you need a little bit more power. And if you've been making your apps accessible up to now, there's some scenarios that have been difficult to achieve.

So we've added some new API in 5.0 that we think will address some of these previously challenging problems. So the first one is how do you tell VoiceOver not to interact with something that it shouldn't? For example, you have some sort of overlay that pops down. VoiceOver still sees the things behind that overlay. And that's because VoiceOver gathers all the elements on the screen and just assumes that if they're visible, that they should be interacted with.

So by using accessibility view as modal, you can tell VoiceOver to ignore everything outside of this view. So say you have an alert that pops down. It's a child of the window. That view is modal. VoiceOver will not interact with any of the other children of that window. The flip side of that is that you can use accessibility elements are hidden. And what that does, it tells VoiceOver to ignore everything in this subtree, including the element, so that VoiceOver won't interact with it. So they're appropriate in different situations. They're both there. Thank you.

We've also added Accessibility Activation Point. So when VoiceOver taps on an item, it usually sends the tap point to the exact center of that object. And that is usually the best case, but sometimes you have a control that might have a big bounding box and a little touch area. You can control where VoiceOver touches by using Accessibility Activation Point.

Finally, Accessibly Perform Escape allows you to implement in your app something that's been in the system since 4.0. So a voiceover user is able to take two fingers and do a scrub gesture, basically a horizontal back and forth motion that is designed to get them out of some sort of state.

So for example, it will hit the back button in preferences. It will escape out of alert. So if you have some sort of modal state up, a voiceover user can know that they can do a two-finger scrub, and then they can get out of the state. of this whatever state they're in. So let's take a look at a different app now that I've designed just for this purpose. It's called the Accessibility New app.

And let's give this a run on our iPad. Accessibility Basic. Double tap to open. and do our accessibility audit to see what we have to do. Accessibility new. Show grid. Button. Show grid. Button. All right, obviously this is a very powerful app. I spent a lot of time making it. If I press this button, So, we didn't get very far with VoiceOver.

If I turn VoiceOver off, I can show this thing and then show you what actually is supposed to happen. I can touch each one of these icons. When I do that, a little X appears. When I touch the X, it goes away. And so on. And then when I'm done with it, I can touch out of there and it goes away. So obviously this app has been designed to highlight the new API that we have. So your mind should be churning away at how we can make this app accessible using some of this new API.

So the first thing we'll want to do is make sure that those icons were touchable. Right now, VoiceOver can't even get to them, say anything about them. So inside of Accessibility New, there's a class called Grid Panel, and here we have the controls that are being created. So we'll make them accessible.

by setting yes on accessibility element. We'll set the accessibility label to be the index that we have, and we'll set that to be buttons. All right, so that's all basic stuff. Now, when that grid panel came down, that's really a great case for the view is modal. The thing has come down. You're not supposed to interact with things behind it at this point. You're only supposed to be in this little panel. So when we show this grid, I can say, panel accessibility view is modal. Tells VoiceOver to stay within this panel.

So another thing we want to do is when you tap on that icon, a little X appears. When you tap on the X, it goes away. So how do we get that same behavior? So this is basically the same scenario as on the home screen when you delete an app. You can bring that up, and the little X box appears. And what VoiceOver does in that case is say, delete this app. And when you tap on it, VoiceOver touches the X button. So to do that, What we want to do is override accessibility activation point.

I'll go down to the bottom and say the activation point of this selected icon is the same as this little close button. So it overrides that. And it also changes a label. So it tells the voiceover user what's going on. It changes the label to say delete selected icon, whatever it happens to be.

So the flip side of that is that we'll also want to reset it, and I already have that code. So when something else gets selected, we can reset the accessibility label, and we can reset the other information that's important there. So the final thing that we want to implement before we take a look at it is how to get out of that modal panel. That thing comes down, VoiceOver could be stuck in there if you can't get out of it.

So we can use the Perform Escape gesture, or Perform Escape API, so the VoiceOver user can do their gesture and get out of it. So I'm going to implement Accessibility Perform Escape. I'll call whatever needs to be called. Panel should close in this example and return yes to say that I handled it.

So with those few things using the new Accessibility API, we'll see if this app is accessible or not. VoiceOver Accessibility New Show Grid Button. All right. : Icon 1, button. Great. So now when we press that, VoiceOver automatically moved into the grid. : Icon 1, icon. If I'm swiping with my finger left to get out of it, and all I hear is this sound, that means-- : Icon. That means I can't get out of it at all. So that's-- our view is modal, working.

Now when I-- : Icon 6, button. --double tap on an item-- : Delete icon 6. --it tells me delete icon 6 right away. If I double tap again, : Icon 1. Button. It will use that Accessibility activation point to tap right in that little icon instead, and send it away. : Icon 7. Button. And finally, how do I get out of it? I can use this two-finger horizontal motion. : Show grid. Button. And it goes away automatically.

So a few new APIs. They're obviously not as intuitive as the basic APIs. That's because we've now entered the realm of handling complex apps. And as developers make their apps more complex, the API needs to keep up with it. I want to talk about some app-specific API now.

So not every app will need to implement this, but if you have an app that falls in this category, it will make VoiceOver user very happy that they can use it, basically. So the two ones that we have API for are for content like iBooks. So, for example, VoiceOver works in iBooks. And then for things that have interactive elements. So, for example, the Chinese keyboard. Very interactive VoiceOver's model of double tapping and selecting really doesn't work well there.

So we've added something called a content reading protocol. So if your app is a magazine or a news article or displays multi-page books, anything like this that has long-form content, then you're able to implement this API to get the same kind of behavior that iBooks offers to VoiceOver users.

Chris Fleizach And what that means is that VoiceOver will touch to speak every single line, so you can get very fine granularity. It will automatically turn pages, and you can continue reading from any line, and it will just keep reading through the whole piece of content. So we think this will be really useful because obviously there's a lot of magazine, there's newsstand coming out, and we're hoping that this will solve VoiceOver's problem.

So how do we do this? Well, we implement the UI Accessibility Reading Content Protocol. And then we have to... implement four methods to return the data. The first one is that we return the page content. So this provides the whole content of the page. So VoiceOver users can just get that whole page content and then speak it.

Next, we want to return the line number for a point. So a voiceover user can touch wherever they are and then have that line spoken to them. So that will give a line back. And then you have to do something with that line. For example, you need to provide the string for that line with content for line number. And then finally, to make sure it looks correct, we need to implement accessibility frame for line number, which returns the bounding box of this specific line.

So that will take care of the content. We also may want to make sure that VoiceOver can automatically turn pages as it moves through the content. So for the content view, you can return the trait, "Trait causes page turn." And obviously, that will tell VoiceOver to cause a page turn. How does it cause a page turn? We need to implement something called Accessibility Scroll.

And Accessibility Scroll came out in 4.3. It allows your app to do whatever it needs to do when a VoiceOver user does a scrolling gesture. And for those of you familiar with VoiceOver know that's a three-finger swipe, left or right, up or down. So if you handle Accessibility Scroll Direction Next or Scroll Direction Previous, you can turn the page in that case and automatically allow VoiceOver to continue reading.

So I have a demo. OK. Accessibility Education Demo: Learning to Write. So I have a book here. It was very challenging to write. And with VoiceOver on, Learning to write. Learning to write. All I can get to is this header. So we'll turn VoiceOver off. VoiceOver off. What else can this do? I can swipe my finger to go to the next page and keep going. There's only three pages. I can go back and so on. So if we implement the right protocol, VoiceOver is designed to make this book experience accessible.

So inside of our project, there is a book content view. And this is where we want to implement the UI Accessibility Reading Content Protocol. So we'll add that to our header. And then inside of our class, Well, let's just look at how this is working. So it looks like I pull out a page out of our bundle, and then I draw the rect by drawing every line in that page. So I separate it by new lines and then just draw at a point.

So how do we make this accessible? Well, the first thing we want to do is return the page content. So I can do that just by getting whatever my data model is for that page. Return that. I trim the characters just so VoiceOver doesn't pause unnecessarily for new lines or other white space. So that's the easy one. Now we need to figure out the line number for a point.

The point is going to come in, and it's going to be in the view space of your view. So if your view starts at 0 and goes to 500, VoiceOver will send a point that is between 0 and 500. And it is up to you to figure out what line is on that point. So in this case, I do that by taking the Y point and dividing by the line height. And that will give me the line. And as long as that is within the number of lines that I have, I can return that line number.

So that will tell VoiceOver, yes, I have a line. Now we have to return what kind of data is at that line. So I'm going to start with content for line number. So I have all my lines, and a line number comes in, and all I have to do is return.

Finally, I need to implement the frame for it. And this is the hard one. So why is it hard? It's because we need to return a frame that VoiceOver can display in screen coordinates. So VoiceOver doesn't know that your view starts at zero and goes to 500. It doesn't know how to turn that into something that can display screen wide. So here's what we do.

We make the frame for what this line should be. So for example, I know it starts at five at the X coordinate, and then five at the Y coordinate, and then I add the line number times the line height, and that gives me the right line. And then I use this size with font to figure out the size. That's on NSString. Easy enough. So I have this bounding box, and I now have this frame with inside of my view.

How do I convert it to the on-screen coordinates? And here's a little bit of code that I copy and paste. I take the window, convert the rec to the window, and then convert from the window to nil. And that gives me the on-screen coordinates. Easy enough once you have that to cut and paste. Let's take a look at how that works. Accessibility. This is an introduction page that should be displayed first. This book is about. This is an introduction page that should be displayed first. This book is about teaching students how to write a letter.

Great. So obviously, I can hear the content for a line. The bounds are correct, because I can touch around. And the page content is returned correctly, because it spoke the whole page. Now my only problem is, how do I turn the page? So with that, we need to go back and look at So, I'm going to start off by implementing Accessibility Scroll and returning the right trait. So, first thing, I'm going to return the trait causes page turn. That will tell VoiceOver when it gets to the end of content, turn the page.

How do I turn the page? I'm going to implement Accessibility Scroll. So for the direction next, all I need to do is turn the page in one direction, and then I'll post a page scroll notification. And this allows VoiceOver to know, yes, the page has turned. I can handle direction previous as well. I'll turn the page in the other direction, post the page scroll notification, tells VoiceOver, page is scrolled.

Okay, so let's run that. And then see if VoiceOver will scroll the pages and read automatically. - To write a letter. - All right, so if I scroll with three fingers left or right, it should turn the pages. - To write the letter, C, first start at the top.

Now you know how to make the letter C. There are also lowercase letters. Great. So I can turn the page with three fingers swiping, and notice it automatically reads as I turn the page. So that's the same behavior as iBooks. It will read the next page automatically. To write the letter. This is an introduction. To write a letter.

If I start reading. To write a letter. To write the letter. C. First start at the top, then make. It will continue turning that page automatically when I start reading all. So, Great. That is implementing the Accessibility Reading Content Protocol to turn long-form content into a great accessible experience.

So next up is interactivity. So as you know, VoiceOver intercepts all gestures on the screen. So it does this to provide a safe environment for exploration. But obviously, that's going to cause a problem when you have a very interactive element. You can't double tap and then double tap everywhere you want to go. So we've come up with a solution that's quite easy to implement, but can provide a very unique and dynamic experience for a VoiceOver user.

So here's what we have to do. If you have a view like this-- and I'll show you an example in a second-- all you need to do is return Accessibility trait allows direct interaction. And for this view, the safety net of VoiceOver will be lifted. A VoiceOver user can switch between exploring and interaction if they want, so they can still discover what's there, but they can also go right through that safety net to interact with an object.

So now we know how to write the letter C, in case you've been following along on this book that I wrote. The final page is... Accessibility Education Demo. Is this thing. Learning to write. Learning to write. And with VoiceOver on, obviously nothing happens. VoiceOver off. But with VoiceOver off, I can do things like-- and it will tell me whether that's wrong or right.

Great. So I can practice my C drawing. So how do we make that experience accessible for a voiceover user? Obviously, this is a great case for the direct interaction trait. So there is a quiz view in the Accessibility Education demo, and inside of here, we need to do a little bit of work.

to make sure that this is accessible. So we'll make sure this is an accessibility element so VoiceOver can see it. We will add an appropriate label. In this case, I'll call it the letter practice area. We can add an accessibility hint. Practice drawing the letter C in this area. And then we can finally add the allows direct touch interaction.

So with those things, now VoiceOver should be able to see it. VoiceOver on. Okay. Letter practice area. So now we have our letter practice area. Practice drawing the letter C in this area. We can practice the letter C in this area. Letter practice area. Practice drawing the letter C in this area.

So I'm practicing. It's working. I have VoiceOver on. It's saying the right thing. There's only one more piece to this puzzle that's missing. We need to provide direct feedback when

So whenever we determine whether it's been a success or not, we can post inside of one of these methods, and accessibility announcement notification with the right text that we want to speak. And now, let's go ahead and start.

Once we've added that, VoiceOver user will be able to directly interact with an object.

So that's all about our Accessibility API that we've added in 5.0. We think we're going to see a lot of great apps come out of it now. So there's one final topic I want to touch upon, and that is something that we call Accessibility Apps. So these are apps that are designed to address a specific need, and there's a growing market for these, and they really make the iOS platform a unique and indispensable device for people with a disability.

So I want to point out two exemplary apps that have just revolutionized a lot of people's lives. The first one is the Looktel Money Reader app. It's just a great app that you can point at any denomination bill, and it will tell you how much it is. So you point at a $20 bill, it tells you it's $20. It's pretty simple. For $1.99, it's revolutionized this field where you used to have to buy a device that cost hundreds of dollars, carry it around, put your money through that to figure out what you're looking at. done with an iPhone app now for less than $2.

The other app is ProLogo2Go, and that's been designed to help people with communication disabilities put together phrases and sentences that they need to speak and to communicate with other people. It's been a game changer for children and adults. It's a very powerful tool, and I want to bring them both up because I think they exhibit a lot of innovation and thinking outside the box for what is possible with an iOS device.

And I bring them up so that we can stimulate your minds to start thinking, how can I make an app that will solve someone's needs in a way that will transform their life? So if you're making one of these kinds of apps, and it's for a VoiceOver user, there's some useful API to know.

One you can know is VoiceOver running, where the UI Accessibility is VoiceOver running. You can know where VoiceOver is in your app by monitoring, did become focused? And then finally, as we saw, you can use the announcement notification to tell VoiceOver to speak a specific string at a specific time.

So a few best practices to go over when you're making your accessible app. One, try to be concise and short with your labels, because a voiceover user has to hear them an awful lot. Don't include the type of information in your label. That is communicated with the trait. No need to put the word "button" in any label that you have. And then finally, if you're going to localize your app, make sure to localize your labels and accessibility information, because VoiceOver speaks many languages.

So we've covered a lot of stuff today. We have a lot of great features in 5.0 that we're really excited about. So the takeaway, obviously, add accessibility to your apps. That's why you're all here. It's going to increase your user base. And if you're one of those kinds of people who can't go to sleep unless someone tells them what a great human being you are every day, you're going to get a lot of great feedback from users telling you that your app really works well with VoiceOver.

I also want to encourage you to think about apps for users with specific needs. I think there's a lot of innovation in this field that we're really excited to see what comes next, and it really just warms the cockles of my heart every time I see a new one of these apps comes out.

So the rest of WWC is all about accessibility. We have two more sessions, one in the afternoon where there may be an appearance by James Dempsey in the breakpoints. There's a web accessibility and automation session tomorrow morning that, while early, is not early enough to be an excuse not to attend.

Chris Fleizach And if you're looking for more information, there's a great voiceover manual that will describe all the possible gestures you can do with voiceover. And there's programming guidelines. There's just lots of stuff on the web about people talking about their experiences and so on, so obviously not that hard to find. Chris Fleizach So thank you for coming out today. It's been a pleasure, and hope to see you at the labs.