Good morning, everyone. My name is James Craig. I'm a member of the Apple accessibility team. Did you guys enjoy Spearhead last night? Anybody go to the party? Woo! Thanks for showing up this morning. I have been a member of, or I've been working in the accessibility field for over 10 years now, and I'm really excited that I get to share some of this stuff with you today.

So just by a show of hands, who here thinks they know what I mean by the term accessibility? Most of you? Okay, cool. Good. We're going to be covering a lot of things today. The first thing is that I bet a lot of you people are here because you've heard that you need to make your applications compliant, web applications or iOS applications, and you're not even really sure what that means. And so we're going to cover how to test your applications using some of the built-in assistive technology. But compliance is not really what I mean by accessibility.

Some others of you are probably here because you're developers, of course, it's a developer conference, and your mind automatically goes to the code when you hear accessibility. You think, "Oh, are my elements labeled? Have I overridden the right methods?" And that's great. We're going to cover some techniques using... HTML, CSS, JavaScript, ARIA to make your applications more accessible.

But that's only part of what accessibility is about. And some other of you here probably heard that automation plays a part, or accessibility plays a part in automation, and so you want to be able to automate your apps. And that's great. That's definitely not what accessibility is about, though.

So the one thing that I want you to remember throughout the whole presentation, even though we're going to be talking about code, even though we're going to be talking about tools, is that accessibility is not just about the code. Accessibility is first and foremost about people, real people with real needs, and these are users of your applications.

So some of them may be using a computing device for the first time in their life. Maybe it's your father, maybe it's your grandfather, mother, and maybe they can't see the screen so well. Some people just want to use the same games and same learning devices that their counterparts in school are using.

Some of them may be attending college and need to watch a lecture, even if that means they need to watch the lecture with the captions turned on. And still others just want to use their devices for the same thing all of the rest of us use our devices for, even if they don't have all of the same abilities that we all do.

And we're not talking about small numbers either. Just in the United States alone, over 50 million Americans have a disability. A lot of you probably think you don't know anyone with a disability, but one of the most common ones is colorblindness. It affects 1 in 12 males. So if you look around, that means there's probably quite a few people in this room that are affected by colorblindness.

Out of that 50 million, 10 million have a severe visual impairment. I'm not talking about people that wear glasses. I mean people who are usually blind and legally blind. And out of that group of people, the ones that need to use screen readers that speak what's on the screen, The money of those are very tech savvy, and they're not just tech savvy with their desktop computers, they're tech savvy with their mobile devices as well. This is a recent screen reader survey from WebAIM, blind users of mobile devices. A full two-thirds of them use a screen reader on their mobile devices, so this is all smartphones, and iOS is one of the more popular screen readers.

And so that's why I'd like to talk to you about accessibility on iOS today. VoiceOver was announced two years ago with iOS 3. We've got some other features as well, but this one in particular has been lauded by the disability groups. We've got quotes from advocacy groups like the RNIB that says Apple has set the standard in recent years. We also were presented with this award recently by the NFB, which is a disability group from the United States, Jacob Bolitan Award.

And while the press is great, that's not the reason that blind users and visually impaired users use iOS. VoiceOver allows blind users to use iOS. Blind users can use iOS because of VoiceOver, but that's not the reason they use iOS. The reason they use iOS is for the same reason that all the rest of us do. It's because of all the things that you can do with it. Make phone calls, watch videos, listen to audio, check your email, tweet, etc. Use all of these fantastic apps, including your app.

And if you haven't gone, you know, a couple extra steps, and sometimes these users can't use your app even though they want to, especially if you have a popular app. They've heard about it, they know what it does, they want to be able to use it just like everyone else.

So the first step to figuring out how your application works with VoiceOver, with the accessibility settings, is to figure out what those accessibility settings are, how to use them. Then we can walk through and test your application. So just to talk about some of the new features in iOS 5, we're really excited about this release. It's probably the biggest release for Apple accessibility since the original accessibility release in iOS 3.0.

We've got VoiceOver, which is a screen reader for the visually impaired. I'm going to demo that one in a minute, but I'll go through the other ones. Full screen zoom allows users to use three finger gestures to zoom in on any app. You don't have to do anything special in your app for this one. Large text allows users to bump up the font size in certain system apps like Calendar and Mail.

White on black reverses the video. Sometimes this helps even with some forms of color blindness. If there's something you're not sure about, you can flip the video. It changes it to a different color spectrum. Speak Selection is new in iOS 5. For those of you who don't need a full-fledged screen reader, you can turn this on, and anywhere you can select text where you get the cut, copy, paste menu, you also get another option that says Speak and Pause if it's already speaking.

Speak/AutoText allows you to hear auto corrections and capitalizations. You can see that the first full screen of these settings is for visually impaired users. We've got a few more sections. Hearing impaired users now can have custom vibrations set to a particular contact in their address book or to all contacts. They can also tap out their own patterns. You may have read about this particular setting on the rumor vlogs this week.

Also new in iOS 5 is LED flash for alerts. So anytime the phone is locked, any of the new phones that have the LED flash on the back, it'll kick off for anything that triggers an alert. Mono audio or custom balance for users that have hearing impairment in maybe one ear but not the other.

We've also got another section for users with motor disabilities. So for example, if you don't have full dexterity and you have trouble using a touchscreen, you can use this thing called Assistive Touch. Did anybody see the demo yesterday in the other accessibility presentation? So you can pair it with a head switch or a joystick and control the device even though you can't actually touch the touchscreen. And for those users, we can also automatically route the incoming calls to your preferred choice of output. So I'd like to demo. So I'm going to go to the settings.

Accessibility, oops, excuse me. So we can see we've got voice over here. I could turn it on from this section. For those of you who are just learning VoiceOver, I'd recommend something else, though. I'm going to go back up to the accessibility settings, and go down here to triple-click home. Can you see the bottom of the screen? It allows you to toggle voiceover on and off using three clicks of the home button.

So I'm going to hit the home button once to go back to the main screen. Now I'll hit the home button three times in a row to turn on voiceover. So one of the ways that, or the way that a blind user is going to use this device is they need to be able to touch the screen and be able to explore what's there without worrying about accidentally activating something. So if you were to touch one of these icons and VoiceOver wasn't on, it would just automatically activate. So now I can touch anywhere on the screen, it's not going to activate.

Settings Newsstand App Store Video Notes Double tap to open. And you can hear what's under my finger as well as a hint on how to activate it. I can also use some gestures, so I'm going to use a single finger swipe to the left and right. Reminders Maps YouTube And back to the previous items. Maps Reminders Notes Double tap to open. There's also a few custom gestures, so I can use three fingers to scroll.

Search iPad. Search field. Is editing. Home. Page 1 of 1. Messages. There's also a special gesture that I'd like to show you that is called the screen curtain, and it uses three fingers with a triple tap. Screen curtain on. So now the screen curtain, or the screen is blacked out completely, but I can still use it. Settings. Photo booth. Settings. Double tap to open. I'll turn it back off.

Screen curtain off. That's a nice feature for blind users that don't actually need to see the screen at all. It keeps people from looking over their shoulder and eavesdropping on what they're doing. Once I have something selected, like this settings application, I can double tap to open it. And so that's the basics of how you use VoiceOver on iOS.

So let's talk about accessibility. Those of you who came to the iOS accessibility presentation yesterday found out how to make your apps accessible in native views. We also know a lot of you are coming to iOS from web development. You know the web technologies a lot better. Some of you are using single frame UI web views to essentially have a web app running in a native wrapper. And so I want to talk to you about how to make those views accessible.

So the way your application works on iOS is that your application will import UIKit, and the user is usually interacting with the Core OS, and through UIKit can command and control your application. Now if VoiceOver is running, there's a couple extra steps. So VoiceOver changes the way the gestures work. So any time VoiceOver is running, the user is always interacting directly with VoiceOver. Excuse me, WebKit also comes into play if you're using WebView.

But anytime VoiceOver is running, the user is interacting directly with VoiceOver. And VoiceOver is essentially driving the system through the UI Accessibility framework, which has hooks into UIKit and your application and WebKit, and can drive the interface. And for those of you that know about ARIA and HTML and how it overrides some of the semantics, WebKit uses that to change the way those accessibility semantics are exposed to the UI Accessibility layer.

The techniques I'm going to be talking about for web accessibility apply across the board. These aren't just specific to iOS, though some of the features are new in iOS. So they will work, most of these will work in sites displayed in Safari and any other accessible web browser, sites displayed in a web view within a native application. Several feed readers utilize this approach. So for example, you don't have to click away to Safari if you want to view a link in your RSS reader. You can just read it right there in the application.

This also applies to web views displaying local content. So even if it's not technically a website that's out on the web, if you're using the same technologies, HTML, DOM, JavaScript, CSS, to display this web content, the same techniques apply. And that's what we're going to be using in the demo here in a minute.

First of all, Accessibility 101. I think I have this slide in every presentation I've ever done about accessibility. Label all your elements, form elements, etc. The way you would do that in HTML here is if you have an image, it's not labeled, the screen reader has to guess at what this one means.

So it's going to use some internal heuristics, and this one it would probably guess that this is an image that is BG 2376. Not very useful, it's kind of a gibberish file name. And so anytime you have an image you want to give it an alternative text label, a human readable alternative text label.

Same concept applies to form elements. Even if you have the label right next to the form element, sometimes the screen readers are going to get a better heuristic value for it. But it's always best to explicitly label that image. In this case we have a label element with a for attribute that corresponds to the ID attribute of the form input.

There's also a bunch of new support for the HTML5 inputs in iOS 5. So for example, date time. If you just say type equals date, of course you need to label this one as well, but if you say type equals date, you get this native picker. You can also use sliders with using input type equals range. And the required attribute is newly supported in iOS 5, and that allows you to specify that these fields are required, in addition to the visual display, these fields are required so that the screen reader can know about that required status as well.

Who here by a show of hands has either used a library that does this or has overridden on touch start, usually for custom scrolling in a web application? Not too many? Okay, that's probably good. You want to avoid using custom controls and gesture overrides if possible. The two main reasons why people were doing this on the web is because single finger scrolling was difficult within an overflow auto, so now you can use a single finger within a div to scroll in a web view. Also, fixed positioning works the way you would expect it to. Technically it worked before, but a lot of people didn't understand how the viewport worked. So both of these things should be a major reason why you don't need to use these gesture overrides in iOS 5 anymore.

You can also use CSS3 to change the screen reader verbosity. The speech module has support for speak property. So, for example, if I have an address here, The way most people would say this is "12345 fake street." The way most screen readers would say this is "12345 fake street." And so we can adjust that verbosity to make it sound more natural in human language. Also, if we have a code sample, we can say that it needs to speak all of the punctuation.

And as a point of note, like all web standards that aren't fully ratified yet, CSS3 is a draft and it's subject to change. Some of this has changed since we started planning this presentation. I'm going to switch to the demo machine. What I've got here is an Xcode project with a nib that contains a single UIWebView. All this WebView does is load up an HTML file, I'll show you the implementation.

So right here it loads up a path of WWDC.html in the web content directory. That file is right over here. So I'll click on it. I'll go ahead and show you the app in the simulator. So this is just HTML markup. You can see that we've got our range slider that's showing up in the view. We've got some tabs at the bottom.

We've got the text that I showed you in the slides. If I click on any of these tabs, it changes. If I click on the test sheet button, A little kind of an action sheet comes up, and we can cancel that or do something with it. So let's hear how this sounds in voiceover on a device.

Great, so I'm going to hit the home button three times to turn on voiceover. VoiceOver on. And I'm going to use a gesture to read all the contents of the page. You're going to hear several accessibility errors in this one. N, gibberish 06582X, PNG, image, denotes a required field. Awesome sauce level, 7, adjustable, phrase, text field, 12,345 fake street. This one equals this other one, test sheet. So I'm just going to stop it right there.

So we heard a bunch of different arrows, starting off with this image. Denotes a required field. Gibberish 06582X. PNG. Image. So that asterisk image is not labeled. It's reading the file name, which isn't very helpful to us. Also on the slider. Seven. Adjustable. It's not associated with its label, so you can't hear that this is the Awesome Sauce slider. Also in the phrase text field. Phrase. Text field. Double tap to edit. We've got a visual indication that this field is required, but you can't hear the difference.

12,345 fake streak. Obviously, this isn't quite how we want it, but it's not a bad rendering. And listen to this sample code. This one equals this other one. It just says this one equals this other one, but those of you who understand programming mean that that's actually the opposite of what this is intended to say. So we want to read more of the punctuation in this.

So here's our unlabeled image at the top. I'm just gonna drag in some alt text. Now it says asterisk for the range label. I'm gonna drag in... I'm going to drag in a label attribute, or a label element that has a for attribute that matches the ID of the element.

Another way that we could label this, this is just native HTML, another way that we could label this is using the aria-label property. And drag that in. This is technically redundant, but this is another way to label something, so you can either have a text string on the label, you could also use the title attribute in this case. If we scroll down to the required text field here, we can see that it's got a class on it.

So I'm going to go ahead and add the required attribute, native HTML5. And here's where we have our verbosity changes. We want to read this as digits when we want to read all the punctuation in the code sample. So I'm going to switch over to the CSS file. Just drop this in anywhere.

Here we've got our address. Anything with an address class is set to speak all of its digits. Anything that's a code block can speak all of the literal punctuation. And I'll go ahead and save this, and... Messages. Double tap to open. Rebuild it. So now that's a labeled image, we hear asterisk. When we go to the slider, we hear that it's associated with this label. So that's much more awesome, I'm going to crank it all the way up.

This one goes up to 11. Text field required. We hear the phrase text field required when we touch this one. And now let's check out the verbosity settings on 12345 Fake Street. 12345 Fake Street. And also the code sample. This one space exclamation mark equals space this other one semicolon. So this is, that's kind of verbose, but that's exactly what you want for source code examples. So before we get into the rest of the slides, I want to show you some additional problems with this.

I'm going to just scoot it up here so you can see. I've got some tabs at the bottom, and you can see earlier in the simulator that when we click on these, it goes between the different tab panels. So I'm going to touch this with voiceover. You guys can't really hear that, but there's a little bonk sound that means that there's nothing selectable there.

So this isn't accessible at all to voiceover. You can click the test sheet. Test sheet button. And I'm going to turn on the screen curtain again. Screen curtain on. And when I activate this test sheet button, I want you to hear, tell me what you think happens based on what you hear. Test sheet.

So all we heard again is a repeat of the test sheet button. So I'm going to turn off the screen curtain. Screen curtain off. Oh, okay. So we can see that this action sheet came up. We probably should have moved the focus to that action sheet. Another thing that's bad about what we've got going on here is that I've got a mask over the rest of the content behind it. We don't want it to be interactable at this moment. We want it to be modal.

But a voiceover user... Phrase, phrase, awesome sauce level. Ask, denotes a required field. ...can still touch all that content that's behind it. So we want to hide the content that's in the main body. And we also want to focus the sheet. So I'm going to go back to slides.

The way we're going to do some of this, fix the tab controller, for example, is by using ARIA. One of the things that ARIA is great at is retrofitting old content without gutting it. You guys familiar with ARIA at all? Little bit stands for Accessible Rich Internet Applications.

So a lot of you probably have inherited some old code on old websites, and you can do things like if you have a link that's acting as a button, it visually looks like a button, but it's actually just using a link tag, you can retrofit the role, so in this case we could just say this is a role of button.

You can also use this to redefine roles where necessary, or define new roles. So, for example, in HTML, HTML5, there's no such thing as a dialogue. There's no such thing as a few different types of containers that exist in the ARIA roles. So you can say, "This particular div represents an application dialogue." There used to be an old HTML5 element called dialogue. It was about dialogue between people, not the same thing. You don't have to set this directly in the markup. you can set it in the DOM via the setAttribute method.

You can also override states and properties. So not just in addition to role, but in this case we've got an ARIA checked property. The reason I'm using this example is because even though there's already a checkbox in HTML5, it only represents binary either checked or unchecked states. There's no mixed state.

So in this case we've got a custom widget that represents the check state of several other checkboxes, some of which are checked, some of which are unchecked. So this one is mixed. Again, you don't have to set this in the markup, you can assign it directly in the JavaScript.

And a lot of people ask me, how do I know when I need to update my application state? When should I set these properties? Usually a good indicator is if you're changing a style, if you're changing a class name, especially if it's something like selected or active or highlight, go through your application and look for those kinds of class names and those kinds of styles, and that's usually where you need to be changing an ARIA property as well. So in this case, depending on the context, we would change this to "ARIA_SELECTED = TRUE".

I also want to talk to you about focus management. Remember when we tested the sheet, the voiceover cursor didn't move into the active action sheet? Did anybody attend my presentation two years ago at WWDC? We talked about... the WWDR rep attended, no one else. So we talked about focus management on the desktop. And on the desktop you have this concept of keyboard focus.

And usually when you set focus, the voiceover cursor follows along. On iOS, there's no real concept of keyboard focus, or there is in a couple places, but I want to make sure that you understand that this is not the same as desktop focus, not the same as keyboard focus. So when we're talking about focus management on this one, I'm talking about moving the voiceover cursor around only.

That said, the way you set focus is exactly the same way you set it in a desktop browser. You get a reference to a button, in this case the first button tag in an element, or in a document. And you just call the focus method on it. Very familiar to you web developers out there.

A note for those of you sharing code between iOS applications and desktop applications: use tabindex if you need. The case here where you might need it is if you were to define a custom checkbox that didn't use an input tag. Div is not natively focusable. So once you get a reference to that div, you've got to make sure that it's focusable by setting the tabindex property on it, or otherwise when you call focus on it, you'll get an error in a desktop browser. Not applicable to iOS, but a point of note for those of you sharing code between the desktop and iOS.

So I'm going to go back to the demo machine. Go back to the HTML file. We saw that these tabs down here at the bottom were not accessible on VoiceOver. So let's look for those in the document. Here we go. It's an unordered list with list items inside of it.

And there's no text attribute, there's no text equivalent, there's no role, there's nothing but an empty list here. So as far as VoiceOver knows, as far as the accessibility hierarchy knows, this is an empty list, so there shouldn't be anything that's there. So what I'm going to do is I'm going to localize those tabs to begin with.

So I'm going to drag in a few labels here. We've got foo, bar, baz, and bop tabs. Also, when I initialize the tabs, I'm going to set the tab list role on the container. So we've got a reference to the UL here. I'm going to set its role to tab list. And for each one of the tabs inside of that, I'm also going to set the -- oops, that was the wrong one.

I'm going to set the role to tab. The other thing that we want to do is we want to check to see which one is selected. So anytime we change which tab is selected, it updates its style. So we've got our select tab method here. Move down, okay, so here's my function.

Sure enough, I'm setting a class name. On this deselect, I'm setting it to an empty string. So that's where I want to set the selected property defaults. And then when I select the newly selected tab, We set the ARIA selected property to true. So let's hear how that sounds. - Messages. Double tap to open.

So I'm going to touch the tabs at the bottom here. Great. Let's hear the selected one. We want to make sure that updates at the same time that we select it. So the first one is no longer selected, the second one? The second one is selected. Okay, so we got a pretty good tab controller here. Now we're going to fix this sheet.

So I'm going to go back to the HTML again. So this sheet is another div. Inside my application, we've got this mask element to gray out the background content. We've also got a wrapper element around the entire rest of the contents of the page. So when the sheet is shown, we want to hide the rest of the contents, and vice versa. When the sheet is hidden, we want to show the rest of the contents.

So in the JavaScript, I have a show sheet function. I also have a hide sheet function. Now we could set focus immediately in this function, but if you notice, there's an animation that occurs when the sheet is shown. It kind of slides up using just a regular CSS transition.

It takes two-tenths of a second. Sometimes if we set the focus to something that's off screen, you're going to get unpredictable behavior. So what we're going to do is we're actually going to set a timeout. You could also use a callback from the animation, but for the sake of simplicity, I'm going to set a timeout here. Call it 200 milliseconds later.

And these are just going to call some functions that don't exist yet. So I'll show you what each of these do. When we focus the sheet, the first thing that we're going to do is we're going to set the ARIA hidden attribute defaults. Excuse me, I forgot one thing. I'm going to go back to the HTML file. On this sheet, because it's hidden by default, I want to explicitly set the hidden property. You could do this in an initialization function, but I'm going to do it right here to make it explicit.

So it's hidden by default when the application launches. Now if I go back to the JavaScript, You can see that we're changing that value to false, so the sheet is now visible to voiceover. It's in the accessibility tree. We're also going to grab the first button within that sheet and focus it.

And then we're going to hide the contents of the rest of the page while this is shown, and vice versa when we focus the trigger that is the button that showed the sheet to begin with. We're going to show the rest of the contents of the page, change the focus back to the button inside those contents, and then hide the sheet again.

So now when we focus the sheet button and activate it, it moves directly to the sheet as we wanted to. Also, if we touch back in the contents of the page, We don't get anything, we don't hear anything back there. Heading level one. So I'm going to go ahead and cancel this. Cancel. Test sheet. Button. Turn off voiceover.

App Switcher Settings. So you can see that with just a few lines of code, literally that was, even including the functions, that was probably 10, 15 lines of code. We've updated our application from a completely inaccessible application to one that's very accessible to a voiceover user. And we're going to find that all of that work that we did, including making those tabs accessible to VoiceOver, is going to allow us to automate the application in a way that benefits you as a developer, allows you to find other bugs that you wouldn't otherwise get.

So, there's a lot of reasons to create automated tests. The first reason is so you can have an easily repeatable regression test. All of you have found bugs in your programs before. It's always a frustrating point when those bugs come back in a later version. So you want to make sure, if you've experienced a bug and you fixed it, you want to make sure that never comes back, so you can write a test that always fails when that bug occurs, and always passes when that bug doesn't occur.

You can also stress your app with longevity tests. Just by a show of hands, who here has tested their app for two weeks straight without sleeping? Anyone? A couple people over here. So the automation team has done it. It's one of the great things that machines are good at. They don't need sleep.

You can also have faster testing for your iOS application updates. As soon as you find a bug, you want to fix it and get an update out to your users, but you also want to make sure you don't break anything new. So all these together give you a higher confidence in your product quality. And as a bonus, we get to find a few accessibility errors as well. Some of you may have noticed that I left at least one bug in that application.

So UIAutomation on iOS is integrated into the Instruments app. It allows automation of any iOS app, though you're going to get the most out of it if you use an accessible app. Typically those that are UIKit-based get a lot of this for free. It leverages the accessibility layer, UI accessibility, that we talked about earlier. And the tests are written in JavaScript, so for all you web developers out there, the syntax is going to look very familiar.

So here's our voiceover and accessibility architecture. The user is down at the bottom of the list, interacting directly with voiceover. Voiceover then drives the application through the UI accessibility layer. So if we take out voiceover, we take out the user out of this scenario, we can put in UIAutomation and these UI tests. So the UI tests are essentially mimicking a user, or simulating a user with a script, and that's interpreted by the UIAutomation framework, which then drives your application through the accessibility layer.

So the way it works, just using JavaScript syntax, you get a reference to the device. Local target is essentially the root element in UIAutomation. This is a reference to your app process. And then we get to some UI views. So app.mainwindow will get a reference to basically everything that's on the screen here.

In the case of our application, we have a web view within a scroll view, so we get the first one of those in each, and it's everything essentially except for the title bar. And within that web view, we can say, "Give me the first text field," and it'll get a reference to that one.

Or select the button that matches the test sheet value, or test sheet name. You can also use it to inspect the interface. So give me the label of the first image. In this case, it's that little asterisk image, and we just added an alt attribute, so it says it returns a string value of asterisk.

You can check whether a button is enabled or disabled, whether a control is enabled or disabled. And you can also check other status of items within the DOM. So in this case, we had that cancel button that was in the sheet. We called checkIsValid, but we explicitly hid it when it was off screen, so in this case it's going to return false. If you called this same method when the sheet was displaying on screen, it would return true.

You can trigger user events. So once you have a reference to an element, you can tap it. You can also do more explicit events or more specific events like touch and hold for two seconds. In this case, it would bring up the paste menu on that phrase field. And you can simulate some other events too, such as shaking the device or changing the device orientation.

Now as you're writing a test, the first thing you want to do is use UIAlogger to start the test. We're going to get a reference to a test name that we can use a couple times when we complete the test as well. We call log start, and if you run multiple tests, it's going to encapsulate all of the output of one test into a container.

At some point, you're going to want to output some information to the console or back to instruments. So you can call logDebug, logMessage, logWarning, and logError. logError will cause an explicit failure. This one is not a UIA logger method, but it can be called on any UIA element. So in this case, I'm just calling it on the entire device.

LogElementTree will output a hierarchical list of everything that's in the view at the moment. You can also capture screenshots or portions of screenshots. And then at some point, you're going to want to, you're going to catch some explicit failure case, so you will call log fail or otherwise log pass. And I'll demo how this works.

And we can choose our automation instrument from this list. Collapse that a little bit. Now up in the toolbar, I can choose my target. In this case, I'm going to select the iPad and the application that we're running here. And also in the left-hand column, we can choose a script. I'm going to add sample.js. Just to show you what it looks like, I'm going to pull down this trace log and move it to script.

And here we've got some JavaScript. So I've got a few references to the app window and web view. A couple of hard-coded strings here. This would be better to pull from a localized strings file. This one I didn't talk about in the slides, but this is essentially a way to, if you get an alert during your automation test, you don't want it to cause a failure, so this can have the framework dismiss that alert for you.

I've got a helper function that just resets the UI by clicking on the first tab. And here's where I get to my tests. So the first one is test labels. What it does is it goes through all of each one of the tabs, checks to see if the tab is valid, then taps the tab, takes a screenshot.

Then once it's tapped that tab, it loops through all of the elements in the view, makes sure they have a label. If we have something that doesn't have a label, it's going to capture a portion of the screenshot that just has where that element is on screen, and then log an error, which is going to fail us. We're going to then cause a pass or a failure.

The second test, Test Sheet, is just going to loop through this for loop twice. It's going to tap the Test Sheet button, And then tap the cancel button and take screenshots after that. And here you can see we're recalling these tests after a short delay. So I'm going to go ahead and switch back over to the iPad view. So you can see this running on the device. I'm going to click record.

Test has started. It's looping through the first page. Taps on the second tab. The fourth tab resets the UI, and it's going to show the sheet and hide it twice. Cancel button. And our test is done. So this is what it looks like on the desktop. You can see that we've got One pass and one failure. Obviously the failure is more interesting, but I'll just show you what the pass looks like. So this is the test sheet test. Takes a screenshot before it tests the sheet, then while the sheet is up, and then wants to make sure that the sheet is gone again.

And that just looped through twice, but that's less interesting. What we want to find out is why did this other one fail? So you can see through here we've got, we made sure the first tab is visible, we tapped it, we clicked tab foo, and then we hit this error.

Right before we hit the error, we got a missing label log, which I put in there explicitly, and we can see that we've got a portion of a screenshot. It's just this text field. So, how did this happen, right? Didn't we hear it, voiceover said phrase text field when we were doing it before? So I'm going to switch back to Xcode, figure out where that error is, go to the markup.

Oh, and sure enough, right here, this input is not associated with its label. So how did VoiceOver get it before? What you're hearing there is some of the built-in heuristics in the screen reader. So it said there's no label associated with this, I'm going to look around and find something that's nearby. Okay, I see this phrase text that's nearby it. That's probably what the user meant as a label.

Our job is to give the user the best experience possible, whether or not the code of the application is in the best shape. So in some cases, we can guess what the label is. It's always better to explicitly label those, though, so we're going to do that now.

I'm actually going to use something different than a label element. So here we've got phrase, it's just a span with an ID. Another way we can label something is using the aria-labelledby property. And this essentially is an ID reference to its label. It's the reverse relationship of the HTML label tag, so which is an ID reference from the label to its element.

So these are just three different ways to label things. All of them work well with VoiceOver. I'm going to save this out. We'll rebuild it again, but first, that's going to make our first test pass. I want to show you how you can catch a logic error as well. So I'm going to go to my JavaScript file, find out where we showed the sheet, and just throw an early return. Just throw an early return. We're going to rebuild this on the device.

And now I'm going to run this test again, and this time I'm going to leave it on the desktop so you can see what it looks like as the test is running from instruments. So our test is starting. Here it takes the first screenshot of the view, taps on the second tab, okay, we made it through that first error, get the screenshots for each one of these, and we've got a pass for the first test.

Second test is about to dismiss the sheet, it's got a screenshot, wait, the sheet's not showing, it's waiting around, and then okay, so we try to tap on an element that wasn't there. So you can see down here we've got an error. We want to correct any script errors, so in this case we actually called a method on an object that didn't exist. So this is a script failure.

So I'm going to go ahead and stop the test, and then just double click down here on the error message. That's going to take us to the script file. We can see somewhere down here, sure enough, we tried to call web.buttons, grab the cancel button label, and tap it.

That cancel button was not showing at the time, it was not even valid, it was in the accessibility framework, so it doesn't exist. A better way to do this is to use the method that I showed you earlier in the slides, which is to check whether or not this is valid, and then tap it. If it's valid, tap it. If it's not valid, and you expect it to be valid, log a failure. And so that's a demo of how to use UIAutomation with instruments on iOS. Thanks.

If you have more questions, Vicky is the Internet and Web Technologies Evangelist, and you guys, oh, yes, let's hear it. Jury is the Developer Tools Evangelist. If you want to find out more about Apple Accessibility, you can go online to Apple.com/accessibility. And if you want to find out more about UIAutomation on iOS, search developer.apple.com. But before you leave the room, guys, I want to give you a few takeaways. We covered a lot of things here. We covered how to make your application more accessible, how to use these devices with assistive technology.

We covered how to automate your apps. But if there's just a couple things I want you to take away, it's that, first of all, fixing accessibility bugs are usually really, really easy. I think in the entire presentation, including the automation scripts that we added, it was probably 30 lines of code total in this existing application that already worked for sighted users.

The other thing is that you should be testing already. We all hate using buggy applications, so one way to catch more bugs is to automate that testing. And accessibility makes it easier. If you make your application accessible, then it's essentially scriptable. But first and foremost, accessibility is not about code, it's not about compliance, it's about the people.

So if you think about it from the terms of code, then you're going to forget something. You're not going to use the device and figure out that you've got essentially a usability error with the screen reader. If you think about compliance, maybe you'll get past it, but you're going to be doing the bare minimum possible, and that's not the right way to use accessibility. Take pride in your work, and realize that all users need to use these devices, all users want to use your applications, and make them as accessible as possible.