Good morning. My name is Susumu Harada. I'm an engineer on the Accessibility Team. And I'm here today with my colleague Patti. And we're really excited to introduce to you a new tool that will make it easier than ever for you to find, diagnose and fix accessibility issues in your apps. So, some of you may be new to accessibility. So let me quickly go over what it means to make your app accessible and some of the issues that you may encounter as you develop your apps.

So, accessibility is about making sure that everyone can enjoy the products and services and the great apps that you all create, regardless of their abilities. Now, some users may have difficulty with their vision or their hearing. And some may experience physical and motor challenges, or cognitive and learning difficulties. So, to support this wide diversity of users, we at Apple have worked hard to make a number of assistive technologies that help augment various users' abilities, such as VoiceOver for people with visual impairments, or Switch control for people with motor difficulties.

And we also made sure that these are available on all devices across all of our platforms, from macOS and iOS to watchOS and tvOS. And while these assistive technologies are great, enabling tools, the heart of our users' experience are the great apps that you create. So to better get a sense for how your apps interact with our assistive technologies and the kinds of issues that you may run into, let's take a look at a quick example.

So here I have a simple app that I created called Math Learner. And it just presents a user with an equation at the top to solve and four answer buttons on the bottom. And the user can tap on an answer button and you get a feedback whether or not the answer is correct.

It's pretty straightforward. And it looks like its functioning properly. But how might a user who is blind use this application? So, obviously without any assistive technology they won't be able to see the screen. So, it's pretty useless. But fortunately, they can use our screen reader called VoiceOver to have the information on the screen read out to them. And they can use various gestures to navigate around and interact with different controls on the screen.

So, if this app is fully accessible, when the user touches on the screen showing the equation, VoiceOver will read out: "5 plus 3 equals." Great. So what does the assistive technology like VoiceOver need from your app in order to present the interface in a meaningful way? So, there are several key pieces of information. But first it needs to know where all the important elements are on the screen. And for each of those elements it needs to know their traits or their types such as what is a button, heading or static text.

And finally, it also needs to know a descriptive label that uniquely identifies that element so it can be described to the user in an understandable way. So with these key pieces of information VoiceOver and other assistive technologies can present the interface to the user in a very usable manner. And all the accessibility information is provided by the app through our accessibility API.

And we at Apple have worked very hard to build accessibility support right into our UI frameworks so that if you build your apps using standard controls all of this accessibility information will be filled in for you automatically by the framer. And you don't have to do any extra work.

But of course, your apps will ultimately contain custom controls. And when they do, you may need to provide some of this accessibility information yourself. And if you don't, you might end up with an inaccessible app. So let's see what the impact of that might be like. So, if I had implemented that equation view at the top using complete custom drawing code like CA Layers or OpenGL, so they don't have any accessibility information by default.

So unless I explicitly provide them, they might be completely invisible to VoiceOver and other assistive technologies. And with these four buttons at the bottom, if I used custom views that act like buttons, but forgot to indicate their accessibility trait as being a button, then VoiceOver and their users will have no idea that they can press on these.

And finally, for the Help button here, if I implemented it using custom drawing code, and forgot to supply the accessibility to the label, VoiceOver will have no idea how to describe it to the user. And the user will have no idea what that button is for. So you can see how if you're missing some key accessibility information the user experience for someone relying on assistive technology could be pretty bad.

So obviously it's very important to make sure your app is fully accessible, to make sure that it has a positive user experience. But there's also another added benefit to making sure your app is accessible. And that is a fully accessible app -- with a fully accessible app you can write automated UI tests for it using Xcode's UI Testing Support. And for more information on how to do that, I encourage you to check out UI Testing With Xcode session from last year's WWDC.

Great. So, how can you make sure that your app is providing all the necessary accessibility information? Well, one method that we've been recommending is to use the actual assistive technologies like VoiceOver and Switch control with your apps. Now, this is still the ideal method, since you can mirror the actual user experience and also catch usability issues beyond whether or not that something is accessible.

But it does require that you be familiar with how to use these technologies. And especially if your app supports multiple platforms, you might find it quite challenging to have to repeatedly deploy your app on a specific device, run through it using each of the assistive technologies and repeat the process across each of our platforms.

And all those platforms also have a number accessibility settings that the user can configure such the Dynamic Type font size. And you as a developer might find it a little tedious to have to go and switch back and forth between changing the accessibility setting, going back to your app, verifying its effect on your app and repeating the process. So, we've heard all your feedback. And today we're very excited to announce a new, an easier, and more streamlined alternative to auditing your apps using the new Accessibility Inspector.

And in the new Accessibility Inspector, with just a single press of a button, it can automatically audit your app for common accessibility issues. It's super easy. And it even suggests to you how to fix those issues. And in the Interactive Inspection mode you can really easily, point at any element on your app and get very detailed accessibility information about them.

And this works whether your app is running on a simulator or on a connected device. And we also offer a live preview feature for settings, which lets you really easily go and change an accessibility setting and immediately see its effect on your app without having to leave your app.

And we're really excited to announce that this Accessibility Inspector now supports all of our platforms. This means that with a single Accessibility Inspector app on your desktop you can audit and inspect your apps running on macOS and iOS, watchOS and even tvOS. And you can do this simply by launching your app on the simulator or on a device and choosing the respective target from the inspector. It's that easy.

And we hope that with this feature it'll make it easier than ever for you to test an audit for your accessibly across all of our platforms. So, to give you a demo of all these exciting features and the new accessibly inspector, I'd like to hand it over to Patti.

Hi. I'm Patty Hoa. And I'm super excited to be the one to tell you all about our new Accessibility Inspector. So, we've been working really hard to incorporate all your feedback. And so I can't wait to show you what we have. So, to note, Accessibility Inspector is now part of the developer package. That means you can simply go into Xcode under the Open Developer Tools section in the minibar and launch the Accessibility Inspector.

So here is a screenshot of our new Accessibly Inspector. I want to bring your attention to the top section first. On the left we have what we call the Target Chooser. That allows you to inspect all the different Apple devices that we have, including your iPhone, your iPad, your desktop machine, your watch, your TV and the simulators.

In the middle we have what we call the Inspection Pointer. Now, when you have this enabled, then the tool can now inspect what is under your finger on your device. Or if you're in a simulator or the desktop, it is what is under your mouse cursor. Then on the right we have these three buttons. These are three major features of the tool. And this is what I'll be spending the rest of the session telling you more details and give you demos of these three features. So what are these three features? First, we have Inspection.

The Inspection feature allows you to debug and analyze the accessibility state of your app. Then we have the Audit feature to help you to find accessibility issues in your app. And last we have the Settings feature that helps you to quickly test how accessibly settings affect your app. So the first one I want to talk about is the Audit feature.

So, ultimately your goal is to make sure your app is accessible to everyone. And to do that you need to first audit your app. Now, earlier Susumu mentioned it would be awesome if you could actually run with VoiceOver or Switch control and audit your app that way. Or you could also hire an accessibility consultant to do the evaluation on your behalf.

But today, I'm going to show you a more cost effective way to get that solution. And that is simply use our Audit feature of the tool. And so, in our Audit tab we have this Run Audit button. And the best way to show you and tell you about this feature is to give you a demo.

So I have here on the screen a running version of the iOS version of the Math Learner app running in the simulator. And so you could play a game. We have equation. We could pick one of these choices. Not that one. This one. There. So that's how you play the game.

Okay. So now I want to launch the Accessibility Inspector. So, I simply go to Xcode. And on the top, under Open Developer Tool, we're going to launch the Accessibility Inspector. So the first thing you want to tell the tool is which device you want to inspect. So in our Target Chooser, I'm going to pick the Simulator.

And then we'll click on this Audit tab here. And here's that Run Audit button I told you about. So I'm going to go ahead and click that. In a split second we've just audited this app. And here are the issues it found. You can now select that. As you cycle through these issues, you'll note that there's a corresponding red highlight over the area where the problem resides.

And of course, if you're inspecting an actual device like iPhone or iPad, this is probably the view you have. So, now if you want to know where the issue is, you will have to take your eye focus away from the tool and look on your iPad or iPhone.

So what we have here is a little nice features, just give you a little preview, a screenshot, of that same element that has the issue right here in the tools. So you could keep your eye focused on the tool itself. All right. So we have these issues. So what's the next thing you'd probably want to do? Go fix it. So let's pick one of these issues here. Element has no description. And click on the disclosure triangle here to expand to get more information.

All right. So element has no description. What does that correspond to the code? Do you know which piece of code to write? Well, if you went online, read all documentation about the accessibility API that might give you some hint on how to fix this issue. But today I'm going to show you a faster way to get that answer. Simply click on this question mark here.

And the tool will tell you which API to fix up. It says: Consider setting the accessibility label. So, at this point, if you know how the API works, you would just go right into your code and change it. But if you want more information about how this API operates, you could click on this hyperlink, which then will take you to -- eventually take you to documentation like this where you could find more information about the API. All right? Since now I know which API to override, let's go right into the code and fix it up.

So, what I'm fixing is those button views with the dots in there. And the way it's implemented, it's simply a custom UI view. So now I'm simply going to override the accessibility label here. And I'm going to return a string that describes those dots. And of course, you should always return localized string.

Okay? So now I have my method written here. I'm going to save it, run and rerun that. Okay, let's go back to Addition. And let's clear out the old issues we have. And rerun the Audit. With that simple change in the code, we've just fixed up a bunch of issues. Isn't that great?

All right. So, note that this same powerful audit feature can be utilized by other teams of your member, like QAs who may want to do an assessment on your behalf. And of course, then they will want to report those bugs. So, to speed up the process in reporting those bugs, they can now just, after they run Audit, they could simply copy and paste it into the bug report. And of course, every good bug report should have a screenshot.

So we have here this Quick Look button here. If I click on that, you should get a nice big screenshot of your entire screen, plus where the problem resides in this blue outline. So again, you could just copy that and paste it into your bug report. And now you're ready to send off that bug.

All right. So what you've just seen is how the Audit feature can help you auto-detect problems in your app. It could give you suggested solution. And even speed up the process of reporting those bugs. As powerful and as great as this feature is in helping you detect issues in your app, similar to the compile warnings in Xcode, having no warning does not mean your app doesn't have any problem. So that's why it's important for you to do some of that manual auditing yourself. And to show you why this is important, let me give you an example.

So, suppose you have a button and its function is to send a mail. Of course, the button comes with a bunch of properties. One thing we tend to like to do when we want to create a new button is we simply take one of these existing button and we duplicate it. And of course, that means all the properties of that button gets transferred. And we go and tweak some of the properties like the actions and the icons.

That's fine. That's all good. Except if you forget to update some of the property, that could be a problem. Especially in this case, it's the accessibility label property. For assistive clients like VoiceOver, the accessibility label property is super important. Because that's the only piece of information it has about how this button functions.

So, if an assistive client like VoiceOver were to land on this button, and you haven't updated the accessibility label, it still says Send Mail. But they're going to press a button thinking that they're sending a mail. In fact, they've just deleted that mail. So that's not a great outcome. And so for that, we're here to help you go through that manual auditing using the tool. And that's why I want to talk about Inspection.

So let me give you some highlights of Inspection and show you how that helps you through the manual auditing process. So, suppose we have a button here that has eight dots in there. In the Inspection tab on the top section there's the Quick Look. And on the left side we have the place where we describe, or give you a summary of how this element's being described. So in this case the eight-dot button, we'll just say 8. And to the right there's the Navigation button that allows you to easily navigate to the neighboring UI elements from this element, from this button here.

Then below we have the basic section that contains properties that's used to describe this element. So for example, we have Label, which corresponds to Accessibility Label. And returns the value 8. And we have Trait. In this case, it's a button, so it returns Button. And as this button is interactable, you'll look into the Action section and see that there's Activate Action for this. And so you could simply click on the Perform button to verify that this action works. So that's a highlight of Inspection. Let me go ahead and give you a demo of that.

All right. So we're back here on our app. And now we're going to go into the Inspection tab. So remember earlier I mentioned there's this Inspection Pointer. Right now it's enabled, which means when I take my mouse over the items in the simulator, the tool will start inspecting that element under the cursor. And you can see that green highlight.

So on the simulator in desktop, if you simply do a single click on that item, it will turn off the Inspection Pointer, which then allows you to now move your mouse around without affecting what is being inspected. So right now we're inspecting the addition. And you see a description for that. Now, I could click on the Go To Next Element button. And you'll see the highlight goes down to these buttons here.

Click again and we go to the next one. And also you get a description. This is the four-dot. Note that we completely bypass something important: The equation. So simply doing this navigation you could kind of find issues along the way. In this case we might have a problem with this equation. Let's keep going. So, we're on the eight dots, nine dots. And down to this Help button here. Except the description doesn't look so great, does it? IMG 0151. Hmm. Doesn't sound right. So, again, that may be another potential area where you need to fix it up.

So, if you could simply use this Quick Look section, navigate through all the UIs in your app and verify that all the description sounds good, then you're on the right track. And of course, if you have any UI that is interactable, like these buttons, you will simply go to the Action Section and, in this case I'm going to click on the Perform button to activate that button to verify the action works.

All right. So what I've just shown you is how you could use the Inspection to help you along with your manual auditing of your app. Now the Inspection is also great to help you do some debugging and analysis of the accessibility state of your app. An example of that is understanding how the accessibility hierarchy works, which becomes quite important, especially on an OS X app where the corresponding view hierarchy tend to be more complex.

So to show you how this feature works, let me go ahead and launch the OS X version of the same app. All right? So I have here the OS X version of the same app. And let's do something a little bit more complex, multiplication. And so then when you play, 3 times 6 --

I want to bring your attention to these four issues here. It says: "Potentially inaccessible element." So the way these answer choices were implemented, they were simply custom NSview. That's drawing those text via CA text layers. Now, by default, CA layers have no accessibility support. Which means if you create them, clients like VoiceOver is not going to be able to see them. And so the tool is telling you that right now, that these are not accessible. So to understand how to fix this up, it helps to know how the accessibility hierarchy works. So let's go to the Inspection.

And again, I'm going to turn on Inspection Pointer. And so if I bring my mouse over and, again, it's going to inspect whatever's under my mouse. What I'm interested in to inspect right now is the entire window. And the reason is because I want to know who are the children of this window.

So, the way I could I look at that is I could look under the hierarchy section here. And then here are all the children of this window. So, I'm on the equation. You can see that red highlight. On the image. Some status text. And a bunch of stuff in the Tool Bar section. So right now, those four crucial answer choice buttons do not exist as children of the window.

And so we know we need to go and fix it up. But today I'm not going to show you how to fix it up. Instead, I'm going to refer you to our sample code accessibility UI example. It's something to download from developer.apple.com. This sample app contains a whole library with different UI scenarios and how you could add accessibly support for those. So I encourage you to do that. And once you fix it up, I expect that those four answer choice buttons should then appear as children of this window. So you could verify it in this hierarchy.

So, what I've shown you so far is how you can use the Inspection feature to help you do some of that manual auditing. And also how you could use Inspection feature to do some more debugging and analysis of the accessibility state of your app. The next thing I want to talk about is Settings.

So, on every OS platform we have many, many accessibility settings. That comes built right in. So for example, on the iPhone, in the Settings app under General Accessibility you're going to find many, many wonderful accessibility features built right in. And settings like Toggle. So let's say if you want to know how Dynamic Font affects your app.

You could go in, adjust the font size to something really large. Then go back to your app. And then visually verify how a large font looks in your app. Then you could go back to the Settings app. Readjust to something really small. Then see how a small font appears in your app.

Now, you could see that this could become quite tedious if you have to constantly switch back and forth between the Settings app and your app. And how do you know which accessibility settings are important to test? Well, our tool will answer that for you. So, here's our Settings tab. Our Settings tab now present you some of the most common accessibility settings that we want you to test. In addition, since it's built right into the tool, you no longer have to switch back and forth between the Settings app and your app.

And here's a demo to show you that can work. So in the Settings tab, now you can invert the colors. And immediately see the effect it has on your app. And we could do the same thing with the font size. Adjust it. And see it directly. So, what you've just seen is how cool the Settings feature is to make it easier for you. So, today I've shown you our three major features. The first is the Audit feature, that help you to autodetect accessibility issues in your app, give you suggestions on how to fix it. And make it faster for you to report those issues.

Then we saw how great the Inspection feature is to guide you through that process of manual auditing and allow you to do debugging analysis of the accessibility state of your app. And last, you just saw the Settings feature that will make it much easier for you to test how accessibility settings changes affect your app. And that is the quick look of Accessibility Inspector.

Great. Thank you Patti. So we really look forward to having all of you try the new Accessibility Inspector. And we hope that you now feel motivated to keep users of all abilities in mind when designing your next great app. And to help you do that, we introduced you today the new Accessibility Inspector that we hope will make it easier than ever for you to find, diagnose and fix accessibility issues in your app.

And it integrates seamlessly with your development workflow so that you can easily inspect and audit your apps, whether it's running on a simulator or on a device. And we're really excited to support all platforms from macOS, iOS, to watchOS and tvOS. So we hope that with this great new tool you will go out and create wonderful apps that can be enjoyed by all users of all abilities.

So you can find more information about this talk along with the video and the transcript at this URL. And we'd love to hear your feedback. So we'd love to hear from you about how we can better support your workflow. And any issues you might encounter while using this Inspector to audit your apps. So you can send us enhancements and bug requests -- or bug reports, sorry, not requests, bug reports at bugreport.apple.com. And also participate in our developer forum at forums.developer.apple.com.

And in case you missed it yesterday, we highly encourage you to go back and watch the videos of these great sessions, including the "What's New in Accessibility" talk and the "Inclusive App Design" talk. And this evening we have the Accessible Technology and Inclusive Design get-together in Buena Vista Park. Not the real park, but the one on the first floor. So we'd love to see you there. So thank you very much. And enjoy the rest of the conference.