Welcome to Thursday afternoon at WWDC. My name's Vicki Murley. I'm the Safari Technologies Evangelist at Apple, and this is session 604, Advanced Effects with HTML5 Media Technologies. So if you're here for a session on advanced media topics, you're probably already familiar with the audio and video elements. These have been around for quite a while now.

They were actually part of the very first public draft of the HTML5 specification. They were two elements that were, at the time, radically different than the headers and paragraphs that people were accustomed to seeing in HTML. This early definition of how these elements should behave had kind of a domino effect.

The first thing that happened was that browser implementers, like the people at Apple who work on Safari, were able to add support for these elements to their web browsers. So Safari was the first browser to support the audio and video element way back in 2007, five years ago.

The next kind of line in the domino effect was that since so many browsers were adopting these elements, websites were able to adopt them without a whole lot of compatibility worries. So if you've watched video on the web in the last couple years, you've probably been using an HTML5 media player. Vimeo is a great site that has an HTML5 player.

On Apple.com, we actually show all of our videos. That looks like a built-in system controller, but it's actually built purely with HTML5. And, of course, these elements have been essential because they let us watch video and listen to audio in web pages on iOS, which is a platform that doesn't support internet plugins.

So today I'm here to talk to you about some new technologies for HTML5 media, new in Safari 6 and new on iOS 6. If you've been using the HTML5 audio and video elements, there may have been times where you were trying to add a specific feature or achieve a certain effect and you have been frustrated.

So today I'm going to talk to you about technologies that solve some of these common problems and hopefully also inspire you to come up with some great new interfaces on your websites. So I'm going to teach you how to do three things today as far as media integration with HTML5. First, I'm going to show you how to synchronize multiple media elements with some new API that's available in Safari 6.

Next, I'm going to show you how to implement full screen interfaces. If you've ever wanted to overlay your own custom controls for your video player and haven't been able to, you're going to be really interested in this topic. And finally, I'm going to tell you how you can augment your user interface with audio. In this section, I'm going to cover the basics of the Web Audio API, which is now available in Safari 6 on Lion and Mountain Lion, and in Safari on iOS 6 as well. So let's get started talking about synchronizing multiple media elements.

There are lots of times when you might want to have two pieces of media on your website in perfect sync. Believe it or not, I created this mockup without any design help whatsoever. It's pretty amazing. But as you can see, I have a video on either side of the page, and I have these sharks kind of swimming from one side to another, and the waves of the water are, you know, they have a cohesive effect.

You may have seen other video effects like this in websites where there's a video across the top of the page and on one side of the page, and something is traveling down from one video to another. In order to achieve these types of effects, you have to keep those videos in perfect sync. So, decoration is one instance where I would need this technology.

Another is if you want to add a sign language video track to the media on your site. There's this guy, his name is Stephen Torrance, and he does American Sign Language interpretations of popular songs. I found him on YouTube and then I was really surprised to learn that he works at Apple, actually. But if you wanted to synchronize one of his sign language interpretations with your own media, you'd need this technology.

If you want to compare two like things on your website, maybe we have a snowboarder, a snowboarding competition, and we want to compare the performance of two different snowboarders coming off of the same jump. Or maybe we want to compare two baseball swings, or two soccer kicks, or two models walking down a runway.

Comparing two like elements in real time, you would need to keep these pieces of video in sync. Finally, you might have a piece of video on your site, and you want to substitute an alternate audio track, like in this case. Hi, I'm Vicki Murley from the Software Evangelism team at Apple.

Sounded a little different when I recorded it in the studio, but thanks to Alan Schaffer for the voice talent here. But this is one kind of, you know, funny example, but there are plenty of other times where you'd want to do this for practical purposes. So how do we do this? If you've tried to do this in the past, you may have thought, okay, well, I'm going to fetch all of the videos on the page, put them into an array, cycle through them, and just call play on all of them. And maybe JavaScript, it's super fast, maybe this will be able to keep them all in sync.

That's not actually what is going to happen. They're always going to be a little bit out of sync. Even if you're doing something fancier than this, like trying to do a little bit of a play on all of the videos, you're going to have to do a little bit of a play on all of the videos.

And then trying to start them all once they hit at the one second mark and then keep them in sync as they're playing. It's really difficult to do. Script execution is always going to take a finite and unpredictable amount of time, even if that time is really small. And it's not as if media playback is blocked by script execution.

So we really need, oh, and that's just playing the media. When we start to think about some of the other issues that come into play with media, it becomes obvious that this is really difficult to do on your own. How do you determine when all of the videos have enough data buffered to play them all the way through? If you wanted to change the playback rate to do like a slow motion effect or fast forward through the video, how would you do that on several videos at exactly the same time? What if one video stalls? You'd have to somehow detect that stall and then pause all the other videos and then, again, start playing them at all the same time. This is really difficult.

Lucky for you, if you're trying to do something like this, there is a new feature in HTML5 and new in Safari 6.0 available on Lion and Mountain Lion called Media Controllers. So Media Controllers basically let you slave any number of media elements, for example, four pieces of video, to a single controller. And then when I call play on the controller, all of the pieces of video start to play. When I call pause on the controller, they're all paused.

Play and pause are probably two functions that are really familiar to you if you've been using the audio and video elements. And in fact, a media controller has all of the same API that you're used to using for the audio and video elements. So the play, pause, and can play type functions, all of the attributes that you're used to, like current time, duration, ended, etc. All of the events that you're used to listening for if you've ever made an HTML5 media player.

The only two attributes that are not supported for media controllers are loop and autoplay. So how do we set up one of these media controllers? Well, if I wanted to synchronize one of Stephen Torrance's sign language tracks to this Bohemian Rhapsody video, the first thing I would do is set the media group attribute on all of the elements that I want to slave to one controller.

And that is going to look something like this. Here are my two video tags. And I'm just going to add the media group attribute and set them to the same value, which is video controller. By putting this HTML, I've just created a media controller implicitly. That's all you have to do.

Once I've declared the media controller, to access that controller, I can get the controller on any of the slaved elements, and that will give me the controller that's common to all of them. So here, I've just used getElementsByTagName to get the first piece of video on the page. I could have also used querySelector. I get the controller object on that video, and then I can just call the play and pause methods directly on that controller. And as I said before, doing so will play and pause all of the slaved pieces of media.

In JavaScript, it's similarly simple. Maybe you have some kind of dynamic effect and you want to start playing two pieces of media at the same time on the fly. You can create a media controller object just by calling new media controller. Set that controller object to the controller on any media element. And then again, just control that media, all those slave pieces of media, through the controller. So here I'm calling play and pause on the controller.

So to show us how this works, synchronizing video playback with media controllers, I'd like to invite Beth Dakin to join me on stage. So I was inspired to create this demo by this beautiful video. This is a video created by a photographer named Joe Capra. He went to Iceland during the time of year when the sun doesn't set for a bunch of days, and he took all of these photographs and then made this beautiful stop motion video.

So this is his webpage where he has the video, and I made my own little mock-up of it for the purposes of this demo. Okay, so I wanted to create a really cool tile effect with this video. So in your slides, you were showing how to sync up two videos, but for this demo, I wanted to sync up four videos. Whoa, wow.

So you're telling me you're about to sync four videos on this webpage. Oh yeah. Oh my gosh. This is happening. I'm really excited. It's going to be awesome. Oh my gosh. Okay. And then you're going to do some cool effects with it? Yeah. I can't wait. Okay. I'm so excited. So this looks like one video here in my page, but it's actually four. So I have four different video elements, each in its own div. I have a different version where I put a border around the div, just so you can see that.

So I have four divs here, and they each are clipping out a portion of the video using overflow hidden. And then they have the video element positioned correctly so that they'll all line up as if it were one video. I see. And so since it's lined up as if it was a single video, I want everything to be in sync. I see. So let's take a look at the code because what I have going on right now is not really doing that. Okay.

So this is what I was trying to do, where I was getting all of the videos and looping through and trying to tell each video to play. And as you were just telling us in your slides, this does not really work. No, it does not. It's not possible to keep everything in sync all the time. This is just very inconsistent.

Beth Dakin, Vicki Murley So I want to use a media controller. So this is my HTML file. I have my four videos here, each in their own div. And I'm just going to add an attribute to each of these files. So I'm going to add each of the videos. So I just want to add this attribute in each of these places.

So I'm going to give them all the same value, tile controller, because I want all four videos to be slave files. Beth Dakin, Vicki Murley So I want to use a media controller. So I'm going to add each of these files. So I'm going to add each of these files. So I'm going to add each of these files.

So this version of the function just starts by getting the video controller. And since all of the videos have the same controller, I can just take any one of those videos and get its controller, since that's the controller for all of the videos. And then instead of telling a video element to play or pause, I'm just going to tell the controller to play or pause. Beth Dakin, Vicki Murley I can't wait. I'm so excited. Should we check it out? Yes. Should we check it out? I'm really excited. All right. This is going to be awesome.

So let's reload. Check it out! They're all in sync! It looks just like one piece of video on a web page. Yeah, but they're all in sync. Isn't that awesome? Wait a second. I thought you said you were going to show us some really cool tiling effects or something like that. Well, but this is really cool too. They're all in sync.

Wait a minute there. I have to say, I'm not going to lie to you, I'm a little disappointed right now. Alright, I'm just going to do two things. Okay, I'm ready. First, I'm going to click this button. And then I'm just going to type my name, okay? Okay, I'm ready. B-E-T-R-I-N-G.

Whoa. What's happening now? I'm going to press F now. You want to know what this is? What's happening here? All right. So here I'm using an awesome new feature in Safari 6 called CSS Filters. And with just one line of CSS, I can apply one of these really cool effects to each of these videos.

And so here I'm just randomizing the different filter effects. And as I click on them, a different filter effect is being applied. And when I press F, it's getting a new filter effect for all of the videos. Cool. So you get to apply a filter with just one line of CSS, basically. Yes. That's it. Pretty cool.

Awesome. So how did you get this to show up and take over the whole screen? That's something I have never seen before. So I'm glad you asked. I used another awesome new API in Safari 6, a new JavaScript API for taking any element full screen. So this is different than the Safari full screen feature that shipped in Safari Online originally, where I pressed that button up there to take the whole Safari application full screen. That's like a Safari feature where all of this is just built into the browser, same on every web page. Here I'm taking advantage of this new JavaScript API where I'm fully controlling this full screen experience and the look and behavior of it.

So you may notice a few ways in which I customize that experience. So obviously a few things are pretty obvious. Like I have it so that the video tiles split apart. And grow apart when you go into full screen. And I also have it so that the background color changes.

And I had all these cool effects. These hot keys are hooked up to different JavaScript functions to do these different things in CSS. And that's all only in full screen. So if I press those buttons normally when we are not in full screen, then we would not see those cool effects. I see.

And another thing is that you may have seen all of those elements. I see. And the elements on the page are flying off in different directions when we go in and out of full screen. And then they fly back on as well. So you're saying that that's not the default behavior for the web page elements to fly off in different directions? No, that's totally customized. Okay, great. Cool. That's awesome. Yeah. Well, thanks, Beth, for showing that to us. Thank you.

So Beth has conveniently brought me to the second part of this talk, which is all about implementing full screen interfaces. If you have been coming to WWDC for a few years now, you probably saw Beth and I on stage in 2010, where we showed you a demo that looked like this.

And in this demo, we had our own custom controls. You can see a play button and then a full screen button over on the right-hand side. But this full screen button was using an older API, and that would just take the video element full screen. And then when we were in the full screen mode, our only option was to overlay the default controls. That was the built-in behavior. It wasn't customizable. You couldn't show your own custom controls in this view.

So, as Beth said, now there's a new JavaScript API which you can use to take any web page element full screen. The element that has your video and the controls inside of it. You can take a paragraph full screen if you really wanted to. And it's really simple and really powerful. It boils down to doing two things, knowing how to enter and exit full screen mode, and knowing how to define a custom or appearance or behavior for the full screen element.

So let's tackle the first part, how to take any web page element full screen. It's actually really simple. All you have to do is fetch that element and then request full screen on that element. So in our demo example, the element that we wanted to take full screen was named main, and so we just fetched that and called WebKit request full screen on it.

I want to take a moment just to mention that all of the API that I'm going to talk about right now has the WebKit prefix on it because these APIs are still part of a developing specification, which you can take a closer look at at this URL. So when you call request full screen on any element, it's going to take over the screen from which the request was initiated. By that, I'm just referring to any kind of dual monitor scenario.

Like if you have Safari on your leftmost monitor, it'll take over the leftmost. Screen, not both monitors. And it has to be triggered by an explicit user action such as a click. You can't trigger full screen, for example, from an onload handler. A mouse move event also does not qualify as an explicit user action.

Imagine going to a website and having an advertisement come up and you move the mouse one pixel and it takes over the full screen. You would not want that experience. One other reminder here is that case. Matters when you're calling request full screen. It's a lowercase s here. I don't know how many times I've mistyped and used an uppercase S, but make sure you use the lowercase S because the specification has changed. And if you use an uppercase, you're going to be using an older version of this API.

That's how you enter full screen. Exiting full screen is really just as simple. You just call exit full screen on the document. This also requires an explicit user action. But in this case, that initial click to enter full screen mode might qualify as the action that you can use to exit full screen mode.

For example, if a user clicks a link, you can set a timeout from that click. If a user clicks a link, you can trigger full screen mode and in that same action set a timeout to exit full screen mode after a certain amount of time. And again, just like entering full screen, case matters for exit full screen as well.

So what about knowing when full screen has been triggered and exited? There's actually a single event listener for both events, and it's called full screen change. Beth Dakin, Vicki Murley So I will just get the element that I'm taking full screen. In our example, that element has an ID of main, and I'm going to add an event listener, the event listener for full screen change. Beth Dakin, Vicki Murley And then when that happens, I'm going to add my listener that listens for those key presses that Beth was doing only in full screen mode, that worked only in full screen mode.

Beth Dakin, Vicki Murley We want to do this because we wouldn't want those key handlers to be in action when we were in line in the page. Beth Dakin, Vicki Murley If a user was entering a comment on my blog or something like that, they would be pretty surprised if video elements started moving around.

So there's a single event for both entering and exiting. The way you determine which direction you're going is to look for a full screen element on the document. So if you have a full screen element on the document, if document.webkit full screen element, then you are entering full screen mode. So at that point, I'm going to add my event listener to handle those key presses. And if you don't have that, then you are exiting full screen mode. So in my example, I then removed that event listener to handle key presses.

As I mentioned, you can take any web page element full screen. An image, a paragraph, a heading, anything you like. You can even take the body full screen if you wanted to. iFrames have a special behavior though. If you have an iFrame on your site, you're often getting content from somewhere else, some third party, and you don't necessarily control that content that you're embedding on your page.

So you might not want that content to be able to take over the full screen on your page. If we take a closer look, this one has a full screen button. You can set an attribute on your iframe that lets you control whether or not an iframe is allowed to take itself full screen.

So I may want to allow full screen on an iframe that is from a trusted video site. I want users to be able to take that element full screen. I want that iframe content to be able to take itself full screen. But I might not want to set the allow full screen attribute on an iframe that contains an advertisement or something like that. I wouldn't want that advertisement to be able to take over the full screen.

The iframe itself knows about the decision that you have made on your site. The iframe can call document.webkit fullscreenenabled to detect your decision here. So they can detect whether or not you have set this attribute to enable fullscreen mode for their content. Beth Dakin, Vicki Murley The iframe itself knows about the decision that you have made on your site. The iframe can call document.webkit fullscreenenabled to detect your decision here. So they can detect whether or not you have set this attribute to enable fullscreen mode for their content.

So those are all the details of entering and exiting full screen mode. Now let's talk about how to define a custom appearance or a behavior for that element that we're taking full screen. So this first explanation really doesn't have much to do with full screen mode, but many of you might be wondering, looking for a little more information about it. CSS filters is a brand new feature in Safari 6, and as Beth said, you can use just one line of CSS to apply a filter effect to any element, including video. So this is the filter property, and it can take one or many of several functions.

You can add a sepia tone, invert the values, add contrast, set grayscale, change the opacity, change the hue rotation, set a blur, change the brightness, or add a drop shadow. You can specify just one of these functions, or you, oh, and this is what they look like. So I've taken this image of the tulips, which is up in the upper left-hand corner, and applied the following filters to it. In each example here, with just one line of CSS, it's pretty amazing.

You can also combine multiple filter functions into one. So if I wanted to see what a hue rotation of 320 degrees with a contrast of two looked like, I could do that. And also, the filter property, like many CSS properties, can be animated. So if I say that I want to animate the filter property over five seconds, I would actually see this effect in my web page, the colors kind of changing over time. Pretty cool.

You may have noticed that the performance as we were filtering these elements and moving them around screen was really great, and that's because in most cases, CSS filters are hardware accelerated, so you're going to get great performance. Okay, so now that that is out of the way, let's focus on how we achieve some of these effects in full screen.

You may have noticed that the interface changed when we went into full screen mode. Our full screen button disappeared, we had a close button appear in the upper left corner, the video obviously got bigger and split into pieces, and all of these other page elements kind of slid off the screen in different directions.

Well, those effects were all achieved with basically one CSS pseudo class that is available in full screen mode, and it's called the full screen pseudo class. Here it has a WebKit prefix, as I mentioned before. If you're not familiar with CSS pseudo-classes, they're applied by the browser, and you define the appearance when the browser applies it.

So a really common pseudo-class is colon hover, and it's used on links on tons of websites. Here I have a link, and the color of the text is gray. When the user hovers over it, I want to color the text white and add a background color of dark cyan. So it's just going to change instantaneously.

I can animate that change by just adding one line to my anchor style definition. That's the transition property, and I'm saying that I want the background color to animate over two seconds and the color to animate over two seconds. So now when a user puts their mouse pointer over this link, it's going to change over time instead of making an instantaneous change.

And that's how we achieved many of the effects that you saw in this interface. The most simple example was changing the background color. Here I've added sort of a yellow outline to the element main, which was the element that I was taking full screen. In the page view, the background color was white, but when this element was displayed in full screen mode, I wanted the background color to be black. So I just select the main element with the full screen pseudo class in this way.

I wanted that to animate over time, so I just added the transition property to change the background color over two seconds. This is how the sliding effects were also done. So we had this logo at the top. It was just a div with a background image. The logo was inside of the element main, so it was inside of the element that we were taking full screen. So we used the CSS pseudo class. We select it differently than we do for the main element.

So here I'm saying that when this logo element is in full screen mode, I want to translate it in the Y direction, negative 300 pixels, so off the page. And again, I just add the transition property so that the change isn't instantaneous, so that it animates off the page.

One case where you're going to definitely be using the full screen pseudo class is when you're actually resizing your element to fit the screen. So by default, Safari and WebKit will preserve the size of the element and its aspect ratio when you go into full screen mode. And this is actually kind of useful. Imagine you have a video. Most videos have a different aspect ratio than the ratio of the user's screen, or they might have a different aspect ratio. So if we just stretch this for you in both directions, it might not look right.

So you can actually customize the width and height of your element in full screen mode. So if I wanted this purple box to just take over the entire screen, I could change the width to 100% and the height to 100%. and then it would do just that, fill the entire screen.

So to summarize, with full screen interfaces, you really have complete control with just a few simple pieces of API. It's really easy to trigger and detect full screen mode. You get to decide how the iframes on your website behave. They cannot take themselves full screen without your permission. It's really easy to define a custom appearance and behavior with the Fullscreen Pseudo Class, the Fullscreen Change Event, and the ability to access the fullscreen element on the document. So that's implementing fullscreen interfaces. Now I want to talk about augmenting your interface with audio.

So if you have been using the audio tag on your website, you've probably noticed that it's great for certain things. It is designed to stream audio. That is more complicated than it sounds behind the scenes. You're getting audio from a remote source, it's buffered, the audio element knows exactly how to handle that changing environment and play that media. It's really easy to use the audio tag to add a background audio track to any website or web app. It's as easy as dropping in an image.

And of course, without the audio element, we wouldn't be able to play audio in our websites on iOS devices without this element. But if you've used the audio tag, if you've tried to use the audio tag to do some other things, you might have found some things quite difficult and in some cases impossible.

So with the audio tag, it's not easy to play a sound at an exact time. You might by chance get that sound to play at the exact time, but if you needed to schedule that sound in the future, you might not get the behavior that you're always looking for with the audio tag.

You can't use the audio tag to dynamically apply effects like reverb and echo. You definitely cannot generate a sound from the audio tag. The audio tag takes existing media and plays it. And finally, you can't use the audio tag to access raw audio data. It's just, it's great in some ways because it's as simple as adding an image to your website, but it doesn't let you kind of see the bits inside and create new interfaces based on that.

So there is a new technology that lets you do all of these things and more that the audio tag isn't designed for, and that is the Web Audio API. And the Web Audio API is available in Safari 6 on Lion and Mountain Lion and in Safari on iOS 6. And right now, I just want to show you a demo to show you what's possible.

So what I've created here with the Web Audio API is just a simple audio sequencer. And on the left, I have a number of instruments, and they're getting highlighted as I mouse over them. If you're looking for sounds for your own audio interfaces, freesound.org is a really great resource.

People have put a lot of time and effort into recreating all different kinds of sounds from all different instruments, and that's actually where I got the sounds for this demo. So right now I'm hovering over the symbol, but if I click on it, the sound is going to play instantaneously. Then maybe I can make it a little louder. Okay, that's better. So we've got the symbol here that I'm clicking on, the snare.

The clap, which is my favorite. If things start going badly, I can just generate applause for myself. And also the kick drum, which is this nice low beat. So you can already see that this is an incredibly responsive UI. I'm just clicking, clicking a bunch of times. It's completely responsive.

This Web Audio API also lets me schedule the exact time that a sound should be played. And that's really important in any kind of game that you're creating for a website and also in this type of sequencer application. So I'm just going to select the kick drum here, and then I'm going to click play to start the playhead on my audio sequencer. And you're going to see the kick drum always playing at the exactly right time.

It's pretty exciting. But it gets even better when we start combining multiple sounds at once. So if you've ever wanted to play two pieces of sound at the same time on iOS, in Safari on iOS, this is your solution for doing that. I'm going to select this symbol and then click play again, and we'll hear them play at the exact same time.

The kind of fine-grained control is really important for something like this. If it were even a little bit off, it would be really noticeable. And of course, it's incredibly important if you're making something like a web game. You need a sound to play as an explosion happens or as two things collide. So let's just make this example a little more complicated so that we can see the sounds just playing perfectly. I'm going to bring in maybe this drum up here.

This audio visualizer here, the spectrum, that's a pretty simple use case of accessing the raw audio data, but you could imagine making an entire music video on your website and synchronizing all of the elements in that video to the exact beats of the song. It would be really cool.

Beth Dakin, Vicki Murley Today, so this API, it's really powerful. It's also very vast. There's a lot to it. So today, since we don't have a whole lot of time left, I'm going to just cover some basic concepts to help you get started playing sounds and fetching and creating sounds.

Beth Dakin, Vicki Murley The other reason that I'm not going in depth on this is because this is an API that is still in development. It's in the versions of Safari that you have with you. Beth Dakin, Vicki Murley We're encouraging you to go ahead and download it. Beth Dakin, Vicki Murley So if you're new to the app, I encourage you to try it out.

Give us feedback. Let us know how you like it, what needs to be different. But today, I'm just going to cover the basics for you to get started. Beth Dakin, Vicki Murley So the Web Audio API, it kind of reminds me of the Canvas API that people use for drawing. An image tag, you just drop it in, and that's sort of it.

It's sort of a black box. But the Canvas tag lets you have a context that you draw to. And so like the Canvas tag, the context that you're drawing to, Beth Dakin, Vicki Murley there's also a context for the Web Audio API. And on that context, you're going to build up a graph of nodes that you route your audio through and then eventually play on your website. Beth Dakin, Vicki Murley So the most basic audio context graph might look something like this.

I would have a source, and that is just routed to a destination. And in almost all cases, there is a default destination, and that is just a destination. Beth Dakin, Vicki Murley So that is just your speakers. There's a built-in default destination on every audio context, and that is the speakers. So let's take a look.

This is a really simple example, but this is where it all starts. So let's take a closer look at how to work with audio sources. So in most scenarios, in a lot of scenarios, you're going to be creating an audio source from a remote resource. And most of the time, you're going to be using XML, HTML, HTTP request to get that raw file data.

So when you're using XHR to fetch these remote resources, you have to do a couple special things. The first two lines look really standard. We're creating a new request. We're calling open on the request with the resource, the URL for our file. We're setting the response type on the request to array buffer. And this is telling the browser, telling XMLHTP request that we just want the ones and zeros from this raw file.

Most of the time when you are using XMLHTP request, you want text as your response. When we set response type to array buffer, we're just going to be getting ones and zeros. I'm going to add an event listener to this request to call process buffer once the load event is set. is emitted, and then I'm going to call send on that request.

And then I'm going to, on that source, I'm going to fill the buffer object with the contents of the file that I have fetched from a remote location. So now we have all of our sound data in this buffer, and we need it to play at exactly the right time. So a user taps the screen, and we want a sound to be emitted.

To do that, there is a function called noteOn, and you just have to tell it when. It looks something like this. This is where we left off on the previous slide. We had just created an audio buffer source node and filled it with our buffer contents from our remote file.

What we have to do next is connect that to the default output. My audio context has a destination object on it by default. So I'm just going to pass that to the connect function. And then I can call note on on my source. And if I provide a parameter of zero, that tells the browser to play that sound right now.

So that covers a really common case of fetching a remote resource and using the Web Audio API to play it. There are other ways to generate sounds with the Web Audio API. You can actually hook up to an audio element and use a media element audio source node to play the audio, to route the audio from your audio element through the Web Audio API.

You can synthesize a sound from nothing. We've been talking a lot about files, fetching them from remote locations. You can use a built-in audio source called Oscillator. This is great for just beeps or simple tones. If you wanted to create a really simple organ, you could use the Oscillator audio source to do that. Create a sound from scratch. And also, similarly, it's easy to generate a completely custom sound from JavaScript using JavaScript Audio Node.

So those are really the basics of the Web Audio API. You have a source, and it's a source node, and it's connected to a destination node, and both of these are on your audio context. I mentioned that you're going to be building up a graph of nodes on this audio context, and this can actually get fairly complex. It can get as complex as you want it to be, in fact.

But it all, so we're starting with three different sources here, routing them through a bunch of filters, and then eventually they're getting to the destination. But even this, most of these things in the middle are just filter nodes that you're adding in. You now know the basics of declaring a source and a destination, hooking them up, and playing a sound. So that's augmenting your interface with audio.

So in closing, I just want to remind you that there are some great new APIs for HTML5 Media in Safari 6, which is available on Lion and Mountain Lion, and in iOS 6. They are going to let you solve some common problems that you've been facing and also let you integrate media in new ways in your web content. The media synchronization and full-screen APIs are really simple, but they're also incredibly powerful. We saw some pretty cool stuff today done with just a few lines of JavaScript and CSS.

And then finally, Web Audio, available in Safari 6 on the desktop and on iOS 6, really enables an entirely new class of web applications. For more information, you can email me directly if you need to, vickieatapple.com. There's also documentation on the Safari Dev Center. The w3.org website has copies of all the specifications of things that I've been discussing today.

And also, if you haven't already, you should join the Safari Developer Program, which is free to join. All you need is an email address, because that will give you access to a Safari section of the developer forums, and many of the engineers from the Safari and WebKit team are there answering questions, and I'm there as well.