Hello, I'm Kevin Calhoune of Apple's Interactive Media Group, and this is session 722, Preparing and Delivering Video for iPhone, Safari, and Apple TV, also known as the session with multiple punctuation marks in its title, which means it must be really, really great.

So what you'll learn here, you will learn how to prepare your video assets for delivery to the platforms of interest during this conference, iPhone, Apple TV, and web, so how to encode it and so forth.

But also, we'll touch upon all of the options that you have for delivering your content to those platforms, including custom applications, including podcasts. But we're going to give most of the emphasis during this session on web delivery. We're going to be talking about the kinds of options that you have for integrating your content into web apps, both what you can do today and what you can look forward to in the future, including how to give behavior to your video in your web apps so that it integrates well with other types of content. And part of that look ahead that I promised pertains to HTML5.

Okay, so what technology frameworks are we going to be related to during this session? All right, for audiovisual standards, that MPEG-4 body of standards that I mentioned for encoding and container formats, we'll be talking about QuickTime's, Apple's implementation of those standards in the product known as QuickTime. We'll also touch upon web standards, obviously, HTML for markup, CSS for styling and presentation, JavaScript for behavior, and we'll talk about Apple's implementation of those standards in WebKit and Safari. And since we're going to be talking about custom apps as well, we'll talk about the application frameworks that you will need to use to build custom apps on Mac OS X, on iPhone, and for Windows as well.

So it's a packed session. I hope you're all strapped in. We're going to start with the raw specifications for video encoding for these various platforms. We're going to tell you the bitrates and everything just because we think some of you are curious and also because some of you are using tools that give you broad options across the profiles that these various encoding formats support.

And you might need to know practically what settings to use with these encoders when you're creating material for these platforms. Also, some of you might be implementers of encoding tools, and you might want to integrate into those tools the ability to target these platforms specifically. So we'll give you information about how to do that. And that's how we start.

So for the next set of slides, I'm going to be giving you the most oppressive and information-laden portion of the session, and I was chosen to do this because I'm the most oppressive member of your presentation team today. Well-suited for the task, let's start with iPhone. The video encoding that we recommend for iPhone is H.264 encoding.

The iPhone supports up to the baseline profile, level 3.0, which by itself implies that the dimensions that it supports are up to 640 by 480 at 30 frames per second. And you may also know that baseline profile also implies that B-frames are not supported on this platform, although you can achieve really good quality video without the B-frames at these resolutions on that device.

For example, if you have video at a 16:9 aspect ratio, typical for high def, you might choose to encode it for iPhone at a scaled down size, preserving its aspect ratio of 480 by 270. You should be aware that when you deliver that video to the iPhone, the user has two options for the way this might look.

The user can choose to view it at its original aspect ratio. And in this case, since the video is a little wider than the device, it will be displayed letterboxed with unused pixels at the top and the bottom not needed for the video display. But the standard transport controls on the iPhone allow the user to zoom that up.

To fill the screen. And what that will look like would be this. Zooming up to fill the screen in this case means, since the video is wider than the display, cutting off the left side and the right-hand side of the video. You should be aware that users have this option, so if you're producing content specifically for the iPhone, you might not want to put vital information far to the left or far to the right of the frame.

Okay, that's the basics of what you need to know if you are encoding content for delivery to the iPhone. Let's move on to something after a punctuation mark in our title. If you are targeting video for Apple TV, this is a different device. It supports up to high-def quality video. It looks really good when you plug it into your high-definition flat-screen monitor.

Apple TV supports same family of video encoding, H.264, up to the progressive main profile, level 3.1. What does this mean? This means that you have sizes available of 1280x720 at 24 fips. That's 720p at 24 fips. Or if you need a higher frame rate, up to 30 fips, you have to encode at a slightly smaller size. 960x540 will work well at 30 frames per second. That profile implies that B frames are supported, and you will definitely want to use them in order to achieve the necessary data rates with resolutions that large.

Audio on the Apple TV. Again, AACLC up to 48 kilohertz is supported on Apple TV, but you also have the option on Apple TV if you happen to be producing content for delivery to other types of hardware for which you are creating Dolby Digital audio. You can deliver that to Apple TV as well, starting with Apple TV 2.0.

Container formats supported by Apple TV. QuickTime movie files are supported. MPEG-4 files are supported. And the M4V variant of MPEG-4 also supported. Same as iPhone on Apple TV, MPEG-4 video, MPEG-4 Part 2 video, that is, is supported. And all iPod playable content is playable on the Apple TV as well. So our table for Apple TV looks like this.

Again, the maximums listed here might not be appropriate for you, and in particular in this case, not all the maximums are achievable simultaneously. You'll have to recall that at 24 frames -- at the largest sizes, the highest frame rate available is 24 frames per second. If you need a higher frame rate, you got to scale down just a little bit.

Other important point here, data rate that you can achieve on the Apple TV. A sustained data rate of 5 megabits per second is what we recommend for the encoding. But just as happens with production video content for other devices as well, if you have a cut in the video and you want to play smoothly across that cut and things are changing, you need a high-quality intraframe right after that cut, the device supports local maximum spikes in the encoding of up to 12 megabits per second. So you can tweak those important points in your encoding up to that data rate.

All right, two hardware devices we covered, iPhone, Apple TV. Now... The third platform that we are talking about in this session today is the web. But we have to remember that these days, web delivery does not imply a specific class of devices. Desktop users can connect to the web by any number of means. High-speed connections, broadbands, lower network connections as well are possible from desktop machines. But whole new classes of devices have become common that also are capable of connecting to the web, and they have a different set of constraints on video and code. coatings.

There's a wide range. However, the standard that we've adopted at Apple for video encoding, H.264, has solutions, has sweet spots across the entire range of devices and network speeds that you will be encoding for for your users, regardless of the device that they are using. So, the encoding varies by device, but the formats that we recommend are the same. Again, H.264 video, give you more details in a moment about the range that we recommend. For audio, again, AACLC up to 48 kHz is supported on all the devices that we're talking about.

Same list of container formats, but on the desktop, those have web connections from desktop machines, actually can make use of additional container formats as well. In particular, all the container formats as supported by QuickTime 7 are available in case you have legacy content in any of those formats. But we recommend QuickTime movie files, MPEG-4 and M4V, just as for the other devices, and 3GP for the low data rate stuff.

All right, a little more information about the range that we're talking about. We're talking about really small to really large. 3GP mobile device on something like the Edge Network, the most appropriate encoding that you might want to use, something like 176 by 144 maximum size at 64 kilobits per second for the video, total of 80 kilobits per second for the audio plus video. That's a pretty narrow pipe, but you can deliver good quality stuff across that connection.

All the way to the high end where we're talking about something like an HD trailer, like the kind that you can see on Apple's trailer site, up to 1080p, 1920 by 1080 dimensions, up to 8 to 10 megabits per second sustained data write. That's achievable if you have a fast enough network connection for delivery via the web. You probably would only have a restricted set of users that are actually capable of keeping up with that amount of data coming across, but it's doable for certain applications. Thank you.

What if you want to deliver just one encoding that can span this entire range of devices and network connections? What's our recommendation? Well, you've probably noticed from the prior slides that all of the devices that are capable of connecting to the web that we're talking about support iPhone encoding. So you can just say, create an encoding suitable for iPhone, you can deliver that in a web app, and that will work everywhere.

Now, that won't take full advantage, of course, of the horsepower and the bandwidth that's available to top-end devices. So we recommend that you can choose to create more than one encoding. If you don't have constraints on server space or on bandwidth, you can create multiple encodings for the same content and make sure that the appropriate users are viewing the appropriate encoding by something that we call a QuickTime Render. So, let's talk about the QuickTime Render. A QuickTime Render is essentially a table of contents.

And when a QuickTime Render is opened, the user agent or QuickTime, whatever happens to be reading the QuickTime Render, will take a look at that table of contents and say, "Hey, if my network connection is this fast, I'm supposed to go to this URL and view this movie.

If my network connection is this fast, I should choose this one." There are other switches that are possible as well in this table of contents, according to device, according to encoding, and according to the reference movies. So, if you have a reference movie that's a good example of a decoding format supported on the end user's device, this reference movie, for example, could give you the option of viewing one URL for an iPhone on an edge connection, another URL for an iPhone on a Wi-Fi connection, and yet a third URL for a desktop machine with a broadband connection, capable of decoding H.264 main profile.

We'll talk about how you can create these reference movies in just a minute. But first, let's talk about how you can use some common tools to create this whole range of encodings without having to know any of the raw detail that we just talked about. So, we're moving away from the oppressive detail, and we're talking more about the cool tools.

QuickTime Pro allows you to create all of the kinds of material, all of the encodings that I've talked about, really easy. It has what are known as movie exporters built right into it that allow you just to choose export to iPhone and the result will be appropriate, or export to Apple TV and so forth.

These movie exporters are really accessible in QuickTime Pro, really easy for you to get to from QuickTime Player. But it's possible for other applications using the QuickTime framework to make these same exporters available as well. So it's possible in custom tools or in tools from other vendors to have these same easy-to-use features for export targeting these particular platforms.

For creating reference movies, once you have the various encodings, we have two options for you. One, we have a feature in QuickTime Player called Export for Web, which takes care of a lot of stuff. We're going to demo that for you in a moment. We also have a tool that you can use called MakeRefMovie. This is available from the QuickTime Developer Tools site that you can navigate to on the ADC pages at Apple. MakeRefMovie allows you to create a reference movie by hand with whatever complex set of switches, no matter how many encodings you decide to prepare for your users.

All right, a little more detail about how to get to this stuff. QuickTime players, export components, as I mentioned, are standard in QuickTime. The ones that we're going to be talking about here, movie to iPhone, that will create exactly the type of encoding that's appropriate for an iPhone connected to the network via WiFi. Same encoding parameters that I just mentioned earlier. Approximately one megabit per second, up to 480 by 360 to take advantage of that data rate for the actual display capabilities of the device.

Movie to iPhone Cellular will create an M4V file that will work if the iPhone's connection is the Edge network. Lower data rate, still good quality. I'll mention Movie to iPod, which is another one of these exporters that's available. From the earlier slides, you know that all the devices that we're talking about support iPod encodings. You can choose this exporter as well if you want to create one encoding that's suitable for all this stuff.

And the parameters of the encoding that will be created are given to you there. Finally, however, if you want, for some reason, if you use one of these exporters and they don't precisely give you exactly what you want, you can use the Movie to MPEG-4 exporter, which gives you access to more settings that you can tweak to suit you. Movie to MPEG-4 Looks a little bit like this.

You open up your asset in QuickTime Player Pro. Then you choose Export from the File menu. Up comes this panel. It includes the pop-up menu in the panel for the exporters that you can choose, the ones that I just mentioned. For example, you can choose Movie to MPEG-4. Those exporters that have additional features, additional settings will display an additional panel. This is the panel for Movie to MPEG-4. Here you can choose the data rate, for example, of your H.264 encoding. You can choose MPEG-4 video instead of H.264 if that's more suitable to you.

For example, if you were encoding to iPhone, you would want to click on the Video Options button and you would want to say, "For iPhone, I know because I went to session 722 that for iPhone, I need to encode at the baseline profile of H.264." Other things are tweakable there as well so that you can create an encoding to suit you.

Kevin Calhoun, Jeremy Farbaugh, Eric Carlson, Mike Czepiel But suppose you're not a video editor and you want to encode to H.264, then you can choose MPEG-4 video instead of H.264 if that's more suitable to you. For example, if you were But suppose you're not interested in all those details, you want to use the one-click feature that takes care of all this for you, that's Export4Web in QuickTime Player.

And Jeremy Farabagh, known as the Code topiarist -- and you know I had a little bit of water before coming up here so I can say that -- is going to show you what Export4Web will do for you in QuickTime Player Pro. Jeremy? Jeremy Farabagh: Hi. Thanks. I'm Jeremy Farabagh.

I'm a developer for the QuickTime Player Pro team. And QuickTime Player introduced recently a new feature called Export for Web, as Kevin said, that allows you to export up to three different platform-appropriate versions of your content. And I'm going to demo it for you right now, just quickly.

Okay, so here I've got a little bit of the movie Last Time I Saw Paris, and I want to make a single version that's appropriate for the iPhone, iPhone over Edge, and for the desktop. So up here from the File menu, I'm going to choose Export for Web, which will pop down a sheet with just a limited number of options. I'm going to call this Last Time, and it lets you choose between three different versions of your exported movies appropriate for the phone, for the iPhone over the Edge network, and for the desktop. The desktop version will be also great for Apple TV.

If you have a movie with a poster frame, you can choose either that or the current frame of the movie from which you'll create a poster image if you're going to use this for embedding into a website. When I click Export, the player is going to start three different exports. It shouldn't take too long since it's a very small clip.

And it'll create those three exported versions plus one reference movie that wraps them all up together. It'll also create an HTML file, which demonstrates how you would embed this movie into a web page. It includes a little bit of JavaScript, a little bit of HTML, and you can just copy and paste these into your web page from whatever text editor you like, or your blog for that matter. And what you'll get is something that looks very much like this. Right now it's just an image. You click on it and it will switch over to a movie. Great scene. Okay. And there you have it. Very simple. Thanks, Jeremy.

We'll be giving you more details in just a little bit about the markup that Export4Web creates and a little more details about the JavaScript that's available. But if you just put that on your website, you're good to go. So that's tools in QuickTime Pro that are available to prepare video for these platforms.

You might be using professional strength video editing tools such as Apple's Final Cut Studio Pro, Final Cut Pro Studio. There, I got it right the second time. These tools have lots more options for the preparation of video. In particular, you can not only prepare audio and video for these platforms, you can use this suite of tools also if you have closed caption information in your source assets.

You can bring that all the way through to your source assets. You can also use this suite of tools to prepare video for delivery to each one of these platforms, iPhone, Apple TV, and the full range of web-capable devices using this suite of tools. If you're preparing material for these platforms with Final Cut Pro Studio, the application you use to encode is known as Compressor 3. Compressor 3 has a similar set of options built into it just like QuickTime Player earlier. It looks like this.

For example, you can choose to encode H.264 video for Apple devices in this particular example. For example, we're encoding for iPhone cellular using the same kinds of settings we saw earlier suitable for delivery over the edge network. Or here's what it would look like if you're encoding for iPhone via Wi-Fi. So that rich set of features is built into the professional strength tools as well.

However, you might not be adopting either of these two sets of tools, the QuickTime Player tools or the Final Cut suite of tools. Instead, you might have a custom suite of tools that you're using in-house for your production process. Or you might be using tools from other vendors. And you might be interested in integrating the same export facilities into those tools. How do you do it? Here's a code snippet for doing that on Mac OS X.

We recommend the use of QtKit together with a Cocoa application for you to be able to manipulate media, in particular, to write it out in these formats that we're talking about. All you need to do to accomplish this in your application is to instantiate QtMovie by pointing it to your asset via QtMovie init with file or QtMovie init with attributes, as you saw this morning at the QtKit sessions. Once you have an instance of QtMovie, exporting it is as easy as invoking the method QtMovie write to file with attributes. The with attributes parameter is a dictionary. Inside that dictionary, you would have a key that says this is an export operation.

And another attribute that says the exporter I want you to use, in this case on the wall, is the iPhone exporter. And with this amount of code, you can add that feature to your applications. So, now we know how to make the media. So, let's go ahead and start.

Question is, how do you get it to the audience that has interest in this media? Well, a wide variety of options, and we're going to cover them all in this one session. First off, Delivery to iPhone and Apple TV. Several options are available on these platforms. First off, in particular, if you have commercial video content, video content for sale, you can distribute it to your users of these platforms by means of the iTunes Store. This is more of a business arrangement than a technical thing. You'd want to get in touch with the iTunes Store folks in order to initiate that distribution. It's available and you should be aware of it.

Another way to distribute video by means of iTunes is to set up and maintain a video podcast feed. You make your video available from whatever servers you happen to have and allow users to browse your podcast together with other podcasts of interest in iTunes and other podcast-capable applications as well.

How do you set up a podcast feed and maintain it? One recommendation is a tool that's part of Mac OS X's server, part of the QuickTime streaming server suite of tools known as Podcast Producer. It has a lot of features in it that help with the preparation of the content and also in the maintenance of the podcast feed. We'll give you a reference later on for how you can find out more about using Podcast Producer.

GarageBand is a very good tool which many of you are already aware is capable of producing audio podcasts. But beyond that, it has features that allow you to associate images with specific ranges of the audio in your podcast. You can use it to create audio-visual podcasts in GarageBand as well. It's something to check out.

Finally, I asked for some recommendations for third-party tools that are relevant for video podcasts. What I was told is that Pulp Motion by Aquafotis is a great tool to check out. It lets you create a lot of interesting content very easily, suitable for delivery as a video podcast.

Okay, one method of delivery then, that's iTunes. Users can browse iTunes directly on Apple TV. They can browse iTunes on the desktop, sync with their devices. That's good distribution. That'll work for lots of content. But another option for delivering video content to your end users is for you to write a custom application. You may want to do this if you need to access specific hardware features of the platform that you're interested in building for.

Or you might just want better control over the integration of video content with other things going on in your application. It has some feature, some tool, some manipulation of data, and you want video to be part of that. What do you use to build applications that integrate video? Depends on the platform. On Mac OS X, just mentioned earlier, our recommendation is QtKit. QtKit allows you to open media and play it back. It also allows you, if you want to make this available, to users of your application.

Some editing operations that are available too. Those of you who are in this room during lunch hour saw the Michael Johnson of Pixar showing some uses of QtKit that allow you to do editing of content, not just playback. Capture is also available in this framework as well, so that you can bring assets in from cameras, for example, and then re-encode them. For more information about how to use QtKit for these purposes, look on the web for the QuickTimeKit framework reference and all the details about the richness of those classes is there in that document.

Let me skip over iPhone for a second since it's the newest of the custom application platforms that we're talking about this week to Windows. Yes, you can take the encodings that we've talked about here and deliver them in a custom application on Windows. You can do that by means of QuickTime. QuickTime 7, everywhere that it's installed on Windows, includes the QuickTime ActiveX control. And you would use that control in your Windows application, just like any other control, to associate content and to manage it.

The QuickTime ActiveX control allows you to control media playback in your app. It also has some features for editing as well, if that's what you need. The QuickTime 7 for Windows update guide has all the information that you need to learn how to use the QuickTime ActiveX control, the features of that class, in your application.

So iPhone, what is available in the iPhone SDK for controlling media playback on that device? The SDK includes a media player, and the class is known as the MP Movie Player Controller, and with that class, you can make use of full-screen video playback with the same transport controls as available elsewhere on the device in your custom application. Because this is so new, it's been covered in other sessions this week, but I want to give you a quick look at what this class looks like and the options that it exposes.

MPMoviePlayerController allows you to initialize from a URL the video content that you want to display. It allows you to initiate playback and to stop it at its current time. Via some properties on this class, you can set or discover the current scaling mode. Remember we talked earlier about the two scaling modes that are available on iPhone for video. So if you want to interact with that in your app, you can.

Also, you have the choice of whether or not you want the user to have access to the standard video controls. Perhaps your application doesn't want the user to zoom in or out or to change the time that's currently being displayed. You have that option as well. Finally, via the notifications up on the wall, you can monitor the state of the media playback. You can discover what mode of scaling the video is currently in. And you can also discover when the video is played through. So if you want something to be initiated automatically in your application when the video is done playing, you can listen for that and respond appropriately.

So, custom applications, multiple platforms. I promised that the bulk of the session was going to be devoted to delivery for the web. And that's what we're going to talk about from here on in. We find that delivery of video to the web is extremely rich and also very scalable across the devices that we've discussed.

We're going to talk about how to use technology that's already broadly disseminated, broadly distributed to end users with QuickTime in order to do that. And we're going to have a look ahead at emerging technology that will make the whole process easier. The first thing I want to note is that this will all work, everything we recommend, on Safari for iPhone.

And it will also work in any browser that supports plugins running on a machine on which QuickTime is installed. Browsers such as Safari for Mac, Safari for Windows, Firefox for either platform, Opera, Internet Explorer. So, lots of users can be reached by means of the recommendations that we have for you here.

What do you need to know in order to set up and prepare delivery for the web? Well, we covered all the details about preparing the media, including providing a reference movie that allows you to deploy multiple encodings that different users will see depending on the type of device we're using. So you already know that.

You'll also want us to cover how to incorporate video into your HTML document. We'll cover that briefly in a moment. You'll want to know what the further integration possibilities are possible in video. We'll talk about the JavaScript methods that are available. We'll talk also about the DOM events that are emitted so that you can coordinate in your page things that are happening in the video with things that are happening elsewhere in the page. And finally, we'll have some practical notes for administrators of web servers to make sure that the video will be delivered in the best and smoothest possible fashion to your users. So, we talked about preparation. Don't need to cover that again.

How to integrate video in HTML markup. You may have seen this type of example in the past using an embed tag, which is commonly used in order to say, "Oh, I want to load a plug-in to manage this content." And this embed example will work in many browsers. However, it is not the most compatible form of markup that we recommend that you use for the full breadth of browsers and browser versions that have been developed over the years.

Our best recommendation for you is instead of using an embed tag like you just saw, to make use of a JavaScript library which Apple distributes and you can redeploy on any site on which you might happen to need it. By invoking this JavaScript library's QT_WriteObject function, the markup will be generated by that JavaScript implementation that is most compatible with the widest variety of browsers and browser versions. Kevin Calhoun, Jeremy Farbaugh, Eric Carlson, Mike Czepiel So, if you want to use a script tag, you would use a script tag in your markup.

And inside the script tag, you would invoke the QT_WriteObject function. And instead of attributes of the embed tag, you would pass parameters to this function that control the display of the video, what URL for the resource that you want the video to come from, how do you want it displayed, scaled, or fixed size, and so forth. All of that you can pass to the function. And the markup, the compatible markup will be generated on the fly as the page loads.

Now, if for some reason you're running in a context in which JavaScript is not permitted, or if you just want to have finer-grained control over the markup that you use in your web pages, this is the markup that that function will generate, the most compatible form of invocation of a plugin that we are aware of.

It's an object tag on the outer part and inside fall back to an embed tag. So browsers that support the object tag will pick up the parameters for the video from that, but browsers that don't will fall back to the contained embed tag and pick up the information for that. That guarantees that things will work in a wide variety of contexts.

So, finally, you want to serve this stuff up so that users can view it without any glitches. How to avoid glitches in web serving for video 101 right here on this slide. If you are a web server administrator, please take note of these recommendations. We find over the years that many of the bugs that end users file against QuickTime when they run into trouble viewing content on the web have to do with misconfigured web servers, in particular, MIME types. So please take note of this. First of all, some of the platforms that we're talking about today depend on byte range access to video.

Video can be of great duration and very large in size. And in fact, on some of these platforms, you can actually play video content that doesn't even fit in the storage capacity of the device. Obviously, in order to load chunks of that video at a time, we want to issue byte range requests against the HTTP server.

So please turn that off. Kevin Calhoun, Jeremy Farbaugh, Eric Carlson, Mike Czepiel You can test whether a particular server supports HTTP requests in the Mac OS X terminal app by issuing that curl command. It'll print out an error if byte range is turned off. If so, please consult the documentation of your web server to turn it on.

Kevin Calhoun, Jeremy Farbaugh, Eric Carlson, Mike Czepiel I mentioned MIME type associations. Please make sure these are right. .mp4 files should be served with MIME type video slash mp4. If your server is serving them with MIME type text slash plain, you can issue byte range requests against the server. your users may not get the experience that you intend.

Finally, video's getting bigger. Resolutions are getting larger. Sizes are growing all the time. In fact, hosting for personal use on the web is getting larger as well. We just announced this week, MobileMe is expanding to 20 gigabytes is the standard amount of storage you can have. End users in their own homes can be creating video that's larger than 2 gigabytes and serving it from their personal spaces on any of a number of services. So please check that you support downloads of greater than 2 gigabytes on your web server, and that will make everything smooth.

That's all the basic information for incorporating video into HTML. What does it look like? Eric has a simple example for HTML4. Eric Carlson is not the only worker in the interactive media group at Apple, but he's one of the primary ones. So Eric is going to come up and show you this now.

Eric Carlson. As Kevin said, I'm also a member of the Interactive Media Group. What we're going to look at here is a really simple example of embedded video in the context of a web page. So we have a web page here about a movie. And so, of course, it would be useful to have a snippet of the movie here.

You've got the standard Playback controls, you can start the movie, you can stop the movie, you can change its volume. And if we look at the source of the page, Let's scale it up a little bit. You can see that we've used the recommended technique that Kevin was just talking about in the part of the markup where we want the movie to show up. We have a call to the QT write object tag. We pass in the URL of the movie. We set the width and the height. We set the scale. We set some various things.

And with that little bit of markup, we have a page where the movie content integrates nicely with the text. For example, if we change the size of the text in the page, the browser relays out the page and tells the plugin element that its size has changed. Using a keyboard shortcut here, I can scale it up and down.

And you can see that it flows nicely as the content on the page changes. We also have some Ahrefs here which link to a simple JavaScript function. So when I click on an Ahref, it changes the time of the movie and really integrates the video content in with the content of the page with really just a little bit of a change. So when I click on an Ahrefs, it changes the time of the movie and really just a little bit of a change.

Okay, so that gives you an example of what you can do with very simple markup in HTML4. We cut for time, can you believe it? We had more content that could fit in 75 minutes. And we had to cut for time constraints. Some richer example is deployment of video using the QuickTime plugin, but you may be aware of them. There are plenty out there available on the public internet. If you are interested in finding out more about the JavaScript interface of the QuickTime plugin or more details about the markup that you would use, you can go to the following resources.

From apple.com/quicktime, navigate from there to QuickTime Pro and from there to tutorials, and there are a couple of interesting things there about embedding that you might want to look at. Or if you want a copy of the full reference for JavaScript scripting of the QuickTime plugin, start at apple.com/quicktime, go from there to developer, to guides, and then to scripting and automation, and we have a document for you there called JavaScript Scripting Guide for QuickTime that can tell you everything that you need to do, everything you need to know to achieve richer integration even than Eric was able to show you with that simple demo.

What I want to highlight, though, is that the same markup that we've just been showing you and this many of the same scripts that are covered in the JavaScript document also work on the iPhone. Safari for iPhone has been implemented to be compatible with the markup for video that we've just recommended. So when it encounters the object and embed tags, that's the same thing that we've just recommended.

So when it encounters the object and embeds tags, it's the same thing that we've just recommended. So when it encounters the object and embeds tags, it's the same thing that we've just And for the first time in iPhone 2.0, we have JavaScript control over video in Safari for iPhone. And also, video will emit DOM events so your page can keep track, your document can keep track of things that are going on in the video on that device in Safari.

So for example, you can initiate playback, you can initiate full screen, stop playback, exit full screen with a JavaScript interface available on Safari for iPhone. And there are, in addition to a subset of the methods and DOM events available on the desktop, there are some that are peculiar, particular to the iPhone that allow you to control or monitor the full screen state as well. So lots of great stuff you can now do integrating video with your web pages for Safari for iPhone.

Now, I promised you earlier in the session, I was going to tell you that we recommend standards for video. But now the last 10 or 15 minutes of the session has been all about this QuickTime plugin thing. And what's that? Well, that's a module that's available from Apple that has its own unique API and its own unique attributes that you treat in its own unique way when you're taking advantage of it in your HTML. Well, that doesn't sound very standard, now does it? It's very rich. It's wonderful to use. It's broadly distributed and extremely reliable. But it falls short of the standard story that I promised you earlier on.

We've achieved interoperability of standards for video encoding and video container formats, as well as interoperability of a number of delivery protocols. However, we recognize that we have not yet achieved, up till HTML4, standards for integration of video content in the web. How are we going to get there? Well, what we've done, together with other interested parties, is pooled our experience, in our case, our experience with the QuickTime plugin, and we put our experience together into a draft specification for what we think video should be like in standard HTML markup, what kind of behavior it should have in the standard DOM. And this specification is now in draft form and part of HTML5.

In other words, HTML5 includes, for the first time, standard video and audio elements that you can treat as just another type of content that any user agent can support, with the same rich set of features that a user agent is able to apply to any other element. We think this is going to be really great.

In fact, so great that we've already implemented a portion of the draft specification in Safari 3.1, and, of course, its follow-on release, Safari 3.1. So all of this stuff related to the video tag in HTML5 that you see here, you can go home and try right away with Safari 3.1 or later. Now, when you have video managed by an element that's built into the DOM, you have very rich integration possibilities, not just the kinds of things that are possible with plugins, but further synchronization with other things going on in the page.

If the user agent is able to do that, if the user agent is aware of time-based media, that it can give you information about the flow through time and synchronize and coordinate things going on elsewhere in the page as time advances. Further, the video element can be treated for composition purposes and other purposes like anything else, so you have extreme richness. The draft of HTML5 is currently available at the URL given up there on the wall.

We would like you to become involved as well to participate, and the further refinement of this draft specification, bring your interest for web delivery of video and audio to the working groups, review the draft, make suggestions, help us refine the specification to make it really great, broadly implementable across the full variety of user agents, browsers that are available to end users.

We think this can be a big success. Well, what does it look like when you use HTML5 for video? Eric is going to start out by showing you something that will look familiar, and then we'll get to it. will move into greater integration possibilities. Eric? So what we're going to start off with here looks extremely similar to the demo I showed you a moment ago.

It's the same text. It's the same content. But in this case, because I've written the markup differently, instead of having the video here be displayed by the QuickTime plugin, it's being displayed by Safari itself. So if I click on the Play button, you'll see that the movie controller goes away. And that's just because that's the behavior of Safari right now. But you have the same basic controls. You can start and stop the movie. You can seek around in the movie.

We can, again, jump to different times in the movie. When I click on these Ahrefs, I'm calling a JavaScript function that seeks within the movie. But if we look at the source to this page, We'll see that instead of just having the call to QT write object, instead here I have a video element. On the video element is the optional attribute that shows the controller. And then inside of the video element, I have a pair of source elements. A source element is what you use in a video or an audio element.

So if you have a user agent, you can load the source element from the user agent and then you can tell the user agent which media resource it should load. In this case, we have two source elements. This is the HTML5 way of doing the equivalent of what you do with a QuickTime reference movie. The browser will only load one of these.

The rule is or the behavior is that it evaluates them in order and picks the first one that's relevant. So the first one here, we have an OGG encoding of the movie. Safari doesn't support that by default, so it continues on to the next one and it's an Mpeng4 encoding.

But the point is that you can write your markup so that So that different user agents which support different encodings still work. So what I have over here is a build of Firefox, an experimental build of Firefox that includes support for the video and audio element. Let me just close this in Safari.

So Let's scale this up a little bit. And when I load the same page that we just saw in Safari 3.1, you see we get the same thing. The playback controls are primitive, but this is just an experimental version of Firefox. The point is, in this case, Firefox does not have an MPEG-4 encoder, but because I Included an OGG encoding, and it has an encoder for that. It's able to load it and display it. And again, as we change the size of the page, it all just works.

Kevin? So a couple of things to note about that markup that Eric just showed. We talked earlier about QuickTime reference movies and the ability to make multiple encodings available that will be used according to the suitability of those encodings to the device and network bandwidth that was available to the end user.

The source element, which is specified as a child element of the video element, plays the same role in HTML5. You can provide multiple encodings, and as Eric mentioned, the user agent will survey these encodings by MIME type or other parameters it can examine. In particular, the specification says that you can include a CSS media query here to define the suitability of a particular resource, and the browser can choose the first one that it hits upon that it knows how to decode and is suitable for the current parameters of the context.

So pretty exciting. The other thing to note about this markup that Eric mentioned is it doesn't just use the video tag. It also has fallback. It falls back in this case to using the QuickTime plug-in if you happen to be running in a browser that doesn't support the video tag. So if you were to launch the current beta version of Firefox 3, which doesn't yet include this support, which is due for a later release, you would fall back to using the QuickTime plug-in instead of the video tag. So pretty clever markup that you've written there.

So we can do that stuff with the video tag, similar to what we can do with the embed tag for video. However, how far beyond that can we go? We took this feature, the video tag of HTML5, together with other new features that are emerging as standards in the web world, in particular, CSS effects that are also implemented for the first time in Safari 3.1, such as transforms and animations.

And we took them to some web designers and said, if you had these facilities in user agents for integration of video and for effects on video, what kind of pages, what kinds of web apps would you build? The first example that we have, we went to some folks who have their hands on some assets that are related to it.

It's pretty good stuff. We know that the same kinds of composition and the same kinds of integration are available in other tools, in other running systems. So we took them to some web designers and said, if you had these facilities in user agents for integration of video and for effects on video, what kind of pages, what kinds of web apps would you build? The first example that we have, we went to some folks who have their hands on some assets that are related to it. It's pretty good stuff. We know that the same kinds of composition and the same kinds of integration are available in other tools, in other running systems.

So we took them to some folks who have their hands on some assets that are related to it. It's pretty good stuff. We know that the same kinds of composition and the same kinds of integration are available in other tools, in other running systems. We know you can use Flash for this. We know you can use Silverlight for this. And those are great tools that are appropriate for many uses. We know you can use Flash for this. We know you can use Silverlight for this.

And those are great tools that are appropriate for many uses. However, our position on all this is, why do you have to stop using your web-based tools, HTML, CSS, and JavaScript, just because you want to make use of video together with images and text? Our answer is, you shouldn't have to. You should have the choice of making use of video and text. Our answer is, you shouldn't have to. You should have the choice of making use of video and text.

shouldn't have to. You should have the choice of making use of the same tool set that you already use for web-borne content when you have video as well as when you don't. You can choose those other options if you want to, but you can stay with the HTML standards and do rich things with video, too.

Well, how practical is it? These are some examples we came up with internally, but we realize there's a transition. There's a challenge. Today, we have broadly disseminated the QuickTime plugin. We hope that many more browsers will come online over time to support these new standards. You want to deliver your video regardless of the capabilities that are available on the particular device that the end user happens to be using.

So we wanted to do some experiments, first of all, in the lab in Cupertino to find out how practical it is for you content providers to begin taking advantage of these emerging standards, even at the beginning of the year. So we're going to do some experiments with the user agent to see how they can help you with the transition. So we're going to do some experiments with the user agent to see how they can help you with the transition.

So we're going to do some experiments with the user agent to see how they can help you with the transition. So we asked Jeremy to build some examples that will make use of the emerging standards when the user agent supports them and fall back to the existing broadly distributed things that you can use if not. Jeremy, what did you come up with? Sure.

So the first task was to create a JavaScript library that would create a video tag when it was available and fall back to an embed tag or an object tag where it was not. And here's a good working example of that. This is just J Random Movie with a standard controller and everything like that. Perfect. It's using the video tag here. If I loaded it in Firefox, it would use the object tag or the embed tag if it was on Internet Explorer.

So once we have that, we have a great foundation. Once we have a piece of JavaScript library that can integrate the two video tags and object tags, we can come up with a common API for controlling those movies with something external. So what can we do with this? Well, my thought was maybe come up with a JavaScript DOM-based movie controller to control the video playback of the movie.

And because I am in the QuickTime player team, I think I know what I wanted it to look like. So here you have a JavaScript-based DOM controller, fully functional, same as the QuickTime player. Play, pause, fast forward, rewind, go to beginning, go to end, mute, et cetera, et cetera.

And the best part about this just being a bunch of DOM elements is that they're completely styleable via CSS. So these buttons along the top, all they do is change the class name of the root DOM element and then pull in a different set of CSS rules. So the browser will just re-render everything the way it's supposed to look. So here's an example of a controller that looks a lot like the one from the Apple.com Pro site or one from an iPod ad, or you might recognize this one.

All just via JavaScript and CSS. So great. Now we have JavaScript controllers. We got a common API. Well, what else could we do? Why don't we rethink the way that movies are embedded into your text? So if you're familiar with a piece of JavaScript library called Lightbox, the idea is it provides a way for you to click on an image and have it pop up over laying your text as like a pseudo full screen kind of experience. But there's no reason we can't do that with a video tag as well, or the QuickTime plug-in where that falls back to. So here's an example of a website with some text and a button that one clicked will pop up a movie on top of the text.

You can see it's scrolling around here in the background, providing you with a JavaScript controller, close button, et cetera. So you don't have to have-- especially useful if you have a lot of these videos embedded into your website. OK, so now we have-- let's see, what do we got? We got a common video tag QuickTime library. We got a JavaScript controller.

We got pop ups. There's got to be something else we can do here. And importantly, if you pull up that same phone-- I'm sorry, you pull up that same web page on the phone, it's just CSS and DOM. That's it. So if I could get the video on the phone, please.

Okay, here's that same link. looks completely different, but has basically the same experience. You click on a movie,

and all the watched ones have shown up as watched. Anyway, some great options for pulling together media on into your web pages. Back to you, Kevin. Kevin Caron: So, Jeremy, I think when you mentioned the latest WebKit, you meant the latest nightly build. Jeremy Farbaugh: Latest nightly build of WebKit, yes.

Kevin Caron: They are hard at work integrating more of the CSS goodness in nightly builds of WebKit and that's what you're taking advantage there. Jeremy Farbaugh: Yes, that's right. Kevin Caron: Okay. So, Jeremy, he works in Cupertino. He can do this stuff. The question is, is it possible to start taking advantage of this stuff on real websites on the public Internet right now? We wanted to run that experiment as well.

And we did. We chose a website that has millions of users and terabytes of video that people access all the time, really useful stuff, rich integration with other content on the website, good navigation, currently using the QuickTime plugin. And we went to one of the developers of that website and we said, can you possibly start taking advantage of this right now with some changes to your JavaScript libraries probably? But once you've made those changes, well, how seamless is the integration? How difficult is it to achieve? The person that we asked is our friend Mike Seeple of the Apple.com developer team, the people who bring you all that great content at www.apple.com.

And he's going to let you know whether he had any success doing that with real content. Jeremy Farbaugh: That's right. Mike Seeple: And he's going to let you know whether he had any success doing that with real content on that actual website. Mike? Mike Seeple: Thanks, Kevin. Hi, my name is -- Sorry. Mike Seeple: I'm sorry. I'm sorry. I'm sorry.

Hi, my name is Mike Czepiel, and I'm a developer on the Apple.com technology team. Last year at the conference, we had the opportunity to come and talk briefly about how we use QuickTime on Apple.com to introduce and promote some of Apple's hottest products. Thankfully, the products keep coming, and so that just means we're putting more and more video online.

Often, the video on Apple.com is the person's first real experience with Apple.com or Apple products, and so we really want to make sure we make a good first impression for the widest audience possible. So it's really important that we take all the advice you guys have been hearing about earlier in the session to heart.

So we use H.264 for the best quality video we can. We hand-tune. We have a mastering code that hand-tunes all of our videos. We use RefMovies to serve the appropriate content, depending upon the platform or the user agent the visitors are using. Of course, we're also really dependent upon, we're also really cognizant of people coming over in varying bandwidth and different devices, so we put a lot of effort into making sure that they get a great experience, whether you're on a desktop, a 30-inch display, or you're using an iPhone on the train or something, trying to, over an edge connection, look at the latest keynote or something.

So of course, with all the video that we have on Apple.com, we're really looking forward to all the improvements that are coming to make our lives a lot easier. We're all born and raised on this web standard such as HTML and CSS, JavaScript, so we're really looking to make anything that makes QuickTime and movies in general a more integrated part of the page, a first-class citizen in the DOM makes our lives a lot easier. And so I'd like to show you a little bit of an existing part of Apple.com, our Find Out How section, where we've started playing with some of this and we can talk you through this If I could get to the demo machine, please.

Thanks. So this is an existing part of our site that you can actually visit live on apple.com/findouthow, where new and prospective Mac users can learn a bit about the platform and how they can actually do all the things that we've promised that they can do. And often, the easiest way to actually tell people about how to do anything is just to show them.

So this is a prime example of where we have tons of video that are just kind of in here. All of the video here is all-- Sorry, this-- --connections, email accounts, and basic settings. All the video here is set up as H.264 with ref movies. I'm familiar with the Mac.

And we're using just the plug-in here. Well, we'll get to that. This is a development version, like I mentioned. So under the hood, there's some special stuff going on. But basically, we have the QuickTime controller here. This is just one of our JavaScript libraries that we use to control. You can interact with it to scrub through the movies, as you've seen earlier. This is akin kind of to Jeremy's stuff.

So it's great that you guys are going to be able to play with this yourselves more easily. I don't have code to give you, unfortunately. So you can interact with movies. You can play them. You can pause them. And of course, you can look for the end of the movie and act appropriately for whatever you're trying to do.

Additionally, we're also using JavaScript APIs. We use things like the setURL function to actually change between movies. So you can just hop between movies just by selecting new ones. Additionally, since it's just this particular demo, like I mentioned, is a development version, that under the hood is actually using the video element and using the fall-through to display the video element in Safari or any browsers that accept that, or it's using the embedding objects to rely on the plugin. The really cool thing is that with a little bit of code and engineering on your part, you can wrap the differences because they're really not that extreme.

You can use the JavaScript interface to do things like pull to update a controller on a page or keep things in sync with the movie. You can use the DOM events for the QuickTime plugin, and you can also use the DOM events for the video element. It just takes a little bit of code on your part to provide a consistent interface to people that are actually building these.

So for instance, the team that's using the code here, they don't need to know about the underlying technology. So it'll act exactly the same as it does in Safari as it does in... Firefox. Exact same thing. This one's definitely using the plug-in. And if I can go to the iPhone, you'll be able to see that it's actually the exact same.

It's going to work exactly the same here on the iPhone as well. Can we have the phone up please? Can we get the phone displayed? Thank you. Okay. So it's pretty much the exact same thing where people can come in and they're presented with a poster frame and if you're serving your content correctly with the right mime types and the phone's able to detect that, picks up a little bit of the movies, realizes it can play it, it'll provide it with a play button that just pops you into full screen. So we're really excited about all the features that are coming in to the HTML5 spec in particular and I'd urge you to, you know, we're actually considering putting this in. I hope to have this stuff live on Apple.com really soon.

And so we're considering it. There's no reason you guys can't, you know, put this on your own sites and start tinkering with it and it's a great way to start getting involved in the HTML5 process as well because if you're starting to play with it, you'll hit some edge cases yourself and you can give feedback either to us or, you know, you can get involved in the spec process yourself. And, you know, we're really looking forward to leveraging all this, using things like different video codecs for different browsers to even, you know, further extend who's able to watch the videos on Apple.com. So thank you for all your work, Kevin. It's looking great. Thank you, Mike.

We're going to start publishing some of the JavaScript that you would use if you want to accomplish some of this on your own sites. Jeremy's example will be published shortly. It's not available as sample code and via the typical path right now. It's going to be made available by the same means that we currently distribute that JavaScript library for embedding that we talk about. So look for that coming real soon and you'll get an idea of the types of scripting methods that you will need to use in order to bridge these different technologies. Okay, told you about standards for encoding and container formats.

We told you that we were committed to standards in general and we showed you that we were moving towards adoption and promotion of the HTML5 standard for video and also the emerging CSS standards for styling, for effects, for mirroring, for gradients, all that cool stuff so that video can be very rich inside of your web apps. And that's basically where we are right now and where we're going tomorrow.

So, where can you get more information relevant to this particular session? Alan Schaffer is the graphics technology evangelist. He can answer questions for you or give you the right references that you need if you want to follow up. But there are additional sessions that you might want to be aware of at the conference this week that are relevant to various portions of my talk.

If you look very closely at this list, you'll note that only one of them occurs in the future. What enables me to put up here sessions that occurred in the past? Well, it's because we're going to deliver video of those sessions to our Apple Developer Connection members on the web in the future using the very same methodologies that we covered right here in this session. So, you'll be able to review those sessions in the future by that means. If you want to know more about Podcast Producer, that's still happening in the future. The arrow is pointing that way.

That'll be in Nob Hill tomorrow at 10.30 a.m. Where can you come and ask us questions? Many or most of us will be in the future. We'll be in the Graphics and Media Lab starting tomorrow morning at 9 a.m. Where we'll have a lab specifically about delivering media in web apps.

But following hard upon that, there is a lab that's specifically about delivering video to iPhone. At the same time, starting at 10.30 tomorrow is the QuickTime Video Lab runs all the way till 3. I'll be there for the whole time. Come ask me questions then. In addition, if you want hands-on experience with Podcast Producer, their lab is tomorrow at 3.30 p.m.

And finally, more information about the use of Final Cut Pro and the full suite of tools available in that package, Friday at 2, back in the Graphics and Media Lab. So, that was an information-packed session. I hope I raised questions in your minds about video preparation and delivery. What can you do that you didn't even think you could do before you came in? Come ask us how and we'll help you develop the things that you want to develop for the future. Start tomorrow morning in the lab.