Good afternoon. Welcome to session 603, Media Integration with QuickTime. I'm Eric Blanpied. I work on interactivity on the QuickTime team. We're going to have a lot of interesting topics today, talking about QuickTime's interactive features and mixing media together. We're going to concentrate on these things from a content point of view, looking at a lot of demos and movies and showing something about doing some content authoring.

And that will fit well with the session which follows this one, when Tim Monroe will tell you a lot about the API work involved in making tools that do that stuff. We'll also spend a bit of time talking about third-party components, talking about a lot of opportunities for developers there, and some new things that will be coming in QuickTime 6 for interactivity.

The basic layout of the session is going to be, we'll start by talking a bit about linear media and how to mix that media and what features QuickTime has for that. We'll get into a lot of the interactive technologies that are in QuickTime, the various media types and other features.

And we'll talk a bit about server communication between movies and servers. And sprinkled throughout there, we'll have some things about some of the new features. and throughout it there'll be lots of demos and we'll be building various things. Helping me out with a lot of this will be Ben Bandt, who will help right now. Thanks, Eric.

To start off, I'd like to explain a little how QuickTime can help in the digital multimedia production workflow, even if you don't intend to ever deliver your content via QuickTime. Could I get actually demo two on AV, please? When talking about the other media technologies available for your use, one might characterize them, say, Reel or Windows Media, as a player application or as a file format or as an ad insertion mechanism.

But it would really be doing QuickTime an injustice to characterize it only as a .mov or only as QuickTime player because .movs are only one of hundreds of file formats that QuickTime can understand and open. And QuickTime player is only one of maybe hundreds or even thousands of applications that you folks have written that rely on the extensive QuickTime APIs to open all these file formats.

So, let's open a few files, shall we? So let's say you were making a cartoon. You might start out by hand drawing the animation and scanning it into your Mac or your PC, and you'd end up with a Photoshop file. Now while a Photoshop file is not the most ideal way to present an image on the web, QuickTime does understand Photoshop files. So what we can do is drop that into IE, and there's a full, maybe five-meg Photoshop file displayed in IE. IE doesn't know how to understand Photoshop files, but the QuickTime plugin does. With QuickTime 6 on Mac OS X, we also are able to understand PDFs.

which is just an amazing vector format for images. So you might take that Photoshop file, trace it in Illustrator, another professional vector authoring application, and QuickTime can understand the media at every step of the way. And with Illustrator 10 on Mac OS X, Illustrator files are now PDF compatible. So QuickTime can understand Illustrator files as well, and so can all of the applications that you have written that rely on the QuickTime APIs. So say BBEdit. can open and understand QuickTime files. Now BBEdit is opening Illustrator files for free. Now, you may be complaining that you don't get to charge for an upgrade, but neither do we. So, back to Eric. ERIC BLAINPIED: Okay. Get the slides, please.

So we've got an idea about what QuickTime is. We spent some time talking about it in a number of sessions today, particularly a fair amount about audio and video, which we're all used to. We've gone over that there's a ton of formats, still image formats and whatnot, and various audio formats.

But what we haven't spent much time talking about is the ways that you can combine these things, and the compositing you can do, and the kind of really dynamic movies you can make that have a lot more than just a little bit of stuff in them. How this works, we've got the basic container being a movie. I could really use a whiteboard for this. Ben, do you have anything? Yeah, I have a whiteboard. Could I get a demo 2 again? Yeah, that'll work. Let's see. Yeah, that's great. Okay.

Great. I got a whiteboard. I got some pens. So, the basic item, the basic unit of a QuickTime movie is the movie. And now a movie can either be a file or it can just be a movie in memory. It doesn't really matter. But most of the we think of it as being a movie file. Then a movie is made up of tracks. One or more tracks, theoretically an unlimited number of tracks, such as a video track, a maybe, Audio Track if we're going to make a typical linear movie here.

A media sample, it's the smaller unit within a track. You have one track per media type in a movie, or a minimum of one track. A particular track, like a video track, can have multiple samples of different durations drawn by different codecs. But you need at least one track for each kind of media you have. Like maybe we'd add a text track to this movie for captions, so it would be... Another type of track with samples that might be of a rather different duration than what else is going on with the text data in it.

Now the media used by a movie can be stored within the movie file or stored externally in other files such as other movie files or any kind of media file that QuickTime can handle, like the audio track here might be in a... And as long as the files stick together, then that's going to work out fine. But a lot of times to be more portable, you want to flatten them together, or do what we call flattening.

It's also in the UI, it's save as self-contained. But that allows you to pull all the data in and have QuickTime lay it out in a logical fashion so that the file is much more portable. So a good tool for picking apart a movie and looking at how it's made is QuickTime Player. So let's go and have a look at that. Thanks, Eric.

So let's just quickly pick some kind of a movie. Let's have a look at, what the heck, how about the whiteboard movie? So a lot of you probably know this, but the basic ideas in QuickTime Player, you can have a look at what's in your movie with get movie properties.

You get a nice little pop-up here with data about the movie at its top level, like its size, what files it references, and so on. And then you get an item for each track, which would tell you things like what file it came from, the format of the track.

All sorts of useful things. We can change the layering and graphics modes and whatnot. And we'll look at this sort of thing more. But QuickTime Player we need to look at as, or QuickTime Player Pro at least, as sort of the first level QuickTime authoring tool. You can do quite a lot of things in it, and we'll mess around some here over the course of the session and show you some of that.

So let's get back to the slides, please. All right, so, so far we've talked a little bit. We've got a number of different types of linear media that we think of. Such as your standard linear video. You may remember CinePack if you've been around for a while. Something from the golden age of digital video.

The first thing you need to do is to create a new video. You can use the "Band" option to create a new video. Well, that's a charming bit of video, but it could really use to be presented or enhanced in some kind of a way. And as we get going in the session, we'll try and give it a little bit of a punch and give it a little custom approach. Other sorts of things that you could do would be to leave CinePack behind and use something like the new MPEG-4 codecs, which you've seen plenty of so far, I believe. There's live streaming and canned streaming for various kinds of broadcast.

And all those still image formats. We've got music tracks, giving you MIDI. QuickTime's got a big sample library from Roland. We've got effects, and you've seen a bit of that in some sessions. Things like video filters, transitions, and whatnot. And tween tracks are a linear thing that we'll wind up using in a couple of ways that allow you to change properties of a movie over time in a smoothly interpolated fashion.

As far as making interactive QuickTime movies go, I right now am sort of thinking of the new MPEG stuff as being good and interesting codecs to have at our disposal. And one of the things that I really like is that for good-sized movies, the CPU requirements for a really good-looking MPEG-4 video seem to be a little lower than for what we were using before, like Sorensen 3, making it a lot better for combining with movies that may have a lot of other stuff going on that needs some processor time, too.

Alright, so here we have an MPEG-4 video with Flash layered on top.

The Flash is interactive, allows us to have some actions which start and stop it. It's composited on top with a nice alpha blend. And if you try doing this with a lot of other codecs, it's just going to bog on you, but this is playing at a very nice smooth rate. And the Flash scaled as nicely as the MPEG-4 did.

Compositing. So the tracks in a QuickTime movie are layered up from front to back, and they can have any number of different graphics modes each. Some types, like a text track, can actually have a keyed setting to make it possible to composite the text over what's below. And all of this is handled by the raw codec, well, a lot of things, but changes in QuickTime 6 to the raw codec and the ICM will make some of this stuff come out in a smoother, more accurate fashion.

You've heard something about media skins, but, and again, media skins often come into play when we think about interactive movies. They allow you to really customize the presentation experience of the content like nothing else, even though they're just a simple mask. By combining that with the various types of tracks that you can layer up to make a strong interface, you can create something which travels with the content.

Every place that QuickTimes plays, whichever OS you're on, it's going to open up and have the look that that movie's supposed to have, as opposed to something where you've skinned a player and you have no idea what kind of interface is going to pop up. And the authoring that we provide for it can be done with very simple XML. So we'll switch to demo two. So let's see if we can't make a simple skin quickly.

The simple and quick and dirty method is just create a two-color pic or gif or basically any image format that QuickTime can understand, export that, then you have Then you have your video file, or your VR file, or your Flash file, or your image file, anything that you want to skin.

Keep that around. You combine these things very simply with a straightforward XML file. And QuickTime knows how to understand this XML file format right here. Basically, it has a movie. The QuickTime project is a very simple project that is very easy to implement. It is a very To only be able to drag the edge of the movie, you might use that as your drag region. Let's open this in player. All we have to do is change it to a dot move.

And there she is. So let's switch to demo one and we'll do a little more advanced compositing and skins. Skins don't always have to be simple shapes. In fact, we encourage you to go wild. It seems like most of the rest of Apple wants to have you standardize on a particular UI or make things easier for the user through standardization.

Eric Blaznpied, Ben Bandt-Horn Skins don't always have to be simple shapes. In fact, we encourage you to go wild. It seems like most of the rest of Apple wants to have you standardize on a particular UI or make things easier for the user through standardization. Eric Blaznpied, Ben Bandt-Horn Skins don't always have to be simple shapes. In fact, we encourage you to go wild. It seems like most of the rest of Apple wants to have you standardize on a particular UI or make things easier for the user through standardization.

But we think that as far as media goes on the web, one of the strongest, one of the most important things you have to accomplish is to keep and hold these web surfers' attention spans because they're going through ten pages a minute. And the most important thing you can do is give them something interesting, a visual feast, and encourage them to explore. So let's do that.

Drag the monkey's head around different chakras. We're just playing different MPEG-4 files, MPEG-4 audio, using a simple script to fade in and out. So let's take a look inside this movie, shall we? We have the annotation data. The monkey was drawn by my roommate Brody. And we have a number of tracks. We'll get more into this later, but we have movie tracks that have an independent timeline. One of the specific fields that are important as far as compositing is concerned is its layer.

It disappeared. And it's graphics mode. There's straight dither copy. and other interesting effects. Probably not transparent is what we want there. Let's see. And we can take each of these tracks and paste or add scale from separate QuickTime windows and do some essential compositing right within QuickTime Player itself.

For more advanced compositing, there are a number of tools out there. Probably the most popular comprehensive one is TotallyHips Live Stage Pro for more of a UI as far as layering and timelines and whatnot. But you can pretty much do most of the essential compositing techniques right within QuickTime Player.

First off, there's the sprite track. Sprite tracks allow you to do animation using a pool of images over and over again, and we've got a lot of wiring that you can do with those. Got Flash, including Flash 5 support in QuickTime 6. We've got text tracks, which we can do all sorts of interesting things with.

You'll see a lot of those today. QuickTime VR, always a favorite. The movie media track, which Ben mentioned briefly, or movies in movies, allow all sorts of interesting opportunities. And then all the linear media that we've talked about and you've heard about in other sessions, it all comes to the party in this stuff.

To talk a little more about sprites, the basic idea of a sprite track is sprites are images that are animated over time. When the sprite track loads up, it's got all of its images at the beginning, a pool of images, which can be shared by as many sprites as you like. And those can be animated by tweens, like I mentioned, or difference frames to change their position or other characteristics, like graphics mode and all, at different points in time. And they can be drawn by any codec that's available in QuickTime.

Another very interesting thing about sprites is that they allow hit testing, so you can do mouse movement and things like that. We use wired actions to provide a large amount of interactivity, a lot of manipulation over the sprites themselves, and over a lot of the different kinds of media that are in a QuickTime movie.

They also can have their image driven, what we call an override, they can have their image drawn from any other track within the movie. So you can do a lot of neat composite things just within that sprite track. And new in QuickTime 6 is the ability to load an external image of any type that QuickTime knows how to handle into that pool of images by using a wired action. So you can do that anytime you like, and then you've got more images to deal with there. So we'll do a sprite demo now on demo 2 please.

So here is an old friend, a basic linear sprite movie. The fish has a number of different poses and positions which are just being changed over time, which sprite is being used at what time is changing and where it is changing. And over the top of it we have a sprite which is being drawn by the Ripple codec and that is taking mouse actions to give us the mouse click stuff. I think the Ripple codec got shown earlier as well.

As an example of the overrides, we have a movie here made by Francis Georges, who is a really... is a really talented QuickTime movie author. There's a simple movie here with a single video track. It's being used as an override for a whole bunch of sprites which are tracking the mouse and getting scaled and having their graphics modes changed over time. To make a really fun sort of thing, I think this was an award-winning movie in one of Totally Hip's contests. Then let's look at Charlie Tuna again. What's going on with that? We're not online.

So let's have a look at... All right, so we'll just make a new project. Oh wait, there we go, we opened it without that. We don't need that anyway. So, um... Here we've got, I've made a sprite track. And in that sprite track, We've got the pool of images I mentioned. The background, a number of different knobs.

The background image, you'll notice, has its codec set to use animation and alpha. This is a picked file with an alpha channel. So we've got a number of sprites that come from those. If we look at the stage, we'll see there's that sprite. And then also, we've brought in the Charlie Tuna video track.

and audio. And that's laid there on stage as well. And we don't see it right now because it's being overlaid by the sprite track, but the sprite track will show it through. And then we've got a skin track in there. And the skin authoring process here is a drag and drop your images into the window process. So let's have a look at this guy. So here it is.

Oh, that's not going to skin it, is it? So here we have the same sort of a thing, but... A little bit of control. We can start and stop. The frame is there. And the whole thing just gives you a little something more to your 20-year-old Cinepac movie. Or 30, is it? 40.

1961. Who knew Cinepac was then? And then we can get further, we can get really fancy with the scripting and do things like add features that we don't really see much anymore like vertical hold knobs, which I really do miss in this era of digital video. And that's the sort of thing that I don't really see in other people's players either.

Finally, there's the new image stuff that we've talked about. This is an extremely simple movie. I've got a text track up here that I can type in. I've got a little button here which takes whatever URL, a relative or absolute URL I put in there and loads that into the sprite right there. And we can go ahead and do something like open up one of these JPEG 2000 images.

Text can't really do a lot with making custom interfaces and highly interactive multimedia pieces without text. And luckily we can do all kinds of things with text in QuickTime. Text, as you just saw, can be editable right there in place. It can also be displayed anti-aliased. It can be keyed over other things. You also can superimpose it over things using the various graphic modes we've got.

You can specify hotspots in certain ranges of text, which can then have wired actions attached to them to do all of the sorts of things that we've seen in these movies so far. We have something called an href track that we can make where the text is full of samples with descriptions of URLs and times on them. And as a movie plays, that makes the entire movie a clickable thing that goes to the URL that's present at the time.

We also have, can use tracks as chapter tracks, which give you an index to a movie and allow you to jump around in time. And then, like I said, we got the wired actions that can be used either triggered from them or we can use wired actions to edit and manipulate the text in a field and do all sorts of things with it.

QuickTime VR is another one of the major interactive media types in QuickTime. It's image-based virtual reality, meaning that rather than doing real-time 3D stuff, you're doing it from photographs or photorealistically rendered images, which gives you an image quality that's really impossible with a real-time 3D kind of VR.

They can consist of panoramas, which allow you to explore a scene, or objects, which allow you to examine a thing. And these different types can be linked together to make multi-node scenes with hotspots that allow you to travel from one to another or pick up an object that's inside of a scene or things like that. And again, you can wire up those hotspots with QuickTime's wired actions and trigger all sorts of interactivity in the QuickTime VR movie or in other tracks that are in the same movie. And there are wired actions to manipulate the VR from the other tracks.

So we'll look at a couple of VR movies just quickly. Here's a pretty nice cubic VR panorama in Grand Central Station in New York. As you see, we can look all the way up here, and this is a pretty high-res movie, so we can really check out the details.

That's a cubic movie. And the other thing is that by combining them with other types of things, we can get into stuff like doing spatial audio, where the movie in this really nice panorama by Janie Fitzgerald has got a sprite track and some audio tracks. The sprite track is monitoring the pan angle and manipulating the volume of the audio track as you go.

Movie in Movie. One of the giant things about importing a movie into another movie and using it as a track is that you get an independent time base, or you get the option of an independent time base, which makes a lot of things possible that you previously had to do in some kind of a dedicated multimedia runtime. You can also load anything. It doesn't have to be our format. It can be anything that QuickTime knows how to play. And you can swap this content around dynamically by using wired actions.

So we'll go back again to demo two and look at a bit of movie and movie stuff in GoLive, which has a very capable QuickTime editor built right in. So we'll switch to the QuickTime authoring palette here. And we will create a new QuickTime movie. And I want to make that 400 by 133.

There's my movie. You'll notice that they've got the controller right on there right away. They've got a dual tab setup where you've got the preview and the layout exactly the way they present their HTML editor. And we can click this here, though, and get the timeline up, which is most useful for what we need to do. So I'm going to start by bringing in a bit of linear video. Drag a video track from there, and it asks me to go and get the thing.

Where do we put those? On here? All right, so we're going to bring in just something to have there in the timeline. Whether we're in preview or layout, we can see it right away. It's just a little bit of staticky stuff. Let's get rid of this color track that they put in there by default. And we'll look at the movie and make it loop.

Keep the standard controller. So this guy is going to just loop around with a bunch of static. I guess not in the authoring environment. So now I want to bring in a movie track. And that guy needs to have, um... I need to set what piece of content it's going to reference. And that's going to be-- actually, let me save this first.

We'll use that inspirational title. All right, so there's that. So the movie track needs its data. We'll click the sample down here. And we'll type in the name, which is--

showing us that sprite track there. And it's set-- what are the settings on the movie track? It's, we've got time, it's slaved, it's off, autoplay is on, we want to loop it though so we can keep seeing the motion and understand what's going on.

Notice that the playback head is not moving, but the animation is continuing. Now, let's change the layering. And we'll take the video track and we will change it to some other graphics mode, like how about... Something really garish. And now we'll start this playing. We've got two tracks that have independent time bases here.

Now, if we want to change what the movie is referring to over time, we can add something else. Let's go back to the QuickTime. I'll add a sprite track. You might notice the icon used for sprite track. This sprite track needs some images, so we will import something.

And when you import it, like I said, you can use any codec you like for your sprite, so we get to choose here. I'm going to try Animation Alpha. Let's see if we have an alpha channel around we could use. And then the sprite that's being used here is being drawn with a default sprite they add, and I would rather use the one I just imported, so I select my sprite, and I'll say image should be the dot. Go into Layout and change where it is on stage.

And let's... The QuickTime app is a great way to create a new video, and you can use it to create a new video.

and I have a question for you. I'm going to ask you to add another action and have this one be LoadChildMovie. That's the URL. It already knows what the URL I was setting it to was.

And hopefully this is going to work when we Save it and... Oh, wait, we've got to set the track to be something we can see through. So the sprite track gets set also to transparent with a black key color. There we go. So that thing is rolling, the static's rolling, we can scrub it all we want, and then when we click the button, we've loaded a different movie into the background, which has a rather different duration than the other things we've had. But still, it has a different time base, so it doesn't matter, it just keeps rolling on us. So that's your basic overview of movies and movies.

Flash 5 now. We've had Flash support in QuickTime for a while. In Flash 4 we had Flash--QuickTime 4 we had Flash 3 support. QuickTime 5 we had Flash 5 support. And now, since we're at 6, obviously we have to have Flash 5 support. The Flash support has always been really useful to give us a really compact media type for doing all sorts of interesting interface work and animation and things like that. The vectors just work very nicely and they scale nicely. They composite really well with other kinds of QuickTime. You just set that track to alpha and you get all of the alpha compositing that's going on in Flash is nicely composited on top of the other QuickTime elements.

The authoring is popular. People are very fond of their tool, so this provides a really nice authoring tool to make interactive QuickTime movies, because you can bring in your QuickTime movies into Flash and add interactive elements with Flash and spit the thing back out as a QuickTime movie. Eric Blaz: We've added a variety of wired actions that we can use to manipulate Flash elements, set Flash variables, trigger Flash buttons, and we can also attach our wired actions to the Flash buttons themselves in another author tool like Live Stage or something.

Oh yeah, we're still on slides. Right. So, um... New with Flash 5, we get some of the Flash 5 features. One of the things that's in there that people like is HTML text, allowing you to do a certain amount of styling your text, and XML handling, allowing you to import XML files with that, and you can use the style tags within there, so you can bring in formatted text, and you can manipulate it dynamically.

And the large amount of scriptability that came with ActionScript in Flash 5 is available within your Flash tracks. And because we can trigger Flash buttons and whatnot from QuickTime actions, you can do all sorts of neat things with using the Flash track as, you know, having complicated ActionScript functions that you trigger from other parts of QuickTime. So now we will go to the demos and see something about Flash 5. All right. Demo 2, please.

Now, like all the other file formats we've been dragging into QuickTime, Flash files just work. A nice way to lay these out would be in Live Stage. Here's our Flash track. Here's a sprite track. You can see if we look at the single sprite in here, we have somewhat of a complex script, but here's the key right here. Track of ID 1.

which would be our Flash track, ID1. We can set Flash variables from within QuickTime. This is Live Stage's QScript notation for Wired Actions, but it's a pretty seamless integration with Flash interactivity. So what we can do with this Flash file, The first step is skinning in QuickTime, something that you can't do in Flash, and exchange data with other QuickTime tracks or with a server or with a video, or you could be controlling a video track at the same time or even incorporate Flash overlays on top of VR.

the slides then. So, third party components are a topic that's been touched on a little bit, but I wanted to cover it some since we're here and there's a number of them that are adding interesting interactive capabilities to QuickTime. The fact that QuickTime is based on this component architecture makes it really easy for developers to extend it and add their own kinds of things to it.

This allows us to come up, people to come up with all kinds of new media types that we don't do at all. And those new things can mix with the other media just as we've been talking about the QuickTime, the QuickTime types. And, you know, Then also we've got the QuickTime Component Download program, which is a program by which developers, Apple hosts, The components from some developers on Apple's download servers, and then when a movie that uses that component is encountered by the player, if it doesn't have the component, it can check the server and offer to download the component dynamically for the user, which in most cases doesn't even require reloading the movie. Often with media types, you need to reload the movie, but with codecs and things, it usually just keeps going.

The third-party components in the component download program right now include Indio, Onto, Streambox, and Zygo Video, who all provide codecs for various sorts of purposes. And BeHere, iPix, Pulse3D, and Zoomify all provide media types that allow you to do various things. And Pulse and Zoomify in particular provide... Media types which have a lot of interactivity and have got wired actions around for you to do things with.

Which we're pretty pleased about. So here we have a 36K movie playing here in QuickTime. It's an interactive, well-designed for the web kind of thing. Using a video texture map here that's streaming off of the Apple website. So, as you can see, the possibilities for this are just great.

We want to switch over to demo two now, and Ben will show you a little bit about the authoring process with Axle. Cool. Well, we've had a 3D component in QuickTime in version 5, the Pulse component, but what makes Axle great is their authoring tool is cross-platform. It runs on 9, 10, and Windows.

It's also extremely easy to use. I really see it as something that even the most novice Flash animator could get their hands on and be able to understand. So you have your traditional 4-view 3D modeler here. Wireframe and Rendered View, and you can preview how that will interact right in the authoring tool. It's really amazing because pretty much every parameter in the application is animatable. and they have an excellent sequencer timeline view.

and I will be talking about the QuickTime app. And this should just blow your socks off. You can create the most complex interactivity in Axle without even writing a line of code. They have a notion of things called sensors or triggers and reactions, and you can have a lot of different You can have a sensor be triggered by pretty much anything, a key, the mouse. You can communicate with the host application as well, including QuickTime.

So here we have time sensors at various times. We play sounds. And the linkages between These sensors and reactions have various properties, how they're supposed to loop, how they're supposed to re-trigger themselves, essentially. create groups of reactions and develop very complex interactivity in no time at all. So this plays an animation that has a path constraint with a number of keys. Toggl's Parameter is visible and invisible. So it's really a powerful tool. So let's see what some people have been doing with this inside of QuickTime.

This is a movie. And the media is being handled by the Axial Media Handler. We're lucky to have Justin here. He did most of the work on this. The great thing about Axle is that they are not only a consumer of the QuickTime technology, but they're also a contributor. By writing a third-party component, they are bringing their own media to all of your apps that are built around QuickTime, and you get this for free.

Particles and other sorts of dynamics built in. Some particles going on. I think we can move the light around here. And I think this is my favorite. Don't forget. Check this out. Let's see if I can remember the controls. Space bar is jump. You can see some of the images and texture mapping going on here.

I'm not positive, but I'm assuming they're using QuickTime importers to handle a wide variety of images. and audio and video, and so on and so forth. So it's QuickTime playing in Axle, which is playing in QuickTime. Exactly. So you could have Axle playing inside of Axle in QuickTime in Axle. All right.

So then we come down to tons of media we've talked about. Well, so it's multimedia when you put it together. So let's mix up that stuff. Just zip right back over to the demo machine, go to demo two here, and look at some, uh, and Eric Blazow, who are the founders of the So here's something that is definitely on the pure engineering demo kind of style, but this shows a sprite track over on this side with a piece of MPEG video slapped on top.

and a Flash track over here. And there's a chapter track, so we can see the different chapters available. And these sprite buttons here change the chapter using some new features in QuickTime 6, some new actions for manipulating chapters. They also have updated the Flash. So the Flash buttons are changing the chapters with the same wired actions, as well as the sprite buttons. And the text that's being drawn here is using Flash's XML import and HTML style text handling, so we can format it and edit it outside of the movie just by fiddling with that here.

So let's just try changing some color of some of that text. Open up the movie again. I forget which sample. Yeah, we just changed that one to white. So that's production-wise a pretty useful thing within the Flash 5 support. Another one to get into a slightly more complex sort of movie.

Here's a... A skinned movie done for marketing purpose for the new Mini. They've got a...

and the little close button. So there we're getting into the sorts of things you can do when you start taking a bunch of different kinds of QuickTime media and layering them up together.

Let's speed on ahead, back on the slides, to briefly talk about intermovie communication. Basically, you can send messages between movies. Any particular object in a movie that can do wired actions can target a movie within the hierarchy of another movie to create all sorts of opportunities with manipulating the various properties and whatnot.

Another thing that we do a lot of interesting things with is XML. As you may be picking up from a number of these places, XML is the text format where QuickTime is sort of standardizing on to do various kinds of descriptions of things. XML in QuickTime is... As far as the interactive QuickTime goes, it happens with QT lists. QT lists are just hierarchical variables that you can have within a movie or a track. And they have this sort of element attribute sort of appearance that you see in an XML file.

But what's neat about them is that you can import them and get them out as XML. So one example of doing this is as config files that you load up to make a movie, a generically authored movie, open to have different sorts of appearances depending on what XML is around.

You can also pass them in now through the plug-in. So the embed tag in an HTML page can have an XML list in there for a QT list. And when that movie opens, it's going to have the QT list loaded into memory and ready to be worked on with your wired actions.

and then you can do server communication with them. You can select any portion of the QT list in a movie and fire it off to a server somewhere and deal with it with your CGI's on that end and do some really neat stuff, which we're going to get into trying to talk about.

So I'm going to show some demos here. I'll just show you a little schematic of the basic layout that we'll get to. Basically, we'll have a single movie, which is looking at a database on a server we've got running over here. And there we'll have some Perl CGI's, which manipulate the data coming in from the movie.

We'll do things like check a MySQL database and send back information, formatted XML, out of the database to allow the movie to then do what it needs to to grab things like other movies and graphics and all. to do its presentation. So let's go back to demo two here. Ben's got up the XML that we see.

Using PHP, we can look at the database that we've got here. So, I mean, this is one of the really fun things about this is that, you know, I didn't know much about any of this stuff a couple months ago, but just downloaded all this open source stuff and installed it on my OS X box and messed around with it and get it working.

So we've got a number of movies here that have got, the database has got a movie name field, the URL to the movie, URL to a thumbnail, a description if we want to use that, and a hits field that we could use for monitoring how often people are viewing the movies and things like that. So let's... have a look at a movie which is going to talk to that database.

So here, this movie, that was kind of quick, let's do it again. This movie goes and hits the database. On open, it sends a server query to the database and it gets back the list that you saw there as XML, which becomes a QT list. And then it does things like imports the, a set of JPEGs and Pings and things that I'm using as new sprite images, which get stuck on these sprites so we can scroll them around. And we can pop open a sprite and view it there and maybe even dump the right text that comes out of the database as well. And we can even go ahead and load that movie into a, right into a movie track here.

or we've got some RTSP streaming there too. That same server's got the QuickTime streaming server on it. And give you the idea of the sort of a dynamic thing that, oh, well. Sort of dynamic applications you can be building once QuickTime 6 is finished. Something that actually you could be doing though with QuickTime 5, here's a project where There's a movie with a bit of XML which

Fire up this movie. Here it is as a skinned movie.

It goes and instead of its built-in graphics, it's loaded all of these graphics in a sort of a, you know, HTML web page style setup. It's loaded in its external graphics and built an interface for us which has a lot of sprite images that do their thing, including the Closed Captioning by The Closed Captioning Project This had the same movie, the same skin point of the movie, different set of graphics. We've edited the text file. We open up the same movie, and we've got a completely different look. The skin changed, the graphic changing in there, the buttons are different.

But again, we've authored, we've done the hard work of authoring one interactive movie once. And then when we needed to change the content it was playing, it was all Photoshop work and a little bit of BBEdit. There was no more of that live stage stuff that people sometimes get scared away by.

That's going to have to happen over here, right? Can we switch back to the slide machine? And we'll look at a movie that does a number of interesting things. Here is the website for Commanding Heights. So here's something that's actually shipping that somebody's really using this stuff for.

So this is WGBH in Boston has made a really sophisticated website to go with this program that they aired last month called Commanding Heights. It's three two-hour episodes. They've put the entire six hours online encoded at various rates, and they've presented it within... Within some interactive movies. Is somebody else using the entire pipe at the convention?

All right, here it is locally. Not all the features work locally. Whoops. So... the... They've got the whole thing laid out.

and Broken Down into Chapters. So there's a different episode for each of the two-hour episodes, and those are all divided into these chapters. They open up in these... In these movies, which have completely sprite-drawn backgrounds and all, with their own interface to play, Can we get demo two, please? I was approached by a government official who had watched the movie. Okay, so maybe I'll do a step or two of that again. So we go and we launch the movie here. And here we have a... A chunk of the video playing.

They've got their own controller down here. They've got features like captions, which are being done by QuickTime text track. They've got their own volume control. You'll notice that bits of information pop up over here on this side as we go. These are links to other sorts of information, like we can open up another text window with the entire interview with that guy.

If we scrub ahead a little, we'll see that we start to get some-- Some interactive information here, which would link into other parts of the thing. We also noticed that this little movie popped up. This little window contains a movie they call The Brain, which has information for navigating between the movies. Let's look at the time map. There's another interesting feature of it.

This is a flash track we can zoom way in on and then navigate around. And as we scrub the time here, we see the dominant economic model for the time that's being viewed. So there happens to be, I realize, a flash point here. Then we have cross links, which are based upon the information here, which would take us into their country reports. Here we have the time map.

which, um, through the intermovie communication going on between all of these pieces, it will be populating, um, with the appropriate data for what we're looking at, although I think this may be one of the things that doesn't work locally. Yeah. All right, so we don't get the text data that we wanted that comes out of XML.

They're using a lot of QT lists in XML here as well. So, um, the cool thing about that is there they are, um, needing to show a lot of information, uh, way more information than what actually went into the show, and they're able to give a whole, uh, a whole set of different ways of looking at the information for anybody, uh, anybody's own style of consuming it.

And what was nice and worked for them is that they created this country report movie once, essentially, and for each time, or each time in the timeline, and each country in, uh... and Eric Schultz, who are the founders of QuickTime, are the founders of the QuickTime platform. They are the founders of the QuickTime platform. Eric Blazpiek, Ben Bandt-Horn, and Eric Schultz, who are the founders of QuickTime. Eric Blazpiek, Ben Bandt-Horn, and Eric Schultz, who are the founders of QuickTime.

It appears that we're getting towards the end, but we do have some ambitious stuff ahead of us, so let's do a quick overview of what we've done, and then we'll try and build something that we hope will work well. So, in conclusion, only QuickTime. supports all of these tons of media types we're talking about.

A lot of things that we have within QuickTime and a lot of things that are just played by taking these other formats in. And only QuickTime can combine them in this elegant fashion and make it easy for them to interact with one another and communicate with the outside world through server communication and things.

Only in QuickTime can you deliver content like that that's going to look the same no matter which QuickTime playing platform you deliver it on. So to get into our closing demo, we'll want to switch Demo 2 to the screen on my left and Demo 1 to the screen on my right.

Okay, so we've seen most of the major forms of QuickTime in this session, or interactive forms of QuickTime in this session. Let's have a look at some of these. We've modified a couple of these movies a little bit so that when we say... We'll open up the sprite fish over here. This guy's swimming around.

Now, the modified version of this movie from what you saw before has got a little bit of a script in it which populates the database. Whoops. and has put itself in with a link to where it is and a thumbnail for it and what its text might be. And it'll sit there playing. And then maybe we'll open up the linear fish, which should also go into the database.

Some actual content that's skinned. That little guy follows us around. All right, now I'm going to, over here, I'm going to... Go somewhere else, say... Did you change that or not? Oops. So, um, I've got a movie here which I can open up, which will look at the database and populate itself with the movies that we've got open. So there's the linear fish and the sprite fish.

Well, okay, so we're being ambitious. Something's not happening with the Axlefish. As I feed them, we'll notice that it's feeding the fishes on screen. Charlie Tuna went to try to get his food when I fed him, but it didn't work. This is happening, although I'm opening it on one machine, it's going out to the server and then coming back. If anyone's out there with AirPort, we've got AirPort going on in here. You'll notice there's a session 603 AirPort network.

And if you do that, the password is WWDC. Go to the URL here, which is 10.0.1.4 slash index1html, and you can open that movie. And there should be QuickTime 6 and 5 versions in there. So if anybody's installed Jaguar, they'll get that by using a ref movie. It doesn't matter which. You don't need to pick anything. You'll just get the right one.

There's the Flash movie. Now, one thing we haven't done here is the live stuff. So let's try and make it even more complicated and get that involved. So I'm going to need a little help, though. Can I volunteer out of the audience? Are you over here, sir? No, not you.

What's your name? My name's Peter. And are you new to Mac? You know, I used to use it a lot, but the last couple years, that whole flying dock, genie thing, it was kind of intimidating, so I'm a little out of date. So what platform do you work on most? XP something.

Yeah, right. Have you done anything with QuickTime before? Yeah, yeah, I have some experience. Actually, it came pre-installed on my BIOS, so yeah, I've got some experience with it. All right, great. Well, I'm going to give you a PowerBook and a Broadcaster here, and we'll see how it goes.

and a camera and a fireware cable and knock yourself out for me. I've got a lot of hands there. Thanks. We can... Has anybody managed to get the fish feeding movie up? Cool. So we got that going. Let's see what the database is looking like. Sure enough, there's a few of the movies. And goodness, we've had a lot of hits happening there.

So that fish is just going to keep feeding, isn't it? Now don't feed the flash fish too much. It'll die. It's serious. That's a wise thing to do. How's that going? Uh-oh. Did I tell you? Did you click broadcast? Should I hit broadcast? . Acts are hard to use, man. . So let's revive him.

Okay, there we go. So now you're probably going to want something to point that at, right? How about this? Now I think we've got another movie to open up for that. Do we? It's over here. Sure. There we go. Oh wait, we need a way to feed him. So let's grab some food.

We've got some electrodes attached to the server here. Could you get a little closer maybe and roll up your sleeves?

Now, if you close the movie head here and reopen it, it ought to... Yeah, it's got the live fish one in there as well. So I don't know if anybody wants to try to get Peter to feed the fish, but... I guess someone is. Well, that pretty much concludes our demo. Thank you very much. Thank you, Peter.