The official title of this is Delivering Content via Interactive QuickTime. To be quite honest, I don't know what that means. What we're going to focus on here is programming QuickTime interactivity. So I'm assuming that a large part of my audience here are software developers, and ideally you will have written a QuickTime-savvy application. What I want to do today is talk about some things you might need to do to make your application interact. active savvy.

My name is Tim Monroe and I'm with QuickTime Engineering. So I want to talk about three things today. One, how to create some of those cool movies that Eric showed us in the previous session. Second, I want to look a little bit at how to use QuickTime API to do some additional sorts of things that you can't do just in the movie itself. And the third thing I want to do, as I said, is to focus on some things that you could do to your application to make them interactive, savvy.

I'm going to do these actually in the reverse order. I'm going to start off by looking at a little bit review about what interactivity is. By now, especially after Eric and Ben's session, you should have a pretty good idea of what it is that characterizes interactivity. Just to review a little bit, objects in the movie can respond to events from the user. Typically those are mouse and keyboard events. We can have different types of media interacting with one another. Objects in the media can be scripted or have intelligent behaviors assigned to them, and these objects can communicate with one another in the same movie or even between movies.

Again, this should be review. We have two intrinsically interactive media types in QuickTime, namely QuickTime VR and Flash. In addition, there are sprite tracks and text tracks can be scripted or wired as we saw in the previous session. And finally, we can use QuickTime functions to add interactive dynamic behaviors to non-interactive tracks.

So let's start by talking about, suppose you've got a QuickTime savvy application. Is there anything additional you need to do to make it work well with the kinds of movies we saw in the previous session? Tim Monroe The answer is yes, and I have outlined here four different levels of interactive awareness. First, I call interactive challenged. Second, interactive aware. Interactive smart. And then, if you're really good, your app can be an interactive genius.

So what are those? An interactive challenged application does not use a movie controller to guide the movie's actions. If you were in the 601 session today, we tried to drive the point home that if you have a QuickTime application that plays QuickTime movies, you want to use a movie controller to do that.

However, it's possible that even if you do use a movie controller, you can try to be too clever. And here's a couple of lines of code where the developer has tried to be too clever. What they've said is, well, if the movie isn't playing, that is to say, if getMovieRate is zero, there's no point in giving the event to MCIsPlayerEvent.

This was fine for linear movies perhaps, but for interactive movies it will fail miserably. A step up is interactive aware. And this sort of just means that you're not interactive challenged. You use a movie controller to guide your movies and you install a movie controller action filter procedure to handle movie controller actions.

I've got electrodes coming out here. Step up from interactive aware is interactive smart. What do you need to do to be interactive smart? Well, you need to handle key-up events, you need to support full-screen playback, you need to support the kind of intermovie communication we saw in the previous session, and you need to support what I call application messages. Each one of these I'll talk about in more detail as we go on.

Finally, if you're really clever and you've really got a great application and you want to be an interactive genius application, you need to do two more things, namely support skinned movies and support contextual pop-up menus. Let's go to the demo machine and see what I mean by skin movies and context menus. We've actually seen quite a number of skins today, so I'll do this quite briefly.

Here is a very simple skin movie. It's got a video track there. Unfortunately, I can't play the video track because I don't have clearance to do that. And I didn't get time to replace the video track. But I can grab in my Apple and move it around. I could click in here and start the video playing. So there's QuickTime Player. It's got at least that much interactive savvy that it can open and display skin movies. But of course, other applications can too. Let's drop that same movie onto This custom application and I also get skinned movie support there.

Now this is a sample application called QtSkins. That sample code has been up on the web for quite a while, but I'll warn you if you go get that today, it won't work under the current builds of OS X. There's some problems with it. I'm currently working on upgrading it, so in about a week or so, you can go there and grab the source code and have a skin-savvy app in no time.

Let's talk about contextual menus. To do that, I'm going to launch Explorer. By contextual menus, I mean the kind of thing you get in a Flash movie. Let's go to... "Just a random website that may have some Flash content on it. So here I've got some, I've got a Flash movie.

And how can I tell that's a Flash movie? If I control click in there, I get this nice contextual pop-up menu. This is also characteristic of the Flash player, but they added it so that you could sort of zoom in, do various other things in the browser." Now, to achieve the level of interactive genius, it would be nice to have that same capability in your application. And here again, I've got a sample application. I'm just going to take a sample flash file, open it in this application.

This flash file, when I use my arrow keys, the little ladybug moves around. This is a QuickTime movie. And again, if I do my control click, in this application, I've got the ability to create a movie. I've got the ability to zoom in. And I get that wonderful flash vector graphics that looks good at all magnifications. I can go back, zoom out, and pull up the About box for that application. Okay, back to slides, please.

So I'll talk a little bit later about exactly what you need to do to support skins and the contextual menu, but let's start at the lower level and talk about how to be interactive aware. The essential thing is that you use a movie controller to control your movies, and the main thing you want to do in that regard is to install a movie controller action filter procedure.

A movie controller is quite a polite piece of code in that before it does anything important, it tells your application about it and gives you a chance to augment its behaviors or, in fact, cancel what it's about to do. We do this with what's called a movie controller action filter procedure, which we install like this.

Now, there's several versions of this function. The one I'm calling here is MCSetActionFilterWithRefCon. What I get here additional is that I can pass an arbitrary four-byte reference constant. In this case, as you can see, I'm passing something called myWindowData. Presumably, that's a handle or a pointer to some data that's associated with this window.

What do I do inside my action filter proc? I'm going to be given MC actions as one of the parameters. And what I want to do, for instance, if I get a MC action controller size changed action, is to resize my window to hold the resized movie. In this case, you'll notice I'm turning false, indicating to the movie controller that I haven't handled that action. That is to say that I don't want to cancel it. If I were to return true, I would be canceling the controller size change.

So that's essentially what you need to do to be interactive aware. To be interactive smart, one thing we need to do is to support the key up event. Now back in 1984 when the Mac OS was developed, they thought key up events are fairly useless. Who in the world would want to do something in response to a key up event? So the default behavior for Macintosh applications is to filter out key up events. Well it turns out that in Flash, you can do things when you've got a key up event. So to fully support Flash, we had to add to QuickTime the capability to pass this key up event through.

So what you need to do is to enable your application to get the event. And how you do that depends on exactly how you're doing the coding. If you have a WaitNextEvent application, you'll want to call setEventMask to every event. If you have a CarbonEvent application, you'll want to register to receive the KEvent raw key up. Now what do you do when you get this event? You just give it to the movie controller and the right thing will happen.

The next thing we want to do to be interactive smart is to be able to play back movies full screen. QuickTime provides two functions, begin full screen and end full screen, that we can use to play a movie back full screen. It's actually a little bit trickier than just calling these functions because when you put a movie into full screen, you have to adjust various movie controller settings. For instance, you have to tell it, you're no longer playing in this small rectangle, you're playing in this big rectangle.

And it turns out that there are actually a number of ways that you can be told to put a movie into full screen mode. One of them is a movie can have a piece of movie user data of type PTV that indicates that this movie wants to be played full screen.

So when you open a movie file, you should look for that piece of movie user data. If you find it, make sure that it says to put the movie full screen and then do so. Another way that you can be told to put a movie full screen is by receiving various application messages, which we'll talk more about later. But let's look at a demo of that. Back to the demo machine.

So here I have four different files that I want to open. The first one is called Full Screen Atom. And this has that PTV atom that tells me that I want to put the movie into full screen mode. So let's drag that onto my sample application here. The movie opens and then immediately goes full screen and starts playing.

If I hit escape, I come back out of full screen. Another way a movie could want to go full screen is if the user says, "Please put this into full screen." For instance, if there's a menu item that says, "Play full screen," again, we'll go full screen. We can start it and play it through.

So there's two ways so far we can be told to go full screen with the movie user, Adam, and with a menu command. Let me illustrate what I call application messages. Here is a simple sprite movie. It's got a single sprite here on the movie. And I have it configured so that when I click on the sprite, it sends a message to the application that says, wouldn't you please put me into full screen mode? So let's do that. Click. And the application, being interactive smart, knows how to handle that message and put it into full screen mode. The final way we can be told to go full screen is by opening a Flash file.

Okay, so that's all I'm going to talk about for the moment in terms of interactive savvy. I've left a few things undone at the level of interactive smart, which I'll come back to later, and I haven't yet talked about the interactive genius level. So what, if we have an application, can we do within the application to add interactive behaviors to our movie? Well, QuickTime has a whole bunch of APIs that we can call to do various things. In this case, the interactive behavior is provided by the playback application. Now let's take a demo of that.

Here I have a QuickTime VR movie and I'm going to drop it on this Custom Application and you'll see that as soon as the movie opens up it starts automatically panning around. In addition, I've got this... the program is smart enough to know that when I click here,

So let's sort of deconstruct that and see how to do that using the QuickTime API.

First thing we need to do is install some sort of timer. Now what timer you use will depend upon your development environment. Cocoa and Windows have timers. In a WaitNextEvent application we could just be in our main event loop looking for tick count. Here I'm installing a CarbonEventLoop timer to fire after a specified delay. I'm passing into the timer an indication of what callback function to call when that timer triggers. And I'm also passing it in a piece of data here, my instance. An instance here is an identifier for the QuickTime VR movie that I've opened.

When the timer fires, what I want to do is get the current pan angle, increment it by some amount, here by one degree, and then set that pan angle. So I'm getting the pan angle and constantly incrementing it whenever the timer fires. You'll notice that I've got a global variable here, g if okay to auto spin. So I'm going to set the new pan angle only if that is true.

Why do I need that? Well, if I get a click in the movie, the movie controller will send my MC Action proc an action of type MC Action Mouse Down. What do I want to do on a mouse down? I want to say stop spinning, and then I want to set another timer to kick off in 10 seconds.

and you can see down below is my second timer which just sets that global variable to true and effectively starts the spinning again. So here with maybe twenty lines of code added to my application I've got some fairly interesting interactive behaviors. Let's take another example of some interactive behaviors I can get by using the APIs. I'm going to open this QuickTime VR movie in QuickTime Player. When I run my mouse over a hotspot and click it, I'll go to a new node.

and you can see that the default behavior of QuickTime VR is simply to jump to the new node. I find that kind of jarring, so what I did was write a special application that When I go from one node to the next, it's going to use the QuickTime effects architecture to do a nice smooth crossfade between the current node picture and the target node picture. So let's do that.

Here I am in my special app. I'll click the hotspot, and I get a nice crossfade when I go from one node to the next. If I come back, again I get that crossfade. Now notice I'm not using a canned linear movie here. If I, for instance, zoom in and change my pan angle and go to the new node, the crossfade happens from the current picture to the new node.

Back to slides please. Okay, how did I do that? Again, I took advantage of several callback functions that QuickTime VR provides me. In particular, I use what are called node leaving procedures and node entering procedures. A node leaving procedure is called when the user clicks a hotspot to go from one node to the other.

So when I get a node leaving procedure called, what do I want to do? Well, I'm going to call these four lines of code, which create an off-screen graphics world. They set the movie to draw into that graphics world, and then I call moviesTask to actually draw the movie into that off-screen graphics world. Nothing on the screen has changed. I've merely taken the current picture and drawn it into a block of memory that the user can't see. When I enter the new node, I want to do a similar thing.

I want to create a new graphics world off-screen, a new G world. I want to set the movie now to draw into that second off-screen graphics world. I want to call movies task to get it to draw. And now what I've got is two off-screen graphics world, one with a picture of the node I'm leaving, one with a picture of the node I'm going into, and I'll run this function, my run transition effect, to use the QuickTime effects APIs to draw a nice crossfade between those two. too.

When I'm done drawing that, I will reset the controller port and the movie port to the on-screen window and then tell the movie that I've changed it. Here's an overview diagram of what I've done. Step one, the movie's drawing into the on-screen window. Step two, I draw it into an off-screen G world. Step three, I draw the entered node into an off-screen G world, run the transition effect.

and then finally reset everything back to the window. So it may look complicated, but in fact it's fairly simple code, and I get a very nice effect from that. Now it occurred to me once I had done that demo that I could apply this same logic to a standard linear movie. And let's go back to demo machine and see that.

Now first I'm going to open this linear movie in just a run of the off the shelf QuickTime application. And I'm going to click down here in the controller bar. And again, the default behavior is just to jump to that new time in the movie. Again, that's OK, but we can do better than that. So let's open that same movie in this special application.

and then click somewhere at the controller bar and now we get a nice crossfade to that new time. Very nice effect. And I've got this little app set up so that I can perhaps, instead of a crossfade, do a random radial transition. This is good stuff. I could do this all day.

OK, back to the slides, please. So here's how I did the time jump transition. Again, I'm exploiting my movie controller action filter procedure. I'm now looking for an action of type MCAction go to time. The movie controller sends me this when it wants me to, lo and behold, go to a new time.

And in that case, I'll call this application-specific function doTimeJump, which essentially does the same sort of thing I did before by getting the two images and running a decompression sequence to make that effect between them. The only difference is that with the VR case, I was using node entering and leaving procedures to trigger it. Now I'm using a movie controller action to trigger it.

Now the QuickTime, I could go on and on for the entire hour here with these kinds of examples. Why? Because the QuickTime APIs are just full of these callback procedures that I can use to get my grubby little mitts into its workings. I could, for instance, use intercept procedures to add directional sounds so that when I'm panning around, I can adjust the volume and balance of a sound that's playing to make it look like the sound is somewhere in that node.

I could use what are called time-based callback functions to do things when the movie reaches its end or is at the beginning or is indeed at any time in the middle there. If I have a text track, I could define a text loading procedure that is called whenever a new text sample gets loaded. A useful thing to do there would be to look at the actual text which is being loaded and do something to that text. And on and on and on. Really the sky's the limit when you have the APIs at your disposal.

The limitations here, however, are that when I use QuickTime functions like this to do special interactive things, for the user to get that experience, they need my special application. So they need this special playback application. Now in some cases, this might not be a problem at all. If I'm putting out kiosks or a CD-ROM title, I can put the playback application right there where the user can get it.

However, it's kind of tricky if I'm in some other situation. For instance, if I want this stuff to work on the web, in a web browser, I can't ship a special playback application. I'm not about to write a web browser that does my special stuff. Well, even here we can get this capability to some degree.

The first thing we could do is take the intelligence out of the playback application and put it into the movie. We saw a lot of examples of this in the previous session. This is what I call scripting the media. Make the media intelligent, not the playback application. The second thing I could do would be to say, "I really want to do it my way, and lo and behold, the current scripting behaviors don't let me do that kind of stuff." I could write a custom media handler that had that logic in it, and then using the new third-party component download program, get that custom media handler distributed to my viewing public.

All right, so that's what we can do with the APIs. Now let's see how we can make the media intelligent. So what is the scripted interactivity? This is where the movie itself contains the logic for these dynamic interactive behaviors. The advantage of that is that that movie can exhibit those behaviors anywhere this QuickTime movie is played back, in the web browser, in QuickTime Player, or indeed in any QuickTime-savvy application.

QuickTime now supports two basic types of scripting. Since we support Flash inside of QuickTime, we can take advantage of Flash's ActionScript capability. Or, as we saw previously, QuickTime provides its own type of scripting called Wired Actions. Let's take a look at each of those. Back to the demo machine.

Let's look at a very simple example of some flash interactivity. Here we have a simple movie where I can move my cursor over these objects and drag them around and make kind of a neat little eggplant. Very simple interactivity, just being to drag objects around. Slightly more complex interactivity,

Okay, how did I do that? Let's go back to slides. This is the entire scripting that's attached to one of those objects in the Eggplant movie.

It says that when the mouse goes down, start dragging, and then the name of an object, and when the mouse comes up, stop dragging. Start drag and stop drag are two functions that are built into Flash. Essentially, they say, you've given me an object, I'm going to make it follow the cursor until you call stop drag, which happens when the mouse is released. Very simple scripting to get that cool drag ability to Flash objects.

The lines is slightly more complicated, and I won't tear this apart entirely, but as you can see, what's going on is they're reading the current position of the mouse, this underscore root dot underscore X mouse and Y mouse. They're doing a little bit of math or trigonometry here, figuring out a rotation for the lines, and then setting the rotation of that object.

Okay, so much for Flash. Again, Flash scripting is the kind of thing we could spend hours and hours talking about. It's sort of this limitless capability of adding cool interactive behaviors to objects in a Flash movie. More generally, we can add what are called wired actions to QuickTime movies.

We do this by creating what are called wired atoms and attaching those wired atoms to various kinds of QuickTime tracks. The wired atom indicates two things: what events are going to trigger it and what to do when it gets triggered. And a wired track is simply a track that contains one or more of these wired atoms.

Action wiring in QuickTime is just about as full-bodied as ActionScript in Flash. We have all sorts of programming constructs that we can use to create wired actions. We can define constants, have variables with changing values, we can set things to happen in a loop, we can branch according to, say, the value of variable, we have operands that allow us to query the environment and base our actions upon, say, what kind of machine we're running on. Thank you.

This is what a wired atom looks like. It's a hierarchical structure of these atoms. Every atom has three pieces of information associated with it. It has a type, it has an ID, and it has some data. Now in this case, the top level atom is of type KQT event type, and its ID is some value that indicates what event is going to trigger this atom. Here we have identifiers for the standard events, mouse over, mouse down, various key downs, things can happen on idle events, fairly rich vocabulary there.

Now what is the data associated with that top level atom? It is another atom, so I'm building up hierarchical collections of atoms. That other atom is of type K action. It's going to specify what action to do when that event occurs. The action here has a type which indicates which action to perform, and it may also need some parameter atoms that give you some optional parameters.

How do I build those? QuickTime provides a number of functions that we can use to build atom containers, that is to say things that hold atoms, and then within that container, atoms. Now here, this slide and the next slide is all the code minus any error checking that we would like to have to build a wired atom.

Let's just briefly run through this. First thing I do is call QtNewAtomContainer to build a container. Then I'm going to start inserting children into that container. The first child I insert is of type KQTEventMouseClick. So this event, sorry, this wired atom is going to be triggered by a mouse click on whatever object it's attached to.

Into that atom, I add an action atom. Then continuing, the action here is going to be K action go to URL. So when I have this finished, this atom, and I attach it to something in a QuickTime movie, when that object gets clicked, it's going to open my web browser and go to whatever URL I've specified. What URL was it? It was www.apple.com.

So, those two pages of code are letting me build a wired atom. Now my job is to put that wired atom somewhere in a QuickTime movie so that it actually does some good. The first thing we can attach it to is a sprite. A sprite is a

Okay, first let's look at a little bit of mouse following. Here's a sprite movie that just follows the location of the mouse or cursor and changes its image accordingly. Nothing exciting there.

We saw in Flash that I could drag things around. I can also do that with wired sprites. Here as I move that over my cursor changes and I can drag that sprite around inside the sprite movie. I can set sprites moving around, and this sprite has some wiring attached to it that's reading its position and determining where it is within the movie box, and when it reaches the edge, it's changing the direction of travel in one or both of the directions.

I can put several of those in a movie and I can make them interact with each other. So here you saw that when they bang against each other they bounce away at the appropriate angle. And finally let's go back to that is the creator of the new, highly-appreciated, and highly-appreciated, web browser.

He is the So there we see in a very simple way how to wire sprites. Back to slides, please. We can also wire text tracks. This is kind of a fun thing. How do I wire a text track? Where do I put the wired atom when I have a text track? A text media sample consists really just of a bunch of text followed by one or more text atom extensions. So I have the text in the sample and I can stick some additional data after that text. One of these types of additional data is called a wired text atom extension.

And within that wired text atom extension, I can have what are called hypertext item atoms that pick out various ranges of text and associate the dynamic behaviors with a text in that range. And the contents of the hypertext item atom is simply the wired atom or what I call the event atom. And here's a little bit better look.

So I The text sample consists of a length, some text, a text atom extension. The text atom extension can look like this. And you can see that down at the bottom there, I've added in my kqt event type atom. is a great example of how to create a video presentation that is as simple as possible.

Here's a text movie that I'm opening. It says, please take me to Apple or CNN. This has in that text sample the kind of structure that we saw there. I have two ranges here of hypertext. When I run my mouse over, you can see it changes from blue to purple, indicating that if I click there, something may happen. Again, if I click the Apple, it's going to launch my browser and take me to Apple.com. So exactly the same text wired atom that I built to put into the sprite, I can put into this text track and have it do the same thing.

Okay, so we've got wired sprites, we've got wired text. There's two other kinds of tracks that we can wire. One of them is Flash. I can't go into as much detail on Flash because the format of a Flash file is horribly complicated. The fundamental idea is that a Flash button or a frame can contain a list of actions, an action list.

This list of actions, or each action in that list, has a tag which identifies which action to execute. Macromedia was kind enough to reserve a tag for us that we can use to put QuickTime wired actions into the Flash movie. And what is the data that goes along with that action? It's simply the action of the atom container that we built earlier. So let's take a look at that.

Here again is our simple little flash button. What I've done is do a little bit of programming to take that Atom container that we built earlier and stick it into the action list associated with this button so that when I click the button, what's going to happen? It's going to open my web browser and take me to Apple.com. Tim Monroe So this exact same Atom, event Atom that I built, I was able to attach to sprites, text, and flash. And let's go back to slides. Now we'll want to attach that to some QuickTime VR.

Now, if we can take these wired actions and associate them with either specific hotspots within a QuickTime VR movie or with the node as a whole. So we could have it set up so that when a hotspot is clicked, the wired atom gets triggered and does whatever it's supposed to do.

Now, the Bottom two bullet points show you exactly where to put that Atom container inside a QuickTime VR movie. Again, the structure of a VR movie is fairly complicated, so I won't go into that in detail, but I will give you a demo of how that can work.

So here I'm opening this movie with QuickTime Player or I could open it in any old QuickTime savvy application. And you'll notice what did it do when it opened up? It started auto-spinning. So the same behavior that I thought a few minutes ago I needed a custom application to get, now I see that I can get with a smart movie. That is to say, by adding some wiring to the QuickTime VR movie.

So let's go back to slides and see how I built that particular atom. Very much the same thing I did earlier. I start off by creating an atom container. Into that atom container, I add an event atom with the ID kqtevent idle. So now I want the action to occur not when the hotspot gets clicked, but just every so often. Okay, whenever this track gets idled.

And I can specify the frequency with which I get idle events in my movie. Then to that event atom, I add an action atom. The action this time is going to be k action qtvr set pan angle. In this case, I need a parameter atom to say what pan angle should I set it to.

And here I'm sending it to my pan angle. What is my pan angle? Back here, it's one. So I'm saying, okay, let's set the current pan angle to be one degree that Now the way I get this to auto spin is by adding what's called a flags atom. And again, to go back, you'll see the flags I'm adding are K action, flag action is delta.

That tells the movie controller, when I said to set the pan angle to one, I didn't mean go to one in the panorama. I meant increment the current pan angle by one. So again, here with about 20 lines of code, I've built up a wired atom. I've attached it to my QuickTime VR movie to get this dynamic auto spin behavior.

Now one very nice feature of Wired Atoms or Wired Actions is that I can target various things in the movie to apply the action to. So every wired action has a default target that depends on the kind of action it is. Set pan angle, obviously the default target there is a QuickTime VR movie. It's possible though that I could target some other object and I do that by adding a target atom to my wired atom that specifies which other object to target. I can in fact even target objects in another movie. Let's take a look at this.

Now at the simplest level... Why doesn't that want to open? "I'm going to open a sheet then and open it in a real application." I don't know why QuickTime Player doesn't want to open it, it just doesn't like that movie. But it is a valid QuickTime movie file. Here I have two sprites. When I click on one sprite, the other sprite will rotate 90 degrees. You can see by clicking on the right-hand sprite, the left-hand sprite is rotating. Similarly, I can click this guy. So each sprite is changing the rotation of the other sprite.

Let's take another example. Here's an example of intermovie communication. So on this side, I've got my ripple sprite. It's just a sprite with a ripple effect applied to it. Over here, I've got another sprite, and I've got it set up so that when I click on it, the ripple sprite will disappear or come back. So here, messages are being sent from the one movie to the other movie saying, yo, Sprite, change your visibility state. And let's take one final example of intermovie communication. This is kind of a nice one.

Here I've opened three movies. Two of them are QuickTime VR panoramas showing the Donner Lake area in the winter and in the summer. And here I have a third movie which is a sprite movie. It's got some controls in it and when I roll over it, it's sending messages to each of those panoramas saying, "Wouldn't you kindly spin around?" And again, they do the right things here. I can zoom them both, zoom them both out. And if you want to get nauseated, you can just run your controller, your mouse back and forth over this and get a real nice earthquake effect.

Okay. Back to slides, please. What do I need to do to support intermovie communication? If you've got a QuickTime-savvy application and you've never heard of intermovie communication before, and you were to open those two movies or those three movies, what you'd find is that they don't communicate. Why? Well, the movie controller that's handling any of those movies needs some help from your application to find the target movie.

A movie controller really just knows about the movie that it's attached to. It doesn't know that you have other windows open that may contain movies. So what it does is say, okay, I've got this wired action here. I want to tell the movie summer.mov to change its pan angle, but I'm not summer.mov. I'm just some sprite movie. What does it do? It sends itself a movie controller action. of type MC_action getExternalMovie.

Because it knows that it's not just out there by itself. It's got an application underneath. So it's essentially saying, look, application, I need some help. I need you to tell me where this movie is. So it's going to either ask for you to give it a movie specified by a certain name or by a certain ID.

Your job, when you get this MC Action Get External Movie, is to go through your list of open movies looking for the movie that it wants you to find. How do you get a movie's name or ID? This is stored in some movie user data. So your job is to look at each movie, look at its user data, see if it has the appropriate kind of user data that gives the movie a name or an ID.

If you find it, then you tell the movie controller, "I found it. Here's the movie you want to send that message to." QuickTime 5 introduced a couple other movie controller actions that you should consider supporting, namely Get Movie Name and Get Movie ID. And these are where a movie controller says, what's my name? And it needs help from you again to look at the movie user data attached to the movie to figure out what the movie's name or ID is.

So again, if you want to be really good, interactive, smart with your application, support these three movie controller actions, and you will have thereby enabled intermovie communication. Now let's talk about application messages. We saw a couple examples of that earlier, namely where the QuickTime movie or the Sprite movie were sending messages to the application saying, I'd really like to go full screen.

So before we had movies talking amongst themselves, now we have movies talking to the application, telling it to do things. So application messages provide a way for interactive media to communicate with the application. Again, there are two types of application messages, what I call QuickTime application messages and Flash application messages. QuickTime application messages were introduced fairly recently in QuickTime 5.

And the way you generate one of these, or one way to generate one, is by using a wired action of type KactionSendAtMessage. So for instance, when I click on a button, that could execute the action, send this application message to the application. And the parameter there would specify which message you want to send.

When the movie controller gets that request, it says, "Okay, I'll send this to the application." How does it do it? It sends the MCActionFilter proc an action of type MCActionAppMessageReceived, saying, "Look, I just got this message. Would you like to do anything in response to that Now QuickTime defines five messages of which three are useful to us, but you can in fact define your own custom messages. So this would be a way for you to create some interactive content that communicated with your application to do special things.

These are the three messages that I think we all should be handling, namely enter full screen requested, exit full screen requested, and close window requested. So you could get a message that says I want to go full screen, or I'd really like to exit full screen, or wouldn't you really like to close the window that contains me? Now you'll notice that I have a comment here that says, be careful here. And I say that because at one point I wasn't careful here and it led to bad things.

How wasn't I careful? Well, I thought, okay, I'm going to get this window close requested message. I know how to close a window, right? I handle the close item in the file menu. I'll just call the same function I call there right here, right? Doesn't that make sense? Well, to me it made sense.

The trouble was the function that closes a window did a couple of things. First, it looked to see whether the data in the window had changed. If so, it would put up a nice dialogue to the user saying, do you want to change or save these changes or discard them? When that was done, it would then get rid of the movie controller, the movie, and finally destroy the window.

Well, I have some news for you. Destroying a movie controller inside a movie controller action filter procedure is not a good idea. You will crash fairly immediately if you try to do that. So, be careful there. So let's look at a QuickTime application message demo. Back to Demos.

Here is a simple one. Here again is a text movie. Again, I've wired up several stretches of text. In particular, the Apple here is wired to send the close window message to the application. And so when I click there, the window goes away. And I didn't crash because I was careful in my MC Action Filter proc. Let's open a couple of movies here.

Now I'm going to, this movie on the right here is a Flash movie again, and I have got it wired to send a message to the application that did this little earthquake thing. Let's do that again. You'll notice that the windows themselves are moving around, right? That's something that the application has to do.

That can't happen from within QuickTime. So this was a very nice use of sending a custom application message from the Flash movie to the application saying, let's do all this window jiggling and play a sound. Actually, if we play this thing full screen and then kick it off, that's kind of cool.

So actually these application messages are very powerful. We have some predefined application messages, but you can define your own custom ones to modify your application's behavior. So, Flash application messages are emitted when you have an action script that has the FS command in it. When the Flash Media Handler gets an FS command to execute, it sends to our MovieControllerActionFilter proc an action of type MCActionDoScript.

Associated with this action are two text strings that I can look at to see how to interpret the action or indeed what it is. So once again in our MCActionFilter proc, I see that I am receiving an action of type MCActionDoScript, and I will call my application function mydofs command. What does that look like? It's fairly simple. I just look at the command and the arguments that are passed to me, and if I get full screen, I'll call myEnterFullScreen. If I get full screen set to false, I'll exit full screen. Very straightforward.

Uh... actually we already demoed the Flash application message, right? When I clicked the Flash button, it sent a message to my application saying, "Let's shake all those windows around." Okay, now we're back with interactive genius. Recall that I needed to do two things to be an interactive genius: support skin movies, and the second one was support a flash contextual pop-up menu. We've seen many examples of skin movies today, and fundamentally, a skin movie is simply a movie with a custom window shape.

Now, if you want to be able to open skinned movies, you need some special code in your application. This is not something that QuickTime can handle. It really knows nothing about the kind of window that the movie is displayed in. It needs help from your application to do that. How do we do that? Well, I'll give you sort of the beginnings of what you need to do, because, of course, how you draw and shape a window depends on what development environment you're using.

In any case, what we need to do is first get information about what shape this window should be. I can do that by seeing if there's a skin track associated with the movie. So I'll call getMovieInTrackType and look for a skin track or actually a skin with the movie track characteristic. If I find a track that has skin data in it, I want to get the track media and also the track media handler.

QuickTime 5 introduced a new function called Media Get Public Info. And so to get the content region of the window, I will call it with the four-character code SKCR. To get the drag region, I'll call it with SKDR. So at this point, I have the shapes that my window is supposed to be in by making these very simple QuickTime calls.

How I then use that information, as I say, is platform-dependent. Windows has a function called SetWindowRegion that is very simple to call. I give it a region that has that shape in it, and it will draw the window in that shape. Carbon has similar capabilities, and so does Cocoa.

Now how do I support the Flash contextual pop-up menu? The easiest way to do it, again inside my MC Action filter prop, is to look for an action of type MC Action Mouse Down. When I do that, and if the Control key is down, I'll call my special application function that pops up the menu, tracks it, and does the appropriate things.

So let's summarize a bit. If you have an application that supports QuickTime movies and you'd like to be able to handle all these interactive behaviors, you need to do these things. Use a movie controller, that's a given really. Support intermovie communication, that's fairly straightforward, it just means iterating through your windows to look for movies that have a specified name or ID. Support the key up event, no matter how useless you may think it is. Flash doesn't think it's useless, so please get it and hand it to the movie controller. Handle application messages, we've seen how to do that either with Flash application messages or with QuickTime application messages.

Support skinned movies, that's a little more complicated but it's fairly straightforward. And finally, support full screen playback. If you do all of these things, your application will be up there at the genius level. People will not be amazed at how little you can do, but amazed by how much you can do.

Okay, let's reflect for a moment at what we've learned today. One thing we've seen is that we can use QuickTime's APIs to embed various interactive behaviors in our application. That is to say, to have our application support various interactive behaviors such as auto-spinning the VR movie or doing the earthquake when I press the flash button. The downside of that, again, is that for people to get these special interactive behaviors, they need my special application.

So one thing that you can consider doing is to export these interactive behaviors as wired actions. That is to say, take the intelligence out of your application and put it into the movie itself. That way these wonderful behaviors can be available everywhere that movie is played back. Tim Monroe Now I've seen several applications that are really quite good at putting together multimedia presentations, but they require a special playback application to play back that presentation.

Tim Monroe And in virtually all the cases I've seen, the kinds of interactivity that they're using could easily be moved from the playback application into the movie itself. Tim Monroe So that I think is a very good thing to try to do if you have one of these applications that allows people to author dynamic interactive media. Thank you.

Second possibility here, as we've seen, we can have the movies send messages to your application. Flash can do it and wired QuickTime tracks can do it. Well, this raises a very interesting possibility. You could have a QuickTime movie be the primary user interface for your application. One of the least fun things about programming on various platforms is getting the UI looking just right on each platform.

One very nice way around that problem is to create a QuickTime movie that has got buttons or pop-up menus or whatever, text input fields, all the things that your application needs to get information from the user, have that movie be the front end for your application. The advantage there is that the UI is going to be the same on whatever platform this movie is played back on.

And that's pretty much what I have to say today. Well, these we've all seen. No point in looking at that. There are three upcoming sessions that I think will be interesting to you. There's a feedback forum coming on Friday. There's a very good session, QuickTime for the web. If you're interested in QuickTime and how it works within browsers, that would be an excellent session to go to. And also we have a technical overview of MPEG-4.

For more information about all this, you can go to the website where we have a large set of documentation. Two other places that I especially like are www.blueabuse.com. This is put together by the Totally Hip People. It's a very nice resource for interactive QuickTime. Another location I want to mention is Mac Tech Magazine. In the past couple of years, they've been publishing a lot of articles about QuickTime, which I think will be very useful.

And sample code, basically everything I've shown you today is using off-the-shelf sample code applications or minor modifications of that sample code. So there's nothing I've done here now that you can't do in the privacy of your own home using sample code that is available now on the web. The one caveat, as I said earlier, is that the skin sample code is slightly out of date, and within the next week or so it should be updated to give you just the behaviors you saw there.

And finally, let me remind you that this week you have a very special opportunity to sit down with members of the QuickTime engineering team, bringing in your code problems and having us hopefully solve them or at least give you some insight as to how you might approach it. And that's what I've got to say. Thank you very much.