that you're seeing revealed for the first time at this conference. We're going to describe what those developments are, review some of the things that you've seen in previous sessions, and amplify on them as well. We're also going to preview some of the more in-depth technical sessions that follow mostly tomorrow, and I'll be directing you to what those sessions are and what content they'll contain. So this is the omnibus for QuickTime in Tiger. What you'll learn is what QuickTime offers to you, the application developer for media services you can use.

We'll give you some examples of applications that already use QuickTime so you can get some ideas of how you can use it as well, talk in depth about what's new and improved, and for the first time on any stage, we'll talk in depth about how you can take advantage of new ways to use QuickTime in both Carbon and Cocoa apps starting in Tiger. year.

Where does QuickTime fit in on Mac OS X platform? You've seen several versions of the architecture diagram for the entire OS this week already, and you've seen QuickTime mostly among the imaging modules in that diagram. Here I've blown up a portion of that architecture diagram to show you that QuickTime sits on top of imaging, video, and audio services on the platform. It's an integration layer that gives you access to digital media services. And it in turn makes use of these lower level services that also have APIs that you can use.

We're clients as well as you. To amplify, what does QuickTime offer? Not just playback. Most of the time when you come to a conference like this and you see QuickTime, we're going to play you really cool content. This session goes beyond that to show that the services that QuickTime offers are complete. It's a full media architecture. Creation of media, editing of media, capture from external devices, and delivery via the web, via streaming, via CD or DVD-ROM. QuickTime offers all of that.

Now, from time to time, we get into discussions about what we mean when we say QuickTime, because QuickTime actually has been around for a while. It was introduced in early 1991, originally available for system Mac OS 607 and system 7. I just had to check with Jim because I'm never sure about which 6 it was. But in fact, it was 607. Over the years, the term QuickTime has been applied to several different things. I've got a list of what some of them are now. QuickTime is, in fact, an end-user product. QuickTime Pro, which is available from Apple's website, you can buy from there. QuickTime is also a media container format known as the QuickTime movie file format. And, in fact, it's achieved great success in the industry as the basis for the MPEG-4 file format and related file formats, 3G as well. Container format.

We can put video and audio in there and some other things, too. QuickTime is also a media solution. There are several products that work together, QuickTime Pro, QuickTime Broadcaster, QuickTime Streaming Server, and the QuickTime Plugin for the internet browsers that allow you to produce and deliver media. But what we're talking about specifically in this session is the technology that you can use in your applications to put media services to work for you. There are several sets of APIs, C APIs, that QuickTime has offered over the years, and And you see some of their names here. Movie Toolbox, set of high-level APIs for playing and editing media. Image Compression Manager allows you to compress and decompress still images or sequences of images. And a number of other component APIs as well, some of which I'll talk about later in the session.

That'll be a theme that I'll come back to. Actually, the themes of this session, if you must know, are what did they say we were? They were classification and reordering. So every time you hear something that relates to classification and reordering, if you could make some sort of sign that you're still awake, that would be good for me. Because it is 5 o'clock on Wednesday, and this is a tough slot to fill. But anyway, this session is about the API. So with no further ado, let's talk about QuickTime APIs at work in existing applications today. So on demo one, I have one of my favorite applications, particularly when I'm away from home, Mail. Mail is a mail application.

It does wonderful stuff. It allows you to find your email messages, to file them, and to read them. But one of the other things that Mail does is it allows you to exchange media from one email server to another. If I could have demo one up, please, that would be handy. Thank you.

So here I have, what have I got in my email now? I just launched this app a minute ago. Ah, something from home. In Mail, and in fact in a number of applications, you can put media, digital media, in line with other content. Mail uses the QuickTime APIs in order to play video and audio. Here I have-- let's see, what do we got? Hi, Dad.

That's nice. Thank you. I'll be home soon, kids. Don't worry. Well, that's just one example of an application that allows you to embed media together with other content. If you have a document-based application and you want to use digital media together with other content, use QuickTime. It's a perfect solution.

Let's see, another application that uses digital media is Keynote. Keynote goes a few steps beyond, however, what an application like Mail does. Sure enough, I can create a new presentation in Keynote and I can take some digital media that I happen to have handy and I can drop it in there and I can play my digital media content inside my presentation. But this is actually not… All that remarkable, this has been done in a number of applications for a number of years. Keynote takes it quite a few steps beyond. What I would actually like to do here is create a new presentation, and this time I want to choose one of those interesting themes, the blackboard theme. And I want to take that same movie that I put on my other slide, And let's make this a blank master. And I think what I want to do here is to make my movie occupy the entire slide. Let's see, that would be 1260 by 768. And I want that positioned at 0, 0. All right, so there I now have a full screen slide with a movie in the background.

that's not quite what I want. I really want it to be in the background. I want to have my text on top of it. The title is, let's see, my fifth grade science project. And let's The red-tailed hawk. This red-tailed hawk has been showing up in our backyard for the last week or so. And one morning while I was drinking my coffee, there it was. So I just grabbed my DV25 camera, and there you go, live from my backyard, actually, stored the red-tailed hawk. But what I really want to do here is have some bullets. I'm going to cover the habitat of the red-tailed hawk. I'm going to cover the markings. I did very well in the fifth grade, by the way, in case you can't tell. and I want to cover the call of the red-tailed hawk. And I want to give these items, oh, I want to give them some... some builds. I'm going to have some drop-ins of my text. This is all very nice. And this is all going to happen over my video.

But the video, you know, it's kind of bright. The sun was shining the day that I shot this video. So what I really want to do is to dial down the opacity of that video, because I really want that to be in the background. Then when I play my video, it's going to show up like this.

You can see in the lower left there a preview of my presentation. Then when I click, my bullets are just going to come in right over my video, which plays in the background. But you really want to see this full screen, don't you? Thank you. I knew I was going to get terminal into this presentation.

All right. This is what happens when it's 5:00 on a Wednesday. Doesn't have to be the WWDC. All right, now we get it. Now when I play my presentation-- oh, I got it wrong. I'll show it to you downstairs in the QuickTime lab. But this movie will play full screen with the bullet points dropping down.

with full resolution and full frame rate. So Keynote goes further than an application that simply embeds media. It combines digital media with other features that it has. It's support for transparency. It's support for builds and transitions. Digital video and audio work seamlessly with that in Keynote. And it's all done by means of QuickTime. OK, back to slides, please.

So I hope I've convinced you that it's worthwhile for you to use QuickTime APIs in your applications to gain that sort of feature that I demonstrated just now. If I had managed to convince you of that only two years ago at WWDC, your application would have gained value without you having to do anything between then and now. Because the APIs that you would have put to work in your application then would have given you access to an increased number of media container formats, video formats and audio formats as new versions of QuickTime have been released by Apple. In just the QuickTime 6 series of releases, we've added support for MPEG-4, for 3G, for JPEG-2000. We've added support for external devices, better support for DV, support for IIDC cameras. That's the Industrial and Instrumentation Digital Camera Standard. Glad I got that right after the failure of the keynote demo.

And we've improved our APIs also in the QuickTime 6 series of releases to make them more accessible to you. Our C APIs are now using modern constructs to refer to files, to refer to other things that are common with other APIs on the platform. So that's what we've been doing. While we've been doing that, what's been happening in the industry? Well, we kind of noticed that video encodings of digital video, not DV digital video, but various formats of video that are delivered digitally, are becoming richer. The quality is becoming better without the data rate going up. And new and more complex means of coding of video are being applied to gain in video coding technology.

Well, what are we going to do to respond to that? You've already seen that we're responding to that by adopting H.264 for QuickTime. What about audio resolution? Audio resolution is becoming ever greater in production and also in delivery to your auditors. Higher resolution audio is becoming more common.

Graphics processing. We found, and we've mentioned several times at the show, that the graphics processor, the GPU, that is in every machine that we sell, and also many of the machines that are sold by other vendors, is valuable not just in the original domain for which those things were developed, but also for other types of applications. Finally, what else is going on in the industry? We're noticing that you guys writing your applications are using these application frameworks and getting a lot of leverage from them. Nice of us to notice that. What are we going to do in response to that? I'll tell you in a moment.

OK. Tough crowd. All right. So let's go through those topics one by one. What's happening with video? By way of analogy, I intend to explain to you how video encoding has become more complex and what we need to do in QuickTime to be able to support more complicated encoding. So I can't get you to respond. I can't make it interactive. Let's play a game. We're going to play which one of these is different.

A flip book, by which I mean a book with each page of which has an image either printed or drawn. And as you flip through it, you get the effective animation. A reel of motion picture film, actual celluloid. They still use it in some theaters. Each frame is on the strip of film. DV25, what's that?

Well, that's the most common format of digital video supported by your DV camera that you can buy down at Good Guys. Or whatever guys you happen to have in the area that you live in. Amen. Finally, step-by-step directions to your local library. You may think that this is the one that's different because it doesn't involve movies or video.

But wait. The step-by-step directions could be delivered as a series of images, even as MapQuest can provide. So which one really is different? Well, it should be obvious to you by now that three of these things are sequences in which the items are each independent of the other items. And those happen to be the first three. The flipbook has pictures which are independent of the other pictures.

DV25 is the digital equivalent of a flipbook. Each frame is independent of every other frame. The one that's different is in fact the step-by-step directions because every direction depends on what came before. You already have to be on the corner of main and third before you turn left and go two Now we're going to play the advanced version. Which one is different this time? Narrative, analepsis, and prolepsis, which of course are the terms for narrative flashback and flashforward.

Oh, thank you. Did you just scratch your head or did you mean to? No. A schedule of automobile maintenance. Thanks. The Romantic Poets, by which, of course, I am referring to the Romantic Poets in the poetic tradition, as described by Harold Bloom in his 1973 book, The Anxiety of Influence, in which he posited that a young poet can actually seek to gain priority over his predecessor by means of several tropes, which he describes, one of which is the trope of substitution by substituting an earliness for a lateness. But I'm going to give it away if I keep going. MPEG-2 video.

Another type of digital video, MPEG-2, another type of digital video coding. Finally, a presentation based on keynote. Well, I'm sure it's all clear since I belabored the point on the romantic poets that what's different here is the schedule of automobile maintenance. Because in that case, you have a sequence of things, each of which is only dependent on the things that have come before. You only go to have the 100,000-mile maintenance after you've already done the 75,000-mile maintenance, right? But these other things are sequences in which items may have dependencies on things that come before or things that come after, or at least the romantic poets hoped that this was true.

So if none of these analogies was effective to you, I will appeal to your stomach, since it's getting close to dinner time. After you learn what all the sharp implements in the kitchen are for, the culinary institute, they eventually reveal to you that you do not necessarily prepare courses for a high class multi-course meal in the same order in which you serve them. And why is this true? They have, they take, well I don't know why I never went to the culinary institute, but I do know about this. The same thing is true of modern video coding. Video frames are encoded and decoded in not necessarily the same order in which they are intended to be displayed. And why is this a good thing? Well, it enables us to get better quality out of comparable frame rates. Let's talk about an example. Suppose I have a sequence of video which starts with one frame and ends with another frame, and all of the intermediate frames are a wipe effect. It just wipes from left to right from one image to the other. Well if I need to encode that sequence of frames, each frame revealing more and more of the final image, then if I can only depend on images that have come before, with each frame I have to waste some of my bits, well, I have to apply some of my bits to the portion of the final image that has become revealed as the wipe progresses. But if I have a video coding which allows me to depend on both the first and the last Well, then that's much easier. The wipe, the intermediate frames, will require many fewer bits in order to code the differences between the first and the final frame. Well, that's basically how we're achieving higher quality with comparable frame rates in video. The bad news is when we shipped QuickTime 1.0 in 1991 for System 607 and 7.0, we did not accommodate this model for video coding in our video support. What I like to say is that there were no master chefs working on QuickTime 1.0. They only came along later.

But in QuickTime 1.0, as you can see, we had this limitation. And this limitation was reflected by the APIs. So you could only compress video for which frames would be completely independent or which would only have dependencies on frames that you had previously presented for compression. But in QuickTime 6.6, the version that you have in your Tiger preview, that SDK that I think they gave out earlier this week, we finally have support for video frames whose decode order and display order can be distinct.

Now, this is a great thing because it allows us to play modern video formats such as H.264, but it allows us to do more than that. I will say we support IP and B-frames for those of you who are technical. But the reason that this is wonderful and amazing is that we have applied all of the richness of the QuickTime media architecture to this type of video encoding. So let me show you over here on demo one what I mean.

Let me open up some of my Hawk movies again. And I'm going to open them up with QuickTime Player 651, both because I want to make a point and also because the new player that's in the tiger seed doesn't quite run the AppleScript that I'm about to run for you. And I have some H.264 encoded video clips of the hawk in my backyard.

And what I can do with these video clips, since I've got QuickTime Player Pro, is I can copy these things and combine them together into another movie. Thank you. I've got different encoding rates on these different clips, but copy and paste of these various clips works just as you would expect it to work with QuickTime. And so now I have a single movie that plays those three clips back to back. "Well, that's pretty good, "but it doesn't quite go far enough "to demonstrate the point." So let's go back to that Apple script that I mentioned. It's called Movie Mincer. And what does it do?

Movie mincer is probably the most fearsome thing to a QuickTime engineer right here at 5 o'clock on a Wednesday afternoon at WWDC. Because what I'm going to do with this script is to select at random-- at random, mind you-- 20 two-second segments from a movie which is itself pasted together from three separate clips. And I'm going to take those two-second segments at random and put them into another movie. I do not know what will happen, ladies and gentlemen. I require complete silence.

It's kind of fast on a G5. I like that. So now that I have pasted together-- well, AppleScript has pasted together these 22 segment segments chosen at random. Remember now, these video frames can have dependencies on frames that come after them or frames that come before them. And there's a very complicated graph of dependencies if you go through the sequence of video frames in these various H.264 movies. Now, I've chosen at random where these clips should start stop that I've pasted from one movie to another? I don't know if I started in an I frame or a B frame or a -- I don't know what frames I started at. If I paste them all together, how is QuipTime going to figure out how to decode these frames which may have dependencies on other frames which may not have gotten pasted into the destination? I don't know how it works. I'm just going to play the movie. Every two seconds, I should see another clip from those various movies. This, I think, is probably the most impressive technology demo that you're going to see at the show. That's not a joke. This is really remarkable. This is live, in-place editing of this very complex video stream and a nice looking hawk, too.

All right, back to slides. So we don't just support playback of B-frame video. We support all that other cool stuff that I talked about earlier. Now, I was copying and pasting with the-- what do they call it? Pasteboard in Mac OS X? But you can also insert media via low-level APIs. An application like iMovie is combining media from multiple sources by means of the very same technology that you saw me demonstrate via copy and paste. This is really pretty cool stuff.

Now, in order to accomplish this, we did have to introduce new APIs because we had the limitations that I mentioned earlier. We have new, in the image compression manager that I mentioned earlier, we have new compression and decompression APIs to support this notion of video frames that can have distinct display and decode orderings.

Also, at the level of media creation and media access, we needed new APIs to carry the data about decode and display order as well. So at those levels, there are new APIs available. The very good news is-- and here's the other reason why I use QuickTime Player 651-- to demonstrate that the high level APIs for movie playback and movie editing are unchanged, even though we have support for this new stuff. So the old player, the one with the currently shipping QuickTime, those features work with the new media content. And the same thing will be true in your applications as well.

But it's not just the APIs that we're changing to accommodate this new cool video stuff. We're also expanding the QuickTime movie file format to accommodate it as well. If you are doing reading of the QuickTime movie file format, if you're opening up movie files and reading the bytes in there and interpreting them, you will definitely want to get on the QuickTime seed. And I believe the email address for the seed is not what I have up here. They both work. You can send to this address. If you want to be part of the QuickTime Seeding Program, in addition to the tiger seeding that you have received, I recommend that you send to this email address.

Now, another point I want to make about the QuickTime movie file format, in the preview of Tiger that you possess, not all of the details that are changing about the QuickTime movie file format are final. So, this preview is not for use for production purposes. Don't go out and start making movies that you want to put on the web and have them playable indefinitely. Use Panther for that.

We will finalize those details, I'm sure, by the time Tiger ships. All right, what about more information? I've given you some of the technology details, but only at a very high level of what's new in video and QuickTime. Where do you go to find out more? I'm recommending two places. There is a document that is available to you in your Tiger seed and is also available via the Apple Developer Connection. It's called What's New in QuickTime 6.6 Developer Preview. There are several sections of that document that I've listed here, B-Frame Video, H.264 codec, and Qt Sample Table API that will give you all of the nitty-gritty details about the APIs that I've mentioned. But if you want to see it explained by living people, actual engineers, come to session 217, Friday at 3:30, supporting new video formats in QuickTime, and you can get the information there as well.

All right, that's what's new in video. What's happening in the industry that's influencing us? Well, what's happening in the area of graphics? I have a graphic. See how I tied that in? Thank you. This timeline reveals in graphical fashion that over the past 25 years, video games have improved in their ability to engross the consumer.

Yeah, I think that's what I meant. In any case, you can see the progression here. We couldn't actually use the images because they're copyrighted, you see. From the line drawings of asteroids to the 3D models of modern video games, you guys must really love this stuff because you're buying it.

Now, it's been mentioned several times during the conference that Moore's Law drives the ability of the processor to shrink, to consume less power, and go faster. Well, we found that there is actually an ancillary law, together with Moore's Law, that has driven this evolution of the graphics processing unit. It's called Atari's Law. Atari's Law states that the game that you produced six months ago isn't good enough.

It has to be better or they won't buy it. And this has driven the evolution of the graphics processing unit, which is now, in one form or another, part of every hardware unit that we sell and is also available from other hardware vendors in their PCs. The GPU is now ubiquitous. Several of you application developers have sat back and taken a look and said, you know, I'm not developing a video game, but every pixel that I draw winds up going through that GPU on the way to the screen. And I know there's a lot of processing power there.

Maybe I can use it too. Well, that's exactly what happened in the development of Keynote. Keynote made use of the power of the GPU to distinguish itself from its competitors in its field by having all these great transitions and all these great effects and great integration of graphics. And for an example, My hawk friend is back again. You know, you can't tell if it's a red-tailed hawk by its tail.

if it's immature. What is QuickTime going to be doing about all of this graphics processing power? Now, these machines have evolved this enormous organ, the GPU, that was originally intended for one purpose, but as you know from evolutionary theory, these organs often wind up being applied for other purposes as well. And the organism gains the ability to do new things by this evolutionary process. Well, QuickTime is going to take advantage of the GPU, too. And how are we going to do it?

We're going to get out of its way. When QuickTime was first introduced in 1991, there wasn't a lot of help among the services available on the OS. What were the numbers again? for playing video to the screen. There was some for sound, but there certainly wasn't a lot of support for synchronizing video with audio.

So QuickTime did it all, and it was necessary for it to do it all, and it was very successful at it. For years, when we talked to application developers at conferences like this, we said, you focus on your area of expertise when building your app. You let us handle the digital media, and you can integrate digital media with other content that you manage. And that story was very good, but something happened along the way. We've noticed that you application developers have actually gained expertise in the area of digital media. And you want to do interesting things to process digital media in your applications. And you don't want to turn all of that over to us. You want what QuickTime offers to work in combination with services that are available on the platform now for really great graphics processing, really great audio processing. Well, we're going to make that possible.

Now you can have everything that's good about QuickTime video and even less. So how are we going to achieve that? For video, we have defined a new abstraction for the destination to which video will be rendered. We're calling it the visual context. It's an abstraction, and we intend there to be several types of visual context. For Tiger, we are making it available concretely for OpenGL textures. And so this means really great stuff. You can get all of that great processing power that's available in the GPU apply to video.

You've seen some demonstrations of it. At a high level, however, you don't have to know anything about OpenGL textures to gain this advantage. If you use This, by the way, is a narrative prolepsis. If you use our high-level modules, the HI Movie View and the QT Movie View, you have the full support from them for visual context without you having to do a thing. And that basically means if you use those modules and you deploy your application on a machine that has Quartz Extreme, then you're eligible for lots of great stuff that the GPU can do with video. However, if you have the kind of application that's doing Manipulations at the OpenGL level, we have low-level APIs for you to use as well.

Where to get more information? Well, the session is 2.15, Thursday at 2 o'clock, Improved Video Rendering and Playback. In the What's New document as part of your seed, look at the Visual Context section, and information will be there for you. Thank you. All right, I said this was an omnibus. I'm going to talk about video and audio in the same session. I can get away with that. QuickTime is a media integration layer. You don't see the core audio guys talking about video, and you won't see the core video guys talking about audio. But QuickTime sits on top of those layers, and we can talk about both. Sound, of course, is spatial. We've had support for mono and stereo audio. At least I'm keeping Jim awake, that's good.

for a long time. And is that sufficient for being able to detect where sound is coming from? Well, I have two ears and I knew it was Jim is the only person laughing at that. But we're finding that it's actually interesting to have more than two audio sources available to create an enveloping experience. And this, of course, was originally available in a movie theater. Does anybody remember the first movie that had support for surround sound?

Fantasia. It was called Fantasound. And there was an array of speakers around. It was all done, it was analog, and it actually all worked. It was kind of cool. And you know whose idea it was? Leopold Stokowski. Anyway, now we can do this in the digital domain. So I've name-dropped Leopold Stokowski and Harold Blum, and I'm only 30 minutes into it. So pretty good.

So, naturally we want to support multiple channels and we want to support higher resolution audio with the same richness that I just mentioned for video. So let's step back over to demo one and I will show you that you can in fact combine some of this rich audio, high resolution, multi-channel audio using QuickTime, the media integration architecture. Let's see. Let me take some video. I'm going to take some video that's near and dear to my heart. This, by the way, is cherry juice.

Those of you who have the QuickTime 1.0 CD-ROM from 1991, you may have seen Cherry Juice. It used one of the original video codecs that shipped with QuickTime 1.0, known fondly as Road Pizza. And this video was produced laboriously by means of a series of still images. and I'm going to take this very old video and I'm going to combine it with some very new audio. I happen to have Well, I'm actually going to open it up in this QuickTime player first. I have some sound effects here that are actually in surround sound. Let me play them for you. Now this demonstrates one of the key things about support for multi-channel audio in QuickTime. Did you know that QuickTime 6.6 automatically mixes down to the abilities of the output device that you happen to have attached? This is 5.1 audio. The room is only stereo.

So I'm going to copy that segment of audio that I happen to have, that 5.1 audio, and I'm going to add it to my Cherry Juice. video, and now we have 1991 vintage video and audio together. Should we stop there? I don't think so. I ran my movie mincer script earlier today in another session. I ran my movie mincer script on this very thing and the same copy and paste, combine, integrate, complete digital media technology was able to produce this.

Man, I hate it when that happens. Was able to produce-- what did I say it wrong? Was able to produce this. Oh, Now, in order to do that, I actually changed some of the parameters to my script. Instead of copying two-second segments, I copied segments of one-fifth of a segment, and I did 40 of them to make eight seconds of media. But the same thing works. That integration technology, that power that's available to QuickTime, when we do multichannel audio, we do multichannel audio, not just playback.

Now, that was a demonstration of some very basic combination of audio and video together, but we thought that it would be valuable for you to survey the current state of production tools. What if you really wanted to do something? You don't want road pizza video in 2004. That's what year it is, right?

You want H.264 video, and what you want to do is to have some full-screen really cool thing, such as the demonstrations of H.264 that you saw at the keynote on Monday, And the question is, how do you do it? Well, fortunately, we have the answer to that question right here. Ladies and gentlemen, introducing the QuickTime Stupid Movie of 2004. Thank you.

I'm going to project what may be perceived as a lateness as a timeliness. I believe that video production requires a digital media architecture as rich as QuickTime. to streamline this process. And now that we're rolling out support for these modern video formats and these modern audio formats, we hope to work with you, the tools developers and the hardware developers, to make this kind of production accessible to people like me.

Thanks to those of you who prepared the video that we showed at the conference, and for those of you who prepared the stupid movie. That was wonderful. Back to slides, please. What was I talking about? High-resolution audio. Okay, we had to introduce some APIs for high-resolution audio, too. Some of them because we wanted to. If you have multiple channels per soundtrack or if you have sampling rates that are higher than our previous limitations, we can now go up to 192 kilohertz, and we can also go up to true 24-bit integer samples.

We would require a new sound description in order to describe those things, but it's only when you have sound with those greater parameters that the new sound description is necessary. If you don't, you can use the old data structures just as well. We have a new sound media handler that's capable of handling these new formats, and it's based squarely on core audio for everything that it does, for decoding and for output. it.

We've introduced some APIs which finally rationalize the ability to get decompressed audio out of QuickTime movies. It's called movie audio extraction. And this is the way we're recommending that if you need to get audio out of movies, use those APIs. We're using it ourselves when we export our movie exporters that can re-encode movies to other formats.

And we also have new support in the Sequence Grabber, which is QuickTime's module for capturing media from external devices, for high-resolution multi-channel audio. It's called the SG Audio Channel. All of those things will be demonstrated and explained in session 213, which is tomorrow at 3.30. And I think that you really want to go to this one, because I understand that there's going to be a live performance of multiple percussionists, a narrator, and a wandering troop of minstrels. And they're going to be recording multi-channel. They're going to be performing live. So please go.

There's no extra charge for admission. Also, there's information about the new APIs in the very same document I continue to reference, the QuickTime 6.6 Developer Preview. The section is audio enhancements. All right, so we talked about video, we talked about graphics processing, we talked about audio. We've noticed that things have been happening to make your lives easier in developing applications. One of them is that you don't call wait next event anymore.

Thank you. It's a long time in coming, wasn't it? But we've also noticed that you are gaining your ability to develop rapidly by using the leverage of application frameworks. Gee, it's kind of great that we notice this stuff, but we do usually have our heads down producing great video and audio like that. It may have taken a few extra years. But now we're aware that you're using high-level modules in Cocoa and also in Carbon to implement your applications. And what we are doing in Tiger is supplying modules that fit with those frameworks to manage most of your common QuickTime needs. For you Carbon developers, we're introducing in Tiger the H.I. MovieView. H.I. Views is an object-oriented view system that's for use by Carbon-based applications. H.I. MovieView allows QuickTime media to play as a first-class citizen in that view framework.

I'm sorry, I have to go around to the door in the front. Let me show you H.I. MovieView in action. Now, there's something interesting about the demos that I give in this session. This is sort of a session that's sandwiched in between the sessions at the beginning of the week that officers of the company deliver, people with titles, people whose pictures you see in trade publications, and even in Newsweek. And the sessions that come towards the end of the week, which are very highly technical, given by people who really know what they're talking about.

If my demos are too flashy, they get moved to the beginning of the week. And if my demos are too technical, they get moved to the end of the week. So I have to do demos that nobody else wants, such as introducing an element of randomness in copy and paste.

This demo partakes somewhat of this in that it's a very elementary introduction to the technology that I just mentioned. The predecessor to the HR view was the Carbon control. That is used by a carbon-based application, iTunes, to display videos in the music store. So if I go to the music store, and let's see, I will go to the Bob Dylan page, and I will go to the main place for that artist. And I will go to the And I know that the music store has video available of a performance by Bob Dylan.

iTunes uses the Carbon Movie Control, the predecessor to the H.I. Movie View, to display its video, and it supports the ability to display video that's downloading from the Could you please get off airport if you're out? Check your mail later. And it can do everything that the Finder can do when it displays a movie as well.

The Finder is using not the predecessor Carbon Movie Control, but in Tiger, the Finder is using HI Movie View to display previews of QuickTime media right here in the Finder. Well, you can see in iTunes, it supports scrolling of video back and forth. We can clip, we can resize. Cool stuff. Good song. Finder using H.I. MovieView also supports all that stuff as well.

I can get a large preview. It'll resize. I can scroll. The HIMovieView can be used in several different contexts within the same application. So for example, if I do a GetInfo, Where'd they move it now? If I do a get info on a media file, the finder also can display a preview inside of the preview pane and that can also be a playable video as well. Although in this particular case, unfortunately, it's not. This also uses the HIMovieView in line with these other user interface elements. If you have a carbon-based app, you're using a view system. HIV support rich composition. Movies can be composed with other elements. Back to slides.

Not just Carbon, but also Cocoa. Oh, by the way, there'll be more information about the H.I. MovieView in tomorrow's session 215. That's the improved video rendering and playback. And several original demos of H.I. MovieView will be available there. But as I mentioned, not only Carbon, but Cocoa too. We are introducing in Tiger the Cocoa QT Kit. And we are making available to you Cocoa developers all of the richness of the manipulations that you can perform on media in a form that should be very familiar to you, using all of the same constructs and all of the same programming techniques that you have learned to develop your Cocoa applications.

Cocoa, of course, already has basic QuickTime support in NSMovie and NSMovieView. But with the Cocoa Qt Kit, we're going quite a bit further beyond that. We have classified QuickTime behaviors and introduced Cocoa classes based on those behaviors that make manipulations of movies, tracks, and media available to you in Objective-C using the Cocoa framework.

And, of course, we have a class for your GUI, the Qt MovieView, that's very rich. it goes quite a bit beyond what you could do in NSMovieView to date. Now, Cocoa not only supports modern applications with graphical user interfaces, it also supports other types of modules as well, bundles, plug-ins, command line tools. But don't take my word for it. Let's look.

Now, when I went through a Cocoa training, when I learned how to write an application in Cocoa, several of you might have had exactly the same experience I had. I learned with the expenses application. Does this ring a bell for anyone? The expenses application is the application that you build if you're working through the learning Cocoa book from O'Reilly, or if you go to any number of training sessions that show you how to do basic application development in Cocoa. And this is a pretty cool app. This allows me to keep track of my expenses like, well, let's see, when I cross the Golden Gate Bridge today, That cost me $5. And when I parked in the Mission Street garage, let me guess, that's going to be about $16. Well, that's kind of good. I can also attach notes, textual notes. Let's see. I'm on level 6. To each of my expenses.

So that's nice. I can browse through my expenses. It'll keep running total. I can save this document to edit it later. The NS table view displaying this set of expenses and the NS text view, which displays these textural notes, are subviews of an NS split view, which allow me to do this kind of cool stuff here. So I can resize the different subviews if I want. When I look at that app as a multimedia developer, I don't see an expenses app. I see a playlist editor.

So what I did was I took the expenses application exactly as I completed it when I completed my Cocoa training, which I did almost as well as I did in the fifth grade. And I made from that expenses app using the same constructs, in fact, the same lines of code, a playlist editor. I added a little bit to it and I changed it around a little bit so I can add multiple movies at a time. So I'm going to add some items to my playlist. And they show up up in the upper pane, which is once again an NS table view. And I've renamed my categories to file name, display name, which may be different. That's a narrative prolepsis. and display the duration using an NS formatter to show the number in seconds with two decimal places. I could have written a better NS formatter to show minutes and seconds, but it was Monday afternoon. What do you want?

Now, the view below has changed from an NSTextView to a QTMovieView. And when I select any one of these items in my playlist, the movie that represents the playable media for the file that that playlist item refers to is displayed in the QTMovieView below. And the NSSplitView works exactly as you would expect. Gee, maybe it'll even work if I play it.

So I can do the resizing while I'm playing, resizing of the window, resizing of the view. I also did a fancy little thing. I'm listening for the notification that comes from the class QT movie that tells me when the movie is done so that I can listen for that notification in my document controller and tell the NS table view above to display the next item in the list and the QT movie view below to display the next movie. Now this NS slider thing, this NS control on the bottom, that's actually functional. It's not just displaying the current time of the movie. I can also click on different-- parts of it to navigate through the movie, and I can drag the thing live to navigate through it as well. Well, how much code is here?

There are places on the web that allow you to download the version of Expenses that's similar to the one that I showed you. There's really very little additional code there than there is for that. We will make this -- unfortunately, it's not available in your preview of Tiger, but we'll make the code for this available in the coming weeks via the Apple Developer Connection. And you'll see that all of the same things that allow you to do rapid application development in Cocoa, you can now do with QtKit using QuickTime. But it's not just about applications that have a GUI. It's also about other types of executable modules as well. With QtKit, you can develop command line tools that you can use in batch processing for media production.

And because it's very difficult to type at this time of day, I have my canned command lines right here. A member of our team developed a command line tool called Recompressor that takes several command line parameters. You tell it, well, here's the movie exporter that I want to use, in this case, 3GPP. Here's the source movie that I want you to recompress. And here's the destination movie that I want you to recompress to. So if I execute this line in terminal-- I told you there was going to be a terminal demo. You saw the graphic earlier on. I didn't lie. It's going to recompress this DV video clip to 3G, and it's now available for me. I can send it out to my 3G handset, and it'll play something like this using MPEG-4 video and AMR audio.

Thank you. So I wish I could tell you more about QTKit in this session, but this is the omnibus quick time presentation. Back to slides, please. There are more details tomorrow morning at 9:00. If you can be here at 5:00 on a Wednesday, you can be here at 9:00 on a Thursday.

New directions in using QuickTime with Cocoa. Go to that session. There'll be lots of great stuff, and I promise the demos there will be very, very rich because you can do really great real-life production stuff with these tools that we're making available to you in Tiger. The section of your What's New in QuickTime 6.6 document that covers this material is called QtKit Framework for QuickTime. We have other documents that are available as well, a reference and a tutorial that you'll have to come to the session tomorrow to get the pointers to.

Okay. So you've seen, I've convinced you now, QuickTime is a cross-platform media architecture. We've demoed only the things that are available in Mac OS X and focused on the things that are new in Tiger, but these same media services for playback, editing, creation, and so forth are also available on Windows, all the Windows 32, the Win32 series of releases.

By means of QuickTime, you can simplify your use of all these variegated media container formats, video and audio formats, external devices for capture, and so forth. But have we done enough for you? Our answer was, no, we haven't. We want to take it another step further, because we realize that people don't just produce media, they classify it as well.

How much did you pay to come to this conference? Remember to get your money's worth. People attach classifying information to media in the course of production in order to identify media that's going to be used in later steps in production or to identify media when it's delivered to the end user. And they tag this media or classify it with several different formats of metadata. There are as many formats for metadata as there are formats for media itself and maybe more. And several of you in the back of the room might be developing your own formats even now. We've noticed that, for example, for MP3 files, there are many more formats for tagging than there are players in the world. What is going on? You can tag your MP3 files with ID3 version 1, ID3 version 2, Lyrics 3, Lyrics 1, APE. I don't know what these things mean. It's just confusing.

But what we want to do is to make sense of all these metadata formats because, hey, we're QuickTime. We are the media integration architecture, and we can do it. So what we are doing in Tiger is introducing APIs that give you access to media metadata. You can be concerned about the specific metadata format if you need to be, or you can simply leave the details of the format up to us and get the information out that you're interested in. How does it work? When you open a movie, you ask QuickTime for a reference to metadata attached either to the movie, to a specific track, or to specific media. You can tell us if you're interested in a specific format, such as iTunes Music Store metadata or classic QuickTime user data, or you can just say, "Give me a reference to whatever it's got." Then, once you have that reference, you can iterate through the items in that metadata by means of keys that are specific to a metadata format or a set of keys that we've defined that we intend to work in common among all the metadata formats that we support. So if you want to get, for example, the display name of the media, you merely have to say, "Give me a reference to the metadata.

Find me the display name, please, and give it to me." And whether that display name is stored in classic QuickTime user data, in iTunes music store metadata or in new QuickTime metadata, we've defined a new format to overcome the limitations of classic user data, we will find it for you and we will deliver it. And I have a demonstration of that right over here. Data potato do what? Do what? I have a demonstration of it. I'm going to back up over here on demo two, please.

All right. So what I want to show you here is that it's possible now for the very first time. for QuickTime Player to display in this demo. It's not the most impressive demo that you'll see at the show, but it's probably the nearest and dearest to my heart. So I'm gonna make sure that it works.

♪ When it's Wednesday and the beach ball spins ♪ ♪ Where do you go, what do you do ♪ Yes, it's late night at the piano bar. Okay, here we go. On demo one, I've opened up an MP3 file, which contains an ID3 version 2 tag. That includes the name, the display name of the file. And that display name is stored as UTF-16. It's Unicode. And for the very first time, QuickTime Player can extract that name from the metadata in the format ID3 version 2 and display it in the title bar of the window. And what is it doing? What is QuickTime Player doing in order to do that? All it's doing-- this is a preview of tomorrow's session-- is calling the method QtMovieDisplayName. And underneath the covers, the QtKit is calling upon the QuickTime metadata services that I described to fetch that piece of information out. It doesn't worry about the format of the metadata. It just says, "Give me what I want," and it delivers it.

But we can go a little bit further than that as well with metadata. And in fact, I have. And I did it in the application that I showed you earlier, my simple playlist editor. I complicated it just a little bit by adding the following feature. This is a Cocoa application. So what I did was extended the Objective-C class QtMovie by implementing a category on that class with the following method, artwork. What I want to do when I instantiate a Qt movie is to fetch the artwork out of that movie wherever it may be. And I implemented that method artwork by using the QuickTime metadata APIs that I described earlier.

What I then do with that artwork is I actually attach it to the movie as a video track. This application is highly visually oriented. What if I want to play some music files? Let me preview these very same music files for you in the Finder. If I click on one of them, you'll see the Finder merely displays the standard QuickTime movie controller because the only media in there is sound media. But in my application, which is taking advantage of support for QuickTime metadata, If I select these audio-only files, which nevertheless contain cover art in their metadata, I can do the following trick. Thank you. As I described earlier, I can actually display the cover art as a video track in parallel with the audio. So if I go through these music files from the iTunes Music Store, I can display the cover art as part of my movie.

You can do stuff like this too. We know that there are many metadata formats that you will be interested in as part of your production tools or as part of delivery if you're displaying media. And what we want to hear from you is what metadata formats interest you? What kind of stuff do you want to read in the course of preparing media or delivering media and present to the user or make available in the production process? As I mentioned, in Tiger, we're going to be supporting QuickTime user data, QuickTime metadata, a new format, that overcomes the limitations, and iTunes Music Store metadata. This MP3 stuff I did by way of demonstration. We're not actually planning to roll out support for MP3 cover art in Tiger, unless you all e-mail us and say that you can't live without it.

Where to obtain more information about QuickTime metadata? I should have done this much earlier in the session. The document of what's new in QuickTime 6.6 does contain information about the QuickTime metadata APIs. We will be posting sample code as the release process moves along towards the eventual delivery of Tiger. And yet, you've come to the right place to find all the information that will be covered in the show sessions about QuickTime metadata.

Now, let's see. It says here I have 11 minutes and 60 seconds. I don't have to talk fast yet. I'm okay. Don't worry. I But I do have more to say, and that is that we've shown you that QuickTime is a digital media architecture, an integration architecture that gives you all these great services. And we're extending it, new media formats, support for new stuff, metadata, not just media. But you don't have to wait for us to extend the QuickTime media architecture, because in fact, the architecture is itself modular and extensible. and the very same coding techniques that we use to add new support to QuickTime, you can use too. We haven't made this point at developer conferences over the last couple of years and I wanted to be sure that we mentioned it this year. Several of you are new to the platform and new to QuickTime. You don't have to email Apple and say, "You know, if only you supported my media container format, I'd like you a lot." Or, "If only you implemented the following video codec, you would be my friend." You can do it yourself because QuickTime ARM defines the component APIs that allow you to implement those things and sample code for this stuff is available at the Apple Developer Connection website on the sample code pages. Many different component types are available. I have an example of a movie import component.

Now, one of my favorite places in the file system-- I don't have to talk faster, but I can't pause any longer. One of my favorite places in the file system is I think I spun up the drive, I clicked on the wrong one, is right here in /library/quicktime. There's cool stuff in there. What's in this particular directory, my /library/quicktime is my modified MP3 import component that implements all that cool ID3 stuff that I just showed you. That's how I extended QuickTime for the purposes of my demo.

I will also extend it slightly further. Now you know that on the platform, we do have support for decoding AAC audio. We rolled it out with our MPEG-4 support in QuickTime 6. However, we have not to date had support for the container format.aac..aac, or ADTS I believe it is, is a format that stores AAC audio packets. And several encoders produce a stream of AAC audio in this format. If I were to drag this.aac file on top of Keynote, however, it would sadly reject it. If only I had a QuickTime movie import component that knew how to read the audio packets from that container format and create a QuickTime movie. Oh wait, I did that.

I implemented an import audio file component. And those of you who are familiar with Core Audio know exactly how I did this. Core Audio, in fact, a service on Mac OS X already knows how to parse these.aac components. I didn't actually have to write any hard code to do this. All I had to do was to make use of Core Audio's ability to do that and to make a movie out of a.aac file. So all I need to do here is to move that cool thing-- there it is-- into /library/quicktime.

And then I do have to quit Keynote and relaunch it so that Keynote and QuickTime can match up together again. Keynote can... register all of the movie import components that are available in the system. But now that I've added that simple component to /library/quicktime, if I now drag the very same.aac file onto Keynote, I get a plus sign and my demo worked. But wait, there's more.

QuickTime uses what's known as the core services component manager in order to load and to invoke these extensible modules, these modules that do movie import and video image decompression and all this stuff. We call them components, but there's nothing really fancy going on in the runtime. QuickTime deploys the component manager and uses the component manager on both Mac OS X and on Windows, But the component manager merely uses the native dynamic library loading mechanism that's available on the platform and the format of dynamically loadable libraries that's defined by the platform. So the question is, why can't I implement these extensibility modules for QuickTime in Objective-C? And the answer is, why, of course you can. Now, there were some problems involving language runtime and stuff that I don't understand too well that made it dangerous to do this with previous releases. The good news is that we've ironed all of this out. And starting with Tiger, you can deploy movie import components of the sort that I just demonstrated or any other of the types of components that you saw listed earlier implemented in Objective-C, taking advantage of all of the great stuff in the Cocoa frameworks. Let me show you what I mean.

Because I don't just say it, I'll actually launch Xcode. I got five minutes and 40 seconds left, I can launch Xcode. Laughter Okay. So the component that I chose to implement here, once again, it was Saturday. I didn't really have a lot of time to implement support for a media format, so I stole support that existed somewhere else. You may already be aware that in AppKit, there's a class called NSBitmapImageRep, and this class knows how to write a number of still image formats. One of the still image formats that it knows how to write is GIF. That's a still image by the way, that QuickTime doesn't know how to write. So all that I did-- this is literally all that I did. We'll post the sample code shortly-- is define a class for a graphics exporter. And I define the following methods: init with image rep. So what's gonna happen here is that an instance of this class is gonna be presented with an image rep. Then the method that's gonna be invoked is export.

And the information about the image is going to be, by whatever means necessary, is going to be turned into the format that this exporter knows how to write to. And it returns that exported image in an instance of NSData. And all of the other work about writing the file out or writing to wherever the thing has to go for the purposes of however it's being used in QuickTime are taken care of at a higher level. And this is all the code that I needed to write in Objective-C in order to implement this exporter. This still image exporter, which is in secret sauce. Oh, actually it got moved. Export GIF component. Move that once again to /library/quicktime.

And this time, when I launch QuickTime Player, it's going to gain the ability to write to GIF files. What I'll do is open one of my favorite movies once again. And I will export, using QuickTime Player, this movie to a sequence of still images. Now you see that we have this support for exporting video to a sequence of images or any visual media. And we have several presets. QuickTime knows how to do JPEG and B-Met-- what's that? Some competitor's image format?

But what I want to do is export to GIF. And only after my graphics exporter that supports GIF was installed was this ability added to the media architecture. What I want to do is export this movie at two frames per second, and I'm going to put them in my pictures directory. And here I go, writing frames of this movie to GIF files. And sure enough, if I open one of them at random, there it is, a GIF file in 256 glorious colors. I think that's the first time in a while that GIF has gotten a round of applause.

We have reached the end. I have tried as hard as I could to jump around to make use of narrative prolepsis and analepsis. I even tried to define for you poetic metalepsis, but now we're done. Our summary is that QuickTime offers high-level integration of media services, playback, capture, editing, cool stuff. You want to use it in your applications. We are continually extending the reach of QuickTime to new formats and to new services as well. But you don't have to wait for us. You can extend it as well. What else are we doing?

I hope by means of demonstrating to you, both here in this session and other sessions in the show, QuickTime's use of core audio, QuickTime's use of core video, QuickTime's use of core image, this also marks a major milestone for QuickTime. We no longer consider QuickTime the only piece of code on the platform that's going to process digital media. It's a great set of services that integrates a lot of wonderful stuff that you're going to want to use. But you, with your intelligence and your need to process digital media and applications, are going to want to use QuickTime in combination with other services that you provide or that are available on the platform.

QuickTime and Core Audio. QuickTime and Core Video. Get the video frames out and spin them around using OpenGL. Get the audio frames out by means of movie audio extraction and do what you like with core audio. That is our model for the future. Rich high-level services at the integration level and powerful specific services at lower levels. And you'll see us doing more and more of this as time goes on. That is the best narrative prolepsis of all and I will end there. Thank you very much. Thank you.