that you're seeing revealed for the first time at this conference. We're going to describe what those developments are, review some of the things that you've seen in previous sessions, and amplify on them as well. We're also going to preview some of the more in-depth technical sessions that follow mostly tomorrow, and I'll be directing you to what those sessions are and what content they'll contain. So this is the omnibus for QuickTime in Tiger. What you'll learn is what QuickTime offers to you, the application developer for media services that you can use.

will give you some examples of applications that already use QuickTime so you can get some ideas of how you can use it as well. Talk in depth about what's new and improved and for the first time on any stage we'll talk in depth about how you can take advantage of new ways to use QuickTime in both Carbon and Cocoa apps starting in Tiger.

Where does QuickTime fit in on Mac OS X platform? You've seen several versions of the architecture diagram for the entire OS this week already, and you've seen QuickTime mostly among the imaging modules in that diagram. Here I've blown up a portion of that architecture diagram to show you that QuickTime sits on top of imaging, video, and audio services on the platform. It's an integration layer that gives you access to digital media services, and it in turn makes use of these lower-level services that also have APIs that you can use.

and other companies. To amplify, what does QuickTime offer? Not just playback. Most of the time when you come to a conference like this and you see QuickTime, we're going to play you really cool content. This session goes beyond that to show that the services that QuickTime offers are complete. It's a full media architecture. Creation of media, editing of media, capture from external devices, and delivery via the web, via streaming, via CD or DVD ROM. QuickTime offers all of that.

Now, from time to time we get into discussions about what we mean when we say QuickTime because QuickTime actually has been around for a while. It was introduced in early 1991, originally available for system Mac OS 607 and system 7. I just had to check with Jim because I'm never sure about which 6 it was. But in fact it was 607. Over the years the term QuickTime has been applied to several different things. I've got a list of what some of them are now. QuickTime is in fact an end user product.

QuickTime Pro which is available from Apple's website. You can buy from there. QuickTime is also a media container format known as the QuickTime movie file format. And in fact it's achieved great success in the industry as the basis for the MPEG-4 file format and related file formats, 3G as well. So, let's get started.

Container format. We can put video and audio in there and some other things too. QuickTime is also a media solution. There are several products that work together. QuickTime Pro, QuickTime Broadcaster, QuickTime streaming server and the QuickTime plug-in for the internet browsers that allow you to produce and deliver media. But what we're talking about specifically in this session is the technology that you can use in your applications to put media services to work for you.

There are several sets of APIs that are available. APIs, C APIs that QuickTime has offered over the years and you see some of their names here. Movie Toolbox, set of high level APIs for playing and editing media. Image Compression Manager allows you to compress and decompress still images or sequences of images. And a number of other component APIs as well, some of which I'll talk about later in the session.

will be a theme that I'll come back to. Actually the themes of this session if you must know are what would they say we were? They were classification and reordering. So every time you hear something that relates to classification and reordering if you could make some sort of sign that you're still awake just that would be good for me because it is 5 o'clock on Wednesday and this is a tough slot to fill.

But anyway this session is about the API so with no further ado let's talk about QuickTime APIs at work in existing applications today. So on demo one I have one of my favorite applications particularly when I'm away from home, Mail. Mail is a mail application. It does wonderful stuff.

It allows you to find your email messages to file them and to read them. But one of the other things that Mail does is it allows you to exchange media from one email server to another. If I could have demo one up please that would be handy. Thank you.

So here I have, what have I got in my email now? I just launched this app a minute ago. something from home. In Mail, and in fact in a number of applications, you can put media, digital media, in line with other content. Mail uses the QuickTime APIs in order to play video and audio.

Here I have, let's see, what do we got? That's nice, thank you. I'll be home soon kids, don't worry. Well, that's just one example of an application that allows you to embed media together with other content. If you have a document-based application and you want to use digital media together with other content, use QuickTime. It's a perfect solution.

Let's see, another application that uses digital media is Keynote. Keynote goes a few steps beyond, however, what an application like Mail does. Sure enough, I can create a new presentation in Keynote and I can take some digital media that I happen to have handy and I can drop it in there and I can play my digital media content inside my presentation.

But this is actually not All that remarkable, this has been done in a number of applications for a number of years. Keynote takes it quite a few steps beyond. What I would actually like to do here is create a new presentation and this time I want to choose one of those interesting themes, the Blackboard theme, and I want to take that same movie that I put on my other slide.

and let's make this a blank master and I think what I want to do here is to make my movie occupy the entire slide let's see that would be 1260 by 768 and I want that positioned at zero zero alright so there I now have a full screen uh... slide with a movie in the background

The Red-Tailed Hawk.

This Red-Tailed Hawk has been showing up in our backyard for the last week or so. And one morning while I was drinking my coffee, there it was. So I just grabbed my DV25 camera and there you go, live from my backyard actually, stored the Red-Tailed Hawk. What I really want to do here is have some bullets. I'm going to cover the habitat of the Red-Tailed Hawk. I'm going to cover the markings. I did very well in the fifth grade, by the way, in case you can't tell. And I want to cover the call of the Red-Tailed Hawk.

And I want to give these items, oh, I want to give them some...

but the video you know it's kinda bright the sun was shining the day that I shot this video so what I really want to do is to dial down the opacity of that video because I really want that to be in the background then when I play my video it's going to show up like this You can see in the lower left there a preview of my presentation. Then when I click, my bullets are just going to come in right over my video, which plays in the background. But you really want to see this full screen, don't you?

doesn't have to be the WWDC.

Now we get it. Now when I play my presentation Oh, I got it wrong. I'll show it to you downstairs in the QuickTime lab, but this movie will play full screen with the bullet points dropping down with full resolution and full frame rate. So Keynote goes further than an application that simply embeds media. It combines digital media with other features that it has. It's support for transparency. It's support for builds and transitions. Digital video and audio work seamlessly with that in Keynote and it's all done by means of QuickTime. Okay, back to slides please.

So I hope I've convinced you that it's worthwhile for you to use QuickTime APIs in your applications to gain that sort of feature that I demonstrated just now. If I had managed to convince you of that only two years ago at WWDC, your application would have gained value without you having to do anything between then and now. Because the APIs that you would have put to work in your application then would have given you access to an increased number of media container formats, video formats, and audio formats as new versions of QuickTime have been released by Apple.

In just the QuickTime 6 series of releases, we've added support for MPEG-4, for 3G, for JPEG-2000. We've added support for external devices, better support for DV, support for IIDC cameras. That's the Industrial and Instrumentation Digital Camera Standard. I'm glad I got that right after the failure of the keynote demo.

Graphics processing. We found, and we've mentioned several times at the show, that the graphics processor, the GPU, that is in every machine that we sell, and also many of the machines that are sold by other vendors, is valuable not just in the original domain for which those things were developed, but also for other types of applications.

Finally, what else is going on in the industry? We're noticing that you guys writing your applications are using these application frameworks and getting a lot of leverage from them. Nice of us to notice that. What are we going to do in response to that? I'll tell you in a moment.

Okay. Tough crowd. All right. So let's go through those topics one by one. What's happening with video? By way of analogy, I intend to explain to you how video encoding has become more complex and what we need to do in QuickTime to be able to support more complicated encoding. So I can't get you to respond. I can't make it interactive. Let's play a game. We're going to play which one of these is different.

Okay, a flip book, by which I mean a book with each page of which has an image either printed or drawn and as you flip through it you get the effective animation. A reel of motion picture film, actual celluloid, they still use it in some theaters. Each frame is on the strip of film.

DV25, what's that? Well, that's the most common format of digital video supported by your DV camera that you can buy down at Good Guys whatever guys you happen to have in the area that you live in.

Now we're going to play the advanced version. Which one is different this time? narrative, analepsis and prolepsis, which of course are the terms for narrative flashback and flashforward.

Oh, thank you. Did you just scratch your head or did you mean to? A schedule of automobile maintenance the romantic poets by which of course I am referring to the romantic poets in the poetic tradition as described by Harold Bloom in his 1973 book The Anxiety of Influence in which he posited that a young poet can actually seek to gain priority over his predecessor by means of several tropes which he describes one of which is the trope of substitution by substituting an earliness for a lateness. But I'm going to give it away if I keep going. MPEG-2 video.

another type of digital video MPEG-2, another type of digital video coding. Finally, a presentation based on Keynote. Well, I'm sure it's all clear since I belabored the point on the Romantic Poets that what's different here is the schedule of automobile maintenance because in that case you have a sequence of things, each of which is only dependent on the things that have come before.

You only go to have the 100,000 mile maintenance after you've already done the 75,000 mile maintenance, right? But these other things are sequences in which items may have dependencies on things that come before or things that come after or at least the Romantic Poets hoped that this was true.

So if none of these analogies was effective to you, I will appeal to your stomach since it's getting close to dinner time. After you learn what all the sharp implements in the kitchen are for, the Culinary Institute, they eventually reveal to you that you do not necessarily prepare courses for a high-class multi-course meal in the same order in which you serve them.

And why is this true? They take, well, I don't know why I never went to the Culinary Institute, but I do know about this. The same thing is true of modern video coding. Video frames are encoded and decoded in not necessarily the same order in which they are intended to be displayed.

And why is this a good thing? Well, it enables us to get better quality out of comparable frame rates. Let's talk about an example. Suppose I have a sequence of video which starts with one frame and ends with another frame, and all of the intermediate frames are a wipe effect.

It just wipes from left to right from one image to the other. Well, if I need to encode that sequence of frames, each frame revealing more and more of the final image, then if I can only depend on images that are not the same, then I can only encode the sequence of frames that are not the same.

If I can only depend on images that have come before, with each frame, I have to waste some of my bits, well, I have to apply some of my bits to the portion of the final image that has become revealed as the wipe progresses. But if I have a video coding which allows me to depend on both the first and the last frame... Well, then that's much easier.

The wipe, the intermediate frames will require much, many fewer bits in order to code the differences between the first and the final frame. Well, that's basically how we're achieving higher quality with comparable frame rates in video. The bad news is when we shipped QuickTime 1.0 in 1991 for System 607 and 7.0, we did not accommodate this model for video coding in our video support. What I like to say is that there were no master chefs working on QuickTime 1.0. They only came along later.

But in QuickTime 1.0, as you can see, we had this limitation. And this limitation was reflected by the APIs. So you could only compress video for which frames would be completely independent or which would only have dependencies on frames that you had previously presented for compression. but in QuickTime 6.6, the version that you have in your Tiger preview, that SDK that I think they gave out earlier this week, we finally have support for video frames whose decode order and display order can be distinct. Now this is a great thing because it allows us to play modern video formats such as H.264, but it allows us to do more than that.

I will say we support IP and B-frames for those of you who are technical. The reason that this is wonderful and amazing is that we have applied all of the richness of the QuickTime media architecture to this type of video encoding. So let me show you over here on demo one what I mean.

Let me open up some of my Hawk movies again. I'm going to open them up with QuickTime Player 651, both because I want to make a point and also because the new player that's in the tiger seed doesn't quite run the Apple script that I'm about to run for you. and I have some H.264 encoded video clips of the hawk in my backyard.

and what I can do with these video clips since I've got QuickTime Player Pro is I can copy these things and combine them together into another movie. I've got different encoding rates on these different clips, but copy and paste of these various clips works just as you would expect it to work with QuickTime. And so now I have a single movie that plays those three clips back to back.

Well, that's pretty good, but it doesn't quite go far enough to demonstrate the point. So let's go back to that Apple script that I mentioned. It's called MovieMincer. And what does it do? Movie mincer is probably the most fearsome thing to a QuickTime engineer right here at 5 o'clock on a Wednesday afternoon at WWDC because what I'm going to do with this script is to select at random, at random mind you, 20 two-second segments from a movie which is itself pasted together from three separate clips and I'm going to take those two-second segments at random and put them into another movie. I do not know what will happen, ladies and gentlemen. I require complete silence.

It's kind of fast on a G5, I like that. So now that I have pasted together, well, AppleScript has pasted together these twenty two-segment segments chosen at random, remember now these video frames can have dependencies on frames that come after them or frames that come before them. And there's a very complicated graph of dependencies if you go through the sequence of video frames in these various H.264 movies. Now I've chosen at random where these clips should start and stop that I've pasted from one movie to another.

I don't know if I started in an I frame or a B frame or a... I don't know what frames I started at. If I paste them all together, how is QuickTime going to figure out how to decode these frames which may have dependencies on other frames which may not have gotten pasted into the destination... I don't know how it works.

I'm just gonna play the movie. Every two seconds I should see another clip from those various movies. This, I think, is probably the most impressive technology demo that you're going to see at the show. That's not a joke. This is really remarkable. This is live, in-place editing of this very complex video stream. And a nice looking hawk too.

All right, back to slides. So we don't just support playback of B-frame video. We support all that other cool stuff that I talked about earlier. Now, I was copying and pasting with the, what do they call it, pasteboard in Mac OS X? But you can also insert media via low-level APIs. An application like iMovie is combining media from multiple sources by means of the very same technology that you saw me demonstrate via copy and paste. this is really pretty cool stuff.

Now in order to accomplish this, we did have to introduce new APIs because we had the limitations that I mentioned earlier. We have new in the image compression manager that I mentioned earlier, we have new compression and decompression APIs to support this notion of video frames that can have distinct display and decode orderings.

also at the level of media creation and media access we needed new APIs to carry the data about decode and display order as well. So at those levels there are new APIs available. The very good news is, and here's the other reason why I used QuickTime Player 651, to demonstrate that the high-level APIs for movie playback and movie editing are unchanged even though we have support for this new stuff. So the old player, the one with the currently shipping QuickTime, those features work with the new media content. And the same thing will be true in your applications as well.

but it's not just the APIs that we're changing to accommodate this new cool video stuff. We're also expanding the QuickTime movie file format to accommodate it as well. If you are doing reading of the QuickTime movie file format, if you're opening up movie files and reading the bytes in there and interpreting them, you will definitely want to get on the QuickTime seed and I believe the email address for the seed is not what I have up here. They both work. You can send to this address if you want to be part of the QuickTime seeding program in addition to the Tiger seeding that you have received. I recommend that you send to this email address.

Now, another point I want to make about the QuickTime movie file format. In the preview of Tiger that you possess, not all of the details that are changing about the QuickTime movie file format are final. So, this preview is not for use for production purposes. Don't go out and start making movies that you want to put on the web and have them playable indefinitely. Use Panther for that.

We will finalize those details, I'm sure, by the time Tiger ships. Alright, what about more information? I've given you some of the technology details but only at a very high level of what's new in video in QuickTime. Where do you go to find out more? I'm recommending two places. There is a document that is available to you in your Tiger SEED and is also available via the Apple Developer Connection. It's called "What's New?" in QuickTime 6.6 Developer Preview.

There are several sections of that document that I've listed here: B-Frame Video, H.264 Codec, and Qt Sample Table API that will give you all of the nitty-gritty details about the APIs that I've mentioned. But if you want to see it explained by living people, actual engineers, come to session 217 Friday at 3:30, supporting new video formats in QuickTime and you can get the information there as well.

All right, that's what's new in video. What's happening in the industry that's influencing us? Well, what's happening in the area of graphics? I have a graphic. See how I tied that in? This timeline reveals in graphical fashion that over the past 25 years, video games have improved in their ability to engross the consumer.

Yeah, I think that's what I meant. In any case, you can see the progression here. We couldn't actually use the images because they're copyrighted. You see. From the line drawings of asteroids to the 3D models of modern video games, you guys must really love this stuff because you're buying it.

Now, it's been mentioned several times during the conference that Moore's Law drives the ability of the processor to shrink, to consume less power and go faster. Well, we found that there is actually an ancillary law together with Moore's Law that has driven this evolution of the graphics processing unit. It's called Atari's Law. Atari's Law states that the game that you produced six months ago isn't good enough.

My hawk friend is back again. You know, you can't tell if it's a red-tailed hawk by its tail. if it's immature. What is QuickTime going to be doing about all of this graphics processing power? Now these machines have evolved this enormous organ, the GPU, that was originally intended for one purpose but as you know from evolutionary theory these organs often wind up being applied for other purposes as well and the organism gains the ability to do new things by this evolutionary process. Well QuickTime is going to take advantage of the GPU too and how are we going to do it? will get out of its way. When QuickTime was first introduced in 1991, there wasn't a lot of help among the services available on the OS.

for playing video to the screen. There was some for sound, but there certainly wasn't a lot of support for synchronizing video with audio. So QuickTime did it all, and it was necessary for it to do it all, and it was very successful at it. For years, when we talked to application developers at conferences like this, we said, "You focus on your area of expertise when building your app. You let us handle the digital media, and you can integrate digital media with other content that you manage." And that story was very good, but something happened along the way.

We've noticed that you application developers have actually gained expertise in the area of digital media, and you want to do interesting things to process digital media in your applications, and you don't want to turn all of that over to us. You want what QuickTime offers to work in combination with services that are available on the platform now for really great graphics processing, really great audio processing. Well, we're going to make that possible.

Now you can have everything that's good about QuickTime video and even less. So, how are we going to achieve that? For video, we have defined a new abstraction for the destination to which video will be rendered. We're calling it the visual context. It's an abstraction and we intend there to be several types of visual context.

For Tiger, we are making it available concretely for OpenGL textures. And so this means really great stuff. You can get all of that great processing power that's available in the GPU applied to video. You've seen some demonstrations of it. At a high level, however, you don't have to know anything about OpenGL textures to gain this advantage.

If you use this by the way is a narrative prolepsis if you use our high level modules the H.I. MovieView and the QT MovieView you have the full support from them for visual context without you having to do a thing and that basically means if you use those modules and you deploy your application on a machine that has Quartz Extreme then you're eligible for lots of great stuff that the GPU can do with video however if you have the kind of application that's doing manipulations at the OpenGL level we have low level APIs for you to use as well Where to get more information? Well, the session is 2:15 Thursday at 2:00, Improved Video Rendering and Playback. In the What's New document as part of your seed, look at the visual context section and information will be there for you.

All right, I said this was an omnibus. I'm going to talk about video and audio in the same session. I can get away with that. QuickTime is a media integration layer. You don't see the core audio guys talking about video and you won't see the core video guys talking about audio, but QuickTime sits on top of those layers and we can talk about both. Sound, of course, is spatial. We've had support for mono and stereo audio. At least I'm keeping Jim awake. That's good.

for a long time. Is that sufficient for being able to detect where sound is coming from? Well, I have two ears and I knew it was Jim as the only person laughing at that. But we're finding that it's actually interesting to have more than two audio sources available to create an enveloping experience. This, of course, was originally available in a movie theater. Does anybody remember the first movie that had support for surround sound?

So I've name dropped Leopold Stokowski and Harold Blum and I'm only thirty minutes into it.

It's pretty good. So, naturally we want to support multiple channels and we want to support higher resolution audio with the same richness that I just mentioned for video. So let's step back over to demo one and I will show you that you can in fact combine some of this rich audio high-resolution multi-channel audio using QuickTime the media integration architecture. Let me take some video. I'm going to take some video that's near and dear to my heart. This by the way is cherry juice.

This those of you have the QuickTime 1.0 CD-ROM from 1991 you may have seen Cherry Juice. It used one of the original video codecs that shipped with QuickTime 1.0 known finally as Road Pizza. And this video was produced laboriously by means of a series of still images. and I'm going to take this very old video and I'm going to combine it with some very new audio I happen to have Well, I'm actually going to open it up in this QuickTime player first. I have some sound effects here that are actually in surround sound.

Let me play them for you. Now this demonstrates one of the key things about support for multi-channel audio in QuickTime. Did you know that QuickTime 6.6 automatically mixes down to the abilities of the device, the output device that you happen to have attached? This is 5.1 audio. The room is only stereo.

So, I'm going to copy that segment of audio that I happen to have, that 5.1 audio, and I'm going to add it to my cherry juice. Video and now we have 1991 vintage video and audio together. should be stopped there. I don't think so. I ran my movie mincer script earlier today in another session. I ran my movie mincer script on this very thing and the same copy and paste combine integrate complete digital media technology was able to produce this.

Now in order to do that, I actually changed some of the parameters to my script. Instead of copying two second segments, I copied segments of one fifth of a segment and I did 40 of them to make eight seconds of media. But the same thing works. That integration technology, that power that's available to QuickTime, when we do multi-channel audio, we do multi-channel audio, not just playback.

Now that was a demonstration of some very basic combination of audio and video together, but we thought that it would be valuable for you to survey the current state of production tools. What if you really wanted to do something, you don't want road pizza video in 2004, that's what year it is, right? You want H.264 video and what you want to do is to have some full screen really cool thing such as the demonstrations of H.264 that you saw at the keynote on Monday. And the question is, how do you do it? Well, fortunately we have the answer to that question right here. Ladies and gentlemen, introducing the QuickTime Stupid Movie of 2004.

I'm going to project what may be perceived as a lateness as a timeliness. I believe that video production requires a digital media architecture as rich as QuickTime. to streamline this process. And now that we're rolling out support for these modern video formats and these modern audio formats, we hope to work with you, the tools developers and the hardware developers, to make this kind of production accessible to people like me.

Thanks to those of you who prepared the video that we showed at the conference and for those of you who prepared the stupid movie. That was wonderful.

We've introduced some APIs which finally rationalize the ability to get decompressed audio out of QuickTime movies. It's called movie audio extraction. And this is the way we're recommending that if you need to get audio out of movies, use those APIs. We're using it ourselves when we export, our movie exporters that can re-encode movies to other formats.

and we also have new support in the sequence grabber which is QuickTime's module for capturing media from external devices for high resolution multi-channel audio. It's called the SG Audio Channel. All of those things will be demonstrated and explained in session 213 which is tomorrow at 3:30. And I think that you really want to go to this one because I understand that there's going to be a live performance of multiple percussionists, a narrator, and a wandering troop of minstrels. And they're going to be recording multichannel. They're going to be performing live, so please go.

There's no extra charge for admission. Also, there's information about the new APIs in the very same document I continue to reference, the QuickTime 6.6 Developer Preview. The section is Audio Enhancements. Alright, so we talked about video, we talked about graphics processing, we talked about audio. We've noticed that things have been happening to make your lives easier in developing applications. One of them is that you don't call waitNext event anymore.

Thank you. It's a long time in coming, wasn't it? But we've also noticed that you are gaining your ability to develop rapidly by using the leverage of application frameworks. Gee, it's kind of great that we notice this stuff, but we do usually have our heads down producing great video and audio like that. It may have taken a few extra years. But now we're aware that you're using high-level modules in Cocoa and also in Carbon to implement your applications.

And what we are doing in Tiger is supplying modules that fit with those frameworks to manage most of your common QuickTime needs. For you Carbon developers, we're introducing in Tiger the H.I. Movie View. H.I. Views is an object-oriented view system that's for use by Carbon-based applications. H.I. Movie View allows QuickTime media to play as a first-class citizen in that view framework.

I'm sorry, I have to go around to the door in the front. Let me show you H.I. MovieView in action. Now there's something interesting about the demos that I give in this session. This is sort of a session that's sandwiched in between the sessions at the beginning of the week that officers of the company deliver, people with titles, people whose pictures you see in trade publications, and even in Newsweek. And the sessions that come towards the end of the week which are very highly technical, given by people who really know what they're talking about.

This demo partakes somewhat of this in that it's a very elementary introduction to the technology that I just mentioned. The predecessor to the HR view was the Carbon control. That is used by a carbon-based application, iTunes, to display videos in the music store. So if I go to the music store, and let's see, I will go to the Bob Dylan page, and I will go to the main place for that artist.

and I know that the music store has video available of a performance by Bob Dylan iTunes uses the Carbon Movie Control, the predecessor to the H.I. Movie View, to display its video. And it supports the ability to display video that's downloading from the could you please to get off airport if you're on check your mail later and it can do everything that the finder can do when it displays a movie as well the finder is using not the predecessor Carbon movie control but in Tiger the finder is using H.I.

MovieView to display previews of QuickTime media right here in the finder you can see in iTunes it supports scrolling a video back and forth we can clip we can resize cool stuff good song finder using H.I. MovieView also supports all that stuff as well i can get a large preview it'll resize I can scroll.

The H.I. MovieView can be used in several different contexts within the same application. So, for example, if I do a GetInfo, Where'd they move it now? If I do a get info on a media file, the finder also can display a preview inside of the preview pane and that can also be a playable video as well. Although in this particular case, unfortunately, it's not. This also uses the HIMovieView in line with these other user interface elements. If you have a carbon-based app, you're using a view system, HIView support rich composition, movies can be composed with other elements. Alright, back to slides.

not just Carbon but also Cocoa. Oh, by the way, there'll be more information about the H.I. MovieView in tomorrow's session 215. That's the improved video rendering and playback and several original demos of H.I. MovieView will be available there. but as I mentioned not only Carbon but Cocoa 2.

We are introducing in Tiger the Cocoa Qt Kit and we are making available to you Cocoa developers all of the richness of the manipulations that you can perform on media in a form that should be very familiar to you using all of the same constructs and all of the same programming techniques that you have learned to develop your Cocoa applications.

Cocoa of course already has basic QuickTime support in NSMovie and NSMovieView. With the Cocoa Qt kit we're going quite a bit further beyond that. We have classified QuickTime behaviors and introduced Cocoa classes based on those behaviors that make manipulations of movies, tracks and media available to you in Objective-C using the Cocoa framework.

And of course we have a class for your GUI, the QtMovieView that's very rich. It goes quite a bit beyond what you could do in NSMovieView to date. Now, Cocoa not only supports modern applications with graphical user interfaces, it also supports other types of modules as well. Bundles, plugins, command line tools. But don't take my word for it. Let's look.

Now when I went through a Cocoa training, when I learned how to write an application in Cocoa, several of you might have had exactly the same experience I had. I learned with the expenses application. Does this ring a bell for anyone? The Expenses application is the application that you build if you're working through the Learning Cocoa book from O'Reilly or if you go to any number of training sessions that show you how to do basic application development in Cocoa. And this is a pretty cool app. This allows me to keep track of my expenses like, let's see, when I cross the Golden Gate Bridge today.

That cost me five bucks. And when I parked in the Mission Street garage, let me guess, that's going to be about 16 bucks. Well, that's kind of good. I can also attach notes, textual notes. Let's see. I'm on level six. To each of my expenses. So that's nice. I can browse through my expenses.

It'll keep running total. I can save this document to edit it later. The NS table view displaying this set of expenses. And the NS text view, which displays these textual notes, are subviews of an NS split view, which allow me to do this kind of cool stuff here. So I can resize the different subviews if I want. When I look at that app as a multimedia developer, I don't see an expenses app. I see a playlist editor.

So what I did was I took the expenses application exactly as I completed it when I completed my Cocoa training, which I did almost as well as I did in the fifth grade. and I made from that expenses app using the same constructs, in fact the same lines of code, a playlist editor. I added a little bit to it and I changed it around a little bit so I can add multiple movies at a time.

So I'm going to add some items to my playlist and they show up in the upper pane which is once again an NS table view. And I've renamed my categories to file name, display name, which may be different. That's a narrative prolepsis. And display the duration using an NS formatter to show the number in seconds with two decimal places. I could have written a better NS formatter to show minutes and seconds but it was Monday afternoon.

What do you want? and the new features of the new The view below has changed from an NSTextView to a QtMovieView. When I select any one of these items in my playlist, the movie that represents the playable media for the file that that playlist item refers to is displayed in the QtMovieView below. The NSSplitView works exactly as you would expect. Maybe it will even work if I play it.

so I can do the resizing while I'm playing, resizing of the window, resizing of the view. I also did a fancy little thing. I'm listening for the notification that comes from the class QtMovie that tells me when the movie is done so that I can listen for that notification in my document controller and tell the NS table view above to display the next item in the list and the QtMovie view below to display the next movie. Now this NS slider thing, this NS control on the bottom, that's actually functional. It's not just displaying the current time of the movie. I can also click on different Hi, I'm Kevin Calhoun, and I'm the developer of QuickTime.

There are places on the web that allow you to download the version of Expenses that's similar to the one that I showed you. There's really very little additional code there than there is for that. We will make this, unfortunately it's not available in your preview of Tiger, but we'll make the code for this available in the coming weeks via the Apple Developer Connection.

You'll see that all of the same things that allow you to do rapid application development in Cocoa, you can now do with QtKit using QuickTime. but it's not just about applications that have a GUI. It's also about other types of executable modules as well. With QtKit you can develop command line tools that you can use in batch processing for media production.

and because it's very difficult to type at this time of day, I have my canned command lines right here. member of our team developed a command line tool called recompressor that takes several command line parameters you tell it well here's the movie exporter that I want to use in this case 3GPP here's the source movie that I want you to recompress and here's the destination movie that I want you to recompress to.

So if I execute this line in terminal I told you there's going to be a terminal demo you saw the graphic earlier on I didn't lie. It's going to recompress this DV video clip to 3G and it's now available for me. I can send it out to my 3G handset and it'll play something like this using MPEG-4 video. And AMR audio.

Thank you. I wish I could tell you more about QtKit in this session, but this is the omnibus QuickTime presentation. Back to slides, please. There are more details tomorrow morning at 9:00. If you can be here at 5:00 on a Wednesday, you can be here at 9:00 on a Thursday.

New directions in using QuickTime with Cocoa. Go to that session. There'll be lots of great stuff and I promise the demos there will be very, very rich because you can do really great real-life production stuff with these tools that we're making available to you in Tiger. The section of your What's New in QuickTime 6.6 document that covers this material is called Qt Kit Framework for QuickTime. We have other documents that are available as well, a reference and a tutorial that you'll have to come to the session tomorrow to get the the pointers to.

Okay. So you've seen, I've convinced you now, QuickTime is a cross-platform media architecture. We've demoed only the things that are available in Mac OS X and focused on the things that are new in Tiger, but these same media services for playback, editing, creation and so forth are also available on Windows. All the Windows 32, the Win32 series of releases.

By means of QuickTime, you can simplify your use of all these variegated media container formats, video and audio formats, external devices for capture, and so forth. But have we done enough for you? Our answer was, no we haven't. We want to take it another step further because we realize that people don't just produce media, they classify it as well.

How much did you pay to come to this conference? Remember to get your money's worth. people attach classifying information to media in the course of production in order to identify media that's going to be used in later steps in production or to identify media when it's delivered to the end user and they tag this media or classify it with several different formats of metadata. There are as many formats for metadata as there are formats for media itself and maybe more and several of you in the back of the room might be developing your own formats even now.

We've noticed that for example for MP3 files there are many more formats for tagging than there are players in the world. What is going on? You can tag your MP3 files with ID3 version 1, ID3 version 2, Lyrics 3, Lyrics 1, APE. I don't know what these things mean. It's just confusing.

but what we want to do is to make sense of all these metadata formats because hey we're QuickTime. We are the media integration architecture and we can do it. So what we are doing in Tiger is introducing APIs that give you access to media metadata. You can be concerned about the specific metadata format if you need to be or you can simply leave the details of the format up to us and get the information out that you're interested in.

How does it work? When you open a movie you ask QuickTime for a reference to metadata attached either to the movie, to a specific track or to specific media. You can tell us if you're interested in a specific format such as iTunes music store metadata or classic QuickTime user data or you can just say give me a reference to whatever it's got.

Then once you have that reference you can iterate through the items in that metadata by means of keys that are specific to a metadata format or a set of keys that we've defined that we intend to work in common among all the metadata formats that we support. So if you want to get for example the display name of the media you merely have to say give me a reference to the metadata, find me the display name please and give it to me.

And whether that display name is stored in classic QuickTime user data, in iTunes music store metadata or in new QuickTime user data, it's not necessarily a reference to the metadata. So if you want to get for example QuickTime metadata we've defined a new format to overcome the limitations of classic user data. We will find it for you and we will deliver it. And I have a demonstration of that right over here. Data potato do what? Do what? I have a demonstration of it. I'm going to back up over here on demo two, please.

All right. So what I want to show you here is that it's possible now for the very first time for QuickTime Player to display in this demo. It's not the most impressive demo that you'll see at the show, but it's probably the nearest and dearest to my heart. So I'm going to make sure that it works.

is presenting a presentation on the new features of QuickTime API. Yes, it's late night at the piano bar. Okay, here we go. On demo one, I've opened up an MP3 file which contains an ID3 version 2 tag. that includes the name, the display name of the file and that display name is stored as UTF-16, it's Unicode.

And for the very first time QuickTime Player can extract that name from the metadata in the format ID3 version 2 and display it in the title bar of the window. And what is it doing? What is QuickTime Player doing in order to do that? All it's doing, it's a preview of tomorrow's session, is calling the method QtMovieDisplayName. And underneath the covers, the QtKit is calling upon the QuickTime metadata services that I described to fetch that piece of information out. It doesn't worry about the format of the metadata, it just says, "Give me what I want," and it delivers it.

But we can go a little bit further than that as well with metadata and in fact I have. And I did it in the application that I showed you earlier, my simple playlist editor. I complicated it just a little bit by adding the following feature. This is a Cocoa application.

So what I did was extended the Objective-C class QtMovie by implementing a category on that class with the following method, artwork. What I want to do when I instantiate a QtMovie is to fetch the artwork out of that movie wherever it may be. And I implemented that method artwork by using the QuickTime metadata APIs that I described earlier.

What I then do with that artwork is I actually attach it to the movie as a video track. This application is highly visually oriented. What if I want to play some music files? Let me preview these very same music files for you in the finder. If I click on one of them, you'll see the finder merely displays the standard QuickTime movie controller because the only media in there is sound media.

But in my application, which is taking advantage of support for QuickTime metadata, if I select these audio-only files, which nevertheless contain cover art in their metadata, I can do the following trick. as I described earlier I can actually display the cover art as a video track in parallel with the audio. So if I go through these music files from the iTunes music store I can display the cover art as part of my movie.

You can do stuff like this too. We know that there are many metadata formats that you will be interested in as part of your production tools or as part of delivery if you're displaying media. And what we want to hear from you is what metadata formats interest you? What kind of stuff do you want to read in the course of preparing media or delivering media and present to the user or make available in the production process? As I mentioned, in Tiger, we're going to be supporting QuickTime user data, QuickTime metadata, a new format that overcomes the limitations, and iTunes Music Store metadata. This MP3 stuff I did by way of demonstration, we're not actually planning to roll out support for MP3 cover art in Tiger, unless you all email us and say that you can't live without it.

The document of what's new in QuickTime 6.6 does contain information about the QuickTime Metadata APIs. We will be posting sample code as the release process moves along towards the eventual delivery of Tiger and yet you've come to the right place to find all the information that will be covered in the show sessions about QuickTime Metadata. Now, let's see. It says here I have 11 minutes and 6. I don't have to talk fast yet. I'm okay. Don't worry.

but I do have more to say and that is that we've shown you that QuickTime is a digital media architecture and integration architecture that gives you all these great services and we're extending it new media formats support for new stuff metadata not just media. But you don't have to wait for us to extend the QuickTime media architecture because in fact the architecture is itself modular and extensible and the very same coding techniques that we use to add new support to QuickTime you can use too.

We haven't made this point at developer conferences over the last couple of years and I wanted to be sure that we mentioned it this year. Several of you are new to the platform and new to QuickTime. You don't have to email Apple and say you know if only you supported my media container format I'd like you a lot or if only you implemented the following video codec you would be my friend.

You can do it yourself because QuickTime defines the component APIs that allow you to implement those things and sample code for this stuff is available. At the Apple developer connection website on the sample code pages. Many different component types are available. I have an example of a movie import component. Now one of my favorite places in the file system, I don't have to talk faster but I can't pause any longer.

One of my favorite places in the file system is right here in /library/quicktime. There's cool stuff in there. What's in this particular directory, my /library/quicktime is my modified mp3 import component that implements all that cool ID3 stuff that I just showed you. That's how I extended QuickTime for the purposes of my demo.

I will also extend it slightly further. Now you know that on the platform we do have support for decoding AAC audio. We rolled it out with our MPEG-4 support in QuickTime 6. However, we have not to date had support for the container format .aac. .aac or ADTS I believe it is, is a format that stores AAC audio packets and several encoders produce a stream of AAC audio in this format. If I were to drag this .aac file on top of keynote however, it would sadly reject it. If only I had a QuickTime movie import component that knew how to read the audio packets from that container format and create a QuickTime movie. Oh wait, I did that.

I implemented an import audio file component. And those of you who are familiar with Core Audio know exactly how I did this. Core Audio, in fact, a service on Mac OS X, already knows how to parse these .aac components. I didn't actually have to write any hard code to do this. All I had to do was to make use of Core Audio's ability to do that and to make a movie out of a .aac file. So, all I need to do here is to move that cool thing into /library/quicktime.

and then I do have to quit Keynote and relaunch it so that Keynote and QuickTime can match up together again. Keynote can

QuickTime uses what's known as the core services component manager in order to load and to invoke these extensible modules. These modules that do movie import and video image decompression and all this stuff. We call them components. But there's nothing really fancy going on in the runtime.

QuickTime deploys the component manager, uses the component manager on both Mac OS X and on Windows. But the component manager merely uses the native dynamic library loading mechanism that's available on the platform and the format of dynamically loadable libraries that's defined by the platform. So the question is why can't I implement these extensibility modules for QuickTime in Objective-C? And the answer is why of course you can.

Now, there were some problems involving language runtime and stuff that I don't understand too well that made it dangerous to do this with previous releases. The good news is that we've ironed all of this out. And starting with Tiger, you can deploy movie import components of the sort that I just demonstrated or any other of the types of components that you saw listed earlier implemented in Objective-C, taking advantage of all of the great stuff in the Cocoa frameworks. Let me show you what

Okay.

So the component that I chose to implement here, once again, it was Saturday. I didn't really have a lot of time to implement support for media format, so I stole support that existed somewhere else. You may already be aware that in AppKit there's a class called NSBitmapImageRep, and this class knows how to write a number of still image formats. One of the still image formats that it knows how to write is GIF. That's a still image format, by the way, that QuickTime doesn't know how to write.

So all that I did, this is literally all that I did, we'll post the sample code shortly, is define a class for a graphics exporter. And I define the following methods: init with imageRep. So what's going to happen here is that an instance of this class is going to be presented with an imageRep. Then the method that's going to be invoked is export.

and the information about the image is going to be, by whatever means necessary, is going to be turned into the format that this exporter knows how to write to and it returns that exported image in an instance of NSData. And all of the other work about writing the file out or writing to wherever the thing has to go for the purposes of however it's being used in QuickTime are taken care of at a higher level. And this is all the code that I needed to write in Objective-C in order to implement this exporter. This still image exporter, which is in secret sauce. Well, actually it got moved. Export GIF component. Move that once again to /library/quicktime.

and this time when I launch QuickTime Player, it's going to gain the ability to write to GIF files. What I'll do is open one of my favorite movies once again. and I will export using QuickTime Player this movie to a sequence of still images. Now you see that we have this support for exporting video to a sequence of images or any visual media and we have several presets.

QuickTime knows how to do JPEG and B-Met--what's that, some competitor's image format? but what I want to do is export to GIF and only after my graphics exporter that supports GIF was installed was this ability added to the media architecture. What I want to do is export this movie at two frames per second and I'm going to put them in my pictures directory and here I go writing frames of this movie to GIF files and sure enough if I open one of them at random, there it is a GIF file in 256 glorious colors. I think that's the first time in a while that GIF has gotten a round of applause.