Good morning and welcome to session 701, Introduction to Digital Media. My name is Steven Tone. I'm a product manager on the QuickTime product marketing team. And we thought this would be a great way to sort of kick off the entire QuickTime and digital media track by having sort of an intro course that explains, you know, just about everything you need to know about digital media from deinterlacing to 3:2 pull-downs. Everything you need to do to take that tape content you have and digitize it and make it look really good.

And so we have a special guest today. We have Hagie Van Dijk from Discrete who knows just about everything about QuickTime. He's even an old QuickTime guy from Apple. And he will be able to take you guys through for about an hour. So, Hag. Hagie Van Dijk Great. Thank you. Hi, everybody.

First thing I'd like to say is thank you all for coming. I appreciate coming down here and talking about one of my favorite subjects, which is digital media. And also thanks to the QuickTime team for allowing me the opportunity to speak. So let's get right into it. If we can switch to the machine please. I want to talk a little bit about what we're doing here today.

Get this started. So what I want to talk about today, sort of digital media basics, real simple stuff is how do we go from this, which is our camera, you know, taking pictures of stuff, very, you know, well, what I will say, used to be analog format, but, you know, capturing pictures and assembling them, putting them together in very creative ways. This is what a lot of you are specialists at, is really in producing this media. So that's the next step. So let's get to the next real level about it.

And then ultimately, how do we get that? And what do we do with it? And where's it going? Increasingly, more and more, I have to play the video to catch up here, but we're starting to see that we need to take our media. You know, they used to say that editing touches every frame. Well, now encoding really touches every frame.

And your media needs to be distributed in a variety of ways. So how do we sort of sort out all of the issues and logistics with our media as we take it from the camera, where we put it in the camera, to the computer? And then ultimately, how do we get that? Well, we have to collect those images, sometimes in random, haphazard, any sort of format ways, produce them in exciting ways using great creative tools like this, and ultimately get them down into formats that we might have in our pocket anytime, anywhere.

So that's the basic premise of my discussion today. And again, hopefully we'll have some fun. And we will take questions at the end if people are interested. We'll have some time for that as well. So, okay, I'd like to switch to the slides, please. Let me find my little switcher here.

Capture, create, output, capture, management, deliver, produce, deliver. How many folks have been to a trade show and seen these words somewhere in the last 10, 15, 18 years? Often times when we go to shows we see words like this on big banners above the convention center drawing us all in.

What does all this mean? These are really terms that are used to describe the media workflow. So in order, what I'm trying to do today is give us a language that we can use to discuss digital media and hopefully that some of these basics will help bring value to you as you attend some of the other sessions here at WWDC 04.

So again, having these sort of catch phrases capture, create, output, they're stages in the production workflow. We use these terms to describe that. What I'm going to be doing is just using that as a frame of reference for our discussion today and kind of breaking up the media production process into those three categories capture, creation and output.

Just to kind of screw things up a little bit, I'm going to start in the middle with the create part. So create, how do we create media? Well, we're already on the desktop in the center thing and the way that we create media is using the very powerful QuickTime format.

So what is QuickTime? There's obviously one of the reasons why I took this talk, I'll have to admit this, is that how often have you been on the receiving end or been the customer who said, you know, what kind of content are you using? And you either said or heard, I have a quick time. Okay, some of you have got to have heard that before. Well, that's sort of one of the things, it's like I'm speaking here in the United States.

QuickTime is a system software. It is a core set of services that allow applications to treat media in a consistent fashion. QuickTime is a time base, a universal time base for multiple synchronous events. What that means is that on my computer over here, I can play a movie.

My computer over there, I can play a movie. That movie has audio and video, maybe a few other kinds of tracks associated with it, but they play back in sync and then I can play them in different places. Now that's kind of a minor point that we all take for granted, but at one time, we had a real hard time making things synchronize up. The fact that QuickTime sort of handles this as a master time base is again something that we might take for granted a little bit.

But the one that we really want to talk about today, the thing that we're all focused on, and the element that we all deal with, is the QuickTime file format. That's really, again, what we're talking about today. So a QuickTime movie, as it's known, is a container for your media. That's the most important thing to remember, is that, again, QuickTime is the services and its timing references, but it's a container for media. And what I mean by that is it's a wrapper that lets your video frame move through the production process.

And sometimes it's transcoded, but that's the central idea is QuickTime is a container. It's not a codec in and of itself. It has codecs, and we'll get to that in just a second. Rather, it's a container that wraps around your media. And kind of protects it as it goes and is used by different applications in different ways. It is track based.

So again, QuickTime is a container of multiple tracks. We're all familiar with an audio and video track. But QuickTime has many other exciting track types, including MIDI tracks, text tracks, sprite tracks. We're always trying to get people to use these other track types because, again, QuickTime is such a technically robust and fruitful foundation. And fruitful foundation for media, we have all these track types.

So we're just not used to using them. So it's always good to be aware of that. And again, obviously, it's identified by the .move file extension. We're all used to what movies are. So let's talk about this a little bit. What is a QuickTime movie? I realize this is really obvious stuff, but again, it's good that we all sort of repeat it, have the same perspective on it.

Video Frame Size How tall and how wide it is. The actual frame is the reference there. Frame Rate How many frames are repeated one after another. Codec or decompressor We'll talk about that in a second. Data Rate Let's talk about these things. First of all, frame size measured in height by width. The frame size I started with is 720 x 480. That's a pretty big size. I think the resolution I'm here is 1024 x 768. It's the pixel size. A computer screen is made up of pixels.

Those are called pixel elements. That's what that's short for, a pixel. A computer display shows video by turning on a pixel. What they use is a value. Already with digital video, we have a resolution component. How many colors do I see? If we had a black and white monitor with a pixel on it, a pixel has two states. That's white or black, off or on. I get a black and white image with a single pixel.

If I take that pixel and I make it 8-bit, I multiply it, get an 8-bit word that I'm using for that one pixel. For a single color, I get 256 shades of gray for that one color. That's a lot of pixels. I can see the color. I can see the color. The color is made up on a computer of red, green, and blue. We add those three colors together at 256 levels of gray for each of them, put it all together, and we get a few million colors that we're used to on our display.

The main thing to realize is that, again, computers display in pixels. The 720 x 480, that's the measurement that we're talking about. Also, switching to the audio here, audio has the same thing. Audio has a sample size. Much like the pixel, the 8-bit pixel that I just talked about, which is determining the video resolution, audio also has a sample size, and that determines the dynamics. For audio, video quality is the difference in loud and softness, and that, again, is controlled by your sample rate.

That should have been first. So sample size, rather. What I'm talking about is 16-bit audio, 8-bit audio, determining that dynamics there. So, for example, if you have a sample rate or frame rate, a picture--so we have a single frame, and a movie is made up of multiple frames that play back in order, okay? And those frames play back at a specific rate.

So as pictures repeat, again, you get frame rate. We have some common frame rates that the video works at. My video camera here is NTSC. That shoots at 29.97 or 30 frames per second. In Europe, they have a different electrical standard. Which relates to a different video standard.

And the video over there moves at 25 frames per second. Again, that's in a camera. But what we're talking about right now as a container is just that video has a rate. So these repeat after each other at a certain rate, okay? Then the last thing is we have--or here we have a codec. So it's interesting.

A codec is a concept that QuickTime brought us. It means compressor, decompressor. Okay. Before, when we'd have a video frame, we'd have to deal with its uncompressed form. So if I just take a picture, that picture is about a meg or the size of an old floppy disk. I was gonna bring a floppy disk today, but I realized some of us wouldn't know what to do with it, so I didn't bother. But anyway, floppy disk is one meg. So at uncompressed video, it's 720 by 480, one meg.

If I run that at 30 frames per second, that's 30 megs a second. Okay. So that's starting to be a large amount of data. Now, QuickTime introduced this revolutionary concept when it came out in 1992 called a codec or compressor, decompressor. What a codec does for you is it takes that data and shrinks it down, often ten times that original size. It also decompresses it. But because that codec is real time or you don't notice that it's there, then if you store your media, if you store your media in a particular codec, it essentially is ten times smaller now.

Okay. Now, we always have a tradeoff with using codecs. The tradeoff is image quality. We want to preserve image quality. And so for that reason, there's really two types of codecs, two general categories for codecs and how they're used. One of those is for the production of media, high data rate codecs, where I want to take high quality images and bind them together. What I want to do is use a high data rate codec for that.

Then there's distribution, where I want to take my media and I want to play it in as many places as I can and get that media exposure. Those are distribution codecs. So there's two different times. And again, the beauty of QuickTime is QuickTime allows you to convert between those codecs very easily and efficiently, because again, it's all within the QuickTime architects that lets you to do that. So the last thing about codec is again, data rate or video size, which is analogous to video quality. Again, the amount of data that we can allocate for a particular movie would be considered its data rate. And that's measured by per second.

So data rates are usually talked about in megabytes per second for production or megabits per second, much smaller unit when we're talking about distribution. And again with audio. So audio has a similar set of attributes. Again, the sample size determines its resolution. Sample rate really determines how good it sounds. And the data rate is very much tied into that. We use less codecs.

There's really less codecs for audio, more the codecs in audio are either uncompressed for production, and then we have distribution codecs for audio. All right. Now what I'd like to do, I believe, is show you just a quick demonstration of what I mean there. I'm going to hide this and bring up...

If you are a beginner or someone who could use a refresher course, this is a must-see session.

Hage Van Dijk When you have a series of movies, so still image frames, determine your frame rate, and then over time, again, multiple frames. So this is going to turn for us and show us that we have multiple frames. So that's the basics of a movie. Again, frame size, frame rate, codec or format, and data rate. So what I'm going to do is bring up – just stop this, hide that.

What I want to do is just quickly bring up QuickTime. And if I go to export this as though I have QuickTime Pro, Let's just take a look at the codecs available to us inside QuickTime. So here I am exporting inside QuickTime Pro. I have video and audio settings.

And if I bring up the standard compression dialog, you see that inside here, QuickTime has a lot of different codecs associated with it. And so what are all of these things? Well, these codecs are different ways of compressing media for different purposes. So for instance, if I'm doing DV, I can select DV Pro or DVC Pro NTSC.

That's a DV codec. It runs at 3.5 megabytes per second. It's a production codec. So I could take my media, preserve the quality, assemble it. What's really important about production codecs is often they're, what we say, fully – they're all iframe or they're fully editable. So as we start to get into distribution codecs, what we do is we start to pull tricks that manipulate the image resolution and all of the data isn't necessarily there in a codec that is used for distribution. For production codecs, what we do is usually try and preserve all of the image quality. It's called an iframe. And again, iframe formats are what are used for production. So again, JPEG or motion JPEG is a real popular format for production. Because every frame is fully editable.

And as we get into DV, DV is a slightly compressed format. But again, it's always editable. Other choices that you see inside here, I do have component video. So again, production codecs. I also have a few other ones in here. I have MPEG-4 video. And I also have Sorenson video. So these are other codecs that are used for distribution. Again, much lower data rate codecs. So two types of codecs. Okay, I think we can switch back to the other machine now, please.

Okay, so that starts to get us into the create part of it, how we create stuff, again, QuickTime movies. Now let's go back out into the real world and talk about how we capture stuff and the formats that we deal with when capturing video. So, I have a DV camera. It shoots at 720 x 480.

Again, if this was a PAL camera, it also shoots at, again, 30 frames per second. You've heard the term SD or standard resolution, standard definition. What that means is that it's a 4:3 aspect ratio, very common standard definition, and again, the size of that is 720 x 480. You can think of SD as like our television.

Again, our television is a 4:3 aspect ratio. 640 x 480. The reason why this is 720 x 480 is because the pixels are not quite square. Again, now I'm on the edge of going less than, a little more than video basic, so I'm going to stay a little bit away from pixel sizes. But again, just realize that you have SD 720 x 480.

That's the size that we're used to. Now, there's a new frame rate in town, and there's been a lot of buzz over a format called HDTV. Everyone says, you know, coming soon, HDTV. So, HDTV is a much larger size. It's taking our SD size, 720 x 480, and basically multiplying it by 4.

So, there's a variety of HD sizes. The two main ones that we're concerned about are the, what we call 720, which is 1280 x 720 in size, and then 1080, which is, again, a huge frame, which is 1920 x 1080. So, a huge video frame. Now, what's also interesting about HD is that it's also higher resolution.

So, again, it can be up to 10 bits per pixel, which means that we can't even display it on our computer screens, all the color information at once, because our computer screens are only 8 bit. So, dealing with HD has a whole bunch of interesting attributes to it. It's very big frame size. It has a lot of color information to capture.

And what's also interesting about HD is that we're back into a situation where with HD and HD production, when you're building a system around that, often your video quality is constrained by your data rate. And a data rate for an HD signal is up around 160 megabytes per second. So, again, a DV camera shoots a steady data rate of 3.5 megabytes per second.

and HD at 160. So again, a lot more data. What's really interesting about HD production is we get more and more into it, is that we start to learn a lot more about HD. HD is, you know, technically a lot of resolution there. But what we're starting to learn a lot is that there's a lot of issues associated with HD capturing. So you want to go out and shoot an HD movie, all of a sudden you need a better makeup person or a lot better makeup.

You need good lights. All of the detail that film and DV cameras was so forgiving about, because we're in the initial stages of HD production, we're really suffering from that. And so we have to blow a lot of smoke on stage to create a lot of depth. The images tend to look very flat. And again, over time, I'm sure that we'll work with lenses and ways of softening that up so it's a little more useful. But today, HD is this huge frame size, very hard to struggle with.

But once you get it, obviously the quality is four times that. Now, we also have frame rates that when you make a movie on your computer, the frame rate can be any frame rate that you want. Again, the idea of starting with the idea of a QuickTime movie as a frame size and a frame rate is just to let you know that those sizes are arbitrary and that you can make them whatever you need to. But out in the real world, sizes are dictated to you by the hardware. So again, my camera shoots full size.

That size happens to be 640 by 480. And so, again, we're starting with the camera. And again, the camera shoots full size. That size happens to be 640 by 480. Runs at 2997. Now, film and HDTV run at 24 frames per second. So that's a little bit different, a little bit slower. And actually, the difference between 24 frames per second and 30 frames per second is really enough that that matters about how the background looks.

That there's actually aesthetic considerations made to film shooting at 24 frames per second. And how you can pan a subject on a camera. And the way that the camera is panning the subject is by the way that that looks versus video, which runs at 30 frames per second. Actually, it's interleaved.

And I'll get to that in just a second. So it's actually showing you 60 images per second. So video is much higher resolution than the 24 frame for film. That's why, again, video is very popular for sports, any sort of content where you need a lot of fast action. Again, video resolution at 30 frames per second allows you to capture that.

Okay? So, again, out in the real world, we have different formats. We have SD. We have HD. We bring those in. So what we need to do at a certain point is capture that video. Now, I used to say that this was... an analog versus digital scenario, and that it was real easy that our cameras were always analog, and that we'd just be digitizing that. More and more, though, it's really portable versus not.

That, increasingly, our cameras digitize on-site and already have converted that. So this is a digital camera. This is a DV camera. It's a mini-DV camera. It's very old. I've dropped it about 14 times from places like this. You can see I need a new one. But the main thing to think-- that, again, at-- So, at 720 x 480, this is 3.5 megabytes per second, so the video comes in.

When I capture this, okay, it's digitized right on the camera, so all I need to do when I capture this is I take the FireWire output. What I'm doing is I'm moving that digital information between my camera and my computer. So, you can think of this as a hard drive.

Again, I can push the video back, but I'm just taking visual information in a digital format and pushing it back and forth with my camera. So, DV is very easy to use because the signal is already digital. If you have an analog format, we need to digitize that.

And again, there's a variety of products to do that. We have the IO products from AJA, again, very high-end cards that let you do component input. If you have, say, a computer that's a little bit more advanced, you can use that. We have a lot of products that are producing on digital beta. Digital BetaCam has a digital output called an SDI jack. And again, that SDI jack you can bring into a DeckLink Blackmagic card, again, an AJA box, and bring that in.

But at some point, you need to take the video from that outside world and bring it in. And again, that's where you capture. And again, what we're talking about there is that you have to be aware of the frame size, the frame rate, the codec, and the data rate of all those different formats. Thanks.

OK, so we talked a little bit about capturing, again, a variety of ways of capturing. So once you've captured that video, what do you do with it? And again, we showed the video of the different content. There's a variety of applications. What mainly happens in that production phase is that you're taking those QuickTime movies, you're combining them all together, either editing, which is a temporal issue-- so you're combining to create a story-- or you're compositing them together.

The main point about production, though, the creation of content, from a technical perspective, is that you end up mastering that format at the same size. Again, a common misconception is that I'm just going to output that video once I'm done to a small resolution and be done with it. And the point, really, of having the discussion today is just to let everyone know that, really, increasingly, you're going to go back to that content and want to deliver it in multiple formats.

So we must make a master version of that content that, again, can be used in many different places. So the best way to do that is to produce it at full size and then to take that master file and start to convert it off to different formats. Also, at some point down the road, you're going to come back and want to make another version of it-- a DVD, another CD, or something like that. So again, having a full ResMaster is really important. So when you're producing your content, again, DV, I'd output a 720 by 480 movie, 30 frames per second, using the DV codec. So at a high, again, a distribution data rate-- or production data rate, excuse me.

So, once we've gotten our video, we've created the master, now what we want to do is start to distribute that video. What formats do we take to communicate our message out to multiple formats? There's really a few different sizes there and again, a few different things that happen with that video as we go through the production process.

The first thing is we want to make a DVD out of it. That's the most popular kind of format or very accessible format for making video. Well, DVDs are interesting. If we take our little formula here, frame size, frame rate, codec, data rate, the frame size of a DVD is set for us. It's full size, 720 x 480. So what's great about a DVD and one of the reasons why it looks good is we've preserved that full frame size of our media.

Now, MPEG-2 is the codec. So that is actually the compression scheme when we talk about MPEG-2. MPEG-2 is a compression scheme and that has a data rate associated with it. And the data rate that we most commonly use for DVDs is 5.2. So that's 5.7 megabits. Again, a DVD has a data rate choice of up to 10 megabits. But I work with a lot of compression products and what happens is a lot of times people have a short 20-minute movie. They want to go ahead and make a DVD of that and we're all concerned about quality.

We want to get the highest possible quality from our media. So if I have less than an hour and 15 minutes to put on my disc and I'm compressing that to DVD, I'm going to have to compress that to MPEG-2. What I'll often do is turn the data rate quality up or what I would think is to turn the video quality all the way up to 8 or 9 megabits per second. Go ahead and encode that media and then go ahead and play it.

Well, there's a little bit of a problem with that and that what we've determined, you know, sort of extensive testing, has found that if you have a real high data rate like that, you're going to have an inconsistent playback experience between the different DVD players. So increasingly, when I talk with people about making DVD content, I really urge them to try the data rate of 5.7 megabits and to try and use filtering or other processes to preserve that video quality as much as possible because what we found is that a 5.7 megabit DVD really plays well in all places.

Now as we move from DVD to other formats, for instance web video, the choices open up a lot more. Again, with web video, what we're trying to do is to get the video to play smoothly. That's where the data rate comes in. But the sizes are rather arbitrary. And really, when we're talking about compressing for video, we're looking at those same trade-offs. So I have uncompressed video that's full size. The first trade-off that I can make is the frame size to reduce that data. And again, common web sizes is 320 by 240.

What we're trying to do with web video is really get a theatrical experience. Whether or not we realize it, we all want a theatrical experience from the web. So we're really trying to get as full-size codec as we can. My work experience has been rather interesting. I've been doing digital video for 18 years. And what we started off with was a technology called QuickTime, where everyone complained that they were little postage stamp movies. And what we did was we worked really hard to, with hardware assist, get that to be full screen, full motion.

And we achieved that in about 1994, the variety of products to do that. Then what happened was about 1996, the Internet became more and more prominent. And we started to get digital video happening on the web. And there was once a time when Peter Hottie, who was the QuickTime architect, asked me to make him a 60K movie. No, he wanted a 30K movie, 30K bit movie. And I said, "Well, Peter, that's, you know, going to be 60 by 90 and not look too good." He says, "I want a movie that played." And so I made the movie. He used it for the demo.

But what was interesting about that was that, you know, I was working with a lot of people. And what I was thinking about that was that we've now, we went to a software world. And then for, after 1996, what happened was slowly we've been trying to get that frame size to grow.

So it's in stretching that frame size, it's in codec developments and increasing the data rate. Can we actually stretch that size out to be full size? And we've almost achieved that. But there was an announcement yesterday, very significant announcement for anyone making digital media. Which is that Apple's going to include a new H.264 codec in the next version of the OS in the Tiger operating system.

Now what's really exciting about that is that right now we can do a fairly good job of stretching MPEG-4 to full screen, full motion. It's pretty difficult to get it. But if we increase the video by a factor of four, that's going to look fantastic. So we really look to MPEG-4 to deliver that.

And we're going to use MPEG-4 to deliver a software version of a codec that is full screen, full motion. Now, the interesting thing for me is just about the time right now that we're getting that together. There's a new thing on the horizon. And that is everyone's driving around in their car with their Wi-Fi, at least in Hong Kong. They're watching video on these portable formats.

So again, what's kind of interesting about this is we work really hard to make it full screen, full motion on software. And they invent a new platform for us. So this is a Clie. This is a TJ37. And what this is is a 200 megahertz CPU. So again, what they've done is they took my Beige G3 or so and they shrank it.

Switch to my machine for a second please.

And so what this is just a picture of relative frame sizes for you. So again, the big white box is our 1920 by 1080 frame size. The next box is our 1280 x 720 frame size. Here's our 720 x 480. Here's our 320 x 240 standard web size.

Here is our 160 x 120 cell phone size. One of the ways we are reducing data is by stretching the frame size down. We can do the same thing with frame rate, but we are a little less forgiving with frame rate because that has to do with temporal motion or motion over time. Again, what you want to be aware of about frame rates is that we have really good eyes as far as seeing deteriorated motion and motion resolution.

What we really want to do is preserve motion wherever possible. You have to really look at your content to see how fast it is moving to determine whether or not you can switch the frame rate. I kind of go back and forth on this and I am currently in a personal phase where I try to keep my frame rates matching as much as possible. I will make my frame rates 29.97 or 24.

You can hit 15 or 7 at lower bit rates, but again, the movie gets choppy. The nice thing about that is, again, with QuickTime as the technology, that choppiness will at least be consistent. You won't get snappy video, you will get choppy video. That is why I am an expert in choppy video. Great, we can go off of there. Back to the video.

Okay, so actually I want to do a little bit of a demo now and just talk a little bit about distribution. There's going to be some seminars. We can go actually back to my machine. There's going to be some sessions during the week about

So I'm going to be using Cleaner as an application to sort of talk for a minute about the process of distribution and how to get the most out of it. The reason why I want to use Cleaner here is because we're going to be talking about pre-processing.

When you spend your time creating your content, you don't really want to think about color values and things like that. You really want to have a time when you're being creative and assembling your media. But at a certain level, you have to come back and take a look at that media in terms of distribution and go, "Okay, now I need to distribute this. I need to shrink this thing down many hundreds of times smaller than its original.

How do I do that?" And that's where compression applications offer you these pre-processing. So the first thing, again, video, when it's shot, it's shot in fields. So we talked about this before where you actually have alternating fields. The reason for that is that when we started, remember that a computer display has a pixel on it.

A single pixel is going to shoot that always on when you see that display on your computer. Well, a television works a little bit different. It's a little more arcane, if you will. A television works by scanning every line. It draws one line in a 30th of a second.

Then it goes back and draws another line in a 30th of a second. That's called interlaced video. Again, a field that's made up of these alternating fields. They're scanned even and odd depending on your video format. So DV, it's even, odd, even, odd, even, odd. And again, that alternating back and forth happens at the point of the camera. So all of your video resolution is caught in that alternating field.

And when you digitize that video, those alternating fields are also captured. And if you produce your video and output it at full resolution, those alternating fields are maintained through that process when you're doing video. So what happens is when you get to the computer screen to view that video, all of a sudden you have a problem because the computer video doesn't display that way. It's always on.

And if you're going to show that frame with both of those, you're going to see a difference there, the interlaced difference in what that motion is. And if you're not careful with how you process that video, all of that difference in motion, is going to be interpreted by the computer, which doesn't know what it is, as noise. It's going to create a, giving you a very blotchy outcome.

So again, one of the things that we have to do when we take video in is start to deinterlace it or reduce that data down by taking away one of those fields and thus capturing that motion smooth. And we're going to use Cleaner to see that in just a second. So again, video alternating and deinterlacing is really critical to data reduction, but it's also critical to getting a good look for your web video, because again, those motion artifacts will be interpreted as noise for you.

Another thing is that if you're using film, film is shot at 24 frames per second. There's a problem with film though, if I edit my film on video, video runs at 29.97, so that's a different frame rate. So there's a difference in the frame rate that film is shot at 24 and the video at 29.97.

So how do we deal with that? Well, there's a very standard process called telecine, which is the process of moving 24 frames per second video up to 30. So it's not like the video case where every field is interlaced, you basically have the 60th of a field. What's happening with telecine video or video that originated on film is we get two progressive frames, so progressive frame, just the whole frame is captured. We get two of those and then we get three interlaced frames.

So we're going to use the telecine process to make up that difference and it's that difference of five over four over time in the telecine process that makes up that 29.97. So we've taken four frames, mapped them to five frames and added three interlaced frames and two progressive frames. So that's where we hear that term three-two pull up or pull down.

Again, it can be two-three or three-two depending on where you start, but the main thing to realize is that there you have two progressive frames and three interlaced frames. Now when you're in that situation and you apply a deinterlaced filter, what you're doing is only, you're only doing the appropriate thing for those three frames because it's mixed of progressive and interlaced frames. Deinterlacing all of that is going to cause a deterioration on those two progressive frames.

So what we need to do when we go back to web video is we need to do a process called intellicine, which is to remove that three-two pull down back to that 24 frames per second video. So one of the really what I consider the pinnacle of web quality is the Apple trailer site. The team in Cupertino does a fabulous job encoding those. And if you look at them, they're all 24 frames per second.

And again, the key to that was that those, all those movies have gone through an intellicine filter to restore that original frame rate. Never know where your media is going to come from today. So again, restoring that original 29, the original 24 from your 29.97 is important. If you're doing stuff in PAL, it's just a 1% raise and lower. So it's a little bit different.

They don't go through quite the intricate process. The telecine process in telecine is really only relevant to 29.97 video. Okay, so again, as I was talking about, the next thing is how to process the video so that it looks good. We're taking uncompressed video that's this big and we need to restore that. And we're running through a hole that's this small. So we want to do every single thing we can to preserve the quality of that video. So again, throwing off the frame rate, making the frame size a lot smaller. Well, that helps. There goes half the data.

Deinterlacing that video takes away half that resolution and actually helps us reduce that data. But we're left with a variety of other techniques to preserve that video. And that's where you're going to see some sessions on pre-processing because the key to good looking web video is pre-processing. And what I'm really talking about now here is web video.

When we talk about DVD, we're not going to do a lot of processing to our DVD because it's already full size 720 by 480. The number one thing that I see people running into with DVD production is they need to go from their interlaced format to a progressive format.

They're going to be playing DVD that will always be played back on a computer. The only problem with that is that when you take your full size interlaced video, and you deinterlace it to make it progressive, you've eliminated half the resolution vertically on that and you've softened that video.

So the one thing that you need to be aware of when you're making DVDs when moving from interlaced to progressive is your videos can go soft. Now there is some tricks that you can do with the green channel and balancing that stuff out. But just remember that, again, video goes soft.

So what you need to do is, you can go to the So I'm going to walk through that now a little bit with Cleaner and talk about what some of these pre-processing principles are and how you use them. So I'm going to use my Bubbles movie for this.

This is a before and after view of my movie before I actually compress it. Let's look at the before view. Again, I've intentionally picked this piece of media because it's got a lot of motion. You'll see these sort of horizontal lines in my video in this before section.

What those are is because what we're seeing is a frame of video of a bunch of bubbles rushing up at us. So the whole thing is moving. And again, all those interlaced lines, what those are is the difference in movement between that 60th of a frame. So this is where I'm talking about deinterlacing your video in order to get the best possible look.

Because again, if I were just to compress this without deinterlacing it, look what I'm feeding the encoder. It's a mess. It's never going to get above this high-level noise. It's never going to get above this high-level noise to get me any sort of picture of an image at all.

So again, deinterlacing as the first step to creating a delivery for your media, an output platform, is extremely critical because with all that otherwise, you're going to get all that noise in there. Now, cleaner is kind of interesting. Cleaner has a feature called adaptive there. You see now if I go to the output also, conversely, notice how much that cleans up, particularly this group right here in the center. See how much just pulling away has a lot of noise.

Half of that field really cleans that video up to get that started there. So again, deinterlacing, probably the most important filter when making web video. Because again, you're able to decrease that video, but you're really preserving the size, the look of this video. Adaptive deinterlacing. If I have a Batman shot, everything's at an angle. I have lots of diagonal lines in my video. And I were to take a picture of that, I'm going to have to do a lot of diagonal lines. and in video and I were to deinterlace it.

Because I had diagonal lines in there and I take away that half every other line, what I'm doing is starting to introduce jaggies into my output. Little jagged lines there. Well, with an adaptive deinterlace, you're only deinterlacing pixels that move, so you're only going after these artifacts to get rid of that motion stuff to, again, prepare your video for a distribution codec. So again, adaptive deinterlace, very important.

You'll notice that Cleaner has a Sharpen filter in there. That Sharpen filter is just to brighten things up again, because codecs inherently dull our content. So again having that Sharpen codec in there is very helpful. And what we have is a concept of noise reduction. So with all of your video, when you go down to lower data rates you're going to be using noise reduction.

Noise reduction filters are very, very cool. What they do is they'll take the edges, they'll preserve those edges, and they'll apply blur to everything in between. So the areas of less detail that our eyes are much more forgiving for, again, can see this noise reduction filter. It helps scale the video down very well.

So again, how to use noise reduction is really keep it on mild or moderate, depending on how much motion you have. Again, we'll do that. The next filters just to talk about for image quality, we have a few of them that you need to be aware of. First thing is Gamma. There is a difference in Gamma between the Mac and Windows platforms.

There is a difference in gamma between the Mac and Windows. A Mac computer is darker than a Windows display. If you are always producing your media on one platform on a Mac and you play it for your Windows friends, they might say that it is a little bit dark for them.

The way that we handle that when producing content for the web is to add a gamma filter for that. What a gamma filter does is just brightens up those mid-tones. You can see here, I'm just going to turn these off for a moment. "Just update that. So if I add just a little bit, I'll turn the gamma filter off. And again, it's just going to take that and brighten that up just a little. Not too much. Go the other way.

The Cleaner has two other very important filters in it that are used for making the video really look the level of quality that we're talking about on the Apple Trailer site. That is the concept of black and white restore. So what these filters do here for you is, again, they'll take and they'll actually regenerate those colors for black and white for you. So this is significant because, again, it's going to really ground your – it's going to give your content good color values.

So you might say, "Well, there's not a lot of black in this video. Why would I use a black restore video on content like this?" Well, if we notice, if we bring up the amount of black restore in the video image, you'll see that this blue that's all over this content really does have elements of black, that there's a darkness associated with this whole background inside this video clip, and that I can sharpen that up or increase the clarity. If you will, of that kind of stuff using a black restore filter.

So how you actually use these filters is you'll notice they have a smoothness control. So if I just turn on the black restore and then turn this way up, you're going to see that at a certain level, it's going to start to get very blotchy. So you'll notice in here it's starting to get – so black is being added and just regenerated into all these areas. use a smoothness control here.

What I'm able to do is just get a little bit of softness and get a little bit of edge and nice lift to that and really get those colors back. So to get good contrast in that video, I can go ahead with a lot of smoothness and softness and bring up that with that black filter. Same is true for the white restore.

So again, same principles apply that all colors have an element of white in them and that by regenerating this black and these white colors, we're able to really preserve those color values. So that when you go ahead and compress that video, you get the highest possible output. So we call this media mastering. The other thing is to realize that when you do audio, again, for audio formats, audio is an interesting case.

For a long time, starting with QuickTime 3, we had audio codec called the QDesign Music Codec. So QDesign was a very good codec for reducing the data rate of our video, but QDesign did have an unfortunate artifact associated with it. It would produce a phase shift at high – at the upper frequencies.

Basically, it would make your audio sound – So you get a swishing sound with the QDesign codec. Now, that was followed up by the very popular MP3 codec. And MP3 was available, I believe, with QuickTime 4. It came in and, again, MP3 – very popular, very portable format.

Okay, so I'm starting to wrap up here and I really wanted to finish again where I started a little bit, which is this concept of video. What is digital video? Well, digital video is a quick time movie or other formats, but it has a frame size and a frame rate. These frame rates are really, and a lot of the size is controlled by the codec, and ultimately that codec has a data rate associated with it, and that data rate really is what determines the quality of your video.

Great, can we switch back to the demo machine please? So I wanted to show you that again, what we've talked a little bit about here, we've introduced this idea of this great new H.264 codec and what's going to be happening there. And really want to talk a little bit about some of the trends here in media. So one of the exciting things that we have here inside Cleaner is we have the ability to go out using the Konoma exporter to the Palm OS. Again, Palm OS devices are extremely good at playing video.

I have my little Clia here that, let's go ahead and play this, and I think more and more what people are seeing is that our video is on our cell phone, that we can take pictures with our cell phone now. So it's just a matter of time before our cell phones will be able to actually display video, moving pictures.

Okay, so we've got the single frame, now we need to get multiple frames. There's also a really exciting component to this technology, well there's two other components. First of all, there's a thing called Blur. 802.11, wireless technology that will be able to allow me to view this video anywhere. And increasingly we're seeing more and more, you probably have it at your house, but increasingly down at Starbucks. You can go to places, just last week it was announced in Washington, D.C., they're actually deploying Wi-Fi in a whole neighborhood under city blocks.

So more and more we're going to be able to take media wherever we want. Now, if you capture a video on your cell phone, and you're using your phone, you're going to be able to take it to your home. Realize that the resolution is this size, it's going to be 160 by 120. So we're going to have an interesting period of time when we try and take that video and scale it up to full size. But again, we're going to have very exciting codecs like H.264 that will help us make that transition.

So again, what's very exciting about that is what will happen, you know, where this media is going. And increasingly, what we'll see is that you'll produce your video in these high-end formats, perhaps in a lot of different formats. Perhaps HD, SD, very high data rate, a lot of creative stuff happening with that. You get to a point where you need to distribute that media. And oftentimes you're going to distribute it in multiple formats. You'll make a DVD of it, you'll make a web movie of it, you'll make a cell phone version of it.

And again, the idea here is that we're distributing this in multiple formats. So you're going to make a master version of that. And you're going to then, from that, derive all of these other versions at these lower data rates. And then you're going to make a version of that that's much smaller sizes. Okay. We can go back to the demo machine, to the presentation machine, please.

So again, we've hit some basics today and just sort of discussed what QuickTime was. QuickTime is a container for your media that's extremely important that it's not just a codec, it has codec attributes associated with it. This is – the reason why I have to draw that contrast is there's some other formats in the world where you don't have this level of technical complexity and this amount of tools and really rich features to use, where the codec is the name of the format. So I shouldn't have to say what they are, right? Thank you. Okay, good. So again, QuickTime is really codec-based. So it really gives you that frame size, that frame rate, codec and data rate.

And then again, data rates determine how you produce it, how that image quality looks, and that we have different codecs for distribution than we have for production. And again, the codecs for distribution are very small, the codecs for production are very high. We have all these frame rates, different sizes of video that we collect from all over the world. We bring those all into our – we digitize those on our computers, assemble those, and then output them at still more smaller sizes in different formats.

So again, lots happening with the media. I think that we're going to now take some questions if people are interested in that. And again, what we do is we have – first of all, who to contact. If you have any questions about the topics that we've talked about today or just QuickTime and QuickTime technologies in general, we're happy to answer them.

And again, we're happy to answer any questions that you may have. So again, we're happy to answer any questions that you may have. So again, we're happy to answer any questions that you may have. So again, we're happy to answer any questions that you may have. QuickTime and QuickTime technologies in general. There's a few very important names you should know.

First is Guillermo Ortiz. He's a developer relations person for QuickTime. And again, wonderful key person to know for any of your questions you have regarding QuickTime and the media track here. In addition, my colleague Steven. Steven is available. He's the product manager for QuickTime, available for questions and follow-up. And then myself, I'm Hegy Van Dyke. I'm available here at Discrete.com.

And my role is I'm an applications analyst. So I'm really a person who works with all phases of media production and delivery and actually product development and delivery. I work for a company called Discrete. At a certain level, we're known for a product called Cleaner, which is a kind of transcoding tool designed for compressing video from one format to another. But Discrete also has a real legacy and a history in high-end media production.

We have a product called the Flame, which is an SGI-based real-time independent resolution system designed to give you very high-quality motion effects. And again, what's interesting about the Flame is those deal with still image formats. So again, large stuff. And we're going to take some questions, I believe.