Good afternoon. Thanks for joining us this afternoon. I'm Brett Halle. I'm the director of Pro Video Engineering. Today we're going to spend a few minutes and talk about how to interface professional video hardware to Final Cut Pro 4.0. One of the big challenges in the video production space is how to actually get all that data into the computer in an effective way, whether it be video and audio and all the various pieces.

And then when you're dealing with that much data, be it standard definition or high definition, how do you manipulate it and manage it and then display it and do things like effects in a very highly efficient and quick way? To talk about how that is done and how you can produce products that can work with Final Cut Pro 4.0, I'd like to invite Ken Carson, the Final Cut Pro engineering lead, to talk about this.

Good afternoon. We've just released Final Cut 4.0 a couple weeks ago. It came out. We introduced it at NAB this year. And with that, we've really shown our commitment to the high-end space for video production, a whole suite of tools. And today, we're here talking about how we can interface hardware with our application and what you can do to help us produce even better tools for the industry.

So we'll be covering a number of topics here. There's a good number of you in the audience who know more about this than I do at this point, having developed a number of products for us. Then there's a bunch of you probably new to this, and I'll be trying to split the difference between those of you who know it all and those of you who don't know much about it.

So we'll be describing two kinds of video hardware that works with Final Cut. Those that just do the input and the output, as opposed to those that do a lot of effects processing on board, beyond what the host computer could do. We'll be talking about RT Extreme, which is our new software model for effects processing. And we'll be talking about the AV playback architecture, which is all built around QuickTime.

As an extension to that, we then use an enabler file, the RT-Enabler, to let Final Cut know about the capabilities. We'll cover that and talk about the opportunities for the developers with our products. Following that, I'll have Gio come up and talk about the uncompressed codecs that we've developed and released with this version, and the Pro I/O protocol and driver set that we've developed.

So with Final Cut Pro 4.0, actually all versions of Final Cut really, it's built on QuickTime. That provides a lot of flexibility for how it works, what kind of hardware can be attached to it, as opposed to everything coming from us and you get what we provide. You can handle with through QuickTime a wide range of resolutions. We can handle, for example, 320, 240 JPEG, all the way up to HD online using hardware solutions.

Software versions, so you buy a computer, Final Cut Pro, that alone is enough to let you work with software versions. For example, the offline RT, the DV, and FireWire out for DV. Adding on hardware gives you the ability to perform SD and HD, and that can either be the I/O or accelerated versions.

So a little history on how the effects have come about within Final Cut. Final Cut 1 came out in the spring of '99, and the ability to do real-time effects was a real obvious missing piece there. So version 2 took us a little while to get there. We released the first version with Final Cut with real-time effects based on QuickTime.

In that model, we were using hardware acceleration to perform all the effects. Later that year, we released Final Cut 3, which introduced the G4 real-time effects. For that, we took the model that we had, developed for hardware with QuickTime, and applied a software engine underneath it, replacing the hardware that would have done the decompression and the effects with software, so that you don't need to add any additional hardware just to do the simple effects.

And this spring, we introduced Final Cut Pro 4.0 with the RT Extreme model, which really enhances what we can do in software. We are now handling full-frame images, different resolutions, a lot more effects, and expanded to be able to use the video out from our software effects. In version 3, there was no monitoring on external devices. And we've also extended the number of effects that are available for the hardware developers as well.

So, to go back through the two different kinds of the video hardware, this is a really big distinction. You can build a board that has a lot of processing power built into it and perform the effects in the hardware. And if you go with a system like that, you can provide a lot of capabilities, just not possible in the base CPU.

On the other hand, you can also build a system that just provides the video I/O. If you take just the computer and the software, The computers come with FireWire, so if you're working with DV, you're all set. But if you're trying to work in standard definition or HD video, there is no plug on the back of a Macintosh that lets you connect direct to your video monitors or tape decks. So you have to have a piece of hardware in between there. That hardware can be as simple as just doing the conversion from the uncompressed video to the analog and digital video worlds.

I'm going to go through some of what's new in RT-Effects with RT-Extreme. One of the things that we've got in there that's with the new effects, all of the effects scripts are now capable of being real-time, with the exception of those that read particular frames back. That's a very small set of them.

The effects scripts is the scripting language that all of our effects have been written in. If you're familiar with the program, the effects builder tool is built into the application. You can see all of these scripts are written. Another aspect of the new effects in version 4 is the variable speed effect.

So this lets you do keyframe speed, real-time speed, and variable rate speed. We've taken advantage of OpenGL to do a number of our effects processing. For those of you who've been catching some of the OpenGL sessions, you can see that there's a lot of capability out there in the GPU, and we're taking advantage of that now.

We added beyond version 3, in version 3 we had just offline RT and DV as our file formats that were supported for real-time. In version 4, we've added uncompressed formats as well. At this point, we are only enabling those when you connect a hardware board to them since there is no I/O directly available for the uncompressed formats.

Internally, the video processing that we do is generally YUV-based. In this version, we've optimized it so that we're using a combination of 422 processing and 4444 processing based on the effects being performed to get you the best performance. Another feature that's been added is real-time 3-2 pull-down insertion.

So with cinema tools bundled in the application and the advent of these new cameras that are shooting, like the Panasonic DVX-100 that puts 24-fps data onto a TV NTSC tape, we can capture that, get the 24-fps data out of that, do true 24-fps editing and playback reinserting the 3-2 pull-downs that you can display it out to an NTSC device.

So I'm going to come over and demo a few things and highlight what's unique in this version that the hardware developers are going to need to know how to get to. So I've got a sequence here. You can see the various tracks and video and audio. I'll just play through it once and then describe a few things that are going on in here.

So you can see that the effects are being performed here. None of this is rendered. Now, first off, over here in the RT pop-up, we've got a number of settings. This being DV, we've enabled a choice of quality to perform for playback. If I switch up to high quality, we'll see that more of the things in the timeline will not be real-time. So if you look at the colors over here, this section being red indicates that it doesn't think it's real-time. This green, it thinks, is if I switch to high quality, this other clip is now beyond what we can guarantee will be real-time and software.

But we've also added another feature called Unlimited RT. Now this removes the time constraints that we have in here for figuring out what's possible on a particular machine. And in general, we have to be pretty conservative with those estimates because Different machines, you have different kinds of disk systems.

If you have short clips, it's possible to play through something that's more complicated, but if you try extending that clip, it won't work. So in the normal model that we use, we are pretty conservative about it. And if I play through this section where we've got multiple clips layered on top, you can see that even in full frame, it is real-time.

And we weren't dropping any frames. So I want to highlight the different render bar colors that we've got up here. A change from version 3. Over here in the left, we've got this section with the cross-dissolve that's being performed in real time. Now that I'm in full resolution, that's coming out in a kind of grayish-green color, because this is full quality. There's no advantage to rendering this. If I switch back to the medium quality mode, you'll see it as bright green, which is what we used to use for the real-time effects.

Over here, this section, now that I'm in unlimited RT, this shows up as orange to show you that it's beyond what we think is really going to work correctly, but we'll let you do it. And this section over here is yellow. This is a proxy effect. So a proxy effect, as I'm scrubbing through it, you can see this has a soft edge on it, but we're not able to perform that part in real time, so we're showing it in yellow. And as it plays, you see it's being performed with a hard edge.

And one of the things that we've got in this version is the ability to work with multiple codecs that handle the same formats. So over here in the system settings under effects handling, we've got on the left side a list of all of the formats that have enabler entries saying that they can be real-time in some form. And then over on the right there's a pop-up that lists the different manufacturers who are providing drivers for that format. So this is an NTSC DV sequence, so we're using the Final Cut Pro codecs.

And I added an enabler that would list a couple of others here so that if I choose Another codec or choose none and come back here. We're going to see that the timeline changes from all those being real-time to being read requiring rendering because nothing is going to perform those effects.

This was a problem in particular previously with DV where a couple of different vendors had DV drivers and we would disable our DV if we found one of them installed. And they needed external control panels to turn on and off their versions in order to turn ours on. And I'm expecting that with version 4, a lot of people are going to be providing uncompressed solutions that are going to be overlapping down here. And this will let people within the application switch who's going to be running the effects.

One other thing that I wanted to highlight over here as a new feature is the floating point rendering. Now we don't do floating point processing in real time, but the option is over here in the video processing tab to choose to use the floating point rendering engine. If you For 10-bit uncompressed video, you'll want to do this as a way to preserve the image quality. Otherwise, you'll be going through 8-bit paths. Alright, so let's go back to the slides.

So one of the important things to remember about ArcGIS Xtreme and using software effects is that, um, They'll just keep getting better. In fact, what we've got here, you know, this is a dual G4 1.42 gig machine. But you can imagine that with what's been introduced here at the show, with the G5 and with the software improvements in Panther, that the capabilities of the system will just keep improving.

On the hardware side, all the things we've been adding for the software effects really translate as well. So, for example, the variable speed effects are available for the hardware. And in addition, the ability to mix formats in a sequence that we've added, which can be enabled with the hardware products at this point.

Previously, if you had more tried to set up a sequence for, say, uncompressed and you wanted to drop a DV clip into it, you would need to render that in order to play it back. Even if it was possible for you to decompress it in real time, we weren't able to get the data queued properly without a glitch in between the formats. In this version, we're able to do that using a new filter that we've developed, which I'll get into later.

So on the floating-point rendering, this is 128 bits per pixel, so it's 32 bits per channel. We're doing this with YUV plus alpha, and it's what we call the R4FL format. This is a variation on the Our 408 format, which we use internally for our YUV plus alpha image processing. Our 408 is 8-bit, pretty much based on the standard 601 style of YUV, but it's a fully sampled 444 format. And this, with 32 bits per channel, is using approximately the same layout.

Of course, videotape machines at this point don't handle 32 bits per pixel. They top out at around 10. So as a more efficient format for storing that kind of data, there's the V210 format. So V210 is storing 10 bits per channel packed in... Four pixels and five bytes, five 32-bit words, something like that.

We've developed a lot of optimized code for doing the conversion between our 4FL and V210, 2VUI formats. And we're making that available to the developers so that we get consistent results. And we thought about leaving it all on our side and having the interface read data from either 2VUI or V210 format and doing the conversion on our side, where it would be a lot simpler. But it means that there's one more buffer copy going on. If we give you the code to you, you can put it in your components and the buffer copy on the decompress can be doing the translation at the same time.

One of the other advantages of using QuickTime format in between here is that Our developers can write drivers to do any format they want. You're not restricted to the V210 or 2VUI formats that we've developed for uncompressed. You can come up with a 16-bit per channel format if you'd like, or you can come up with other more highly compressed formats as well.

So here's a block diagram that shows how we're performing the effects. Now this is pretty much the QuickTime model of how you do things. On the left-hand side, we've got the video path, and in the center is the audio path. So the video goes through the QuickTime movie, and then you've got the codecs that process that and send it out to the hardware, and then the hardware interfaces to the external world videotape machines or monitors.

The pieces shown in gold are the parts that you would be writing. The blue parts are part of the system software, and the application in Final Cut's up there at the top. The audio path in the middle is sort of the same. In the standard QuickTime model, the audio would be driven directly off the QuickTime movie. Within Final Cut, it's kind of a roundabout path. In the QuickTime movie, it's being called driving it all, but it's going back through Final Cut to provide the data directly to Core Audio so that we get the multi-channel output at this point.

Now, standard real-time effects that you'll be using, the first three that I'll talk about are effects that QuickTime has defined. So there's composite, alpha gain, and cross dissolve. The composite is selected within the Final Cut interface through the layering of the tracks. If you've got more than one layer of video, they get combined through the composite effect. Alpha gain is really the opacity filter. So that takes a full 100% opaque track and adds some opacity to that so that if you then layer it on top of something else, it's blending into the background.

And the cross-dissolve transition lets you take two clips, put them next to each other, and cross-dissolve between them. So building your effects off of the standard QuickTime Codex makes it a lot simpler to develop these, and this is really the core set of being able to do anything in real time.

In addition to those, we've developed a set within Final Cut.

The Speed Filter lets you define variable rate speed and play it back in real time. Now this can be done either "As a single track where we're just holding frames and playing them out in the new durations, or you can also "We do a dual track playback and blend the results.

So in a dual track version, we provide you the frame that's a little bit before the point in time that we really want to be showing, and the frame that's a little after that point in time, and you can blend the two based on the ratio of how far it is away from the optimal time." The RT data filter is something we developed as a way to get around the problem I was describing before about using multiple formats in one sequence.

And this also simplifies the development of the playback as well. Because in a standard QuickTime model, if you're playing normal video without effects on it, you're going to get a call per frame to be decompressing each of the frames. And then handling it and passing it through. When you're performing an effect, the effect is really treated as one frame of very long duration. Or one sample, really. And you're responsible for producing all the in-between frames based on the timing provided.

With this data filter, you can construct a, well, Final Cut will construct the sequence for you so that everything in the movie is flowing through the effect model. That if you have no other effects, you always have the data filter at the base. It also provides a way for us to pass off some data that is otherwise not easy for you to get.

For example, what's the frame rate? It's one of those tricky things where QuickTime's so flexible that they don't really care what the frame rate is, but in the video world, the frame rate really matters and doesn't change. Your hardware really needs to know what it is. And sometimes it's not as obvious as it should be when you're trying to figure it out based on the QuickTime movie. So we can give you that directly through this.

The RT-STILS effect lets you include STILS in the sequence in real time. Now, the STILS are set up So that we render them when we construct the movie and put them into RAM so that on playback there's no delay accessing them from disk or creating them if they're generators.

And the interface provides the ability to control how much of the applications memory you're going to dedicate to the still cache. And we've got a lot of code in there to handle releasing and reallocating the stills as you switch the sequence that's in the front. Of course, if you're using more stills than fit in memory and you're switching, it's got to be regenerating them each time you switch tabs. So there's a price to that.

And then the motion effect implements the controls that are in the Final Cut motion tab. So this gives you the ability to do scaling, centering, "Rotation, cropping, distort, drop shadow, those are the sorts of things that are in the motion effect." Now, when we take the effect that you've described in the Final Cut timeline and Convert that into a QuickTime movie.

We have to break down the hierarchy of the effects as they're going to be processed. So this is an example where we've got a moving still over cross-dissolve of two color-corrected DV sources. So in the upper left, we've got the still effect, which is essentially a generator in the QuickTime model because there are no sources to that. Then below that you can see the two video streams which feed the color correction effects. Those two color correctors go in the cross dissolve and the motion still with the motion goes through being composite on top.

But within the QuickTime movie structure, it actually looks a lot more like our timeline, in that you've got a base video track, and then you've got tracks for the various sources. So over in the movie world, you're going to see just the composite there, and the two DV sources.

So, for example, the composite that's over there on the right is the only thing that you really see directly in the movie. And when they try to play back the movie, QuickTime goes around and asks all of the installed components who handle that format, or that particular effect, in this case the composite blend effect.

They're looking for anybody that can handle that, and if you're writing these components, your component should say you can. But when that happens, you're then responsible for handling all the things that are below it. So you need to examine the tree, make sure that it's all stuff that you understand, and then you have to expand that out. And map that to whatever hardware or software components you have that can perform these pieces.

The capture architecture is very much like the playback architecture, just a few pieces that are replaced. Data is flowing the other direction. You've got the sequence grabber that's really driving the QuickTime side of things. And then the pieces that you need to write is the video digitizer and the Core Audio device.

So in the audio side, it's flowing through the sound manager and Core Audio. Again, the audio is the center path and video is on the left. And as usual, the device control is off to the side as a separate path that really, in most cases, won't affect anything that you'd be writing. The application would talk straight to the devices through something like a USB to serial converter. Or if you're using a FireWire protocol, it could be A/V sync. control.

So again, this is pretty much the standard QuickTime model for how to do things, and if you're trying to figure out how to write the QuickTime drivers, there's a lot of information available on the QuickTime website. And I/O Kit layer for how to talk to the hardware is also standard for the device drivers. But there are some things that you need to know to work with Final Cut that are a little different. One of them is that getting the AV sync on playback accurately.

For those of you who made it to the QuickTime session yesterday afternoon, Jean-Michel was describing the new APIs that'll make this a little better in the next version of QuickTime. Unfortunately, that's not part of Final Cut 4 at this point, but we will be switching to that at some point.

But as it stands now, since the audio and the video are running through those two separate paths that we saw, they're independent, and you have to do some work to make sure that they come back together and are in perfect sync when you play back. On the playback side, it's pretty straightforward in that as playback begins, the first frame of audio that you receive and the first frame of video that you receive should be lined up, even though they come at separate times. We know and you know that those are the two things that need to be played together to begin the playback.

For capture, it's a similar sort of problem, and that's not yet addressed in the new version of QuickTime. So the way we've worked around that one is to have you insert a known pattern into the audio, since the audio is streaming before the actual video frames start to appear.

You just insert, we provide a call to give you the pattern to use. You insert that into your audio up until the point where you begin the capture. And once again, in your drivers, you know exactly which video frame you're starting the capture with and what audio corresponds to that. So when you get to that first frame of audio that corresponds to that first frame of video that you are capturing, stop inserting our pattern, put in the real audio stream, and we'll add that to the video.

After the capture, walk through, look at the audio at the beginning, strip off all the parts that match the pattern, and line up that, the next sample of audio, with the beginning of the video. As I said, the VTR control would generally be through external devices like USB.

Now, one of the other areas that is different from the standard QuickTime model is the RT enabler. This is what's used to let Final Cut know the capabilities of your RT system. These are implemented as FX scripts using the FX script language, and they're run as startup scripts as opposed to the normal scripts that are interpreted and run to perform effects in real time.

The first important aspect of the enabler is the effect mapping. And what the effect mapping accomplishes for us is the ability to use the same filters and UI and effect scripts for real-time as we use for not real-time. That means that the users don't need to learn different sets of filters or choose different filters depending on their hardware configuration.

They just edit the same way they always do. And within this effect mapping, your script will define which of our software FX scripts you've got hardware or other acceleration for. And through this effect mapping, we describe what QuickTime components of yours get called in place of the FX scripts.

One of the big advantages of that, as I said, is that you've got only one set of filters and effects that people need to learn to use. But another aspect of that is that you can take a project file that you built on a high-end system with lots of hardware and can move it over to, say, a PowerBook and work on a portable system, which doesn't have access to your hardware, and the filters and effects still perform correctly because they were all built off of the effects scripts.

We just fall back to the effects scripts. You can see the right results. And it works the opposite way, too, that somebody could be developing their project in offline mode without the hardware and then going into an online suite and using the new features and capabilities that you've developed to get things in real time.

Second big aspect of the RT-Enabler is the effect costing. The effect costing is necessary to be able to predict before we play it back whether something's going to require rendering or can be played in real time. At first glance, that seems like a simple problem, but in fact, it's not at all obvious when you get to the middle ground as to whether something will perform in real time or not.

If you've got a piece of hardware that's got three physical decoders on it, you can pretty well guarantee that a fourth stream of video is not going to work. That's a pretty simple aspect of the costing. But if you're doing it in software or a lot of the hardware these days, it's more general purpose and it's more a matter of how many other things is it trying to do that limits the capabilities.

So we've got two models that we can use. One of them is based on a time budget and the other is a resource budget. And you can use the two of them at the same time. So the resource budget is like the example I had where you've got physical decompressors and you can't handle more than three streams of video. So you can put a limit saying you can only do three.

On the other hand, the time budget is a model where you describe how long each of the effects takes. And how much total time you're willing to take to perform the effects. And before we construct the QuickTime movie, we run through it, add it all up and see whether all the pieces exceeds the limit that you've set. And if it exceeds the limit, we'll put up the red render bar saying, "Sorry, this needs rendering. We can't do that in real time." Otherwise, we'll give you the green render bar and see it play back in real time.

So a couple of changes that we made in version 4 to the enablers. One of these is providing the manufacturer name to show up in the Effects Handling tab. So if you don't add the new entry to the enabler, over there where I showed you in the Effects Handling tab, we won't know what to provide as a name. We'll just put the manufacturer 4cc, which isn't very useful to your customers.

Another enhancement that we made to it, if you've looked at the Final Cut enabler, the one that's inside of our application, you can see that it's a pretty big file, and we've added some commands that help save the redundancy, make it easier to write the pieces, and maintain the code that you're putting in there.

For example, if you're defining all of the capabilities for DV, NTSC DV, you then have to duplicate all of that again to do PAL, and with the new commands, it's pretty easy to just duplicate and then tell it what's different, rather than repeat a lot of the data.

Another new thing that we've added in this version is a debugging enabler, which Just use data to counsel about what's going on while we're interpreting the effects and figuring out what we're going to be building in the QuickTime movie. One of the trickiest things has been when you build your effect, you've got all your components installed, you think everything's set up, and you put it all together and you get a red render bar.

It's like, okay, why isn't it real time? There's a lot of reasons it could not be real time. And short of you getting together with us using your debugger and our debugger, it's been pretty tricky to figure out some of these cases. This way, we're providing more information about what we're doing on our side and what we thought didn't work. So I think this is going to help a lot. And we'd like feedback on other sorts of things that we can be putting in here that'll help you out.

So, here's an example, a very simple example, of what's necessary to enable video out and the uncompressed software path in version four. So, the first line is setting the enable to view IFX flag, and that enables the software processing of uncompressed video in real time within the application. The second switch, the enable software RT video out flag, lets us call your video out with our software effect.

So, This makes the RKE Enabler look simple. And if you're building just an I/O device, this is all you really need to be able to do. Because all of the effects processing will be handled through our software, we've already got a complex enabler that describes how to do all of the effects that we perform.

In this case, the parameter 1 at the beginning of the enable software, or T-video out, is a description of what the relative cost of your video audio is compared to our standard version. So you need to do a little profiling to see what scale factor to put in there so that our costing works out right. But otherwise, it's pretty straightforward.

So the opportunities in the area to support Final Cut, hardware acceleration is always going to be able to provide more effects, better quality. In the SD world, the SineWave, for example, now can do a number of streams of full-quality SD with a wide variety of the effects that we have in Final Cut. In the HD world, the bandwidth requirements make hardware a real necessity still.

There's a number of products already that provide 3/2 pulldown in real time from the video outs, so we're playing it back as though it's 24 and they provide a spicket that converts that to 30 for playback. Another aspect of what you can do with hardware is provide other kinds of codecs, so MPEG-2, motion JPEG, there's Bixlet, who knows? You can do whatever you want.

So with RTXtreme, that really demands an I/O style of device. If you're going to use the software effects built into Final Cut, all you need to provide is the interface to the video hardware. That can be done either through PCI or FireWire. There's a number of boards out there now that are doing this with PCI. Gio's going to be talking about how you can do this with FireWire now.

One of the things that the I/O device provides is a way to connect to VTRs, but another aspect is to give you true color monitoring on NTSC or PAL video devices. Another big area of opportunity, this isn't exactly hardware specific, but effects scripts and after effects plugins fit into the effects model within Final Cut nicely.

So to get back to HD, this is really the biggest opportunity area that I see coming up. And that as RT Extreme gets faster and faster, taking over the SD video world is within sight. We're not there. There's still a lot that hardware can do in that area. But in HD, it's going to be a while. There isn't anything that's going to get you high quality HD out of the computer as it is today. Advantage, it's the same basic architecture as SD, it's just bigger. Wider, higher, faster.

And of course, the new hardware that we've been introducing, the G5 with PCIX, will assist you with the high bandwidth needs in this area. And the XSERV RAID is a good way to get a lot of data quickly off of the hard disk. All right. I'd like to introduce Giovanni Agnole to take over and talk about the uncompressed codecs that we've developed.

Okay, my name's Giovanni Agnolli and I'm going to be talking about two things today. The first is going to be the uncompressed 422 codecs that we've included in the Final Cut Pro release. I'm also going to be talking about Pro I/O after that. In the 4.0 release of Final Cut, we've added standard 8- and 10-bit uncompressed YCBCR422 codecs, and these are also available to developers for inclusion in their products. Say you wanted to ship a PCI board that worked with After Effects.

The main reason we did these codecs is so that we could facilitate the exchange of uncompressed data and remove the need for vendor-specific codecs. We saw a lot of problems with users trying to exchange uncompressed data, and they needed to go and send their friend who wanted to read the data the vendor-specific codec that had created it. Of course, these codecs are tuned to take advantage of the latest hardware. We've done a lot of work to improve the speed of them by writing some AlteVec code. And another big win with these codecs is that Final Cut 4 supports them natively.

So the formats of these codecs are pretty straightforward, and they've been around for quite a while, actually. They were originally described by Chris Perazzi in December '99 in QT IceFlow '19, and I put the link up here so you guys can check it out. And we commonly refer to the codecs by their 4CCs. So Ken's talked about the 2VUI and the V210 codecs. That's essentially what we're talking about here, 8-bit and 10-bit uncompressed YCBCR.

The codecs have a lot of capabilities, but the things that you guys are most interested in is what they compress and decompress to. And the 2VY codec supports the R408 and V408 FCP formats, as well as 32-bit ARGB and 2VY. The 10-bit adds support for the FCP R4FL, which is the 32-bit floating point format, and V64A, which is the After Effects 16-bit RGB format. We also support the QT gamma level APIs in these codecs. And so if you have a destination PIX map that has a different gamma than the source image, we'll do the gamma conversion for you. And as I said before, they've been optimized for the latest hardware.

So the main reason we're telling you about this is because we want you to use these codecs. And just, you know, the benefits are highest quality. It's uncompressed. It's not a lossy codec. We want to get a common file format among our user base so that we can ease the interchange of data among our users, which we believe will improve the user workflow dramatically. And it's also important that you support these formats if you intend to use RTX Stream, because RTX Stream is tuned for these codecs.

We use the 2VUI format internally to render. So if it has to do a conversion to another format, that's an extra copy there that means loss of performance. And best of all, you're getting free optimized code. And it can't get better than that. We're going to give these codecs to you to distribute if you'd like. And that means less work for you as a developer. You can focus on doing something cool with your hardware instead of writing codecs.

So the next thing I'm going to talk about is ProIO, which is, I don't know, a new way of getting audio and video in and out of the Macintosh. And first I want to talk a little bit about the philosophy behind ProIO and why we went about doing this.

As Ken talked previously, we see this trend of ever increasing processing power in the CPU and GPU and decreasing cost of storage. And so that led us to developing RTX Stream, which takes advantage of those capabilities in the host machine. And with that, we needed a mechanism for easily getting audio and video in and out of the Mac, primarily interfacing with professional devices such as high-end VTRs and cameras.

And so when we went and tried to figure out what kind of bus we were going to use, we decided on FireWire because of its easy plug and play capabilities. And it's also really simple instead of like having to go to a PCI card or whatnot, you just plug it in similar to a DV camera.

So what is a Pro-I architecture? It's basically got two parts. We have a data transfer protocol specification, and we also have FireWire unit specifications. And the data transfer protocol specification mainly deals with how to send uncompressed audio and video over the 1394A bus. And so in that uncompressed stream, we're sending synchronized audio and video. The video can be NTC or PAL, 8 or 10 bit, YCBC or 422 video. And we also send eight channels of uncompressed audio, which can be 16 or 24 bit, and a variety of sample rates.

And that all comes out to about 230 megabits per second. And then once we add, like, pro-I, we can send that to the Pro-I. Brett Halle, Ken Carson, Giovanni Agnoli And so in that uncompressed stream, we're sending synchronized audio and video. The video can be NTC or PAL, 8 or 10 bit, YCBC or 422 video. And we also send eight channels of uncompressed audio, which can be 16 or 24 bit, and a variety of sample rates.

protocol and packetization on top of that were really pushing the limits of the 1394a spec. But we believe that it was important to fit all this into the 400 megabit per second spec versus just because of the wide variety of machines out there that have these connections on them.

So getting back to the protocol specification also covers things like packetization and the framing of the data and some flow control techniques. The second part of the architecture is the FireWire unit specification. And I don't know if you know anything about FireWire, but there's a lot of resources on the web. A unit specification kind of allows a device to define its capabilities. And so we have three units.

There's the AV unit, which is the -- it does most of the heavy lifting. It handles all the transfer of uncompressed audio and video. And we have a serial unit, which handles -- allows us to transmit RS-422 over the FireWire. So this is really useful for deck control. And the third unit we've put into the architecture is the firmware upgrade unit, which allows you to upgrade your device at some later date. So if you want to upgrade the firmware, it's -- there's a unit that allows you to do that.

So, what are the developer opportunities with Pro I/O? Well, what we're trying to get is people, you developers, to develop Pro I/O devices. We really think that there's a broad market for these things and it really, you know, we've had a lot of success with the first devices to date. So, you can develop an analog-only device that would, you know, convert component or composite signals, you know, XLR into digital data and send it over the FireWire bus.

Or you could develop a digital I/O device that would take in SDI, AES, ADAT, et cetera. Or you could do a device that did both. Or maybe you have an idea. Maybe you have an idea of some other connection that we haven't thought of and we'd be happy to talk to you about that and, you know, update the specifications to include that.

So the first Pro IO device that is out there today is the AJIO. And it's kind of the Cadillac of devices. And I just put this up here as an example to show you what these devices look like. Basically, we have a bunch of analog and digital connections. Pretty much anything a professional video user could want to connect to their existing VTRs or cameras. And then you'll see in the lower middle part of the screen there, there's a FireWire connection.

That's the bus connection to the Mac. So we're basically capturing data or playing back data through all these different connections and then transferring it over the FireWire bus to the Mac. There's also an RS-422 connection there for deck control, which if you're in the Pro video space, you know is absolutely necessary. Amen.

So, along with the architecture, we're also providing drivers. The ProIO driver release is a standard set of QuickTime drivers for ProIO devices. So, if you make a device that follows a spec, it should work with these drivers. In that set of QuickTime drivers, there's a VDIG and a HAL plugin for capture of audio and video, and also VOUT and the VOUT transfer codec and Clock, Core Audio, HAL plugin output device for playback.

And it's basically the QuickTime driver set that you need to interrupt with any QuickTime-capable application. So, if you make a device that follows a spec, it should work with these drivers. So, the idea is that these drivers should work not just with Final Cut, but with other applications as well. Additionally, we support the Final Cut specific APIs that Ken talked about earlier, the Cover, A/V sync, and a Deck Control. And those are, you know, we're the Final Cut team, so of course we do the extra bit to support our app.

In addition, there's Apple FireWire Serial Driver, which... allows you to send serial data over the FireWire bus to the ProIO device. And, of course, all this stuff's been optimized for the latest hardware, lots of good AlteVec loops, etc. And best of all, it's out there right now. It was released yesterday.

I took Ken's capture architecture diagram and kind of grayed out all the non-ProIO parts just to show you what we're providing in the driver package and also what you as a developer need to do. If you look down there, there's the ProIO hardware. That's what developers need to develop.

And then you can look and see that we provide a ProIO VDIG and ProIO HAL plugin to get audio and video into Final Cut or any QuickTime-enabled application. And then we also have the Apple FireWire serial driver to do deck control. And all those talk through the I/O FireWire family, which sends queries across the FireWire bus.

So playback is very similar. We have a transfer codec, a V-out clock to send video out, and we have a HAL plugin to send audio out. And again, they communicate through the I/O FireWire family to the Pro I/O hardware. And there's the Apple FireWire serial driver there for any deck control.

So obviously I can't tell you how to build a device right here. So if you want more information on uncompressed 422 codec licensing or ProIO licensing, or want to take part in an upcoming ProIO kitchen, you should contact Jeff Lowe, who's our WDR representative, and he will help get you the right information. I think I'm going to call Brett up. He's going to wrap up for us. Thanks, Jill.

So we spent some time going over all the various ways that you can be approaching or get involved in developing hardware for Final Cut 4. We strongly encourage you to do that. There's a number of other sessions that you can get involved with. Some of them are coming up here or have happened early in the week, and if you've missed them, they should be on the DVD.

We will have a session tomorrow for dealing with plug-ins and encourage you to do that if you're interested in creating, for example, effects scripts or things like After Effects modules, things like that. We will be talking about the various plug-in models there, and we will have a feedback form for Final Cut on Friday afternoon at 5:00.

As already mentioned, Jeff Lowe is the guy. If you're interested in any of the pro professional film and video area, he can help you. As mentioned, we will be having a kitchen later this year to get into details about actually how to make some of these devices and what the protocols, wire protocols and stuff are.

There's a number of places you can get more information. And obviously, it can take a while to scribble all this down, but I believe all this information is up on the web for you. But in addition to the kind of standard stuff that's out there in terms of video formats, if you haven't had a chance, there's QuickTime sessions and audio sessions and I/O Kit sessions that will be very important for you in terms of driver development.