General • 51:40
This session focuses on the formats and methods available for the exchange of media and metadata/project data with Apple's suite of professional applications. Topics discussed include the use of new XML-based data formats, interoperability with existing industry-standard formats (AAF, OMF, EDL, etc.), and conventions for the structure and contents of QuickTime-based media in professional production environments.
Speakers: Brett Halle, David Black, Harry Plate, Michael Johnson
Unlisted on Apple Developer site
Transcript
This transcript was generated using Whisper, it has known transcription errors. We are working on an improved version.
Good morning. Today, basically, as Brett said, we're going to talk about how to get data in and out of Final Cut Pro. And really this discussion focuses a lot on project data formats and media data formats, how to make everything tie together so that you can actually not only work with Final Cut Pro and Apple's other professional applications, but also build a set of tools that other people in the industry will find useful to actually get production done and do things like high-end Hollywood films and television production.
Certainly, and also various strategies for extending sort of directly Apple's professional applications. We're going to talk a lot about specific project formats and media formats and again, sort of what type of tools and some examples of some things, solutions you might actually be able to build using these things with your software.
I'd like to start off talking about project data and sort of begin the discussion with what is project data. It's sort of used different terms in the industry. Avid will refer to things like its compositions. Final Cut Pro will get projects and sequences. If you go into a compositing application, it might be a composite. Or if you go into Shake, it might be a Shake script.
And really what the key thing here is, this is the data that your users or our users spend the most time creating. Certainly they'll capture media assets, they'll get stills from the Photoshop artists, they'll get music from their audio person. But they'll spend the most time in software aligning these elements, putting them together, applying effects. And so this is what everyone focuses on.
So in a certain perspective, when you go out into the industry and talk about putting new tools in people's hands, they get very emotional about this. They're spending this time creating this and they want to make sure that they can keep it, they can use it, and that they can move it around if they need to. Of course, since this is certainly the thing that people spend the most time on, it's also the most unique between different applications.
And so it's traditionally been very difficult. There are some very basic formats that have been around for a very long time. More attempts in this direction have been made in the last few years. But it's not an easy process. And certainly a lot of the work we've been doing for the last year or two has been to try and make this process easier.
So we begin talking about some of the legacy formats that some of you may or may not be familiar with. And a lot of these formats were designed like 15 to 20 years ago and originated with a lot of hardware-based systems. Things where you would have a project, you would have your show you're putting together, and it would be, of all things, a floppy disk. Not quite back to paper-tape days, but still pretty primitive.
One of the lowest common denominator here is what a lot of people call batchless. Really what a Batchless boils down to is if I have a bunch of source footage, like on a videotape, I'm going to have someone, the assistant editor, maybe the DP for the day, sit down with a tape deck and basically shuttle through that footage and write down the time, the time codes, of the particular pieces of media, the particular clips that we're going to want to use in our production.
Just indexed by time code. When this moves off of pads of paper in a digital realm, it's really just a delimited text file. So you'll have some time codes, some name, maybe an identifier for the source tape, and it's just stored in a plain text file on disk. The users actually love this, and if you talk to film editors today or television editors, they still use them all the time because it's just a text file.
They can pull up TextEdit, they can pull up Word, they can pull up Microsoft Excel and change these numbers, and they're very comfortable with that because they control exactly what's in there. And because it's a tabulated text file, it's very easy to generate, very easy to parse. So pretty much any modern non-linear editor at least in the prosumer and professional space will support these. But they're very limited.
Here's a very basic example of a batch list. And it's talking about before just a tabulated text file with a first line with a header with all this information on it. You see clip name, time code, and real identification. And sort of moving beyond just identifying pieces of media that one might work with, you move to an EDL.
Again, this is designed for hardware-based systems of the past where you'd have a huge bank of tape machines on one side, a record deck on the other, and monitors, and you sort of queue everything up so that it would record at the same time. And just basic edits, sort of an event-based model with times and effect identifiers. Although it's limited, sort of two tracks of video, four tracks of audio, a lot of this based on sort of switcher technology of the day.
And sort of like batch lists, a lot of users out in the field love this stuff to this day because you can open one up in a text editor, and if you train your eye, you know exactly what to look for. So if I put together an hour-long show and it's just not going out to tape correctly, I can open it up and fix it.
It's a bit more difficult than a simple batch list to parse, however, because there's a lot of data in there, and there's also a lot of different formats. Sort of everything started with one or two de facto standards about 20 years ago, but now you've got maybe about a dozen different sort of formats for different manufacturers' gear, different tweaks for different hardware, and even different effects processing gear. Here's a very simple EDL, and as you compare the batch list, it's kind of frightening. A lot of time codes there, comments for effect IDs, things like you compare a dissolve with a D and a wipe with a W012. Not very friendly to parse to understand.
So, sort of moving on, Batchless and EDL sort of got more contemporary formats. And certainly 10 years ago, beginning of the nonlinear digital revolution with Avid, everyone saw that these formats really were lacking in a lot of ways. And so more contemporary formats were created to go in their place and sort of address a lot of capabilities of modern systems, things like, well, you can go beyond two layers of video or you actually want to get down to effect parameters.
You might want to be doing things like throwing a Gaussian blur where you're keyframing the amount of the blur over time or doing that fuzz filter where you're fuzzing someone's frame out. But unfortunately, due to when these--due to a lot of these efforts started, these formats were generally binary formats. So you'd have some structured storage model and you would encode all this in binary because it was cheap and it was small.
But at this disadvantage of now a user or even a power user or a developer can't directly look at it. So you introduce a lot of library requirements. And it's pretty complicated to generate. So if you look at some examples we'll talk about in a moment, there will be sort of a small set of tools out there to work with these files, but really not a community of large general tools to do things.
So the best known example of a contemporary format would be OMF, object media format from Avid, created when they actually tried to unify sort of audio and video products about seven or eight years ago now. It was based around some Apple standards at the time, OpenDoc and Bento.
And actually sort of describes for two models. So you have sort of what they term a composition only OMF file, basically just a description of your edit, your effects and the elements within, versus a combined format where you actually store your media all in one container. Really big file, really hard to move around.
Of course at this point it's more or less abandoned. And part of that was Avid's move to more of a cross-platform system and some issues with basically bringing Bento libraries on different platforms. And really where it's used today a lot is in audio post-production systems. And that you'll have Pro Tools, one of the most popular nonlinear auto-editor audio systems. And I think there are a lot of popular nonlinear audio editors today that actually still use that format.
Moving on to the next slide. Moving forward from OMF, in order to address this situation, the Advanced Authoring Forum was founded four or five years ago by a consortium of companies including Avid, Microsoft, and Adobe. Basically, they tried to take the lessons from OMF and build a new format that still was based around a binary container format, this time Microsoft Structured Storage, but actually have it more be of an industry-standard format so that you could take media and projects between different applications and different systems.
Unfortunately, it was sort of a mixed success in that while a lot of the basics, a lot of the elements that would be sort of the equivalent of an EDL made it into the public specification, a lot of things really didn't. And those are all the really interesting things today, things like layering information, compositing information, filters and effects.
A lot of this is sort of termed as black metadata by people using AAF and where sort of one company will have to find their own little black box inside of this open specification so you really can't get to it. A lot of companies do support the general version of the AAF standard, Avid, Quantel, and Apple actually joined the Advanced Authoring Forum last year and is working on AAF tools using the efforts of AutomaticDoc. Thank you. Thank you.
So moving on to the future, there's been a lot of movement, especially with the open source movement the last few years, toward more open formats. AAF and OMF are really interesting, but because of a lot of the issues, it's very difficult to join the club of working with them. It might be a lot of effort to use binary formats, there might be library issues depending on the platform, and the whole black metadata problem.
And sort of the key concept of a lot of these formats has been open access, fully documented data, everything open, no real issues with data access. And also trying sort of getting back to the capabilities of batch lists and EDLs where a technical user could open one up and see what's inside of it, and actually make changes, or simply have some faith that this project that they spent a month creating will actually be useful in another year if they move their media to a different system. And also of course trying to minimize platform and library requirements.
And to address this, we've actually come up with a format we're calling XML Interchange Format, working on it for the last two years. And we introduced it, announced it with Final Cut Pro 4, just uses XML. Everyone loves XML, it's easy to parse, you can use it in pretty much any platform. And it's relatively self-documenting, and there are a lot of tools out there to use it.
And power users can open the thing up in text editor or even a structured XML editor and make changes to the data. For our XML Interchange Format, we've gone to a lot of effort to sort of try and make it a superset of all formats. The goal is really to expose all of the functionality inside of Final Cut Pro, sort of no black data, no weird undocumented little hooks, and just open everything up.
And a lot of this was the goal of trying to be able to base other formats behind XML. So working with Automatic Duck, we have import and export support to AAF, but going through XML in the middle. Since XML is a superset of what's at least publicly available in AAF today, then it's relatively trivial to do the translation and bring that in Final Cut Pro.
The one issue that our XML format does not address is the issue of embedded media transport. So the idea that I could have one Mondo file, which certainly is easier to move around today than it was seven or eight years ago, but one container with my media and my composition information in sort of one box. And we really didn't feel that that was an important issue to address right now.
Certainly there are a lot of different workflows that that model does not necessarily meet everyone's needs. And it also tends to make the cross-platform, cross-application issue much more difficult. Different hardware, different software may have different requirements for streaming data or moving data around, and trying to sort of encapsulate it in one standard can be rather difficult.
Here's a very basic example of an XML file. It's similar to something that would have been defined in the Batch List example earlier. As you can see, it's content-based, so the various keys are things that if one was familiar with an online editor, they'd be familiar with: a clip, or a name, or a duration. And sort of multiple formats as well, so if you look at timecode definition, you sort of have both frame count and frame rate along with additional string defining that data. Now, of course, Apple's not alone in -- oh, excuse me.
There's actually one slide that was missing. Actually, at the moment, what I'd like to do is invite Harry Plate up to the stage to show off tools that Automattic Deck has been building with us to support AAF inside of Final Cut Pro. Very good. Okay. Thank you, David.
Okay, make sure we're looking at the same story here. First of all, Automatic Duck came on the scene here just a couple years ago. My son and partner is a video editor and got sick and tired of messing around with EDLs as he would move his sequences and compositions from an Avid into After Effects, and that was the product that we initially started out developing. What I've got here -- well, let's see, in fact, we want to delete that because I want to import the Chrysler example.
Underneath literally the import menu, you'll see a pick that allows us to bring up a dialog that allows you to select. In this case, I'm going to grab a Chrysler example and an AAF file. Again, this was a short commercial piece that was done a long time ago. This process opens an AAF file, parses it, now goes off and searches for footage. By the way, this footage now is represented by OMF files. This entire sequence being created by an Avid Symphony.
I think it was about two years ago, it's been one of our favorite examples to test things. So it's fairly straightforward. We create an XML. The XML now is being read by Final Cut. And we should see a Chrysler example now up here in that project window. And we'll open that up.
And sure enough, here's all that metadata. We had a couple of audio clips. Let's make this thing look a little more... attractive. We'll fit it, show you some more. Some more information. There's markers, dissolves, video, etc. And this is something that when we did this manually some years ago, this took an editor four hours to reconstruct in order to get all of the time codes, etc. done correctly.
So fairly straightforward. Again, you open an AAF file. Through the magic of Microsoft COM and a binary library, you may or may not succeed in getting everything. But... It's a challenge. You track down all the metadata. You track down all the media. You open up OMF files. We reconstruct QuickTime reference movies so that we don't move any video. So the reference movies are very fast.
And then ultimately Final Cut is able to import that. And the entire description, of course, is passed via XML. Now... The opposite case is fairly straightforward. I can take that same example and I can turn around and export it. We'll just drop that thing out there and now we take that XML. Well, let's see, we better replace that. We don't want that one anymore.
Reconstruct an AAF file based on that XML description. Fairly straightforward. In this case, it comes back and says, "Hey, wait a minute. You've got some effects there that we can't represent in AAF." And that's fine. Our documentation will fully tell you exactly what you're doing wrong. So let's see, I grabbed the wrong one, but we don't need Internet Explorer. But we do want After Effects. And we'll find a site of After Effects as soon as I close down that.
This is a simple import process yet again. And I can grab that Chrysler example, import it. And it's the same process now in the Adobe After Effects world where you open up this file, parse it, grabbing all of the media references as necessary, using them directly in the case from Final Cut because we already have the QuickTime movies. And we create a composition, again just a different nomenclature, no longer a sequence as in the Final Cut world but something slightly different.
Now, a timeline in the Final Cut world is quite different. They show layers each on a separate horizontal row versus the timeline that you might be used to inside a Final Cut. But nonetheless, people like to finish inside of After Effects for a variety of reasons, whether it be the effects, the color correction, or whatever.
But again, I emphasize, XML is under the covers. And the ability now to get into Final Cut, of course, we've had customers for a long time crying for that kind of capability. So it gives Final Cut now a tremendous edge now in the complete workflow of any post-production facility. That's it. Back to you.
Thank you, Harry. It's a very powerful example of these sorts of tools. But sort of talk about maybe what other people have done in the industry surrounding XML, because certainly it's not a new format. It's not a new idea. Actually, excuse me, wrong slide. A little bit more detail on our XML interchange format. Sort of the big question that we've gotten since NAB of this year when we announced XML is when will it be available? Well, today we have a public beta available on the WWDC site. Simply go to connect.apple.com and log in.
And it's an early build, but shows up, gives you import and export XML commands inside of Final Cut Pro. Documentation there on the format. Again, preliminary stuff. We'll be updating it on a regular basis until we hopefully release it later this year. And we're also basing a new plugin API on this data format.
Certainly the commands that Harry was able to use with his software were written using this new API, and we'll be getting that out to you as soon as possible. But more importantly, sort of one thing that we've done is we've been able to do a lot of work on this.
One point to emphasize is we really want your feedback. We've been working on this for a long time. We've talked to a great deal of people in the industry about that. Another gentleman I'll actually bring up here in a few minutes that has another sort of compelling tool based on this.
But we really want to hear what you think is missing or what can be done better, or sort of what your needs are. There may be various workflow scenarios or various tools that you want to build that we just haven't thought of. And any of that feedback is incredibly valuable. And we will be releasing this in a final form to end users and developers. by the end of this year.
[Transcript missing]
Some strategies for your application should you want to play nicely doing data interchange. Any form of data interchange you can do is incredibly helpful. Even if it's primitive, just something that at least is documented or maybe in a plain text form. Brett Halle, David Black, Harry Plate, Michael Johnson A lot of people in the post-production community are very creative and amazed at the amount of pain people will endure to sort of have that safety feeling of looking at their data.
And so certainly anything you can do is good. Conventional legacy formats, if you can support them, that's great. It can be a lot of work. It can be a lot of debugging. If you can't, then we encourage you to at least look at doing some more modern open formats. And sort of allowing other people to write translators in the same way that Automatic Duck has.
Sort of opening that door gives you more leverage without necessarily spending a lot of engineering effort. And certainly open formats are great. They level the playing field for everyone. And in the end, everyone wins. Developers sell more software and users get the tools they need. And it's a very good thing.
For Apple's professional applications, Final Cut Pro 4 at present has native support for EDLs, Batchless, and of course XML very shortly, and OMF and AAF through the automatic duckguys. Shake-through, which I want to mention for a moment, is a little bit different story in that at the moment there isn't really an XML representation or direct OMF or AAF support, but their project file format is at least sort of a plain text file.
ShakeScript is something very similar to C or C++. It sort of defines the structure of a project. And it's an example of something that is rather technical, but at least is in a form where other tools can generate and parse this information. And that's one of, sort of one of the details that has made Shake incredibly popular and powerful in the high-end post-production industry.
So I'd like to move on to exchanging media. And so we talked about sort of the project composition data side of the universe and sort of getting that data that-- and you just spend a lot of time creating move back and forth. But media is also incredibly important as well. Why do you want to do it? Well, there might be formats you don't support in your application. It may be that you support still image files, but you don't support movies. Or you're based around video or audio, but you really can't deal with container formats.
By allowing users to import and export data in different formats, you just increase what's possible and what users are able to do. By exchanging data, you can leverage things in other applications. You might have a wonderful compositing engine, but you really don't do 3D animation. If you can import and export data, then you can bring in 3D animation from some other application and work with it.
Hardware support is an interesting one, in that hardware support, sort of the ability to output to decks or capture video from a deck or a camera, isn't always the easiest thing to do and can be a lot of effort, and yet users need it. If you can import and export media, then you can leverage someone else's software to do that for you.
And finally, in a real production environment, certainly there are some tools that will fit the entire bill for production, but most times there really isn't. A lot of high-end facilities like to pick and choose between different tools for different uses. And by exchanging media cleanly, you allow yourself to be inserted as part of that pipeline. And there are a few other details that you need to think about to make that practical, and we'll get to those in a few moments.
Some of the additional difficulties had in sharing media: lack of public format. Certainly there are a lot of formats, but when you move into the higher end of the spectrum, you get some more private things. You get codecs that, the details aren't necessarily there. Certainly you can get a piece of code compiled for one CPU, but not for another. And some formats are more difficult to work in than others.
An example of OMF and AAF, you basically have to wrestle with what libraries to compile and build and load to use that data on different platforms. There's also a big issue which isn't always necessarily considered is quality and conversion when transcoding media between different formats. The computer world, RGB is great.
That's pretty much what we all use. When you move up into the high end film market, you've got sort of nonlinear, high bit depth RGB. But when you're in the general production market, you've got YUV video to deal with. Conversions to basically YUV to RGB and back again, you lose color.
Because there isn't complete overlap in the spectrum there. And that can be an issue in some circumstances. Traditionally it's been identified as an issue for sort of the broadcast online market. They're a little bit more open to that these days, but it's still a problem, especially when you move up into the high definition market.
And licensing issues. Certainly if you have, you may have this wonderful codec and there may be some source code floating around the web, but you can't really use it because of licensing issues. And so. Sort of by supporting more open formats, you can help to get around that by at least getting a format that will go into another tool that maybe does have a licensing situation worked out.
Just sort of some common formats everyone's familiar with. You've got all the still formats that have been around for a while, for the most part open. Of course, they aren't really that great for video. A lot of people early on sort of took those still formats and just numbered them, put them in a folder, and played them back.
And that can be rather CPU-intensive, and certainly there are a lot of Unix workstations that that's just the best way to deal with it because you can get to any frame really quickly, but it's kind of a pain to move around. Certainly QuickTime is a great container format for just stringing together a lot of video images. You can do it uncompressed, do it in various codecs.
In fact, we actually have a lot of sort of open codecs at Apple. In Final Cut Pro 4, we introduced some new open uncompressed codecs for 8- and 10-bit video where the details, sort of the bits, are completely documented for free. Of course, we have our own codec implementation, but should you be on another platform or another piece of software and you can't actually use QuickTime, all the details are there so that you can parse those out and use those.
And certainly we have Microsoft formats, AVI, and Windows Media, traditionally more used in the delivery space than the editing space, although certainly some low-end tools make use of them. OMF and AAF, again, dual designs sort of for media container and composition container, so you can embed data in there.
Cineon/DPX is a format that Kodak came up with about a half-dozen years ago for the film industry for sort of high-resolution, high-bit depth, nonlinear RGB images. Details for that are out there. You have to dig to find them. But a lot of high-end tools like to use those frames for sort of final finished film renders and effects work. And of course, MPEG-2, just pure element streams you might spit out of Compressor or DVD Studio Pro.
Not really used, sort of, there's some movement toward using MPEG-2 in the editorial space, more for iframe only, basically in-frame edits or files, as opposed to the long-op stuff, traditionally used for broadcast, but still a lot of that stuff out there. Again, really quick, some common audio formats, AAF and WAV, uncompressed, everyone knows them, more or less the same except the bits are flipped.
SD-2, and I actually bring this up because it's an interesting example of sort of a partially public format. Certainly there are a lot of tools out there to read SD-2, it's not a hard format to reverse engineer. Unfortunately, this is one of those licensed formats that I mentioned earlier, where you can't exactly create a product that will write SD-2 files without a license from Avid, actually. It's interesting in that Pro Tools in the past has made very heavy use of SD-2. It makes it a little bit interesting to get data into Pro Tools sometimes, but just sort of be aware.
Of course, QuickTime, Windows Media, OMF and AAF, containers for audio formats as well. And of course, there are a lot of popular compressed formats now, MP3 and AAC, more for delivery, but we tend to see a lot of these in post just because someone gets a track that that's the only format they have and need to bring that in.
Talk about MUX formats for a moment. Formats where basically you have audio and video, potentially ancillary data, combined together. Very handy container you can just move around, something you can just open up and play back. Of course QuickTime's great for this. You can put pretty much anything you want in a QuickTime movie. Of course when you get to applications like Final Cut Pro, there are some conventions that you have to observe, things like timings, video times, etc. AVI, Windows Media, again more delivery formats, but they do support MUX data.
OMF and AAF as well can support this. MXF is something that's interesting to mention in that it's a format that Sony has been working on for the last few years with SMPTE. And MXF has actually been designed as a subset of AAF with the goal of creating a MUX data format that hardware devices could implement cheaply. So that you might have a camera that's recording direct to disk. And you need some format to describe that data and interleave it together in a completely, ideally, platform and software independent manner.
You can get the specs for that through SMPTE. It's starting to gain momentum with some new devices that have been coming out that have been announced this year. Still sort of ramping up. Interesting to see how common that becomes and sort of if it scales from the low bitrate compressed formats up into uncompressed. And just for completeness, MPEG-2 transport streams that you might get off of an antenna. For HDTV or program streams that you might have on a DVD.
I also want to talk about metadata for a minute. We've been talking about sort of the core, audio and video data, the bits that represent the things that you listen to and you see on the screen. But metadata is actually really important and becoming a lot more important as time rolls on.
So if I've gone out into the field and shot 40 hours of footage, I need some way to identify that footage. Certainly by simply being on a tape-based device, there's time code. But when that media is brought into the digital realm or brought onto a computer hard drive, there needs to be some way to track it other than the file name.
And this is really where metadata comes in. Traditionally, pretty much any non-linear editing system has tracked time code. So basically you can embed a set of numbers in a media file that track where that came from on a tape. And you may attach that with a source tape identifier as well.
Traditionally, it's been stored in pretty strange locations just because it's really been an afterthought. It's remixed formats together. It's great. It's playing back. Wait a minute. Where are we going to put time code? Fortunately, with QuickTime being a very general architecture, it's actually easy to drop time code in a logical location for a change and even put other information in.
You can utilize things like text tracks and user data bits just to drop data into a file and at least be confident that it's writing consistently with that media on disk. And that's the way if you go into a professional environment, it's required. They need it there. So it's very important to be able to read and process that. And it's just becoming more important as the amount of media grows, as sort of multi-user systems grow. The value of having that data is just all the more important.
So just some quick strategies on things that you can do to support to deal with media interchange correctly. You want to support as many media formats as possible. That might involve using a rich media engine like QuickTime to do the translation for you. It's quite handy. Certainly there are other options out there as well. When you're processing media or importing it into your system, you really need to try and keep it as close to the native format as possible.
Sort of the biggest thing that I get is YUV to RGB conversion. The reds will get weird, the color will get muddied if you don't do it right. And that's kind of a big deal if the native acquisition device is YUV and the output device is YUV, but in the middle you're going through RGB. So if at all possible, you need to try and keep that in mind.
Audio is another situation where if you've got a 24-bit audio recorder out in the field, you want that audio to stay 24-bit when it gets to the audio post house at the very end. And again, metadata. At the very least, pass it through. Ideally access it and display it and edit it. But if nothing else, if the data simply passes through your system cleanly, then your users are going to love it.
Apple's applications, certainly QuickTime. We love QuickTime. It's pretty much our universal container format. We use QuickTime as a translation conduit for other formats. We have a lot of great codecs. I mentioned earlier the uncompressed codecs we can use with Final Cut Pro 4, the pixel codec announced here at the show, and traditional things like animation and CinePack and Sorenson.
And time code and metadata. We do currently Final Cut Pro 4 supports multiple time code tracks and QuickTime files, multiple tape source identifiers. And as time goes on, we're going to be expanding that. We're going to have more support for metadata and not less, just because it's come up so often from all of our pro customers. And we also now have support for 24-bit 96 kilohertz audio along with multi-channel audio in Final Cut Pro 4. That all comes in at the moment through AFF files. The number of formats we bring in will be extended. expanded as time goes on.
So we sort of talked about project exchange and media exchange, and I sort of want to bring them together and sort of plant some ideas out there for how you might make use of these. It's not always obvious if you're not in a post house or at a film studio sort of what the opportunities are to sort of enhance and extend things, and sort of where new and unique applications and small tools might fit into the picture.
So one of the biggest things, one of the biggest requests we get now from post houses is sort of media asset management, where a post house is only sort of getting more work as time goes on. They may have six projects going at once, and to have sort of project data and media scattered across 40 different machines is really kind of a pain.
It's a pain for backup. It's a pain for tracking. And it's just a pain for workflow, and that one machine might be locked on a project. And sort of by building a central archive, you've got one place where you can collect and track things. And you can do things like track the revisions of a particular element. You might have something you shoot on video. It comes in. It's captured.
Great, you've got revision one of this element. Now it needs to be color corrected. Then it needs to have maybe some effects applied to it. Maybe it needs to have a lower third title applied to it. Sort of almost do versioning like we're used to in CVS, but with media elements, so that if something goes horribly wrong in the process, you're not going back to the original tape to bring the thing in and run it through. You can actually go back to maybe step number three where the issue came in.
Review approval loop is very important. Sort of the concept. Traditionally, if working on a show, I'd have to get approval from an executive, approval from a client. I'd have to take the project, dump it out to VHS, give the VHS to a courier, send it across town, have them look at it, send something back, give it a call.
Having sort of automated or at least electronic review loops is incredibly, incredibly handy. So certainly you could send an email to the client and say, "Hey, the spot's up. Take a look at it. Log into a URL. Look on a web page. Play it back. Make some notes." And sort of cutting the time all of that review processes take.
And also, come back to sort of central location, backups, and even just status, looking at a project, seeing what elements are completed, what elements are left to be done. And it only sort of primes things for multi-user products in the future. This is interesting where there are products in the marketplace that many people today consider to be multi-user, but really are just sort of media sharing and sort of the idea of having sort of direct clients and direct collaborative workflow where two people might be working on the same sequence at the same time is something that's not necessarily around today but is coming. And when it gets here, it's going to take the market by storm. And by encouraging this infrastructure and building tools that tie into this infrastructure, it's only going to make this easier.
Another very powerful example of this will be linking specialized applications. No one piece of software does everything, and it's not really even possible. There are things that might be specialized to one particular company, things that might be specialized to one particular market segment. And it makes sense in circumstances to use specialized software for specialized tasks and then be able to bring data back and forth. And it also makes use of different skill sets. You might have people who are less technical who just need to organize something, and people who are more technical who are going to finesse things and give it that final polish that it needs.
And even down to allowing for low-end hardware at one phase, high-end hardware at another, or even going to really dumb hardware at some point just for an executive to view something. And at this point, I'd actually like to ask Michael Johnson to come up and give a demo of an example of sort of linking specialized applications using XML and media exchange technology.
Hey folks, so this is a little tool that we wrote last year and we've actually continued to work on it and it's one of these things where we heard about the XML stuff that the Final Cut Pro folks were doing a little while ago, but then they have a product to get out and so you're like, "Oh, that sounds really great.
We're looking forward to that. We'll be very excited when that comes along." And so they got us some stuff a few months ago and we looked at it and gave them some comments, but really weren't using it or anything. And then last week they stopped by and said, "Well, you know, we're going to have this talk there.
We'd like to do that thing and we'll get you that spec, that code that'll let you read the XML stuff in your Final Cut thing there." And we had some, you know, little kind of going back and forth on that and I think I actually got the thing from David, I guess it was about 45 minutes ago. Yeah, so we actually did it over there in the session in the back there.
We actually said, "Oh look, it actually works." So I'm actually here more as, you know, as a developer, as a sort of a fellow developer here and just When you do things like this, when you deal with custom tools, it's so nice when it's a file format. Because we deal with OMF and things like that, and OMF is just like, you know, it's just this thing.
It's just this blob of stuff, and there's all these libraries, and you have to compile them, and you have to go get them. And yes, you have the source code for them, but then, oh, you've got a new compiler, and oh, we've got a link against that. Oh, no, is this C++? Oh, yeah, okay, we've got a link against that. Oh, no, recompile that part. You know, this is like, it's a spec. It's a file format. It's text.
And what's also fun with it is that, so, David and the Final Cut folks, they're at Apple, and so they have all these people who know all sorts of stuff about programming, like all the core foundation things and stuff like that. And, you know, I'm an old Unix, Next Step, Packer guy. I don't really know from core foundations. So if it's not Cocoa, I'm really kind of at a loss. But so, it's XML.
So I just, you know, go out in the net, and I get XPAT, you know. And that actually, that's for parsing it, but just for this little thing here. So this is, let me just show you this. You know, we've got this simple little program here, which just lets you go and, you know, time things out for your whole, you know.
You know, we're good on that. And then if we actually want to--let's see, put it out there. It's in my debug menu. I got the idea from Safari. And so let's see. So if we just say--let's call it demo. And then--so all I'm doing there is writing out strings and then telling the NSString to write itself out. So I got me a little XML file there. And let's go and say import XML.
And, uh... let's just see... that looks good, yeah. Uh-huh. And... So now we've got that and it's got our timing on there. And now actually there's one really neat thing about this which is not at all obvious to this, which is that this is actually going out and those are actually film res storyboards.
And so that's actually a really nice thing about stuff like Final Cut is I just spec'd in the XML thing. I said, "Oh, well, I'm going to give you some media and it's actually this big." Even though the media is actually like two megapixels, I actually just do it as a you know, whatever it is, 720 by 540 or something.
And it just deals with it. It's just good. And so let's actually just go look at the Let's go look at what that looks like here, because there's a couple of interesting things in terms of -- let's see, how big do we have to make that? You can actually read that.
So this is what we wrote out, and it's just -- it's just text, right? And I'm one of those, you know -- and you don't have to, like, have all the intention or anything like that, but I'm one of those people who, when you write the code, you might as well make it so that it's readable, because unfortunately, David is completely correct in that text file formats, you know, will be edited by power users.
Sadly enough, you know, we tool makers like to say, "No, we'll give you a really nice way to just -- don't open the file. Don't edit it directly." But -- and again, this is where the whole file format versus the 17 libraries that you have to link against is a great thing, because in a production environment, in a real production environment, you have all sorts of people who, you know, you cannot keep them to the purity of your code. You know, you want to write in C or C++. Or Objective-C or whatever your particular thing is.
Well, too bad. You know, they have a cartoon to make. And so if they need to write a Perl script that's going to go over this thing, grovel this thing, move this thing into here, or they've got to write some Python that's going to do this, or some C shell, or an Apple script, or a whatever, you know, God bless them. They have work to do. And so this is actually what's really neat.
And let me -- I just want to point out one thing that's really, really neat in here. And I don't know if this is actually true, but I will take some credit in giving them the -- saying at least that this was a really good idea, which is this little guy here, the path URL, which what that means is what -- in this XML file format, your media, it's a URL. And I'm pretty sure that, you know, like the Final Cut, the version of this as it's going in here, it's -- it really wants, you know, file URLs.
But from our perspective, we write custom things that are in this file format. And, you know, we might have a web server in the way of that media there, and it's going to serve up that piece of media to you. And because it's a URL, you know, if you're writing custom things that's using this file format, you know, you're all good. You know, you can just put an arbitrary URL in the way there. And if you want to have your version control system in there, if you want to auto-generate that piece of information, you know, there's all sorts of interesting things you could do with that.
And like I said, the format is really, really simple. You can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format, and you can just go to the file format.
And it's really simple. And for parsing this stuff, I used XPAT. Nice thing with XPAT is it's out there, and sort of most languages have sort of access to that. It's just this very simple C library. And XML is just -- I mean, it's not a very complicated thing.
It's
[Transcript missing]
So sort of one final example I'd like to talk about today. Sort of the concept of production pipeline. And this is not so much an editorial, so much as actual final rendering and production where you're trying to take a series of disparate tools to get a certain look for something.
Earlier I mentioned the concept that there may be one application that does a blur really well and one application that does a composite really well. You just want to smash them together. And a lot of people do production pipelines this way. They may render and shake. They may pull it into After Effects. They may output it using Compressor. And sort of allowing tools to be hooked into that pipeline is incredibly compelling.
And part of this is project files. XML is great. Certainly you can endure the pain of OMF and AEF. I don't recommend it. But if at least you can sort of concept something to drive your software from the outside, that's sort of the beginning of it. And then, of course, we move on to media interchange. And trying to bring media through a system without loss in quality or at the very least known loss of quality.
It may be that you have a tool that doesn't make sense to work in YUV, but if at least sort of you can document what the effects of it are, that's going to make someone more willing to give it a shot and see what it is. And scriptability is great. At the very least, just dropping something on and having it go. But technologies like Apple Script are very useful. They're very interesting to implement at times, but they're incredibly useful.
And try and summarize things. Project data interchange, import, export, wonderful thing. Open formats, wonderful thing. If you can move data through an application, actually, and something I sort of mentioned a moment ago, if you can move data in and out of an application, it's actually going to make it easier to be adopted than harder. One of the challenges we've had on Final Cut Pro has been convincing people that, you know, that $200,000 thing you bought three years ago, it's really not all that. You need to try this PowerBook.
And their response is, well, that's great, but that thing in the corner, I know what it does, and I know I can count on it, and I'm just really nervous about bringing my project into it and not being able to get it out again. If they know that you can bring data in and out, they're a lot more willing to try it.
They're a lot more willing to throw it in. And if it's great, they're a lot more willing to adopt it because they sort of have that safety layer. Media interchange. Again, just like project data, if you can pull data through an application, know what the results are, it's just going to make people more willing to try it. And combining the unique abilities of your application with which others just makes it all that much more powerful.
And this is not strictly data interchange, but we alluded to it a few minutes ago. Extensibility and scriptability is very important, both sort of in workflow systems and in production pipeline systems. If you can very basically control how something--sort of spawn something, spawn a process to go do a render via script, via just an import file with some extra commands in it, it just makes the whole process easier and makes it possible in some cases.
And scripting is great because if you have those wacky power users out there who are just bitching because you haven't written that one tool to do the special clip move that they've been demanding for three years, if they can script it, then they can just go off and write it themselves.
And a couple strategies against the same messages. Open project media formats are great. Scriptability is great if you're doing an application. If you're writing a little tool, you want to hook into that data flow somehow. You might write an integrated plugin. You might write just a small application that takes the output of one app and converts it to another just like the automatic duck guys have done.
And if you can integrate tools into an application, it's very compelling. I mentioned a little while ago that sort of based around XML, we have a new plugin API coming soon. And we'll actually be discussing that on Friday. And being able to sort of like the automatic duck guys have done, being able to insert a tool directly into the UI of a larger application like Final Cut Pro or Shake or something else just makes it easier for users who may be very artistically talented but not technically talented to get their head around something and use it.
Our direction for professional applications is we like open formats. The whole concept is there's an ecosystem. There are tools of varying sizes and the users don't really care where they come from, they just need them to work. Open data formats allows everyone to play and everyone to talk in the same space. We like QuickTime a lot.
We're pushing for QuickTime to provide open, high-quality data interchange methods. So new codecs like our uncompressed codecs and Pixlet. Basically trying to set the Mac up such that if you're doing high-quality video processing, it's just easy because you don't have to deal with things like file I/O and gamma conversion and hardware output.
The pieces are there for you to build into your applications. Extensibility, extending application through plugins, through scripts, etc. That just makes it all the more powerful because then power users can do their thing, tool developers can use their things. In the end, there's just a much more compelling solution.
And finally, as mentioned earlier, we really want your feedback. Anything you feel we're doing right, we're doing wrong, tools we may not have thought of, things that you're trying to do with Final Cut Pro or other applications that you just can't do, we really want to hear about that so that we can adjust our direction accordingly to try and address your needs and sort of make this space just a lot better.
And let's see, one other session we had this week you might be interested in, we actually had a session on Friday talking about all the different plugin APIs of the Apple Pro Application Support-- Shake, Final Cut Pro, Logic, Soundtrack. That's on Friday at 3:30. Also be going into--also be giving some initial details on the plugin API we're hoping to wrap XML around later this year.
If you need to contact people, first person's really Jeff Lowe, he's the evangelist at Apple for Pro Video Applications. He's an excellent starting point, he'll vector you off to whoever you need to get your question answered. If you have questions on Final Cut Pro Data Interchange, feel free to email me and I'll get back to you as soon as I can and again vector you off to the right person if I don't have the answer to your question.