Good morning, everyone. Hopefully everybody had a good time at the event last night on campus. I'm Glenn Bulycz on the QuickTime marketing team. And we're very, very privileged to have this speaker here all the way from London. I've known Mark and Dulcy, his partner, for quite a long time. And if there has been a developer in the work that we've been doing that has consistently over-delivered and painstakingly made products for their customers that just astound in every area, technologically, aesthetically, it's certainly bluish. So I'll introduce Mark here and he'll come up and present to you some of the work that they've been doing and then we'll hold Q&A afterwards. And thank you for coming and here's Mark Griffiths.

morning We've basically been building these skins for the last two to three years, really. And we found them a really amazing way of delivering an experience for the user. And this is really what we're about, is to try and create an experience for the people who come and look at these things.

So what I want to do today is to really go through the steps that we take to create these things. And now, they do touch on a lot of areas of what QuickTime can do. So I'm going to talk a little bit about the encoding that we do, how we do it, the choices that we make when we're designing the skin and thinking about what we're going to include in it. Now obviously The actual construction of these skins is critical in creating that experience. We want this to be seamless. We want people to feel like they're having something different out of these.

So to get them to run properly, things like the graphic construction of those is critical. The other thing that we've gotta do is to think about how the people in our audience, what kind of technologies do they use to connect to the internet? How do they come to our material? And certainly in the UK, we still have a very large dial-up audience. You know, a lot of people are still on modems. It's changing, you know, we -- broadband is starting to come along and really replace that. I mean, dial-up is really feeling like it's on the way out, which is great for us because we can start to really push what we we can do with the video and the encoding and so on. So again, I'm going to talk through some of the things that we have to do to make sure that the modem user still has some degree of experience compared to the best broadband. you know And I've mentioned a bit about performance, and I'm going to talk as well about how the performance relates to the-- the aesthetics of the presentation. And then I'll have a quick look about how we actually go about deploying this on the web, so that we can create a destination for people as well, because the thing with these media skins is they're files. And we don't want to just embed these in a web page, you know. There's plenty of sites that already do that. And Again, we're trying to be different from this. We want to create a differentiator with this stuff. And if we have time at the end, I'm going to talk a little bit about how you can make these things dynamic, how you can update the content. Okay, so we started doing these back in 2001, not long after QuickTime 5.02 came out and gave us these capabilities. these.

So the first one we did was actually for Pink Floyd. We were designing this website that was very dynamic. It was all Flash-based. And again, we were wondering, how could we include this video in this very dynamic Flash website. At the time, Flash didn't have any video capabilities of its own. And it just didn't seem right to have this rich, interactive environment. And so, I think, and then just popping open a new window and embedding QuickTime movies in it. It would have been all right, but it didn't really allow us to achieve our goals. So, we're looking at this and what we wanted to really do was I'm going to keep going back and talking about aesthetics and quality in this because A lot of our work is around that. And that was very much part of the brief, the Pink Floyd. Pink Floyd have always been about their visuals, about creating, again, an experience. The skins that we made for them, they needed to be emailed. We wanted to send stuff out to the mailing list, to the people who were interested in the band. And in the same way, we wanted it to be part of the website. Again, stand out from the crowd. We wanted it to be unique. And at the time, back in 2001 when we did this, it was pretty special.

it did create something new for the visitors on the website. And certainly with Pink Floyd, when we did that, it was, as is usual with record companies, the content wasn't there. You know, we knew kind of what to expect, but it was... I think with about a week till launch, we still only had one piece of video ready. So the skin had to be able to talk to a database at the back end. And we could bring in new content and so on.

So what I'd like to do now is just to show you the first Pink Floyd skin that we did. So this is what you get when you started our player up. Now to begin with, this list on the right was rather sparse. But as time's gone on, we've managed to dig more and more video out of an archive and put it through this system. So I'd just like to show you something running in this. We don't need no education We don't need no thought control ♪ Oh ♪ That's kind of a summary of where we started. It was, the main thing for us is also, it's the quality of the video presentation. I mean, we see a lot of QuickTime stuff on the web, a lot of things get encoded, you know, around the available bandwidth that people can use. But here we were really interested in pushing full motion video because It was a new thing. We'd just got Sorenson Video 3 come out and it really did change our ballpark at the time completely. We were now able to deliver these full motion videos at a larger size and encapsulate them in these players. So this is where we started. Over the last few years, we've done quite a few of these now. We're probably on our 12th or or 15th player now. And there's been a range of creative executions. And so, here's a play we did for Ed Harcourt. It's a similar thing, I mean, We have the, here we have the 16 by 9 video, which we certainly in the UK is what everyone has been moving towards. And so again, we wanted to create a presentation to really show that off.

Okay, so you see now, you're probably wondering why I've got this nice wide screen area and I have letterboxed the video. The reason being, some of the other video that we were using were completely different, it was a completely different aspect ratio. And so the skin had to be programmed in such a way so that it could, so that we can handle both these videos and try and again create the best aesthetic that we could in the presentation. The other thing that we were able to do with this, which we thought was pretty cool at half an hour. I was convinced I was going to get a first bite. And then I had to pull this box out of the cellar, which was fine. But it was just when they got the-- We were able to add a commentary track to the video as well, which is something that we see in DVDs and that sort of thing. But here we were actually able to interleave that into the main video stream and to give ourselves a commentary track so that we can actually get Ed in to talk about the video and and experiences he went through. Could I go back to slides?

Okay, so I've talked a bit about what the QuickTime Media Skin is, you know, it's, and you've seen a couple of examples. It started off in QuickTime 5.02 was the first time we were really able to create these custom window shapes. Before that, it was still rectangular movies. and You know, they were good. You could certainly brand those up, but they didn't feel that different.

What we have been able to do is to create this kind of now electronic flyer, digital flyer that really the shape of the window is as much part of its branding as the content. Each one of these skins uses an awful lot of what QuickTime offers us in terms of its video handling, the audio handling, and being able to create interactivity through the use of wired sprites. And obviously, we make full use of QuickTime's networking and internet capabilities.

So that brings us on to where we are today, and the most recent skin that we produced, which is the Robbie Williams Nebworth skin, which you may have seen, you know, Now the brief on this one was very much to make an impact. We wanted to kind of shock people with this one. I don't know if you're aware of Robbie Williams as an artist, but in the UK he's huge. He is probably one of our biggest, most successful artists. His music doesn't really translate.

He's tried to kind of move out into a bigger international market. But he's such a typical British cheeky chappy, you know. And it's just, that's what he's about. And so we wanted to kind of try and touch on that anarchy just a little bit. And we wanted to communicate the energy of his performance. He is a very, very energetic performer. Um... And so we wanted the skin to reflect that. The branding of the player needed to be integrated across the whole DVD campaign. We wanted to have some degree of recognition with what they'd done in the DVD artwork, what they'd done in the tour program for the whole series. I mean, Robbie's tour across Europe was seen by millions of people. I mean, I can't remember.

I think it was close on 2.5, 3 million people who witnessed this concert as it traveled around Europe. And also we wanted to include some of the videos, the DVD extras, and artwork from the package. So if we can go back to the demo machine, I'll just take you around the Nebworth skin in a bit more detail.

So as you can see, we have a very, very complex shape. This took a lot of work in Photoshop. We didn't really have anything that was the right shape or-- that could hold video. So this whole pattern was comped together from various assets from the artwork and to try and create a frame that we could play back the video in. So... You get the idea with the video there, but there's a lot of additional functionality in here as well.

we wanted to carry across a lot of the traditional interactivity that people are used to from viewing video in QuickTime sorry, in QuickTime Player so while being able to actually kind of play about you can also scrub through the video here you know choose any point you like We can completely go full screen, and so the user can focus just on that, clear out the rest of their desktop clutter, and just concentrate on this. Also, obviously, because some users are more capable, we have to account for a range of skills. So we've included some degree of instruction for people who were confused with what all the various iconic controls did.

uh... one of the things i think that that i really think it's quite cool is is in terms of handling all these different bandwidths of video so we have obviously this is our big stream now I'll talk about this a bit more in a bit, but if anyone's ever tried any, you know, done encoding, you've probably noticed this is quite challenging footage to encode. We have a lot of crowd shots, a lot of pans and zooms.

And the bandwidth that we were used to encoding our video at really were not cutting it. It really needed everything to be pushed up a notch for us to be able to convey the quality of experience. So if you're down at the modem level-- So what you can do here, at the moment you see I've zoomed the video up to fill the entire window, which may well be what some people want to see.

However, that is actually its native size. So you can actually choose to have a higher quality at a small size. If you're on a lower resolution screen, that's still going to be fairly reasonable. But this will download as a file much more happily over a modem. Now we also wanted to kind of hit a middle ground.

So you can see here, this is a bit larger. And this is the stream that people on 512 kilobit ADSL may choose to see. It would stream in real time. The big stream over here, your broadband speeds are much more impressive than ours back at home at the moment. That big stream would still require a little bit of waiting to get it buffering before it'll kick off and start playing. So this one, again, you can zoom it out to fill the whole player. And thanks to the kind of video smoothing inside Sorensen, you still can achieve quite quality presentation of it. So aside from all the video in here, We also wanted to show people how to use some of the what they can expect from the DVD package. So we've got this other QuickTime movie that could be brought in. And I don't know if you noticed, but when it came in, the controls changed to allow people to interact with this content instead. So you can step through and see some previews from the artwork there. We have some of the DVD extras. when you came in on a chopper. Um... And we wanted to give people a proper kind of, some liner notes really from the DVD so that they knew what to expect from the package overall. Now this was, This is quite a big deal for EMI who we did this for because it gave them a way of getting this out into all sorts of people's hands. We could promote it through Robbie's website. We could email it to the mailing list. It really gave us a lot of flexibility. So what I'm going to do now is really sort of get into the nitty gritty about how we made this. the steps involved in actually producing the video, the interactive part of the Quake time, and the eventual deployments on the web. So, if we can go back to slides.

So the first step of the process was trying to gather the assets. Now this is always a challenging process, particularly when record companies, artists, management, there's a whole host of people that have to clear this stuff. And more often than not, we find that we end up with really the raw deal at the end of the process. Everyone's taken their stuff. We see all this media produced and we can't work out where these assets have come from because we never saw them. And But with Robbie we were quite lucky. Robbie's management company were editing the video themselves. And so they were actually able to provide us all the footage on DVD as QuickTime files. So that gave us a really good head start with this, and one that we're not used to seeing. Normally we're not a large company, Bluish. So the idea of us being able to go out and afford DigiBeta decks and-- that sort of thing to allow us compatibility with this is really beyond us. So we normally tend to... Expect things to be delivered on mini-DV.

We even get stuff delivered to us in MPEG-2 on DVD, which comes with its own set of problems. There's a lot of transcoding issues from MPEG-2, audio tracks that seem to just disappear. But in this case, we were pretty well sorted. The artwork that we were given in this case as well was pretty awesome.

We were given everything that they had. We were given the graphics that they used to develop the, to print up all the backdrops on stage. We had all the artwork from the DVD and the tour program all in high res. So we pretty much could do whatever we liked with this. So we started to think about how we could actually pull this artwork apart and build a player from it. So the first thing I want to talk about here is the design, really.

Because we're a strange lot. We're just as much into the design as we are into the development of these things. And like I said, I keep going back to talking about the aesthetics of this. And this is very apparent in this player. Now, while it's a very elaborate player design, the thing that was really key to us to get good playback was to actually make the underlying player incredibly simple. We found through our own bitter experience that the moment you start to try and get too clever with the way that you're handling media in these things, these things tend to really slow down. But with a bit of thought at the design stage, you can really sort this out. We were, and obviously we wanted to include a lot of interactivity, so this was already forcing us down a particular path with this.

I've already spoken a little bit about the context sensitive navigation. And again, I A lot of the time we don't know what is actually going to end up in this. So there had to be levels of code written inside QuickTime to enable that functionality. Now the window mask shape caused us no end of problems. We knew we wanted to create this very elaborate baroque feel to the way the shape of the player. And so we thought, well yeah, we'll just take the artwork, we'll stick it through levels in Photoshop, bit threshold, and it'll be fine. And oh no it wasn't.

It was horrible. I mean the first time we tried that and started to play video back in it, everything was being jerky. You tried to pick the player up and move it around the screen. It would be jumpy. It just wasn't very pleasant. So, we had to kind of go back to scratch and literally draw that shape by hand, concentrating on getting as many pixels in a horizontal line as we could. Because that really helped QuickTime. That meant the drawing routines on both Mac and Windows could really deal with it. And I can show you-- That's the shape we ended up with. So as you can see, it's pretty clean. I mean, there aren't many stray pixels in there. We've painted them all out. So that gave us our shape.

Inside that, we needed to lay in a background, something that we could build on top of. Um, so what you're seeing there is the actual background JPEG from the QuickTime skin. Uh, the--we use a range of, of, you know, image file formats to encapsulate these things. So, you know, we use JPEGs where we have large, complex backgrounds, um, or photographic backgrounds. We still use GIFs occasionally where we want something to be really small. Um... And more and more, we're now getting into using PNGs and that sort of thing, 'cause it-- they have certain advantages themselves. So... We then, you know, here you can see what would happen if we tried, if we laid the video into the design.

which wasn't really doing what we wanted. So we figured, well, maybe we can take these extra bits and we can float it over the top of the video. with alpha channels and so forth, but again, we lost all our performance. The video would not play back smoothly at 25 frames a second with those in place. So we kind of thought about this, and we actually went back to a technique that we used to use years ago when doing CD-ROMs in Director, and we actually laid this framework in place into the video. We produced a mask, and as we were encoding it in Cleaner, we were applying this mask to every single piece of video so that when it came back to be displayed by the user, it would seem seamless. We tried to make sure that whatever state the player was in, that that edging would be in place so that you never saw the actual rectangle that held the QuickTime movie.

So again, talking about the aesthetics, one of the things that we find really frustrating about QuickTime is the fact that it wants to put this funny white rectangle with a little QuickTime loading icon. And we find that quite jarring. So, What we did here is to create an overlay, an image that we could bring back to the front whenever a video stream started playing that would cover up what we figured was a bit of a, something that didn't really fit with the overall feeling that we were trying to create.

So now we start looking at laying in the interactive elements. Now again, this was critical. One thing that we, certainly I did it. The first time I tried to create one of these, this is exactly what I did. I would create a sprite track. I'd try and lay it in over the complete design, knocking out the background by setting it to transparent in live stage. But again, that just does not work. Every element needs to be, it needs to be separate, it needs to be its own thing. So what you start to do is to build up a whole range of tracks for each interactive element. So for instance, the progress bar here, you know, the transport control here, the bandwidth control, the main navigation, which is embedded as a flash track, and the kind of window controls. They're all separate elements. They've all been kept small, self-contained, so that we can have a whole range of different functionality around the player without it impacting performance.

So now the flash track at the top, that gave us some very interesting issues. Because the-- Again, just like the video, we couldn't stick any graphics over the top of that track because it would kill performance throughout. So using the track masking features of Photoshop, we added a mask for that top bit of navigation. would define the area that the Flash movie would occupy. So this enabled us to lay the Flash track on top, but to create the visual appearance of a track that was behind the rest of the design.

OK, so once we've got all those components together, they all add up and create the impression of-- the player overall and give the impression of it being very multi-layered when in fact it's not. It really is quite quite simple everything is is in its own space. So those are kind of the design considerations we set out with, which is basically based on our experience of doing these in the past. So we'd already equipped ourselves with these few bits of knowledge about how QuickTime would respond when laying these elements together.

So now I'm going to talk about the video. In this instance, we didn't need to capture anything. We already had been given files. And I'm sure different people here will have different technologies that they use to capture their video and bring it into their workflow. For us in this instance, it had already been done. So we could move straight into the encoding phase.

So, to do this, we're still using Cleaner. I know Cleaner seems to have kind of gone very quiet and we find that a shame because that was historically our favorite tool. I've been using that right back from when it was a piece of shareware called Movie Cleaner and that really had been my main tool for processing and sorting out video.

Where clean is going, I have no idea at the moment. And we're certainly going to-- But, so, yeah, we'll be looking at other tools to help us do this. The trouble is, is that with Cleaner, it... Cleaner was special in as much that it enabled us to do everything in one go. The cropping the video so that we could get rid of all the extraneous sort of video signals and stuff from our source. It enabled us to do all the color correction.

set up a batch that would just run through all these files in one go. Now, Like I was saying before, it was a real... It was horrible. When we first got the video in and we tried our first test encodes at the bandwidth that we were used to encoding this material at, it was a disaster.

We just thought, oh, how can we ever send this out? Because it's... Again, the aesthetics of the presentation just wasn't there. So we kind of moved everything up. We normally kind of overall used to use kind of 100 kilobit, 300 and 600 to carry the video. In this instance, that just wasn't going to be possible. So we moved all the video bandwidth up a notch, up to one megabit at the top end. And then we found that all of a sudden, you know, it was a lot of trial and error. But eventually we found a point that we were happy with the file sizes compared to the quality and the motion we were getting.

So as I said, it's all encoded in Sorensen 3. And Robbie's one is... one of the first ones that we started to use AAC audio in, simply because the quality of that presentation is so much better. You can probably hear the difference between the Pink Floyd one where that's actually an MP3 soundtrack. It's all right, but the moment you start using AAC at the higher bandwidths, it really starts to shine.

So these are some of the things we considered as we were encoding this thing. Now one of the things that we've always taken as a choice is I want to, be true to the original timing, the temporal quality. I want to try and carry through that 25 frames a second of the source material because It's music. Music is about rhythm. You know, and it's called full motion video for a reason. And we found that the moment you started to drop the frame rate down, you would either start to have all sorts of horrible sort of half frame problems, and the rhythm, the rhythm of the performance would be lost.

So we chose to go for a temporal quality over spatial quality. I wouldn't mind seeing a bit of blocking or artefacting in these streams if it meant that I could keep this thing running at 25 frames a second and to carry through the... the vibrancy of the original performance. We also use progressive downloads rather than streams and we do this for a very particular reason. It allows users on all sorts of connections to access this. If we try and start to stream one megabit movies down to the user, we're going to be immediately leaving out a whole segment of our audience.

Whereas with a progressive download, they can just wait. If they really want to watch the high quality version on dial-up, they can just wait for it. It also means that we can do additional functionality like the scrub control on the video so that people can randomly access any point of that file once it's downloaded.

As I already mentioned, the style of footage that we had for this gig was such that it's an encoding nightmare. You've got kind of crowds jumping around, you've got these long shots showing 50,000 people. It's kind of the worst possible situation for encoding. And like I've already said as well, we needed to burn those elements, burn the edge of the frame design into the video, which was all done as we encoded the video. So I guess the point I'm trying to make there is that the design considerations up front were really important for us because we had to work out how to put all these different things in place as we were doing it. So here I can give you some of the settings that we used.

So the Pink Floyd material and the Pink Floyd player and the Ed Harcourt player that I showed you are very much that middle column. So you can see the jump that we had to take with Nebworth to achieve what we felt was a similar level of aesthetics to the look and feel of the video.

Now, again, a lot of these numbers came out through trial and error. You know, I sit there with clean air, just batching out different qualities over and over again and we found there are certain sweet points in that that where the the quality of the video and everything would just come together so So that's that. And you can see that as far as the audio goes, that's pretty simple. We've found a set of values there that we're quite happy with over a whole range of-- different things that we're given. And obviously at those bandwidths, AAC audio just, it blows MP3 out of the water.

So now we get into the... So what we have now, we have a folder full of encoded video. We have an idea of the design that we are going to use to... you know, to present this to the users. And so now we get to the authoring stage. So I think fairly obviously, as it's probably one of, well, the only tool that can really do this stuff, it was all built using TotallyHips Live Stage Pro 4.

And this is a pretty awesome tool for us because it's different. You know, we, I mean, historically, I'm a director developer, and I've seen my workflow move away from Director into tools such as Flash and particularly to QuickTime. And Live Stage really gives me as the author, an amazing amount of control.

The language inside LiveStage is called QScript. It's an odd language. It's not perhaps what you'd be used to compared to today's modern scripting languages. But you can still get the results if you're prepared to kind of work on it. So again, every interactive element was laid up into a track so that we could get the performance that we wanted. So we had to break that design down into the little pieces.

so that we can actually achieve the interactivity without it interfering with the video itself. So then, I'm going to talk a little bit about where we put all this code. Because you get into these authoring environments. And it seems like you can stick stuff just about anywhere. But in this instance, we normally now try and concentrate all our code into just one of these elements. In... in this case particularly, is actually all held in the progress bar. Because that is where I need to write code to manage the playback of this video as it streams down. I want that progress bar to be the thing that is dynamically updating as the video downloads.

So that's where most of it's kept. All the other interactive elements tend to send events back to that progress bar to trigger certain events and that sort of thing. It also means that I have one point inside the entire movie that I can go and see all my code. So, oh, and the other place that tends to end up with a lot of code is the flash track. The way that a flash file or a Swift file will work in QuickTime requires you to attach code to buttons.

you have to kind of pass things backwards and forwards to the Flash. You can't, I mean this is something that really pains me is the ability that, or not having the ability to call a function. So in Flash 5, you can define your own custom functions anywhere, any movie clip inside the tool. But, QuickTime doesn't work in it that way. I have to expose variables at the right at the root of the movie and put data in those. And literally, I have to have little buttons off screen so that once I've populated these variables with the right data, I can click that button from the script and the appropriate actions can take place. So-- The most complex code in this is all about playback management.

When you normally sort of see a QuickTime movie, you have a route timeline. And that's what you're controlling. When you play and stop that, it's is all happening at that bottom level. So we actually have quite a multitude of-- you know, a couple of hundred lines of code in this thing just for managing the playback. If the user's network stalls, if, um... you know, for whatever reason the video stops, I want the player to be able to recognize that and react accordingly so that when the data is back in place, it can just pick up and carry on through. day. So I now have -- I've now got a folder full of encoded video. I've got a design. I've started -- and I've authored that. I've put the code in the right places. And that's sort of starting to -- that's behaving itself now.

we now get onto the the step of actually publishing this thing you know we have this quick time movie is is is doing its thing but I want to actually move it to the web The main reason I wanted to have it on the web is to create a destination.

You can't just-- well, you can. You can obviously give somebody a URL that goes straight to the QuickTime movie. But then you're not going to have any of the-- the wonderful sort of shape to the window. It's just going to be embedded there in the page, and it's not the most aesthetic of experiences. So we actually have to try and launch the movie from the web page, and we do that by embedding a second tiny little QuickTime movie that handles that.

The other thing that we want to do from the web page is to give people the ability to download this thing and keep it. It's one of the great powers of these. They are like digital flyers. And it's all well and good for you coming to the page and seeing it, but not everyone has QuickTime Pro. Users with QuickTime Pro obviously can save this thing off their drive and keep it for later.

So we wanted to give people an easy download so they can just keep this. So if I can go back to the... So we actually have a link to this from our website. So you'll see it automatically starts launching the movie. And there we go. So we can kind of create that whole seamless experience in one step. That little embedded QuickTime movie in the page is the secret to moving it into QuickTime Player and kicking the whole thing off. And now we're here. Obviously, we can interact with the skin. I can go back and I can choose to launch it again.

And indeed, I can click a link and take it home for my own, you know, to keep it for later. So that's kind of pretty much the process we-- an outline of the process that we went through to create this thing. It's--you know, it does touch on just about every element that QuickTime has to offer. I guess apart from VR, you know, but we couldn't really see a good use for that.

So that is pretty much it in terms of Neverworth. Neverworth was-- we always knew what the content would be. It was very much a static target. The clips that were authorized for us to redistribute were very clear. And so we could build that player in such a way so that-- The content was just there. We didn't need to hook anything up to the web. What I'd like to show you now is another skin that we actually did for Robbie Williams, an earlier one, where we had really no idea about the content. Thank you again. So again, a similar thing, you know, the video's there. We have a range of video available to browse. You can see that As I select things, it gives you descriptions about what those pieces of video are.

We even have this top level menu where we can actually choose to see what area of content we're interested in seeing. And now all of this is dynamic. The player really is just a completely empty shell that needs to connect to an XML file on the server before it knows what it's going to do.

So this gives us the ability to create these players as empty shells and then choose what content we're going to play into it and how the player can be updated over time as new video becomes available. I guess really that's about it. I mean, the... I guess, in conclusion, I can say, I mean, this is all pretty exciting to us. You know, the ability to create these experiences for people and to... Yeah, you know, it's cool, you know? It's something different, it's something new. There's a lot of power to these players. And we really feel like we're just starting to scratch the surface of this. And we're very interested to see how QuickSign is evolving and the things that we can do in future. Okay, so...