Ladies and gentlemen, please welcome Senior Director, Interactive Media Group, Peter Graffagnino. Hi. Good afternoon, everybody. I know it's been a long day, a lot of interesting and exciting announcements today. But this is the last session. I promise you it'll be worth it that you stayed here for the last session of the day.

So let's get started a little bit. We talked a lot about Leopard and all the innovations this morning. But we in the system software team have been busy on a few other projects this year as well. Both Apple TV and iPhone contain a lot of the media and graphics technologies that we work on in Leopard as well.

And so, if you look across Apple's entire product line from Macs to iPhone, iPods, and Apple TV, we really have a broad opportunity for you guys in a number of different areas. And I want everyone to kind of think beyond just addressing the desktop and thinking about some other platforms as well.

So, one way to kind of look at this universe of evolving Apple products is to think about kind of desktop platform, the web platform, and the media platform. And these are, you know, not unique to Apple. Everyone out there is sort of talking about these things. And a bunch of people think that, you know, these are their forces at work and these are all going to become one thing. It's going to be a grand universal runtime to rule everything and it's going to play your media and do your desktop computing and your web computing.

And I guess my take on all that, and maybe I'm just old, but I don't think that's going to happen. I have an analogy for this just from thinking about, you know, how we go about our daily lives. My desktop, at least my laptop, I kind of feel is like my home. I mean, it's my stuff.

The people I, you know, people I let into my house, I know I kind of treat them a little bit differently than someone I might just meet on the street. The web, I kind of think like the... of the pub or the Village Green where it's very social.

You go there, you may meet an old friend, you may find a new friend, you may do a little business with the bank. But you're kind of out and social and connected. And it's a great platform for that, but, you know, it's not where you live. Although I guess some people live in the pub, but... Not everybody. The media is like a very different kind of modality for people.

You sit in a, you know, in a movie theater and all the chairs face the same direction. You watch the screen, it's kind of a very passive experience where you're transported by a story. And so I think that these are, you know, for all time have been kind of three different, different ways we are. So I think that it's unclear to me that these platforms are all gonna merge. It's also unclear for some technical reasons.

Let's look at the desktop. The desktop is really all about performance and responsiveness. You have your hard disk drive with lots of storage. You've got 64-bit computer. You've got all of the power of the GPU. You've got potentially fiber channel connections. You've got multiple core CPUs. All of this power is really there to drive the user experience and make the tool or the task that the user is doing with the computer be performant and responsive. And so there's a big difference when your data is kind of microseconds or milliseconds away than when it might be minutes or hours away.

Device connectivity. This is another thing where the desktop is really essential for dealing with things like scanners, printers, MIDI keyboards, cameras of all kinds. So again, you need rich frameworks to be able to take this data in, use it, and put data out again. And trusted execution. The desktop is an environment where it is your stuff on your hard drive, and the applications you run you trust to help you to manage your life and to build your tools in your personal thing, just like you might take care of things in your house.

So let's talk about the web. The technical aspects of the web that I think make it unique are, for one, its connectedness. I mean, it is a big cloud out there. Every site can connect to every other site. It's very linked. It's interconnected. It's always on. Again, it's like a social environment where people go to meet and communicate.

The other important thing about the web, and this is critical to the web in order to meet that need of being a social place, is it's really platform independent. It doesn't matter if you're running Windows or Mac or your iPhone or Linux or whatever. You can get on the web. You can take care of business. You can meet people.

And also an important leg of the whole ubiquity of the web is that it's based on standards. So you can have an implementation that runs on a device or a desktop or multiple browsers. Content Media So this is a platform that has the widest reach. There's a wide variety of devices that can play back content and media, come in all form factors and different trade-offs. Form factor, performance, battery life, connectivity, and all of those things make it practically any form factor device can play media.

And media's really a narrative experience. So it's usually a storytelling thing. You sit back, you get transported, you go where the story's going to go. And you're in a different receptive mode. You're not participating. You're really being moved by the story. And lastly, it's really limited interactivity. I mean, people tried to do interactive storytelling with CD-ROMs and things like that, and people didn't really want that. You really just want to take where the director and the storyteller wanted you to go. So maybe you can time shift or play, pause, stop, those kind of things, chapter navigation. But really, you turn it on, and you just want to be entertained.

So those are some of the reasons why I think that, yeah, we'll see a little blurring of the lines, but basically these three platforms remain distinct. And obviously, they remain critical to Apple on all fronts. So for example, in the world of Apple devices, you could have a great app on OS X on Leopard.

Then you could develop a website, which would be great on OS X, but that website would work just fine on Windows as well, especially with WebKit on Windows now and Safari on Windows. Or that website could work on your phone. You saw from Scott today how great the phone is in being real high-fidelity web experience.

So all of these platforms form an ecosystem for content, for the web experiences, and for desktop experiences. So in summary, platform richness is really an area where the desktop wins because it's closest to the hardware. The web is pretty rich, and Darren's going to have some great demos later of web experiences. And of course, media is everywhere but not as rich. Platform connectedness is really where the web excels. Desktop is pretty connected as well, but there's not as much of agreement on the protocols that people use for file sharing and things like that.

Media is typically not a connected experience. People are playing with this area, but frankly, I'm not sure that's what I want to do when I'm being entertained. Now, if you really want to talk about hitting that broad variety of devices, whether it's DVD players or your home theater system or an iPod, then media is really everywhere. And so that's where media really has an advantage in terms of reach.

So those three platforms are the way I set up the talk today to talk about the desktop, we're going to talk about the web, and we're going to talk about the media and kind of how all of these things create opportunity for you and your applications. First up is going to be the desktop. We're going to talk about Mac OS X Leopard and all the graphics and media technologies we have for you there. And to do that, I'd like to invite John Stauffer up to the stage to take you through making great desktop applications.

Thank you, Peter, and welcome. So today, I'm going to be talking about making great desktop applications. And more specifically, I'm going to be talking about something more dear to my heart, which is making great desktop applications using graphics and media technologies inside of Leopard. So let's get started.

So the technologies that I wanted to highlight, that I wanted to encourage you to learn more about, that we wanted to have you look at in integrating into your applications is Core Animation, ImageKit, Core Image, OpenGL, and QuickTime. These are the technologies that we're going to talk about.

We're going to run through some demos. We're going to look at some examples. And what's important to understand as we walk through and we talk about these is that these technologies not only are for enhancing your application, for enhancing the user experience of your application, but they can perform the heavy lifting of your application. They can do a lot of the work that your application needs to get done.

So to start with, I'm going to read a quote from a developer, Will Shipley of Delicious Monster. And Will has had the chance to use Core Animation since we seeded it, and here's what he has to say about it. "The revolution coming from Core Animation is akin to the one that came from the original Mac in 1984. We're going to see a whole new world of user interface metaphors with Core Animation." summarizes how we feel about it, and that is that it brings -- it enables you as the application developer to rethink and refactor your applications to build new capabilities into your application.

So let's talk about it a little bit. So at Apple, we've been doing exactly that. We've been rethinking our applications, our user interfaces, and-- We wanted to bring that capability to you. We wanted to expose it in a way that you could do the same. And that's why we built Core Animation. We built it for you to do the same kinds of things we're doing at Apple.

So Core Animation is a layer-based framework. And what that means is that it's -- a layer is 2D planes arranged in 3D space. And Core Animation handles the animation of those planes So Core Automation comes with a very easy programming model. It makes it easy for you to leverage it.

Core animation supports transparencies in these layers, and it supports all the filter effects that come with Mac OS X, all the filter effects that are part of core image. It supports multiple media types in these layers, such as text, 2D graphics, OpenGL can be embedded on a layer and animated, and the same can be done with QuickTime.

So a layer is a fairly expressive type of media. And Core Animation is GPU-accelerated and multithreaded, so it gives you the performance you need to handle the most demanding animations. It's a very powerful API. So Core Animation is built to deliver high production value experiences. That's what we built it for.

So Time Machine is a great example of using Core Animation. Time Machine takes what normally would be a mundane task of backing up files and makes it into an interesting, and usable application for a user. It takes the content, it takes the function of that application and makes it foremost.

It makes it the thing that everyone, that the user is focused on. It makes it clear to use. So it's a good example of the type of rethinking, refactoring you can do with an application. So let's do a demo of Core Animation, and let's show some examples of diving into it a little bit more.

So many of you have seen this demo in the keynote. But what I want to do is I want to talk about the mechanics behind it a little bit. So we talked about layers and space. So as you can see, these are just layers arranged in space. And the power of core animation is in its animation, right? So as I type a search key, I see that these layers are animating. And what's important to understand here is that LayerKit is handling that animation for you. The application is defining the beginning and the end state of those layers in space. And Core Animation is moving them there for you. It's animating those layers in space.

That is all just part of Core Animation and free with Core Animation. So we also talked a little bit about different media types that can be put onto a layer. Let's look at that a little bit closer. So we see video here, but we talked about still images. So it can handle still images. It can handle text.

But what's interesting here is there's another layer behind this. There's another nugget of functionality here. And that is that Core Automation can handle levels of detail as well. So I can sit here and zoom into this PDF that has text on it. And what happens is Core Animation calls back to the application and asks it to redraw this text.

at full resolution. So as I zoom in, I'm getting full-quality text, even though I'm zooming in on a document. And I can zoom back out, and Core Animation, again, is handling the level of detail. So, you know, the same thing can be done with other types of PDFs or rendering. You know, so here's a map. I can zoom in on this. And again, you can see I'm getting full-quality rendering. So let's go back to slides.

So that's Core Animation. So at Apple, we've also been building robust image viewing and thumbnail viewing capabilities into our applications and into the operating system. And this also led us to build ImageKit. ImageKit, is built for you to be able to do that same thing in your products, your applications.

So ImageKit is a Cocoa framework that handles your common image handling tasks. It has a robust image view for viewing images, but it's not just for viewing them. It handles common editing tasks like rotate, crop, scale, and it can also apply all the filter effects of core image. So you can take the image and you can perform a whole wealth of operations and filtering tasks on that image in the view.

It also comes with a high-performance browser, thumbnail browser, and we've spent the time to optimize this such that it can handle a large number of images. In fact, it can handle up to 250,000 images, so that no matter what your user is doing, if they happen to If you pick their iPhoto library on their file system, your application won't all of a sudden come to a stop. It'll actually handle it robustly and manage that.

It comes with a slideshow and a picture taker for giving your users full-screen picture viewing and integration with the camera. And it leverages Core Animation. So this is a technology that layers on top of Core Animation for some of its functionality. So let's go back and let's look at a demo of that.

So here we see a fairly basic application. On the top, where the fish are, that is the image view. The bottom section of the screen, where the thumbnails, that is the image browser. So as I said, you can perform basic operations like rotates. You can perform scales. And that's all just built in.

At the bottom here, we have the thumbnail browser. And you can see that it gives you smooth scrolling, but it handles other types of tasks as well. For instance, if I select some images and I move them, the image browser automatically handles the animation and the reordering, so it gives you a smooth, polished look of managing your thumbnails. So right now I have 46 images in here. Let's throw another 5,000 in there.

Just drag it and drop it in there. You can see very quickly it imported those. They're very responsive. And again, I can sit here and just scroll these. And as I scroll quickly through these thumbnails, ImageKit is off building the thumbnails for me, building the thumbnails and displaying them.

But what it can also do is, let's say I scale these thumbnails. Core animation, I'm sorry, ImageKit is off building the thumbnails at the new resolution. So it's all very easy for you to have this kind of capability built into your application. OK, let's go back to slides.

Core Image. So Core Image is not new for Leopard, but it's a technology that we have continued to invest in. It's a technology that we feel is a key technology, and it is the backbone of image processing inside the operating system.

We've made it extensible through custom image units so you can write your own effects and extend the capability of core image. And it's based on a full floating point implementation, meaning that you get professional-grade image processing in your application.

Like our other technologies, we spent the time to GPU accelerate and multi-thread core image. And Core Image, like I mentioned before, it is used by some of our other technologies, like Core Animation and ImageKit. It is the filter effects technology in those APIs. So new for Leopard, we've added 25 new effects. We've added 64-bit support for handling those large images. And we've enhanced performance. In some cases, we've enhanced performance by as much as 10x. So we've been investing and improving this, and it's going to be a great technology for Leopard.

OpenGL. So OpenGL, we look at it as the bedrock of graphics on Mac OS X. It is the harder abstraction layer at which you access the GPU. It is that lowest level API. Many of the technologies that we've been talking about, and many others, directly or indirectly, leverage OpenGL to accelerate through the graphics processor. So OpenGL is an industry standard.

Apple works with companies like NVIDIA, ATI, and Intel to continue to further OpenGL, make it better for developers, easier to leverage functionality, more efficient, but also to expose new hardware functionality. We make it such that as new hardware comes out, we work with the standards body to get that exposed functionality in the standard.

New in Leopard is OpenGL 2.1, and that includes GLSL 1.2, the high-level shading language. And like many of our other technologies, we've spent the time to multithread it to make it efficient on your system, optimized so that it runs at maximum performance. We've also built a 64-bit version, so you can integrate this into your 64-bit applications.

And part of the new shader language is what we call Generation 4 shaders, and that includes geometry shaders. So now we have a new functional programmable block on the pipeline, which is geometry shaders. And we've improved the tools. We've spent the time to improve OpenGL Profiler and OpenGL Driver Monitor for debugging and optimizing our applications. And we've improved Shader Builder for building your shader effects.

So some of you have probably seen John Carmack's technology demonstration in the keynote. This represents his-- Fifth-generation game engine. And we think this is a really good example of what can be done with the power of OpenGL. We think this is a great example of building an immersive 3D cinematic application. And this is the kind of applications that we hope are around the corner, that we're going to start seeing more and more of on the platform as the graphics processor and the APIs to drive that get more flexible and more powerful.

So let's talk about QuickTime. QuickTime is another key technology that we invest in at Apple. We've been investing heavily in improving. An area we've been investing in is QtKit. QtKit is the Cocoa framework for accessing QuickTime functionality, and it makes integrating QuickTime functionality into your application easy. It also supports core animation.

Like I had mentioned before, you can embed a QuickTime movie onto a layer, allowing you to perform animations. And we've added some new API in QtKit for Leopard, and that is it supports device capture. You can capture video from iSights and many USB and FireWire cameras. And it also supports 64-bit. This is the only way to get QuickTime support in a 64-bit application.

So let's look at a demo. And to do that, I'm going to invite Geoff Staahl, the 3D graphics engineering manager, up on the stage. And we're going to look at a demo of World of Warcraft. And this is an interesting demo because it combines two technologies, OpenGL and QuickTime together to bring new capabilities to an application, new user functionality. And we thought it would be interesting to look at this and see how that was done. So with that, I'm going to hand it over to Geoff. GEOFF STALL: Thanks, John.

We can go ahead and switch to the demo machine. And what we're going to show here is a little bit different than last year. Last year, we showed how Blizzard has taken advantage of multi-threaded OpenGL to greatly enhance the performance of their application. In this case, we're going to show how Blizzard has taken a synergy of some of the technologies we have here to greatly enhance the user experience, allowing players to create great machinima productions in-game. What we're demonstrating is a new update coming very shortly from Blizzard for Mac OS X that will allow in-game movie capture and compression on the fly. So let's jump right in and show you what you're going to have.

So if we look under Mac Options, now we have the ability to control movie recording. We're going to be recording for the demo at 1024x768, 30 frames per second, and using an H.264 codec to compress on the fly. So as we run around, one of the keys here is to actually integrate this into your player experience so it doesn't really affect your gameplay. So I've just enabled the video capture. And if you notice, the gameplay is still extremely fluid.

Using the power of the GPU and the multicore on this system, Blizzard has been able to put the video capture in and the on-the-fly compression without detriment to your gameplay. So once you finish your adventure, you can head back to the campfire to tell tall tales. And you can show who really was the hero for that day.

You see the compression is finishing up here as I'm heading back out. And this will compress into a QuickTime movie that we can then send to your mom, sync to your iPod. Send to your Apple TV or to your iPhone or even make a great website. So let's hop down and look at the movie we've just created.

Launching in QuickTime, this is the movie we just created, 1024 by 768, 30 frames per second, on-the-fly compression, and the gameplay still is tremendous. So this is a great example of how Blizzard has taken, again, a synergy of these technologies, OpenGL, QuickTime, the GPU and multi-core CPUs, combined them together to create a great, enhanced user experience. Thanks a lot, and back to John. JOHN STAFFER: Thank you, Geoff.

So that's a great example of combining two technologies to bring a new functionality or a new capability to your application. And that brings me to Quartz Composer. Quartz Composer is a technology that does just that. Quartz Composer integrates many of the interesting technologies in Mac OS X into an easy-to-use framework.

Quartz Composer is for building stunning motion graphics and visual effects. That's what Quartz Composer is built for. But further, Quartz Composer comes with a tool that allows you to build these visual effects, and we call them compositions, in a visual programming model. You can launch this tool, and you can build visual effects without writing a single line of code. And that's one of the really powerful parts of Quartz Composer, is the tool that allows you to do this. It makes it an artistic process, not a programming process.

So these compositions are easily integrated into your application. And so new for Leopard, what we've done is we've added the ability for you to write your own patches. Patches are the functional blocks of a composition, the logical functional blocks. So you can write your own patches so you can extend the capability of Quartz Composer.

And what we've also done for Leopard is we've built a system-wide composition repository where compositions can be shared between applications. So, for example, Photo Booth, iChat, and Automator use the compositions in this shared repository. You can build a composition, add it to that repository, and automatically extend the functionality of those applications. Those applications will pick those compositions up and they'll become available in those applications. So that's a great way to share compositions between your applications or other applications.

So we've added new and updated the current patches. We've been putting a lot of work into enhancing those, the patches that come with Quartz Composer. And we've completely rewritten the visual editor. We've rewritten it from the ground up, making it even easier and more flexible for you to build compositions.

So that's Quartz Composer. So something we're doing that's interesting this year is we have a display wall. Some of you may have seen it already, but it's something you have to go see. It's down in the Graphics & Media lab. It's a massive video wall display. And this is based on Quartz Composer, and it shows the flexibility that Quartz Composer has. Quartz Composer brings very flexible technologies into a single package.

So we talked about a few technologies. We highlighted a few. Don't forget, there's a whole array of technologies in Mac OS X and Leopard. And we encourage you to go to all the sessions and learn about the graphics and media technologies that are built right in, technologies that will not only enhance your application, but make your application optimized for the system and do the heavy lifting for your application. These are fantastic technologies that should make your jobs easy. Easier. So with that, I'm going to hand the stage back to Peter. Thank you. PETER GRAFFANINO: Thanks, John.

Thanks, John. So that's a little brief tour of Leopard and some of the new things we have for you. As John mentioned, there's tons of stuff, lots of sessions, so please check out all the great Leopard stuff. I'd like to talk a little bit about Mac OS X to kind of set the stage for the web section. Mac OS X is a layered architecture, and that's really stood us well over the test of time with the applications and frameworks on top of graphics and, of course, the OS stack.

So we've got a lot of stuff underneath. And if you think about it and squint your eyes a little bit, the web is sort of designed in the same way. There's websites, which are kind of all the applications that run on the web. There's an HTML engine, which provides the framework.

There's a graphics layer underneath that that supports the HTML engine for text and graphics and an operating system, obviously, for the device that the browser is running on. And really, if you think about Apple's implementation of the web, it really is all about... And one thing we haven't mentioned so far is that actually the Quartz 2D API moves with WebKit wherever WebKit goes.

And so both iPhone and the Windows implementation of WebKit rely on Quartz 2D. That's why we get such great rendering fidelity across platform. And, of course, QuickTime, the QuickTime plugin, plugs right into WebKit to provide rich media services as well. So if you look at all the things we're doing in this space, it's a pretty compelling platform. And to talk about that... Across all of our products. Sorry, I missed that click. But to talk about this, I'm going to invite Darren Adler up to the stage to talk about Apple's web technology. Thank you.

Thanks, Peter. So even though I'm the manager of the team that developed Safari, I'm not really talking just about Safari today. I want to take you on a quick tour through the world of web applications and web technology. So first, let's get a little bit of context. Here's a picture you've seen earlier today a couple times, desktop as of 1984. The state of the art in terms of the look with those snazzy rounded corners around the screen and in the user interface.

The desktop has changed a lot. If you take a look at desktop today, you see a lot more media, pictures everywhere, graphics, and also a lot more richness. Well, the same kind of thing has happened with the web. Here's the web as it looked in the Mosaic browser at the dawn of the web in 1993.

And websites are mostly about text and links. There are plenty of pictures in them, but occasionally. And the web has also changed. Alongside the desktop, the web is also now full of media. Even a website like the New York Times that's still basically about some information and some links is full of elements that have media, have interaction. And this is what it looks like in a modern browser like Safari 3.

Both of these platforms, the web technologies and the desktop platforms, are obviously going to keep moving. And what happens is that capabilities that were something you'd find only on the desktop in years past become something that's achievable with web technology. I'll show you what I mean by that in a moment.

So what are some of the distinguishing things about the web? Well, we talked about the connectedness of the web and also the ubiquity of the web. If you take these four, these are the top four browsers by market share, Internet Explorer, Firefox, Safari, and Opera. If you take these four, they cover a huge variety of devices and platforms, everything from a kiosk in an airport, desktop computers, game consoles.

You're going to be able to get web content all over the place because of these. And what makes it work across all these different browsers is the way that we cooperate and use standards. But even if you just think about Safari 3 and WebKit, this is something that you've got on Mac OS X, you've got on iPhone, and also on Windows. So there's a lot of reach to web content.

So what is a web application? Well, a lot of times you might hear the term Web 2.0. And this was coined a few years back by Tim O'Reilly, the famous technologist and book publisher. And it's really about a new generation of websites that are different from the websites on the early web.

The kinds of things that we're talking about when we talk about Web 2.0 are websites that involve participation and also websites that start to have a feel that's less about moving from page to page. And more like what you associate with a desktop application. So now let's take a look at some of these web applications to show what I'm talking about.

Wikipedia is an interesting example. You take a look at this page full of information like an encyclopedia page. And there's nothing there that particularly makes it Web 2.0. But the key is the interaction and participation. If you know more about the Adelie Penguin and you want to add it to the site, you edit it using this link. Wikipedia's editor.

One of the seminal web applications that had a lot of impact on what everybody thinks of being possible on the web is Google Maps. Again, this is an application that when you're using the map and manipulating it and searching for things, you're not moving between web pages. You're using the kind of interaction and clicks and gestures and user interface that's a little bit more like a desktop application.

37signals is a company that's done a number of web applications, and this HiRise application is an example of a contact manager. Now, these are applications for keeping track of contacts, very popular for people in businesses like sales who really need to keep track of who they talk to.

And these are traditionally desktop applications. Here's one built completely with web technology. And then you get to some applications that are really traditional desktop applications like word processors, and Google Docs, formerly Rightly, is a complete word processor implemented with Web Technology. So I'd like to show you another web application. So let's go over here. One of my favorites.

and this is the mail from .Mac. So if you're familiar, if you haven't seen this before, you might not even realize that we're looking at a web page. The main clues are the fact that it's inside Safari and has the bars from Apple.com. But what's going on here is that you have a mail application with a lot of the same qualities as mail applications on the desktop. It looks a lot like the OS X Mail app. You click on a mail message and see the message down below. These fancy ones look really great.

And there's a lot of interaction here that's exactly like what you would expect in a desktop app. For example, if you choose a couple messages, you can delete them or reply to any one message, mark them as unread, all the same kind of commands you'd expect in the desktop.

And also, the user interface is very similar to the desktop. The user interface has elements that you might not expect to find in the web at all. For example, I can just take this message, drag it over and drop it into a folder. This is not what people think about -- what most people think about when they think about web pages and web user interface. So let's see. We can search for a word like "team" and -- oh, that was the wrong word to search for. Let's not do that.

Or I can search for somebody's name in my address book. And just choose it here and use it to address my mail. So really what you've got here is something that has a lot of the properties of a desktop application, even though it's implemented entirely with web technology.

So let's go back to the slides. So even though this is a web application, it's reminiscent of desktop apps. And it really is the kind of thing that it looks like a desktop application, maybe not a desktop app of today, but certainly a desktop app of a couple years ago.

The techniques used to implement this are a big part of a lot of the different Web 2.0 websites. And one interesting thing about this is even though I was demoing that in the brand new Safari 3, it works just fine with existing browsers. It doesn't require brand new browser capabilities. It works great in Safari 2. So if you're putting together a web application, there are a few things you want to do.

One is that you want to make sure that it works in any browser and degrades gracefully. And another is that just like any software development, you'd like to reuse your code. You don't want to have every single web application start from scratch. And to make that happen, one of the most important things to do is to get a JavaScript library.

JavaScript is the programming language you use inside web pages alongside the markup languages, HTML and CSS. And there are some really excellent JavaScript libraries that have been a big part of the Web 2.0 thing that's happening right now. Dojo, Scriptaculous, which is built on prototype, are some of the richest and most popular.

You can also build your own JavaScript library. It's great language and it's pretty easy to make reusable code. Here at Apple, we do both of those things. And if you take a look at Apple websites, you'll see all kinds of use of these libraries. One of the nice things about the web is you can see the source code just with view source and all. And so see that we do both of those things.

Now, when the people build these sorts of libraries and you build your site, there's really a rich set of technologies to build on. A whole lot of stuff already built into every browser, all the modern ones. And just to give you an idea of how this relates to desktop technology, I picked out a few of the fundamentals and listed the classes in AppKit or the techniques in AppKit and the web techniques. And I'm not going to go through all of these, but, you know, there's, you know, if you want to do a table on the desktop, with AppKit, use NS Table View.

There's the table element on the web. But let's talk for a moment specifically about 2D graphics. So, because of the Canvas element, which is an innovation originally created by Apple for Safari around the time of Dashboard, which is now in most of the modern browsers, you can do graphics operations. Pretty most of the graphics operations that are in Quartz 2D are available to you within the web.

So let's look at an example here. If you looked at Apple applications recently, you may have noticed that to create the illusion of a floor, we often reflect an image. In fact, I think of this as the rounded corners of the 21st century. So to achieve this in a web page is straightforward with Canvas. You simply have the image, draw it twice with the proper settings to make the reflection look right.

And so I just, that's an example of achieving these things that you think of as doing with desktop technology. Now another example is rich text editing. I list here one class, NSTextView, as the Cocoa way to access rich text. And it is true that there's only that one class you have to deal with to do the basics, but there are really tons and tons of classes when it comes to rich text editing in Cocoa. And the reason for that is it's a really complex class. It's a complicated problem.

It's one of the trickiest things to do in an application framework. Now with Safari 3, we have a full implementation of that rich text editing based on what is done in other browsers and some web standards. Let me show you a little bit, a little example of that.

So this ties together the idea of using a JavaScript library and also a little bit of an example of rich text. So imagine that this is a piece of a web page that's editable. And because we want it to be what you see is what you get, we don't want to put a lot of editing controls on the page all the time.

So instead, what we do is make it so that once you make a selection in this text, you get this attractive editing toolbar that appears around it. Now, any time you have editable text in Safari, you have this whole menu full of editing stuff that the browser supplies that's always there. So I'm going to use that. Just pull up the font panel, make the text a little bigger, about 20 point.

But in addition, this toolbar gives you something that will appear in any browser that lets you do these editing operations, bold some of the text, make these three words italic, or even take a whole paragraph and right justify it. And what I think is particularly cool about this example is that it's a little bit of JavaScript to create this toolbar. And it's really beautiful. It's nice looking, nice experience, and works with any browser.

So that's an example of the kind of building block that you might want to use when you're putting together web applications. This particular one looks good. We're going to give you source code. There's a session on HTML editing later in the week, I think Wednesday, where you'll hear more about this.

Now, as I said, Safari 3 now has full rich editing. The basics were there in Safari 2, but now it's really all there. So one thing I'd like to ask you to do is please, if you have a site that does rich editing in other browsers, turn it on for the new Safari.

A lot of times, we go to sites where there's rich editing, but we just get plain text because they say, oh, it's Safari. They don't have this stuff. So please turn it on. And also, if you're doing that, please check for the new WebKit version, not the new Safari version, because we don't want to lock out other web browsers that are using WebKit.

So I talked a little bit about some of the things you use to put together a web application, but there's another really interesting category, what I'm referring to here as hybrid applications. Now, there are really these applications that take advantage of web technologies and desktop technologies. Now, there's one category like World of Warcraft that uses the Internet and networking on the back end and desktop-type stuff on the front end. And that's really one type of hybrid app.

But the kind I'm talking about is one that wants to use other web technologies like HTML and JavaScript. And this is really straightforward to do with Cocoa. You just create a web view, which is an NSView subclass. There's all kinds of ways to use it, rich text view if you're authoring HTML, to incorporate web pages. Let me just show you a couple examples real quick.

IChat in Leopard uses WebView for its chat transcript. And what does that do for IChat? Well, one of the things it does is you can have a bunch of different styles of transcript, and you don't have to write separate code for those because the people writing IChat didn't have to write separate code for those because they can use CSS style sheets.

So just by changing the style sheet, I can make this chat instead look like this or like this. And all of that is done with the style sheet language. You get that for free just because you're using a WebView. Another example of sort of the premier application using WebKit is Mail that uses WebKit for authoring all the mail.

A great example of a third-party app that's a true hybrid is NetNewsWire, which is an application to help you read lots of news feeds. And here, what WebView is really being used for is to be a mini web browser inside an app. And depending on the type of app you have, you may need one of those.

It's really easy to get one. Finally, I thought this was an interesting example. There is a web chat program called Campfire. And so it's just, it's a web application. It's on a web page. But they wanted to have a desktop version of it. And so what they have here is this application called Pyro is just really a single-purpose web browser.

And I've seen a lot of these cropping up recently, things that are called applications that are really just a web browser that points to a particular site, a real junction point between web technology and the desktop. So let me show you now one of the most interesting and the rest of the world. This is actually something built entirely with web technology.

and others have been working on a new web kit that is built into WebKit. It's not yet in the versions that you get, say, on the Leopard DVD, but it is going to be in nightly builds very soon. And that's the Web Inspector. Let's say that I want to take a look at this website and figure out how it's built.

And I'm interested in how this Delete button is implemented. So I choose this menu item here, Inspect Element. And because I have a build of WebKit with this additional feature in it, it'll appear any time I click on anything. And immediately, I get a view of the web page. This is the DOM tree.

So what you see here, and you probably can't see, I can make it big just for a second. Let me put it back to the normal size. This is the entire DOM tree. The way HTML works is a tree. And I have this image element selected right now. And over to the right here are all sorts of details about the style of that element. And that's very useful if you want to find out why something is a certain color, what rule made it so.

And I'm not going to take you through all of this, but you can find out what style sheet a particular rule came from. If you click on that link, it'll take you right to the style sheet source code. There's some other information you can get. For example, what's the size of all the various boxes, which is really good for understanding the layout of the page.

Now, as this page loaded, the document itself is only one of the things that was loaded from the network. All sorts of other things were loaded. For example, all these images were loaded. And so we can take a look. DeleteDim.gif is the one on this one. I can click on this link. It'll take me right to it. And here's the image itself. And that way, you can take a look and say, you know, if you see the name of a resource, you can go over here and view what it looks like.

But there-- oops, wrong button. Oh yeah, that docks it in the window. Sorry, didn't mean to demo that. So let me reload this again. Does it help me show you this? So if you have the inspector up while you're loading the page, it'll also report any errors in your HTML. And this page has some tags here, has some examples here of some tags that were closed without being open. So that's great if you develop a site and you want to see what mistakes you made.

This same console is a place where you can evaluate any JavaScript exceptions. So for example, I'll just type in something here just to show you that the whole language is here. Get the Delete button and get its inner HTML. And there, it's evaluated the expression. And so there's the string that came out of it.

So that's really good if you want to test some JavaScript functions you've written. There are so many features. I want to show you a couple more. If you type-- let's say you're-- interested in this Delete button, you want to find out all the code that's dealing with it. So let me just type Delete button into the Search field.

And I'll immediately get all the different places Delete button is referenced. The original HTML source, which has this cool syntax highlighting here. The DOM, which is the parsed version of the source with all the information we saw before. Here's a style sheet. This is a style rule that applies specifically to the Delete button.

The JavaScript code, here's all the places in any JavaScript code loaded by the site that refer to that button. It's real good when sleuthing and trying to pick apart a site and figure out how it works or any kind of debugging. So the last thing I want to show you here with the Web Inspector is this little view we have that shows you how your page loaded. This is a graph over time.

And so we see each color corresponds to a different type of thing. So we see here the document loading, and then the style sheets loading, and then the scripts loading, and then the images loading. And you can learn a lot about why it takes a certain amount of time to load your site. This is really good for figuring out how your site will perform on a slow network, too. So we think this is going to be really popular for people doing web development, whether they're targeting the desktop or iPhone.

So this tool that will help you build those apps, it was created entirely with web technologies. And because of that, it's going to work in any WebKit application, and it also works in Safari. We didn't have to do a whole new version just to make it work on Windows as well as OS X. And it's going to be available soon in WebKit Nightly Builds.

So Mac OS X is a really popular platform for people who are developing content for the web. And I just wanted to dig a little bit into the reason why it's so popular. This is something I've heard for a long time. And I think one of the biggest reasons is all the tools that are side by side.

What's built into OS X are all these unique server technologies, the very same server technologies that many, if not most, websites use on the back ends. And also there are these beloved client-side tools that are really what people building websites love to use. And they're both there on OS X.

So if you take a look at all these built-in tools, I just picked a few here to give you the idea. But these are a few of the things that are built in that you can actually run on your OS X machine. And a lot of web developers I know sort of run a small version of the site on the very same machine they're developing on.

And you can do that without having to add anything, because all this stuff comes built in. And then as far as client-side tools, there are these crucial tools that you use both for content creation, manipulating your images, editing your HTML, and all that sort of thing. And both of these are available on OS X.

But I think there's another reason. And I went and asked some web developers why is it they like to develop on OS X. And this is what Jason Freed told me. Jason is the CEO and founder of 37signals, the company that created and popularized Ruby on Rails and also the application, a lot of web applications.

And he said, working on OS X reminds me that beauty, clarity, and workflow matter. That's a valuable reminder when you're building a web app. about. So I think that there are some things that go beyond just the concrete issues of exactly which tools are available that affect people choosing OS X.

So how is it that these web technologies are put together and work across all of these different web browsers? The key is web standards. And I'm just going to say a word or two about web standards. This is an open process that involves all the different people who make web browsers.

And in addition, lots of people are interested in web technology. And the way the standards process works helps prevent us from really diverging when we have these multiple browsers all trying to work with the same web. And it also gives a way for innovations to spread from one browser to another.

The process is managed by standards organizations. And two of the most important standards organizations are the W3C. This is the original organization that does the HTML standard and all sorts of vital web standards. And also, more recently, there was a group called the What Working Group formed a little while back.

It was formed by the browser vendors who, at the time, were a little bit frustrated with not enough standards that really were relevant to the future directions of browsers. There's a recent standard that's being developed that's actually being worked on by both of these groups. It was started in the What Working Group and now is being worked on in both W3C and What Working Group. And its original name was different, but it's now called HTML5.

And this standard is going to be a really big deal for the future of the web. One of the things that HTML5 is doing is defining, previously undocumented and undefined things about the web very clearly so that the browsers will all be able to implement them the same way. So it's really going to help for compatibility.

A lot of the new features that really would help pushing Web 2.0 even further-- I don't know if I was more of a marketer. It might say Web 3.0 there. And also, some relevant Apple points are it includes Apple's Canvas element, which we created and was quickly implemented in some of the other major browsers. Some recent news-- we contributed a design for video and audio elements.

We really want to bring media out of the plug-in world into the browser world. And our proposal was very well received. And it was a great example of bringing a lot of the expertise that our media experts here have at Apple. Last thing I want to say about HTML5 is that this is one of the easiest standards to participate in. It's even more open than most. The mailing list where the technical issues are discussed is something you can subscribe to. If you're interested, go to the W3C website.

So what's coming in web technology? Well, there's going to be more innovation. For us at Apple, this standards and this whole space of web is a place where we want to do a lot more. And as you can see, we're really serious about web technology, having just brought Safari to new platforms with a bunch of new features and an open-source approach. And I think there are some opportunities for developers here. And one of them is, if you have an existing website, turn it into a richer Web 2.0 website. Or if you have an application, think about what kind of web presence really makes sense for your app.

With the web technologies, there are a whole variety of different ways to kind of add the web to your desktop app. And there's also this situation where Web 2.0 has undergone this explosive growth, and I don't think the development tools have really caught up to that yet. So I think while there's some really exciting new ones that have come out recently, I think there's room for web development tools. So for some of you, that may be an opportunity as well. And as you heard this morning, the skills that you develop developing web applications are going to be directly relevant to making great applications for iPhone. So with that, I'd like to hand it back to Peter.

Thanks a lot, Darren. Thank you. Great. So Darren talked a lot about the web. And I'm going to take you through the media section of the talk, where we're going to talk about the media platform for Apple. So really, if you look at kind of the web 2.0 application phenomenon, you know, kind of Google Maps being the archetypical starting point for that, more and more of these sites are starting to-- like YouTube and Google Video, other sites starting to add video and media into their websites. So it's a really important kind of development in web design.

And Apple plays a role in that, certainly, with all the great technology we build right into Mac OS X. It makes a great foundation for the wealth of content creation apps that you see out there that some of you have built and Apple's as well. At the heart of all of this media platform for Apple is QuickTime and H.264.

So let's talk a little bit about H.264. It's the next-generation codec from the MPEG community as well as the phone standards groups. It comes in a variety of profiles. It's really meant to scale all the way from phone-type applications all the way up to HD and cinema. It comes in three basic profiles, extended, main, and baseline, and I won't go into all the details. But it's been well-received in the industry and adopted by many, many platforms from PSP to Xbox and PlayStation 3 and platforms like that.

And of course, Apple uses it for a number of applications. We use it in a real-time teleconferencing video chat, the application we ship with the OS, iChat AV. So it's great low-latency codec. It's also the backbone of our media business with iTunes, with the content we distribute, both movies and TV shows.

And of course, with Apple TV, with recent announcement with YouTube, what we've done is we've had them give us a web feed of their content in 264, and we put a user interface on the Apple TV for that. So we're using 264 with the YouTube content there. So really, across all of Apple products, it's really the core of our strategy to leverage this effort on this codec and implement it up and down the product line. And some specific examples of the bitrate.

So we have sites that we use, for example, to get full HD on a decent performing Intel Mac. It might be 1920 by 1080 HD. For Apple TV, we have 720p video, also HD. iPod and iPhone are megabit and a half of the baseline profile. And on iPhone, we have format as well that is tuned and optimized for the edge data. network, the cellular data network.

So all the way up and down, we can get great video. So let's talk a little bit about media on the iPhone. We haven't really talked a lot about this yet, but it's real important that you guys understand what's coming with iPhone and media. So iPhone can play back media in two forms. First off, it's an iPod, so it can take any content that's synced via iTunes. If you have a podcast, it can get synced to iPhone and actually stored like a video iPod. Or you can stream via Safari.

We've extended Safari with a simple embed in object tags that can do streaming over the internet. The thing to remember about iPhone is it's always connected, so it has both Wi-Fi and Edge connectivity. In fact, it can switch between them if you go out of range of a hotspot, Edge automatically comes on. So it's real important to realize that there's pretty different data rates there, and so we're recommending about a megabit per second.

So it's a simple profile for Wi-Fi and about a 90 kilobit stream for the Edge network. The more speeds and speeds, the 264 baseline that is supported is level 3, and it's up to 640 by 480 at 30 frames per second. We also recommend AACLC, which is our standard audio codec, and for media container formats, it's .move, mp4, m4v, and 3gp.

So this is not in your seed, but we are going to be adding to the ever-growing Movie 2 exporter list for a quick time. We're going to add Movie to iPhone, which is appropriate for streaming over Wi-Fi or hotspot connectivity. We're going to add Movie to iPhone Cellular, which is for the Edge data network. It's compatible with the 3GP file format. And of course, Movie to iPod, since iPhone is part iPod, that data rate works as well.

And one of the things you may not know if you haven't been around the quick time media space is understanding the concepts of ref movies or reference movies. And these are movies you put on a website that actually point off to a bunch of different representations of the same video.

So for example, based at runtime on conditions like network or whether you're connected to Edge or Wi-Fi, we can pick the appropriate data stream to feed. And this works both on desktop and iPhone. So with that, let me go over to the iPhone and give you some more demos of media.

So if I can get the iPhone up. So let me-- I'm going to go into Safari on the iPhone and I've bookmarked a site here that we've created with a bunch of demos for you. First off, we took one of Jeff's rehearsal demos with his World of Warcraft and made it work on the phone. And there's a little blog, Jeff and His Quest. And you'll see what happens with iPhone, with media on the iPhone. There's an embedded experience, but then media always goes full screen. So there it's playing. Let me see if everything's working here. There we can rotate.

So you'll notice that we have a full-screen experience on iPhone, so we don't actually play video embedded in the web page. It just goes to full screen, so we can use all the real estate. Let me go back. There's a heads up display there that I'll show you that in a second.

So let me go back to my page here, and here's an interesting one. The apple.com guys, you know, they launched a new Web site today, a new redesign. They're also preparing for iPhone. So they're actually putting up ref movies that can stream both edge and Wi-Fi. And I've got the Ratatouille trailer here, which of course you have to play.

I know there's some Pixar people out there, and Chris Harkin are the founders of the Paris France, home of the finest restaurants and the greatest chefs in the world. You can see I do a fit or a fill just with a double tap. Go back to my web page.

And rather than just showing sites we created, we thought we'd go find some that work pretty well on the iPhone. This is a video site. It's like a soap opera, but it's only on the web. So you go watch this site. And it's a site where the basic setup is it's a bunch of folks having breakfast, and they kind of review their week and what they're doing. It's kind of fun.

But if I go to the View Episode link, you'll see they come to their episode viewer. And you see the QuickTime logo there? That's their plug-in for their embed tag. And the one thing that they haven't done to be ready for iPhone is we have, there's a way to do a JPEG URL for the poster frame. And that's kind of the one thing they're missing. Other than that, they'll be ready. If anyone who works on that site's here, maybe they'll be ready tonight. But basically, you can see the content.

and David I can show you the scrubbing as well. We have HTTP 1.1 support. In fact, you need to have an HTTP 1.1 server to feed iPhone video. But I can even seek outside of the range and hopefully it will catch up. There we go. Honest, Mr. Wiley, it was a ride home. So if you've already seen the first few minutes of the show, you can go watch the end.

But anyway, so that's that site. And like I said, the only thing they're missing is the embed tag poster frame rather than putting the poster frame in the movie. So they're pretty close. Let me go back out, and I think that's, I think I'll just show you that for now and then save the other one for later. So that's a few things about iPhone and media.

So let's talk about Leopard and H.264 and what we're doing there. So we've improved the encoding speed. We know that this is a sometimes painful process. You want to get your content up on the web. So both the single pass and multi-pass is about a factor of two faster now, and we're continuing to work on that to get it even better.

QtKit has gained some capture capabilities. John talked about some of these, including capture from multiple cameras simultaneously, which people have been asking for as well. We have alpha support in the 264 codec, so you can create compressed alpha movies. And we're going to turn on full screen for all.

and I'm going to kind of show you a best practices site we've put together. that brings together all of the richness of web technology plus Quicktime. So what we have here is a site that we-- Ducati of America is actually a neighbor down in Cupertino, so they were kind enough to let us take some of their assets and play around a little bit. So we've developed the site here that we think is a best practices site for embedding media. So right here we have a media-- a plug-in instance, and I can play the video directly, have an inline experience.

I can stop that. I have a watch now if I want a more full-screen experience of this particular entry. And what that does is it switches over to a 720p feed, and it goes full browser width. And you'll notice we've got chapter tracks that are marked, and you can see as the video traverses the chapters, the HTML at the bottom is changing to respond.

So this is all done in JavaScript. And you can see as I hover over the chapters, I even have little chapter marks that help me navigate. So that's pretty cool. And you can see as I hover over the chapters, I even have little chapter marks that help me navigate. So that's pretty cool.

So that's... it was really easy to write the JavaScript controller as well because of the DOM events. So let me go back to the site here and point out a few other things. We have some download buttons. We're recommending that people support downloads direct of, you know, if you have a video site like this for both iPod, Apple TV, and this download goes directly to QuickTime. This is the 720p content downloaded.

and the rest of the team. Peter Graffagnino, John Stauffer, Geoff Stauffer, Darin Adler and the rest of the team. Peter Graffagnino, John Stauffer, Geoff Stauffer, Darin Adler and the rest of the team. Or you can actually drag the XML directly into the iTunes window. That works as well if you just want to manage iTunes to manage it and it's not through the podcast directory. So anyway, that'll sync down all the content. All the content on this site is in the XML. And so, for example, I've synced a couple of these.

This is an HD version that then I could sync to my Apple TV. So we think podcasts are real important. If you're putting up an episodic video site or a video blog or something like that, to have that XML. Because I think that's going to help you reach a lot more platforms that are more in the sync model than the streaming model.

Okay, so that's the Ducati site. Now, what I wanted to do was show you the Ducati site also running on the iPhone. So on the iPhone, as I said, it's the baseline profile, so some of the media embeds need to have the proper ref movies to bring up the appropriate formats for the iPhone since we support the baseline profile. If I can get the iPhone up again.

There we are. So we'll go to the Ducati site. That is the same bits from the same server. And you'll see it's basically the same design. I can zoom in and you'll see it render. And we've done some conditional browser detection to remove the download things because they don't really make sense on the iPhone. And zoom back out. I can play the video and the video goes full screen. And this is now streaming the iPhone version of the video.

And to thank them, I had to actually play the whole commercial for you just because I thought it's cool. Anyway, so the other thing I wanted to point out here is the web pages are all, if you do the right CSS, you get kind of the semantic zooming where you can get the appropriate element kind of coming to the front.

And so that's another thing to be aware of when you're developing websites. If we can kind of figure out from your CSS the important areas of your site, that really helps navigation. So anyway, so that's it for the iPhone demo. And I believe I left my clicker over here.

So again, that was the same bits coming from the same server, which is just with a few tweaks and the right content on the site. So let's review what was on that site. It used reference movies to be able to address the appropriate data rates for desktop and for iPhone. It used a poster JPEG and the object embed tag. That's why the serialized guys didn't have a poster frame showing for their content.

One thing on iPhone is you need to be aware of your viewports. Some websites are very particular about their viewports, and on iPhone, you need to be aware of that because it's not the same size as a desktop. There are ways to detect the user agent, and so we were doing that to kind of disable the download buttons because they don't make sense on the iPhone.

Also, we recommend including an RSS feed for offline viewing, and that's your way to support Apple TV and iPods. And don't require JavaScript for media playback. The media extension that we have in iPhone right now does not do any JavaScript. It just gives you that full screen experience. So don't require JavaScript. Have an alternate.

So that's the combination of 264 and web technology. I think you can see you can make some pretty compelling websites with that. And again, you can support the wide variety of Apple platforms that are out there. And I think there's a lot of opportunity here, both in the authoring space for tools, as well as in the content and application space.

So those are the platforms. We gave you a little tour today of desktop, web, and media. And my message to you is to really get ready and kind of think beyond just desktop applications. Think about this whole bigger ecosystem of devices and websites and content and how to support developers with rich-media experiences. And of course, keep iPhone in mind and build great Web 2.0 experiences on iPhone.

So there are some sessions I want to point you out. I can't list them all here because there's too many, but I will point out the tracks. There's the Graphics & Imaging track, which is about all the APIs for graphics and media, of course, emphasizing the new ones in Leopard.

There's a Content & Media track, which this year is focusing on web development and content creation. You'll definitely want to go to that. There's a session in here tomorrow talking specifically about iPhone issues, and there'll be some discussion of both the kind of the the web technology piece of that as well as the media piece of that.

And in closing, we decided to have a little fun since it's at the end of the day, and we put together a brief video for your viewing pleasure. I'd like to roll now. You have no doubt been hearing a lot about Web 2.0 recently. Today on WIT TV, we sit down with management at a local company focused on integration of media with Web 2.0 technologies.

Thanks for letting us visit CashCal Media. Before telling the viewers about your unique approach to media and Web 2.0 integration, tell us a little bit about your company. I'd be glad to. CashCal Media combines expertise in web technologies with a passion for media. Our facilities feature state-of-the-art equipment, allowing us to deliver results to our clients and partners.

And to better serve clients and partners, we're committed to ongoing employee education in all the most relevant web technologies. Tell us about your focus on Web 2.0 and the opportunities you see. Yes, certainly. CCM wholeheartedly embraces the social networking phenomenon. Through our partnerships, we're involved with the destination sites. Also, in this results-oriented world, it's not about what you're doing, it's about what you've done.

Consider microblogging. Who has time for sentences, let alone whole paragraphs? We're pioneering nanoblogging. Words, they're the future. Mm-hmm. Please tell us about challenges in producing rich media. Well, we see many of the same challenges producers of mainstream media see, but also face some unique ones. Take, for example, finding the right talent for projects.

I see you've only worked in low-bitrate video. And we have other challenges in finding the right talent for our web media efforts. Finally, let me show you something we just put together demonstrating the synergy possible between traditional markets and Web 2.0-enabled media. In the '70s, math was hard... ♪ ♪ ...until the arrival of the calculator. The lumbering is still hard, but now there's SearchBar.