Thank you for coming. It's going to be a great week. Today, in this session, we are going to talk about all the great technologies that we make for the graphics and media needs of your applications. Apple delivers products that give users incredible immersive, vivid user experiences. We saw that today with the release of iPhone 4. iPhone 4 with its new retina display and a high resolution display is the perfect platform for really bringing vivid user experiences. And of course, we see this with the iPad, the revolutionary iPad, and the Macintosh.

At the heart of all of this are the Graphics and Media technologies. We deliver over twenty frameworks, over twenty different graphics and media frameworks for you to leverage in your application. These frameworks deliver a huge amount of graphics and media technology that you can leverage to build this vivid, immersive, user experiences right at your application. These are the same frameworks that we use when we build our products. They are the same technologies that you see when we demo on stage, right.

These are really fantastic technologies. These are technologies you should learn about and use to their fullest, and you will be able to build applications that really make your users love your products. So we deliver these technologies at different levels of API. At the high level on the iPhone, we have a Cocoa Touch API which is an Objective-C API that makes it easy for you to get access to the Graphics and Media technologies and easy for you to incorporate into your product.

We, of course, have low level technologies such as Quartz for high-quality 2D rendering, OpenGL for high performance 3D, and Core Audio for professional grade audio. These low level technologies give you the flexibility to do custom things to get access to all the capabilities of the platform. And then we have technologies such as Core Animation that let you mix and match between the different technologies. They let you take 2D graphics, 3D graphics, and video, and blend them together into really custom-immersive user experiences.

Core Animation is a very powerful technology for you to build really immersive vivid user experiences. So what we do at Apple is we build the graphics and media technology of a common platform across all of our products. So across the desktop Mac OS and across IOS, it is the same graphics and media technologies, the same foundation. This means that we've been able to take years of our experience and we deliver that across all of the platforms, giving you that same high-quality professional grade technology.

So what this also means is that we have optimized this technology for the hardware. So whether you are trying to get high performance or get access to all of the features of the hardware, whether it be microphones and speakers and cameras, this technology has been optimized for that hardware device.

And what is really important about that is that the world is going mobile, and optimized means that when you use our technologies right, you also are optimizing for battery life. And battery life is going to be something very important for your users. You want to make sure that when you are building an application, you are giving your users a good experience with battery life when going mobile.

So looking at some of the applications that you have built, there is BoinxTV. BoinxTV takes our graphics and media technologies and incorporates them in a way that brings a live TV studio right on to the Mac. It is really cool. Or Pinball HD uses an intuitive Multi-Touch interface combined with OpenGL ES to take an arcade classic and make a really great application for the iPad. OrFour Track. FourTrack takes Core Audio and makes a multi-track recorder right on your iPhone.

That is pretty amazing. So you have taken all our technologies, all the graphics and media technologies, and you have used them to build over 225,000 applications. That is a huge number. And of course, I'd like to say thank you because you guys have built some applications that are absolutely amazing.

So for the rest of the talk, what I want to focus on is I want to focus on iOS 4. We are going to talk about some of the new and innovative features that we are building into iOS 4, specifically, OpenGL ES and the new extensions we are offering there, AVFoundation for the audio and video needs of your application, HTTP Live Streaming, the best way to stream live video over the internet, and Game Center for bringing social gaming into your game. And to start, I am going to bring Gheoff Stahl up to talk about OpenGL ES.

Thank you.

OpenGL ES, it's the open standard-spaced technology that allows you to access the power of the GPU. When we conceived iOS, we chose OpenGL ES as a graphics foundation, and John talked about layering OpenGL ES as a full participant in this layering. Core Animation, for example, is built on top of OpenGL ES.

So no matter what high level API your application uses, you get the power of OpenGL ES and the power of the GPU. So before we go into the new features in iOS 4, lets first take a review of OpenGL ES and what it is. OpenGL ES is a low level graphics API that uses vertices, triangles, and triangular meshes to express geometry. When that is combined with textures and pixel data, you can render compelling 3D images such as this Sergeant Shock [phonetic] who would look right at home in a 3D game.

But it is more than this. OpenGL ES can be used for high speed compositing as we see with Core Animation, and also can be used for image effects. And when we go to OpenGL ES 2.0 and we add shaders, OpenGL ES can really unlock the power of the GPU.

So what are shaders? Shaders are small programs written in OpenGL shading language that run on the GPU and can either affect the vertex or fragment parts of the rendering pipeline, thus allowing you to really customize your drawing experience and unlock that full power of the GPU. In this case, we replied dynamic lighting and shadows to Sergeant Shock to really immerse him in your application. I think the best way to show this is to look at a demo. So I'd like to have David Biterman [phonetic] help me with this demo called Extreme Ball.

So this demo is created by Digital Legends for Imagination Technologies, and it is a immersive 3D world here, I made a futuristic arena, and what it shows is the power of OpenGL ES 2.0 and shaders. First, we are looking at the triangular mesh that makes up the world as you would expect. It makes a structure of the world. And now, let us look at some of the fine details in the simulation.

So you have the reflections in the floor, you have the water with animated normal maps with that specular highlighting, the character itself, his facial expressions, a physics-driven hair model, she is animated on a bone structure with motion capture and vertex shaders, animated bones and skinning the mesh, fragment shader provide that lighting detail. When we look at the character, we can see dynamic lighting, we see reflections in the floor, and we see shadows cast.

This simulation drives over a million and a half polygons per second. The models-- the balls themselves are very simple geometric meshes covered with textures, normal maps, specular highlights, create a more-- it looks like you got more geometry than you would expect. Really, all in all, this is a great demonstration of the power of OpenGL ES 2.0, what you can do with shaders, and all of this is running on a GPU that you can hold in the palm of your hand. Absolutely fantastic, amazing technology.

Again, written by Digital Legends for Imagination Technologies.

Thank you, Dave.

We have reviewed OpenGL ES. Let us talk about iOS 4. So last year, we introduced OpenGL ES 2.0 and shaders. This year we are going to introduce some really fantastic new features to add on top of them.

So let us jump right in, starting off with low cost full scene anti-aliasing. We've added a full scene anti-aliasing which reduces the aliasing artifacts in your scene for very low cost while maintaining optimum frame rate. So it markedly improved the rendering quality.

For example, here, you see, if you look at the ball from the previous simulation, you will see there is an aliasing artifacts in the under side of that ball. If we turn on full scene anti-aliasing, what you can see is this aliasing artifacts are markedly reduced. This, and the last demonstration you saw, was running with full scene anti-aliasing on. You can see the frame rates and throughput we achieved. Low cost full scene anti-aliasing, new in iOS 4.

One of the really important things that you saw on the demonstration is the way your 3D scene interacts with lighting and shadows. So what we have done for iOS 4, we have enhanced shadows. We have added support for two shadowing techniques, both shadow mapping and shadow volumes. We have added depth texture support for shadow mapping, and shadow is a technique.

It is image-based technique where you take the light and you render the scene from the point of view of the light into the depth texture, you accumulate these results, and then when you are rendering the scene from the user's point of view, the user results, they guide you to what is in shadow and what is out of shadow.

Shadow volumes are-- use a stencil buffer and show-- and allow pixel accurate shadows, like you see in this example here. If you are interested in either of these techniques, the more advanced rendering techniques, we are going to be talking about these later this week, the advanced OpenGL ES rendering sessions, where we talk about exactly how they created this demonstration right here, enhanced shadows.

So beyond shadows, a really important thing in 3D rendering is textures. And we have a standout example we used earlier, Pinball HD, fantastic use of textures to give you that immersive 3D Pinball experience. So how do they do this? Well, it is actually fairly simple, and it is a common and convenient technique that many of you may already use if you are using OpenGL ES, and it is texture atlas.

So what is a texture atlas? A texture atlas is a large texture that accumulate-- you store all of your smaller textures into, and then during your rendering, you reference out of that. As you can see here, we have textures for the bumpers, for the flippers, and for the table art, all of that is contained in one texture and it's-- and you reference that when you are drawing your scene.

For iOS 4, we have added a new extension texture max level which allows you even more control over texture filtering, allows you to have pixel accurate texturing, and ensures you have the highest quality scenes possible. And we also added shader texture LOD, which is level of detail control for shaders.

This means you can write a single shader that you can control the texture filtering. So whether an object is near to you or far to you and far away from you in the scene, you can ensure smooth alias-free texturing across your scene, allows a shader to select the texture, select the mipmap bubble, and it's interesting when texturing. That is shader texture LOD, two new advanced texture management routines now available in iOS 4. The next thing we have added are two additional texturing techniques. The first one is all about integration with media.

YUV 422 textures allow native support of video on the GPU with OpenGL ES. This allows you to maintain-- do not have to do costly format conversions, maintain an optimal memory layout for video textures, and really allows you to integrate video with your 3D rendering in one space, integration with media YUV 422 textures. The second thing we have added for texturing is floating point textures.

The floating point texture is a little bit different. Floating point textures allow some new rendering techniques you could not do before on a handheld GPU. In this case, I will talk about two of them. One case is you-- let's say you have a really complex calculation that you want to run on a per pixel basis. So, that-- running in a shader.

What you can do is you can precalculate your data at a granularity that is set for-- that's right for your application, store that in a floating point texture, upload it to the GPU, and then during-- when your application is running, that complex shader calculation collapses down to a single texture lookup ensuring your application maintains optimum performance. Great technique there. Also, floating point textures, of course, can be used for uploading high precision numerical data.

Let us say you have hype field that you want to upload to the GPU, put it in a floating point texture, upload it, you will be able to reference that at floating point resolution. So we have added a number of new things to iOS 4. We have also ensured that it is optimized for performance. We think this is the most optimized release ever for our graphic system.

We have added stencil wrap for high performance, stencil shadows, we have added vertex array object that allows improved vertex efficiency, especially for multiple vertex objects, we have added discard frame buffer which allows you to-- allows the GPU to optimize its memory bandwidth use, and we have paid particular attention to texturing.

We have improved both GL tech subimage, and we have also improved the overall texture upload speed throughout the system. All in all, this is the most optimized release we have ever had. So how do we bring this all together? Well, let us show one more-- let us show another demo that shows some of the power of OpenGL ES 2.0 and the use of some of the new features of iOS 4. I'd like to invite up Torsten Reil, the CEO of NaturalMedia to give the demonstration.

I would like to show you something today. I've been very excited about it. It is something that we have been working on at NaturalMotion with our partners at MunkyFun.

We are trying to push the boundaries of what is possible in terms of interactivity, realism, and believability of a character on a handheld device. To do this, we are using advanced OpenGL ES 2.0 features, as well as our runtime animation engine, Morpheme. Now, the character that we are working on is a horse.

It is a horse that is much less about my little pony and much more about creating something that is authentic, believable, and something you can interact with in a natural way. It forms the core component of a game that we are working on together with MunkyFun, and I would like to introduce Nick Pavis [phonetic], who's the CEO of MunkyFun, to show us what this looks like so far.

So what you see here on this screen is running live on iPhone 4 on iOS 4. You see here first of all, starting with the close up of the horse, we are using high resolution textures for the horse, as well as advanced OpenGL ES 2.0 shaders to make the coat look natural, but also for details such as the eyes on the horse. Now, what you will see in a moment is that Nick can interact with the horse.

You would not see his fingers here, but he is stroking the horse, and the horse is reacting to that naturally and interactively. We are using procedural animations for this driven by the morpheme animation engine rather than canned animation. Now, what's Nick done here is send the horse on to the paddock and have it run around on the paddock.

So the scene itself is about 16,000 polygons, the horse is another 10,000 polygons, we have got full scene anti-aliasing running on this scene, and you can see that the horse itself is running naturally, again, using a morpheme animation engine. So what we want to show you now is really, how we are building this scene out starting with the beginning, with the basic lighting which you see here. We are now adding textures to this, and the textures themselves are high resolution.

So for example, the grass alone uses four different high resolution textures to avoid repetitiveness and tiles. Next up, we are adding anti-aliasing, as you have just seen in to the scene, and particularly, is useful for areas like the fence, for example, especially when we do the slow camera pans, we get rid of the jaggies. Again, that is something that is possible now in iOS 4.

And moving on, we are now adding vegetation here. So you can see the trees have just been added in the background, as well as grass in the foregrounds close to the fence. This is done using Alpha, again on OpenGL ES 2.0 and iOS 4. And next up, we've added fog. So the fog is used to make all the scene components sit together in a natural way.

And we have implemented that again, not really in the old style, but in a new style. We are using desaturation shaders for this again, to give us that extra realistic look. We are now adding the horse to the scene. There it is, obviously, right now, without any textures.

But what you see is again, that we are using dynamic shadows using the shadow buffers in OpenGL ES 2.0. And also, if you look carefully, what you can see is that the green from the grass itself is bleeding into the horse itself. And the same is true for the sky as well.

That again, gives us an additional level of realism which is only really possible now with OpenGL ES 2.0. And Nick is now going to add the texture on the horse here, as well as the shaders, again, which gives the coat a realistic look. Next up, I think Nick is going to send the horse off.

Not 100 % realistic yet. But I think, as a next step, he is adding the animation engine to it. And now, the horse is running. And I think Nick's just actually triggered that jump itself. You can see we get very smooth transitions from one gate to the other one, and we also have sound to the background running as well, which is triggered off the animation events.

Now, Nick's just called the horse back to the gates, presumably to give it a few more strokes for having screwed up the demo. So this is really where we are right now. We are amazed at the power that is now available on the iPhone and in the handheld device. And we are really excited about the possibilities that we now have, and that NaturalMotion together with MunkyFun, for creating just amazing games.

So thank you very much.

Thank you, Torsten. It's absolutely stunning what they can achieve with a handheld device, OpenGL 2.0-- OpenGL ES 2.0, and iOS 4. Think about where were three years ago. Think where we were five years ago. This is amazing. This is a device that you hold in your hand, this fits in your pocket, absolutely amazing power. So what do we talk about? OpenGL ES 2.0 and shaders unlock the fantastic power of the handheld GPU. Full scene anti-aliasing, very low cost.

Enhance shadows, shadow mapping, shadow volumes, new texture management techniques, new texture formats about integration, unlocking new rendering techniques, and finally, all of these is optimized for performance, delivering what we think is the absolute leading handheld graphic system. I thank you very much, and I will turn the talk over to Meriko to talk about our media systems.

This is awesome.

Thank you, guys, for being here. I know it is a little bit late on a Monday night, and you all got here early to see Steve. We are thrilled to see you all here. And I am incredibly excited to be talking to you about the media systems on iOS 4 and iPhone 4 today.

I would like to start by talking about AV Foundation. AV Foundation is a modern Objective-C 2.0 API that is designed for maximum flexibility, professional grade access to time-based media-- and professional access to time-based media with dropping ease of use. To set the stage, I would like to talk a little bit about what we had in iOS 3. In iOS 3, we gave you UIKit API to access the camera, record data, trim it, and save it off.

We gave you a set of AV Foundation API for working with audio. This allows you to record audio, process audio, and save it off to a file. You guys responded with fantastic media-based applications, things like Pro HDR for taking beautiful HDR photography, Reel Director for recording and editing media, and Skype for awesome quality audio conferencing. We said, "Yes, please.

Could we have some more?" So we spent the last year dramatically extending AV Foundation. So first off, we are giving you great access to the camera. We are giving you control over the camera and access to its output. We have fantastic professional grade frame-accurate editing services, and high-quality performance playback. And finally, a really easy to use set of export API. We use this technology to power our applications inside of iOS 4, from our iPod media playback to our camera applications to record and trim, and save. These technologies are at the heart of Face Time.

Did you guys all get to see a Face Time this morning? We are-- yeah.

We are really, really, really proud of that feature. And Randy showed you iMovie. iMovie for iPhone 4. iMovie is built entirely on exactly the same public API and AV Foundation that we are presenting to you in iPhone 4.

It is pretty cool. So because we're the graphics and imaging and media teams, we built AV Foundation hand in hand with our foundation technologies. It works beautifully with OpenGL ES, it sounds great with Core Audio and OpenAL, it's best friends with core animation, and it is a first class citizen in our multitasking operating system. So I am not kidding when I say it is extensive. We have more than forty new classes in AV Foundation for you to work with. I would like to talk to you a little bit about the media you can access when you are using AV Foundation.

In all of these classes, you can access camera directly from your-- data directly from the camera, from your user's camera roll, or from their iPod library. This means you have direct access to their media libraries, their asset libraries, and anything that you save off in your application space.

I would like to take a deeper dive into some of these areas. We are really proud of the new camera on iPhone 4, and we are giving you great access and control to it. We are giving you frame level access, we are giving you control over the hardware features, and we are also giving you drop-in recording API.

To talk a little bit about recording, what you are going to do is open an AV capture session, and all you need to do is pick which preset you would like to record for your application's use. We have high, medium, and low preset that automatically adjusts to your users' iPhones, so you do not have to query and figure out what your aspect ratio should be or what the right bit rate should be. So on an iPhone 4, we will record 16 by 9 high-definition video. And on the iPhone 3GS will record standard definition 640 by 480 video. We also allow you to record stills using the AV capture session.

You can record in JPEG format, and we also allow you to record in-- uncompressed. We are going to record that in the native format for your user's iPhone for maximal hardware acceleration. So in iPhone 4, that is five megapixels of uncompressed data for you to work with in your photography applications. We give you excellent device control. On all iPhones, you can set the point of interest for exposure and for white balance. I'm really looking forward to seeing what you guys do with HDR and various kinds of image processing applications.

We also, on an iPhone 3GS and an iPhone 4, give you access to set the focal point or the interest point for the focus. And on iPhone 4, you can choose between your cameras and whether or not you would like to enable the flash. All of these functionalities are available in automatic mode, just like our camera uses, or in manual mode if you would like to set that within your own application. You guys have also shown tremendous access in working with the data live off of the camera as a live source of information for your application.

And in iOS 4, you have got it. What we are doing is giving you direct and unfettered access to the frame streaming directly off of the camera to process and work with as you wish. So this is great for things like object tracking, it is great for bar code scanners, and it is really great for augmented reality.

So we thought we put together a little demo for you. So what we have got is an iPhone 4. We are using the capture classes to put up a live preview of the data streaming off of the camera. In addition, we are doing image processing on all of those frames to look for this tag. And this tag is really just a unique black and white bar code that we have coded into our application.

You will see that when David moves it into frame, and you can see it, we are pegging this OpenGL ES 2.0 tower of blocks onto that tag. We are using that tag and we are doing image processing on that tag to see it shape. So we can see it is tilted and we know where the floor is. So if David tilts the tag, you can see that the blocks track, moves around the base, so it sticks there. Mostly, totally live, high frame rate, and great performance.

Because our games team put this together, they could not help but put a physics model in. So you can actually knock the blocks over.

And they're still sticking to the floor, tilt it, they'll continues sticking to the floor. It is pretty cool. I hope this provided some inspiration for your applications using the camera. So in iOS 3, we gave you really basic access to editing on the iPhone.

We basically gave you trim and save. We have dramatically expanded this inside of AV Foundation in iOS 4, and we are really proud of our models. Again, it is professional grade, frame accurate, access to editing. And I would like to take you through a little bit about how that works. So the first thing you are going to do is you are going to open an AV composition.

You are going to pick your clips, and then you are going to set the areas that you like. Once you do that, you will sequence them into a composition. Once they are in a composition, you can start working with them live within the composition, rendering them out. We have got easy drop in opacity and affine transforms for great transitions, super easy to use. We also love Core Animation. So if you want to do something a little more custom, titling your own transition, you can do that too, lay those in.

And again, we really like audio here in this group. So you can do all kinds of things with audio. It is professional grade, you can lay in a new track, you can mix down the existing tracks, you can pass them through, you can overlay an audio track if you like. It is pretty cool.

So once you have that composition, you probably want to be able to play it back in preview. We have high performance, beautiful playback engine on iPhone. Since iPhone 1.0, we have been compositing beautiful, animated, blended alpha user interface on top of this. It sticks to your fingers, operates completely in real time, does not drop frames. This is really important to us, and we are very glad in iOS to extend that capability to you.

Now, how do you do that? You do that by picking up an AV player. AV player is the way you inspect your media, understand its structure, and work with your media. You can observe and control your playback state, you can enable and disable tracks, you can set in and out points, you can even control your playback rate. So now, you have got a whole lot of information about your media and the ability to control it.

The next thing you want to do is probably lay in a custom control.

Now, the way we do this is that all of our media plays directly back into a Core Animation layer. It is a first class citizen in your Core Animation rendered tree. So you can draw your own custom controls, track it on in, and then you can use something that we call AV Synchronized layer. We think AV Synchronized layer is super cool. What this does-- if you will bear with me a minute-- is you can take the time base of your movie and use it to drive the animation of your core animation layer tree or subtree.

And what this means is that no matter what is going on, whether your user is playing back and forward, backward, scrubbing, you can peg exactly the animation point you want in your UI to that spot in the movie. We put together a little bit of demo to show you this. We call it CoverPlay. We took the movie that Randy made this morning in the keynote. That is 720P full frame rate video.

We can start it playing. So what we have done is we have used the AV Asset Image Generator to pull out all of the key frames from that movie and lay each one of them into their own layer. We are using AV player to play back the main movie, full frame, and then we are using Core Animation to animate those layers of thumbnails.

And those layers of thumbnails are actually our scrubber bar for this movie. These thumbnails stick to the same spot in the movie at all times, and the way that works is that the user input gesture is controlling the time base of the movie. The time base of the movie is then controlling that animation using an AV Synchronized layer. I think this is pretty cool.

Our users love to share their media, and we want to make it really easy for you to help them. So what we have got is a set of AV export sessions. Again, just like in recording, we have presets. You can choose high, medium, and low, and we will keep track of what kind of phone your user has. We also will allow you to pass through tracks. We can set trim on pass through tracks, or enable or disable any particular track you like.

So it is pretty flexible. We also have fantastic support from metadata in our export classes. We support ID3, iTunes, and Quicktime metadata. We will propagate your data from source to destination, we will allow you to add or edit metadata as you like, you can even do something like take your core location, get your GPS data, and lay that into the Quicktime metadata location inside of a movie track. I challenged the AV Foundation team to make a demo that uses many of these classes as we could. So what they built me was a knock, knock joke generator.

I am not sure if they were trying to tell me something. But what you have got here is a series of movies. Each of them has a thumbnail, and a set of themes. And what we are going to do is [inaudible]-- I have not seen Roger in a couple of days. Let us get him to tell us a joke. This is going to use AV composition to edit these clips together and play them back.

Knock, knock.

Who's there?

Sam.

Sam who?

Sam Francisco, here I come.

You guys like the joke?

Okay. Okay. If we can go back to the beginning, we will dive into a little more detail. So what is going on here is when we made these movies, every engineer read the entire knock, knock joke start to finish.

And then we use the AV Foundation API to mark the in and out points. When we use the composition, we are using the built-in transitions I talked about to make those cross fades and pushes. So if you want to do another one, you guys can watch for that.

Knock, knock.

> Who's there?

Interrupting cow.

Interrupting cow moo.

That sounds like a thumbs down.

Finally, they really wanted to show off the power of Core Animation and how well it integrates with all of the AV Foundation classes.

So in this middle reel, what you have got is a series of clip art and a series of descriptions of the Core Animation transitions. When you watch this next movie playback, you will see the badges dancing. All of those are rendering live at full frame rate on top of 720P video that we took on iPhone 4.

One more joke.

Knock, knock.

Who's there?

Candice.

Candice who?

Candice be the last knock, knock joke?

Yeah, okay.

So those arms are animated in and out with Core Animation. We also created some custom playback controls. We have got a play button, a rewind button, a scrubber, and even a share button. Maybe we will save off the metadata about whether you like the joke before we send it out.

So I wanna talk for just a moment about multitasking. It is one of my favorite features in iOS 4. I think it is hugely powerful. We built the AV Foundation classes hand in hand with the multitasking system, so that your users will get the behaviors they expect with the media system. Just like audio sessions, you can set your multitasking category directly inside the AV foundation API, and we will take of starting and stopping system services for you as your users expect.

So for instance, we do not allow you to record in the backgrounds, so we will shut down the camera preview and bring it back up. You do not have to keep track of that. We will maintain a list of what is changed in the state of your application while your users multitask out, provide that information back to you when your user comes back into your app, so you can update your user interface to match.

We have also enabled export in the background, which is pretty cool. So AV Foundation, we have more than forty new classes. We have great access to the camera and control over it, we have professional grade editing services that are frame accurate, we have performed a playback, and easy to use export. We are really excited about AV Foundation. So I would like to change tracks for a moment and talk about HTTP Live Streaming, which is something pretty near and dear to my heart.

Last year, we stood up on stage with our friends from CNN and MLB as we introduced our open standard for high-quality, internet-based streaming media. They showed you an introduction to what they were planning to ship with iOS 3. And in the last year, your response has been overwhelming. These are just a few of the folks who have fantastic applications up in the store.

Our folks-- our friends in the broadcast industry who have signed up as well, inlet-- has got built-in support for streaming media and their encoder boxes and optimized streaming a ton of data over the network everyday. So to review how HTTP Live Streaming works, I would like you take-- take you through a few of the features. At its heart, it is adaptive. We are serving you the bits you need for the feed that matches your speed.

We are continuously analyzing your user's bandwidth and determining what the highest speed they can get that is going to match their bandwidths. So you get full frame video and seamless audio. If your network pickups or they change state, they might drop down the 3G, we will move them to a lower stream without ever interrupting their audio, or rebuffering video in a way that they can see. If your network comes back, we will bring it back up. This is really, really beautiful.

I am hoping some of you guys have played with NetFlix on an iPad and seen a little bit about how this works. Streaming is great for live content, things like live sporting events and concerts. It is also great for on-demand content, whether you want to rebroadcast lectures, watch TV shows, watch a movie.

And we are built on the industry's best standard codecs for beautiful video and crystal clear audio. It's the H.264 for video and AAC for audio. We have stream level encryption to keep your content partners feeling secure and happy. And on the networking side, we are built on top of the HTTP.

And that sounds pretty simple, but it's really pretty profound, because what it gives us is the maximum compatibility with the existing web infrastructure. This makes this really firewall-friendly. If your users can download a webpage, look at a webpage, they can probably get you stream without having to reconfigure their router or their Mac.

I think this is pretty cool. We have some new features in iOS 4 that we think are fantastic. The first one is Stream Insertion. And what this is a programmatic way to inject or add content to the beginning or in the middle of one of your streams. This allows you to take a series of bumpers or interstitial content and inject it into a stream live or on-demand so that you do not have to burn a copy of that content into every single stream that you stream to each user.

We handle various frame sizes, different aspect ratios with no problem, just jump in your trailers and we'll stream to your users and play them back, and we have added the ability to include a metadata track. And you can put whatever data is useful for your application into that track, and it will stick to the time base of your movie.

You can put GPS data in, you might want to tie some slides to a lecture, or inject live player data or stats if something interesting happens in the baseball game that you are streaming live. And we have also got some new tools. We have given you final segmenter so that you can work with your existing on-demand content without having to re-encode it, we have given you a file validator so that you can make sure that your streams are compliant and adhering to our best practices, we have also given you a file index generator to help user posting workflow. So we also have a demo. I would like to invite up Matt Johnson from Engaged Sports. He has been working on an application that he plans to launch next month that uses a bunch of iOS 4 features.

Thank you. At Engaged Sports, we are building new applications so fans can experience live sporting events in entirely new ways on their iPhone. We have a really exciting new app that we want to demonstrate for you today for one of the world's largest sporting events, the Tour de France.

But first, a quick bit of background. This project started [inaudible] the passion for cycling.

Also, my day job is running one of the teams that's in the Tour de France, and they'd been killing us for years so we'd be at the race, and we really had no ability to follow the action live on our iPhones. We've been in development for the last couple of months on a 3.0 version of the application when the 4.0 beta was released. We had a number of features that we dreamed up, but we now realized were possible with 4.0. We're really excited to demonstrate some of those features for you here today.

Joining me on stage is Jay Graves from our development partner, Double Encore. And Tour de France is very different than the traditional stadium sporting experience, because it turns the entire country of France into its stadium. It happens over three weeks every month in July, and it travels almost 4000 kilometers. One of the first features we developed with 4.0 was using MapKit. One of the things that you want to do when you're looking at the Tour de France is you want to be able to follow the action along the race course.

We now have live GPS data streaming to us from the race as it's happening that we can now show the user on their iPhone. At the top of the map here, you see the start town for this stage. This is the last year's stage 20 of the Tour de France. The yellow line indicates the progress the stage has made.

With 4.0 and map overlays, we're able to integrate this data directly into the Map View. So both the route that the race is following, the progress the race has made, as well as the position of the relative leaders on the race all integrated into a map overlay later. And with Core Animation framework, we're able to pan and zoom, and our map overlays will animate directly with the map. Now maps are great. But when you have a sport as visually exciting as cycling, you really want to be able to see it live.

With HTTP Live Streaming, we can do exactly that. And what's really exciting about 4.0 is we have an entirely new level of control over the video layer. So one of the first things we did, and something we'd always want to do for cycling, was overlay the elevation profile of the race course. So here, you can see on the left is the start of the race, and the right is the finish.

And in between, you can see the elevation games or the climbs or mountains that the racers will go over during the course. Now, we can make this come to life. With synchronized metadata, we're embedding the live GPS data, the distance the race has traveled, and the individual position of the leaders on the race directly into the video while the video is still playing live in the background. Now, the Tour de France is a long race.

It's three weeks long, but each day is also six hours a day. So there are times I need to leave the app and go do something else. Now, you may have all seen what multitasking and the ability to have background audio, but you've probably never seen it done with a video-centric application. So as you can hear, the audio track from the race is still playing. If something exciting happens, I can jump right back in, and the audio-- video is perfectly in sync with the audio.

So the last feature--

So if you thought that was something, this is even better. So the last feature is something we've always wanted to do for cycling, and it really completely changes the user experience of how you go and watch a bike race. We've converted that elevation profile into a video scrubber.

So in case you want to-- you missed some action, you know, you wanted to go back to the start of a climb or you wanted to make your own highlights and replay the race later, you can now scrub along the profile, release, and both the video and the audio will go perfectly in sync with that geographical location along the race course.

Thank you very much.

So I guess I shouldn't go too far because I'm wrapping up. HTTP Live Streaming, it's our open protocol for high quality adaptive media over the internet. In iOS 4, we brought you new tools, stream insertion, and synchronized metadata. And if this is any indication of what you guys are going to do, I'm really excited to see the next year. With that, I'd like to bring out Mike to talk about Game Center for a minute

There are over 50,000 games and entertainment titles on the App Store today.

There's no doubt that our platform offers a revolutionary gaming experience. Games are huge in our platform, and we want to make them bigger and better. Therefore, we're introducing Game Center. At its core, Game Center is a social gaming network that enables your games to take advantage about social gaming, structure, and allows the players to enjoy. Game Center is actually made up of three pieces. One piece is a Game Center application.

Game Center application is used for people to discover new Game Center games, to make new Game Center friends, and to compare their progress versus their friends. Game Center also includes a powerful and flexible framework called Game Kit. Now, Game Kit is what your apps use to take advantage of the features we're again talk about today. Game Kit is also tightly coupled with our Game Center services.

Our Game Center services are a flexible, scalable set of services that really fast. And what they do is they bring the people to your games so that your games can enjoy all the people in our social gaming network and have great time. I wan to talk about some of the features in Game Center. So, one of them is Leaderboards. Now, Leaderboards are great way for players to compare how they're doing versus everybody else in their-- in your game all over the world.

Our servers take care of storing, sorting, and filtering all the Leaderboard information you send up to us, and giving it back to you at the appropriate time and in the appropriate format. We've made it really easy for you with a few simple lines of code, to have one or more Leaderboards to your game. Whether your game is a single-player game or multiplayer game, you can take advantage of the Leaderboards. Every Leaderboard comes with a couple of built in filters. There are filters for friends and everyone, and there's still just for time for today, this week, this-- and all time.

We also provide a basic UI, standard UI, you can use to make it really easy to add Leaderboards to your game, or you can choose to customize the look of your Leaderboards to make them fully immersive within your game. I think it's a great feature that we've made super easy to use.

I want to talk about the next feature now, which is Achievements. Achievements are a way for rewarding a player for accomplishing a goal in your game. It adds depth to any game, whether your game is a racing game, a strategy game, a first person shooter, it doesn't really matter, Achievements add a whole another dimension to the game. The completionist will want to collect every single one of your achievements, and everyone will enjoy comparing their progress in your game versus their friends.

Just like with the Leaderboards, we've provide a standard UI you can use so that Achievements look the same within your game as they do in our Game Center app. Or you can customize the user interface so that it fit seamlessly within your game. That's Achievements. And I want to talk about another feature I'm really excited about, Multiplayer. We've taken something that's normally very, very complicated and difficult to do and made it simple. So the first part of Multiplayer I want to talk about are game invitations. So with iOS 3, we introduced push authentications, real powerful system to get a message to any device.

We leveraged that with our game invitations. So now, when a player invites their friend to play a game, that friend will get a push notification. Whatever they're doing, wherever they are, they get a push authentication to play the game with optional personalized message, and here, they can accept the game.

They accept it, launches the game, they play Multiplayer game, but we also recognize that not everybody has the same games. So in this case, if I invite one of my friends and my friend doesn't have the game I'm inviting him to, they'll still get a push notification--

-- but in this case, we replace the Accept button with an App Store button.

So it allows him to go to the App Store, purchase the game, and then play with the person inviting him. This is a great way for your game to leverage our social gaming network and increase the distribution and visibility of your game. We have a lot of features like this built into Game Center, and this is just one. But this gets us friends inviting friends. We want to make so that any Game Center players can play against any other Game Center player. So we introduced this great feature called Auto-matching. Let me talk about what Auto-matching is.

What Auto-matching does is players who want to play your game in an auto-match, they let us know, and it goes up to our server, our server takes all the people all over the world asking to be matched to your game, figures out the best possible groupings of matches out of all those people, and returns that information back to your game.

It's really fast, it's really simple, our servers take care of all the complex work here.

So Auto-matching is a great way for anybody to be matched with anybody, and we also add a way-- we realize that even though we think our logic is really great for putting people together, every game has its own set of requirements. So we've also offered some flexible API's for you to tell our service exactly how you want to-- the players to be matched for your game. That way, we return the exact proper matching your game requires.

So, now we found the people for your game. Another very difficult thing normally in multiplayer games is connecting the devices so they can talk to each other. We also offer the peer to peer connectivity. Now, when we find the players, we will connect them over Wi-Fi or 3G on a peer to peer network, we have a real simple API for doing this to make-- take all the complexity out of this process.

So now, we got a game. We've taken the people, we found them, we've matched them together, and they're talking. But if you're like me, when I'm playing a Multiplayer game, I like to talk to the people I'm playing with. I like to strategize, I like to trash talk, whatever. And so, we've introduced in-game voice chat.

This is tightly coupled to our-- all of our matching and our peer to peer. So it works almost seamlessly. Few simple lines of code, your game can have everybody in the game, talking to each other. Or if you want, you can have team A talking amongst themselves and team B talking amongst themselves. There's a lot of flexibility in here for your games to take advantage of. So I just want to go over, there's-- Game Center's made up of three main pieces.

Our Game Center application, the Game Kit framework, the Game Center services, and all those provide these features and more, Leaderboards, Achievements, Invites, Auto-matching, peer to peer connectivity, and in-game voice chat. Right now, Game Center is available as a developer preview and will be available to the general public later this year. Thank you.

Back to you, John.

That's just fantastic stuff, isn't it? We're really happy about what we've been able to do in iOS 4. So we've gone over a little bit of OpenGL ES, AV foundation, HTTP Live Streaming, and Game Center, and the new features and innovations we're bringing to iOS 4. And to bring that together, we want to do something fun. We, at Apple, built a technology demo.

It's a game that brings a bunch of these technologies together into a really interesting demo. It's a demo that we're gonna be using through some of the sessions this week, and teaching you and showing you how we use these technologies, and how we incorporate them into an application. So with that, I'm going to bring Graeme Devine up, and we're going to look at a demo-- a technology demo called Quest.

I'm really, really proud to be here today to give you, well, the world's first preview of Quest.

We spent the last couple of months at Apple writing a highly performant game prototype that runs on all of Apple's embedded platforms at 40 to 60 frames per second using Apple frameworks and technologies. So let's jump in and take a look. One of the things we did is integrate Game Center. But one of the things we also did was on this iPhone 4, it is made of resolution.

So let's see if we can zoom in and just take a look at this gorgeous display. That is full frame OpenGL ES 2.0 on top of a Cocoa Touch user interface. It logged in to Game Kit and it uses AV Foundation to provide a three dimensional sound environment, because a lot of people play games with the headphones on, and we wanna be able to provide them environment.

So let's take a look at my Achievements. You see that is a Core Animation flip. By using Cocoa Touch to drive all of my UI, I can easily in-- a few lines of code, just do large flips like that and flip around the display. You can see, I've not quite run-- I've not quite won the demon slayer achievement, so I think you can see where this demo is going.

So let's sit down and lets dive in. Now, the Quest environment is lit per pixel. As the character runs around and runs into areas and out of areas, the world around him affects him. The world itself is around 50,000 triangles, putting a few million triangles per second on each of our platforms. As he runs through the red area, you can see that he is getting lit by the environment around him, and the environment around the character is affecting him.

One of the other things that we did--

Thank you. But the other thing that we did was we are using the GPU on our devices to animate using skeletal animation and skinning. He is completely running on the GPU. A lot of talks this week about how we're doing that. But one of the other things we wanted to do with Cocoa Touch was really make our environment interactive touching the world around you.

So if touch my character, I can make up a contextual menu and I can do all sorts of things to my contextual menu. But if I slide to the right, I can call in my pet [noise]. And then if I touch again on top of my contextual menu, the environment-- that contextual menu has completely changed. It's changed because of my character. My character now has his little robot friend. So the menu is completely changed.

So that's how you dismiss him. And you can see, I'm still able to run around, I'm going to [noise]-- I think the demon is pretty hungry. So maybe we'll have to have a bash on him. So by interacting with the demon and pressing on top of the demon, I can actually start the fight [noise], making the world interactive, I want to interact with that demon and fight him right now. The text lying off there is all Cocoa Touch again. Everything in the user interface is running on top of OpenGL ES 2.0 using Cocoa Touch.

We didn't write our own UI at all.

And I got that achievement. Let's do one more thing. Let's touch in the character's face. I used to put him on a drive home. This Cocoa Touch interface and how integrated it is to the game environment.

By taking the game and skinning in such a way that it looks like it's part of the game environment, I can't tell the difference between the game and the interface. But to our game players, it is exactly the familiar interface they use on the hundreds of thousands of other iPhone app that are out there. There is no difference between this application and the photos application. We just skinned the UI. So let's press done and quit on out.

Thank you.

Thank you, Graeme. So we talked about just a few technologies available in the Graphics and Media frameworks. We have a wealth of technology for you to use. I encourage you to go learn about all of them. And we have 26 sessions this week just on the graphics and media technologies. And we have 24 labs.

The labs are really useful for you to be able to go and ask the hard questions to the engineers who actually wrote these things, these frameworks. They will be there to answer all your questions, so we encourage you to go to these. With that, thank you and enjoy your week.