Hello, I'm Allan Schaffehr, the game technologies evangelist at Apple. There's a new framework in iOS 5 called GLKit, which can help you to move your apps onto the programmable graphics pipeline enabled by OpenGL ES 2.0. So in this video, I'll introduce GLKit and its four main components. The first are some new classes to integrate OpenGL ES into the view hierarchy and control your rendering loop.

The second is a texture loader supporting a variety of formats that takes all the work out of loading textures. The third is a math library that's blazing fast. And fourth is an effects library to help you move existing code from OpenGL ES 1.1 to 2.0. So there's a lot to cover. Let's get started.

So GLKit will help you create OpenGL ES 2.0-based apps more easily, and this is going to give your apps more artistic headroom. I get asked all the time about the benefits of moving to OpenGL ES 2.0, and here's what it's really about. When you combine this API with the advances in the hardware on iOS, you open the door to more advanced effects, more realism, and amazing-looking visuals like in the apps you've just seen. So that's what this talk is all about, introducing you to GLKit and helping you understand how you can move your apps onto OpenGL ES 2.0.

GLKit consists of four major parts. The first part provides new classes for managing your rendering and draw loop, the GLK View and GLK View Controller. These make OpenGL ES a first-class citizen in the display system and view hierarchy, with all the semantics and behaviors you're familiar with from UIKit. The view handles its drawing and handles all the setup of frame buffers and render buffers and so on for you.

The view controller integrates into the system to provide you with a rendering loop, handle display orientation changes, backgrounding, and so on. The second part is a texture loader. Almost every OpenGL ES app loads textures, and this was another aspect of OpenGL ES that had been left to developers to implement on their own.

So previously, this meant loading the texture as a UI image, converting it to a CG image, and then giving OpenGL ES a copy of that data. All that was prone to memory overconsumption, and it increased your startup time because most developers were doing all that work on their main thread.

Well, now we're providing a texture loader that simplifies all of this and lets you load textures in a variety of formats with just a couple of lines of code. And it can leverage blocks and Grand Central Dispatch to move the loading off your main thread. The third part is a huge math library called GLK Math. So this is a high-performance math library for 3D graphics. It provides vectors, matrices, and quaternions, and all the math operations you typically would need for those objects. And it's also providing a matrix stack, which is really helpful for those of you transitioning from ES 1.1 to 2.0.

This was another thing that had been left to developers to implement themselves, but now you can just use this library. It'll make that a much easier transition. And finally, the fourth part is an effects library. The main feature this provides is a very simple interface. It provides a group of effects that implement the features of the fixed function pipeline, but using shaders.

It gives you an almost one-for-one mapping for the code that you might currently have with a replacement that's using OpenGL ES 2.0. And then this brings your app up to the level where it becomes interoperable with other shaders that you might want to develop. So now let's dive into each area in more detail. I'll start with the GLK View and View Controller.

The GLK View provides your app with a view directly compatible with OpenGL ES. It's a UIView subclass, so you get all the familiar behavior of normal views, but what it adds is all the handling of your drawable surfaces. Your color render buffer, your depth buffer, stencil, multi-sampling, and the best practices for discarding buffers at the end of the frame. And this replaces dozens of lines of code from the old OpenGL ES application template, so you don't have to do all this manually. Now you just set the properties you want, and the class takes care of the rest.

It also provides you with two options for drawing its content. You can subclass GLK View and override DrawRect, or you can set up some other object as a delegate and have that object implement GLK View Draw in Rect. Now the reason for having both options is that many game engines, for example, prefer to use one method or the other, and this provides everyone with that. that flexibility.

So let's look at the API. Initialization is just like a regular view using init with frame, and what's added is the Eagle context. So we still allow you to manage creation of the context in case you're doing any clever things with share groups and so on. Then there's these properties here for specifying either a 16 or 32-bit color buffer, your depth buffer, stencil, and enabling multi-sampling. Then there's properties for height and width that you can use for things like your aspect ratio calculations.

Then here are various methods. Most of these are actually taking care of you if you also use the GLK view controller. But if you're not, and you want to get at these directly, you can tell the view to display itself. You can manage the drawables, for example, for render to texture, or as you transition to and from the background. And there's even a method to take a snapshot into a UI image. image.

Now I'll show you the code for this in just a minute, but first let me introduce the other class, the GLK ViewController. So, this is a ViewController. It manages the drawing of its view, and gives you all the typical ViewController behavior. But what it adds is an OpenGL rendering loop, in sync with the display hardware.

So, you tell it your desired frames per second, and it will call the display method on the GLK View. It also provides an update method. This is very typical in many game engines, to separate the draw method from the update for your simulation, or physics, or sound, etc.

And the GLK ViewController also provides several statistics. But most importantly, this gets OpenGL ES-based applications onto the path of using a ViewController to manage display orientation changes. And this is very important. Because most OpenGL ES-based games aren't completely stand-alone anymore. They need to work seamlessly with other system behaviors, like Notification Center and Game Center, and the screens that those other APIs present to you. They need to manage orientation for a second display if they're using AirPlay mirroring, and so on.

Now, we used to say that you should manage orientation yourself in OpenGL ES, and avoid using a ViewController. And that's not true. That's no longer the case. You should use a ViewController now, and this object will do all of that for you. So here's some of the UI. Typically, you'll subclass GLK ViewController, and then override methods like ViewDidLoad and ViewDidUnload.

And you'll use ShouldAutoRotateToInterfaceOrientation to specify which orientations you support. And then finally, you'll override this update method for any of your per-frame updates. And this update here lets you specify your frame rate and how you handle transitions to and from the background. And then there's various statistics, like a frame counter, the time since the last frame, or since the app resumed.

So let's show you these objects in action. Here I've subclassed GLK ViewController, and this is my implementation for ViewDidLoad. I start out by creating my OpenGL context, recommending OpenGL ES 2.0, of course. And here I'm getting my GLK view, setting its context and delegate, and configuring the drawables that I want. So I've asked for a 32-bit color format, 24-bit depth buffer, and 4x multisampling.

Those are the view properties. But now back on the view controller, I'm requesting 30 frames per second. And as far as setup goes, that's it. This replaces all that code that you used to have to set this up manually from the old OpenGL ES application template. So it saves you a lot of work.

And all that we have to do now to render our scene is to implement our update and draw methods. Update is where we calculate any per frame updates. In this case, I'm getting the time since the last update from the view controller, and using that to calculate some rotation value.

And then here's my draw method, glkViewDrawInRect. Before this is called, your context will be made current, and the frame buffer will be bound. So now you can just start making regular OpenGL calls to draw this particular frame. And after it's done, we'll call presentRenderBuffer and present that to the user.

So, the GLK view and the view controller give you a lot. If you're using these classes, we will take care of the render buffer setup, we'll set up a drawing loop, call your update and draw methods, all synchronized to display updates. We handle orientation changes and transitions to and from the background, and give you various statistics and manage memory for you.

So that's the GLK view and view controller. So next, let's move on to the texture loader. The GLK texture loader makes texture loading really simple. We support all the common image formats from UIKit, like Ping, JPEG, TIFF, and also PVRTC. And it's also really flexible. We can load textures from a file or URL, and we can also load textures from data in memory or from a Core Graphics CG image rev.

There's support for regular 2D bitmaps as well as cubemap textures. And the best feature of all is support for loading textures asynchronously. Then we provide a few options for your convenience to automatically generate bitmaps from a base texture, to flip the origin from bottom left to top left, and to do alpha pre-multiplication.

So conceptually, here's how you'll use the texture loader. First, before you do anything else, you have to make your context current. And I mention this because you're typically doing texture loading just as your app starts, so be sure to create a context first. Then you'll specify some options, like whether you want mip mapping. And then you'll call the method to load the texture.

This will load the image, define the texture, and return to you a GLK texture info object, which contains a lot of information. It has the texture name or its ID, its width and height, whether or not the texture has alpha, the texture's origin, and whether it has mip levels.

So let's take a look. We've already created our context here, and we make it current if it isn't already. And in this case, I want to load the texture from a file, so we define the path to the file we want, foo.ping in our bundle, and define our options.

I've specified here that we want the texture loader, so we want the texture loader to generate mip maps for us, and now we load the texture. We pass the path, the options, and check for errors, and this returns a GLK texture info object. So all this is done during the texture loading phase of our app, and then later, when it's time to draw something with this texture, we just make regular GL calls to bind the texture passing in its name.

Now, that was how to load a texture synchronously, but you can also load them asynchronously using a GCD queue. This lets you start loading textures in the background and then get a callback when they're done. So, conceptually, it's very similar. We'll create a GLK texture loader object and initialize it with the shared context. Then, we set up our options and call its loading method, along with a GCD queue and a completion handler. Then, once the texture is ready, the completion handler will be called, and we're past a GLK texture or info object that's all ready to go.

So here's how that works. We've already created our context and we get its share group. And again, we set up our path to our file, and now we create the texture loader and initialize it with the share group. And then here's where the work happens. We're calling texture with contents of file, passing the path, any options, a GCD queue, and a block as a completion handler. Note that a queue of null means the default queue. And when it's all finished loading, our block is going to execute, passing the texture info, which I'll use later to bind the texture.

So let's summarize the Texture Loader. This gives you a really easy way to load 2D and cube map textures from a variety of sources and in a variety of formats. You can load on your main thread or as a background task, and it can generate mipmaps, handle pre-multiplied alpha, and flip the origin for you.

So, that's the texture loader. Next up is GLK Math. GLK Math is a math library made for 3D graphics applications. It has over 175 math functions with two, three, and four component vectors, three by three and four by four matrices, and support for quaternions. But what sets it apart is the performance. We provide a scalar implementation that runs on the simulator, but on devices, it's written to take advantage of the NEON instructions to use the CPU as a vector processor. And this allows it to be just blazingly fast.

The entry points are written in C, and the NEON instructions themselves are inline in the header file, so the compiler can perform additional optimizations for you. Then, on top of all that, it also provides a matrix stack implementation. So, those of you coming from OpenGL ES 1.1, you'll notice that the compiler is running on a single-core, one-by-one, single-core, single-core, single-core, single-core, single-core, single-core, single-core, single-core, single-core, single-core, single-core, single-core, and single-core. So, if you're coming from OpenGL ES 1.1, have an easy way to migrate all of your matrix stack code, and other matrix operations directly into this library in a way that's ready to use with OpenGL ES 2.0.

Now, here's one example. This is the actual implementation of GLK Vector4Multiply. And starting with this lower section first, you can see how a four-component vector multiply works. We have two vectors. We're multiplying the corresponding elements with each other and storing the results in a third vector, which becomes the return value.

This is the code that executes when you run in the simulator. But up here, you can see the code that runs on the device itself. It's a NEON instruction, VectorMultiplyQuadFloat32, passing in the two vector arrays by value and returning the result by value also. So it's really fast.

And that's just one of more than 175 math functions in this library. It's super powerful. There's routines in here for the different matrix types, for quaternions, for vectors. There's distance formula, cross product, dot product, length, minimum and maximum, normalize, project, multiply, rotate, translate, scale, and the list goes on.

Now, as I said, we also provide a matrix stack implementation, called GLK Matrix Stack. So, this provides the common matrix operations you're familiar with from OpenGL ES 1.1, like Push and Pop Matrix, Load and Multiply Matrix, and the Euler transformations, Rotate, Translate, and Scale. And this works hand-in-hand with the other GLK math types. Its purpose is to make it really easy for you to maintain your own model view and projection matrix stacks, and then use them in shaders along with OpenGL ES 2.0.

So let's summarize. GLK math has over 175 math functions dealing with vectors, matrices, and quaternions. It's blazing fast, inline, neon, and fills a common need in most graphics apps, and particularly in apps that are moving to ES 2.0. So that's GLK math, and last are the GLK effects.

Our motivation with GLK effects is to enable your application to get the great visual effects of OpenGL ES 2.0, but with very little effort. So, this provides you with an effects library that helps you move up from OpenGL ES 1.1 and make your app ready to adopt programmable shaders on its own.

There are three main classes: the Base Effect, the Reflection Map Effect, which I won't be covering in this video, and a third one called the Skybox Effect. So, let's start with the Base Effect. The Base Effect mimics the features of the fixed function pipeline of OpenGL ES 1.1 and implements them using shaders.

This means that by using the Base Effect in your code, you can have an almost one-for-one mapping of the effects that you're going to be using. So, let's get started. There are three main effects you're using with ES 1.1 and with a replacement that makes you compatible with OpenGL ES 2.0. It supports up to three lights, and each can be directional, point, or a spotlight, each with its own transform and color and so on.

Then there can be materials for each object, 2D and Cubemap textures, fog, a transformation stack, and a lot more. So, using the Base Effect breaks down into six steps. There's three things we do during the setup of the effect. The first is the base effect. and then three things we do each frame.

The important thing to remember about the base effect is that it's handling your graphics state and you handle all your geometry. So, during setup, we'll create our base effect instance, set its properties like the color of the lights and the transformations and so on, and then you apply that to your existing vertex attributes and vertex array state.

Then, each frame, we update anything that's changed, like the transformations, then bind the base effect state and draw your objects. So let's take a look. Here's the first two steps. Now, it looks like a lot of code, but it's really just setting a bunch of properties. So we create our base effect instance, and now we set all the properties we want. I'm specifying per-pixel lighting with one light. I set its position, diffuse color, and ambient colors. And note, we're using the GLK vector for routines to set these values.

Since we have a light, we need to find a material. So I set the diffuse, ambient, and specular colors, and the specular shininess. So that sets up the properties, but I imagine if you're using OpenGL ES 1.1 right now, that you can see exactly how your existing code maps to this.

Next is step three, which is code you would have already had to set up your vertex array attributes. So just for illustration, here I'm generating and binding a vertex array object, which is a best practice on iOS. And then I'm setting up a vertex buffer object, binding the buffer and defining its data. And the one thing that's changed is the call to GL vertex attribute pointer.

We need to indicate which array this is, the position, the normals, colors, or texture coordinates, using a built-in type from GLKit. So that's what you see here. And then we repeat the same process to define other arrays for our normals and so on. So now everything is set up.

Now for each frame, we update any base effect properties that have changed, like the current transformation. We tell the base effect to prepare to draw, which binds its state. And then we just draw normally. We bind our vertex array and call GLDrawArrays or GLDrawElements. And with this, we can move an entire application from ES 1.1 to 2.0.

Now, the last object to talk about is the skybox effect. A skybox is a textured cube, and the idea is that you put the eye point inside the cube. So then the faces of each side of the cube become your backdrop in all directions. Skyboxes are really typical in 3D environments, and ours is really simple. You define its center and size in a transformation, and you specify the textures to show on each face. And it draws a cube for you that acts as your backdrop. So that's all.

So here's how it works. We alloc and init our skybox object. Then we load a cubemap texture that we'll apply to the various sides of the skybox. And here I'm using the texture loader to load a cubemap with contents of a pvrtc file. Then I set up some properties. I use the ID of the cube map and set its center and size. And then at the start of each frame, I just call the methods prepareToDraw and draw.

One last thing to mention about the GLK effects is how they modify the ES 2.0 pipeline state in case you're managing state in your own scene graph or your game engine. The base effect is managing state, so it modifies the current program, the current shader, and the currently bound texture if you've enabled texturing.

The skybox has a much broader impact because it uses both state and provides its own geometry. So, it modifies the current program, the currently bound cube map, then also the array binding, buffer binding, and the enable for the vertex attribute array for positions. And I didn't cover the reflection map, but it's very similar to the base effect. It's actually a subclass of base effect. And it also modifies the current program and the current cube map.

So let's summarize GLK effects. These provide you with OpenGL ES 2.0 level effects with really minimal effort. It drops right into your code to replace existing OpenGL ES 1.1 code and make your app ready for you to develop custom shaders of your own. So that's GLK Effects. All right, so I've covered a lot of ground here. The GLK View and View Controller to make OpenGL ES a first-class citizen in your view hierarchy. The GLK Texture Loader to make texture loading really simple.

The GLK Math Library for fast math and a matrix stack library to use with your shaders. And the GLK Effects Library, which can help you to move your app onto OpenGL ES 2.0. So if you have questions about GLKit or any of the content I've presented here, please contact me at [email protected]. We also have some great documentation available in the iOS Dev Center, and you can come talk to us in the developer forums as well. Thanks for watching.