[Applause]

Thank you. So good afternoon, and welcome to the enhancements to SceneKit session. My name is Thomas Goossens. If you don't already know what SceneKit is, here is an overview of the different frameworks on iOS 10. SceneKit and SpriteKit are the high-level APIs. SpriteKit for 2D and SceneKit for 3D. You can use them with other game technologies like Model I/O, GameplayKit, and GameController, and give you access to the low-level APIs, which are OpenGL and Metal.

SceneKit was introduced on OS X on Mountain Lion and last year on iOS in iOS 8. Last year, we added a few new features to SceneKit to make it ready for casual games, like particle systems, physics, force fields, and the integration with SpriteKit, for example, for all the 2D game overlays.

This year, the two main additions to SceneKit are the transition to Metal and the new Scene Editor. In this presentation we will start by showing you the new Scene Editor, then explain why we are transitioning to Metal and what it means for you in terms of adoption. And we will finish by presenting a few new features that we added to the engine this year. All right.

The new Scene Editor. It is available in Xcode 7, so we can run it in El Capitan or Yosemite, and it is available in the force field. To open it, simply open a 3D file with SceneKit, for example, DAE, OBJ, olympic, or you can create using template chooser.

In that case, it will create a new empty SceneKit document with the extension .scn, and an scn file is simply an scn scene instance archived with an archiver. That means that you can produce your own SceneKit document programmatically if you want, and this is very handy for the production flow.

Then the main goal of the Scene Editor is to let you build the levels of your game. You will see that it's really easy to combine multiple assets from different files into a single scene by keeping references to the original assets. The Scene Editor also includes some visual editors for some of the key features of SceneKit, for example, to let you configure particle systems live in a 3D scene, configure your physics, inspect and configure the physics shapes of your objects, and preview a simulation for example, sensing for force fields, and it includes an action editor to let you build action sequences with a graphic user interface.

Scene Editor also includes some more advanced features, for example, if you want to editor your shared modifiers live in your scene, and some baking tools, for example, to bake ambient occlusion, and for this please refer to the Model IO session for more information about this. Okay. But nothing better than a demo to show you this, so let me bring Amaury on stage for presentation of Xcode 7. Thank you. [Applause]

Hello. My name is Amaury, and I am a software engineer with the SceneKit team. We dramatically improved the SceneKit editor in Xcode 7, and with its new editing capabilities, it's now really easy to build scenes from the ground up in a visual and interactive manner.

Later on in the session, I will show you how we used SceneKit in editor to build Red Panda, the puzzle game, but right now we will start with something more simple, a small game of physical scale, which consists of building a tower of blocks that will eventually collapse as we remove blocks from it.

So what you see here is a SceneKit Scene Editor showing an empty scene. On the right side, you have the usually utility area, with its inspectors and libraries. In the object library, you can find many SceneKit objects, such as modes, lights, cameras, parametric geometries, as well as physics fields. Now, for our wooden block, we use a simple box geometric. To customize its appearance, I can drop and imagine it.

Okay. So now just like in Interface Builder, inspectors as well can edit all the properties of your 3D objects. For instance, in the attributes inspector, I can change the size of my geometry to be 2 by 1 by 6. And to center it in the scene, I can use the node inspector and set its position to zero. But actually, there is a more simple way to do that. Using the 3D manipulator, I can freely move the object in the scene.

Okay. So now that our wooden block is ready, we will use it as a template to build other blocks. Using the 3D manipulator, I can easily make copies by holding the alt key. I can repeat the operation, make a selection, duplicate that selection, rotate it, make a new selection, repeat the operation, and one last time to finish the tower. Okay. We are now ready to use the physics engine that's built into SceneKit to make the scene more interesting. I will simply select all the objects, head to the physics inspector, and add a dynamic body to each block.

We are now ready to run the simulation. As you can see, our blocks are now subject to gravity and fall indefinitely. To prevent that, we will add a floor to the scene, so I open the objects library, put a floor, position it, and I need a static body so that it participates in the simulation but is not allowed to move. If we run the simulation again, nothing happens. But as we remove a few blocks from the tower, we get a fair idea of what the game could look like.

[Applause] One additional thing I would like to show you is a scene review. You can review the scene review in the lower left corner of the editor. The scene review is where you can see all the nodes in your scene, and it is where you can edit the node hierarchy.

For instance, we have our floor, the many wooden boxes, but we also have a spotlight. I placed this light in the scene right after I created the project. It is currently hidden and has no effect, so let's show it. To further improve the lighting, and to avoid these completely dark areas, I will simply add an ambient light to the scene.

And that's it. Just through the editor, we built an entire scene with 3D objects, textures, lighting, and dynamic bodies that could make a nice game after we implement gestures. So the SceneKit Scene Editor really allows you to build more of your game up by writing less code. And there's actually much more things that you can do with Editor, and to cover that, let's switch to the Red Panda demo, and let's see the demo again.

So Red Panda is a puzzle game in the 3D world, where you can manipulate the camera with pan gestures and control the character on his path. The goal of this game is to collect these flowers and here is the first one. On his path, Red Panda will also have to collect items.

See how the world is rich and animated with many details? We have lots of particle systems and animated vegetation. Now, Red Panda can go everywhere in the level. For instance, he can climb hills. But on his path Red Panda will also have to avoid obstacles. Notice the sound became louder as we pushed. It's because we used one of SceneKit new API's 3D audio. Many of the features are in play here.

We have the physics engine for navigation and collision detection. Lots of particle systems. And shadow modifiers and actions help to bring the scene to life. So we just collected our seventh flower, and by moving the camera, we can see where objects hide. We have a quick tour under the arch, and the level is completed. [Music] Thank you, Thomas.

[Applause] So we are now going to see how we built this game using the SceneKit Scene Editor. So this is a starting point for our game. A quick and easy way to immediately improve your scene is to add a skybox. A skybox is simply a collection of six images that represent the environment. After we add a skybox to the scene, we have a context in which we can build the level.

So if we now take a look at the resources in our project, we can see that our artist provided us with many building blocks, and by that I mean that we have separate files for the bamboos, for the flowers to collect, for the pearls to collect, as well as the many structural elements.

So now how do you use these external building blocks in your scene? Well, you go to the scene, and if you drag and drop an external building block in the scene, you can see that the reference node has been created as indicated by this little arrow icon. It is very nice and convenient to have these external resources as external files because that way you can design the level and still have a chance to save the assets, and you can also reuse these assets to build other levels for your game.

So as you can see, I've already included other reference nodes for other industrial elements. So this is how you typically build a level. You would add your building blocks one after the other and iterate. You would do the same thing for the flowers to collect. The pearls. And finally, the vegetation.

So here you can see how far we went with just a simple operation. So as you remember, our scene is rich and animated with many details. We have a lot of particle systems, but we also have animated vegetation. Many were animated with skimming which is a fairly Complicated animation technique that has to be done in a special tool. But for we use shadow modifiers. Actually, there is one last element to animate here in the small waterfall.

So to do that, I will open the scene for our waterfall, and the SceneKit's editor now allows you to add modifiers right into this scene for immediate feedback and live compilation issues. You can add and edit each of the modifiers in the assistant editor. So here I select the waterfall and choose to add a geometry modifier.

I add the modifier, and you can immediately see its effect. You can play with your shadow modifier and try to tweak the final rendering. You can try to find what the final modifiers are. Once you are happy with your scene, you close the assistant editor and go back to your main level.

As you can see, our mountain is now animated. This is because we used reference nodes, and when you edit assets, they are automatically reflected in the scene. One last thing we want to do here is to add an enemy in the scene. So this is a scene for our enemy. As you can see, it is static. It has no behavior. To add the behavior, we will use actions. The Scene Editor now allows you to add and edit actions right in the scene. So this is the action editor. I will select my enemy. Choose a row detection.

Add it. Set it to rotate along the Y axis. And make it repeat forever. So this is a simple behavior that we have for our enemy, but the action editor allows you to build some complex scenarios, and for more information about that, I invite you to watch the What's New in SpriteKit session. We can now go back to our level, add a reference to our enemy, and our scene is finished.

So just -- we just saw how the editor allows you to build an entire level from external building blocks and how you can bring life to the scene with shadow modifiers and actions. And with that, I hand over to Thomas, who will talk about the architecture of the game. [Applause]

So yeah, we are very happy to give you this nice demo as a sample code. It's available on the developer website. You will see it's very simple and straightforward. It's about 1,000 items of code for both iOS and OS X, it includes everything like the sound, the controls, everything you just saw.

So Amaury showed you what was done in Xcode 7 to build this demo. I will quickly present the rest, what was done on the inside and the outside. The first thing we did was to define a direction and a simple game play for our game. So we started with our main character. Then we defined the shape of our level, including where the collectible items should be and where the enemy should be. And we defined the final appearance we wanted to achieve using SceneKit.

Then the next step was to scope the 3D models in the 3D tool, to paint the different textures, configure, and animate our character. And the final polish was done by adding some particles, vegetation, and fog, and we finished the game by adding some 2D overlays on top of that. Then the first challenge on the code side was to manage the collisions. For example, we don't want our character to cross the walls.

To do this with collisions, we work with collision meshes. A collision mesh is a simplified version of a rounded mesh that has a similar shape so that the collision feels consistent with what you see on the screen. It's very important to work with collision meshes and not the rendered mesh when you deal with physics and collisions for performance reasons.

And as for the ground, we want our character to be able to progress everywhere in the level but always stick on the ground. So we need to know what is the elevation at any point in the 3D world. For that, there are several solutions. What we did here was to again work with the collision meshes.

And we do array intersection, vertical array and meshes, this I was go us an intersection point, and this is where we can place our character. Then whe we move the character with the touchscreen of the keyboard, we simply move it on the X and Z plane. Then we do a new intersection where the character is. This gives us a new intersection point, and we use that point to adjust the Y position of our character. We redo that at every frame, and this is how simple it is to animate our character in the game.

Then animations. There are several animated elements in the game. The character is animated when it is idle or working. The grass and the bamboos are animated to simulate wind effect. And as a nice side effect, you can see that the shadows casted by the bamboos are also automatically animated.

And all these animations are important because they contribute making your game more live. And technically, the character and the bamboos are animated using skinning. So the artist already built the skeleton and the animations in the 3D tool and exported that to a DAE file, so they are absolutely no good to do here for the developer.

And the grass was animated with a shadow modifier directly in Xcode 7. Related to animations, particle effects. So there are several particle effects in the game, and again, they were all done entirely in Xcode 7, no code here. The pearl and the flower with some sparkles emitted. And the enemy throws some fire. That was done with a -- modulated with a color gradient. The particle was done with four images of dots more or less blurred with random sizes and an initial velocity to simulate the wind effect.

Now, one note about our materials and how we achieve this great-looking materials in the game. So here is our looks like in the game, but if you just take its geometry and render it with flat colors, it would look like this, which is a little bit too simplistic. So the first thing we did was to set a diffused texture, and this gives us a base color of our objects. Then we added a normal map that added very fine details to our surface without adding many polygons. So this is quite cheap.

Then we added a light map using the self-illumination property, which is new in this release. And the light map adds very fine details about global illumination to our objects. You can see in the corner of the stone, you can see some occlusion, for example, that is baked in the light map.

Then the last thing we did was a reflective cube map to reflect an environment and, in particular, the sky. So the reflective cube map looks like this. We attenuate its effect using an attenuateer to attenuate the reflection on the surfaces that are facing the point of view. Now, one note about the light map. It only baked the global illumination and not the main direct light and not the main shadows because we want the main lighting to continue to be fully dynamic.

That way if we want, we can animate the position of the sun if needed, but most of all, a character can progress in the game, and its shadows are live and automatically, you have nothing to do here. So that's about it for this demo, and I hand over to Sebastien to talk about the transition to Metal. Thank you. [Applause]

SEBASTIEN METROT: Thank you, Thomas. So I am Sebastien, and I am working on SceneKit, and I am going to talk to you about the transition to Metal as well as a bunch of new features that we have added this year. So SceneKit is transitioning to Metal. Metal, as you know, is very low-level and low-overhead graphic API that brings better performances and in much more modern API to the platforms, and most importantly for you SceneKit users, it just works.

It also brings some new features to SceneKit, like compute shaders so you can just use a Metal -- or the geometry source or Metal texture as material property. It also allows us to do automatic batching. So if you have the geometry used by multiple objects using the same material, we can dramatically reduce the number of [Inaudible]. I will show you that with a demo.

So what you see is a simple SceneKit application running on the OpenGL ES editor. It is displaying a planet with orbiting asteroids, and on the OpenGL is eating up much of the CPU just displaying the asteroids. As you can see, we cannot add many more asteroids without maxing CPU and reducing the frame rate.

But if we move to the same application running on the Metal renderer, we can add many more asteroids, as you will see in a moment. Yes. Of course, the CPU starts to be eaten too, but we can raise the number of asteroids dramatically. I think it's quite awesome. [Applause] Thank you.

So how do you adopt Metal with your SceneKit applications? Well, first is the default on iOS 9, so you just have to recompile your application with the new SDK, and it will use Metal on all the hardware that supports it. It's also backward compatible, so if you have an existing application that you didn't change and didn't recompile, it will work on OpenGL by default.

And you can opt for using the OpenGL renderer, if you want to, for example, if you have custom shaders that are OpenGL specific. Choosing the renderer is really easy, if you have a built-in application with built-in renderer, you just have to use this small button to change the default.

You can also do it programmatically. So if you created your scene view at runtime, you will give it the right option, use Metal or OpenGL, as you want. And what's new also is that we have added support for Metal shaders. So how do you deal with shaders in SceneKit with the new API? The first way to use custom shaders with SceneKit is to use Metal shader modifiers. Shadow modifyers used to be only in GSL. Now we support both languages.

If you add on GLSL modifier, it will work as it is on OpenGL, and it will be translated to Metal. It works in most situations. If you write your shadow modifier with Metal shadow language, it will not work, it will be ignored, but it will work as Metal.

The second way to use shaders in SceneKit is to use SCNPrograms. In this case, there is no automatic translation, so you will do it by hand. Will you have to translate all your code. It's supported -- it supports both GLSL and Metal shading languages, so you can support both in the same application. You can compile them offline. You just have to provide the function names for the shader for the fragment and vertex shader, or you can compile it at runtime. In this case, you just have to give the source code in addition to the function names.

I will show you now how you do it by creating a very small shader that would work in SceneKit. The first operation that you will have to do is to include our SceneKit header so that you can have access to some semantics and, for example, the default metrics. Here we define our custom vertex structure that is in the correct data in SceneKit thanks to the semantic definition. We can also define the use of a metrics that is, for example, the model view projection.

And we can define our own structure with our own data for the algorithm we are trying to create. Then we just have to create the vertex function by giving the arguments for the structure that we have defined. Really easy. Then in the program, we just define the same structures, we have to use the same types and the same layout for the structure as we used in the Metal definition. We created a program, give the vertex and fragment function names. And then we can give some data by using the structure and wrapping it around time stamps. Very simple.

The last way to use shaders in SceneKit is to use techniques, techniques were introduced last year. It's a great way to do post-processing for all your applications on the role scene, and in this case, you just have to give the correct function names for the Metal shaders in your pass definition, and that's it. It will work just like the OpenGL version. That was it for Metal. Let's move on to some new features that we have added for you this year.

The first one is the integration with Model I/O, so we have added the support for Model I/O, both for importing and exporting of your models and scenes. As you can see, not all the file formats support all the data that we are supporting in SceneKit, so choose the one that is most suited for your application. We have also added scene transition with one line of code, very easy, you can create transitions between scenes, and with great animations. Really easy to use. We have added also debug options, so you can visualize your boxes, your physics shapes, everything at one time for debugging purposes.

We have also added blend modes to material so you can choose how transparency works for each node -- for each material in your application. You can create new effects with that. And we have also added an audio engine that works for everything in SceneKit. We can create 3D sounds, attach them to nodes, and they will move with the nodes in the scene. It also works with ambience and music, so you can digital position audio and loop the sound. The listener is placed on the camera node, but you can move it to any node in the scene. It's really easy.

It's based on the AVAudioEngine, so you just have to annotate from AV audio node, and you can do your own audio processing if you want to if you have an existing audio engine. And last, we have added also a way to created sound action so you can integrate sound into your action sequences. That's really easy to do.

And here is how you use the sound API. First you have to instantiate an audio sound from the file fame. Then you create a player and attach the player to any node in the scene, and it will start to play right away and stop when the sound is finished.

If you want to do ambience or music, you have to just flip a switch, like digital positional audio and start looping. And last, it's really easy to create a play song selection. It's just one line, and you can run the action or add it to an existing action sequence.

This API was used in the sample code for all the sound, and we used the collision mesh as -- that Thomas showed you sooner to detect kind of ground the character is working on. For example [scratching sound] when he walks on the grass. Or when he walks on a rock. It was really easy to do. So that's about it. And I give you back to Thomas for the wrap-up of this session. Thank you. [Applause]

Okay. So that was an overview of what we added to SceneKit this year. So a great new Scene Editor that will save you tons of lines of code and do more for your game insight tools, which is usually a good idea. A smooth transition to Metal which should boost the performance of your game with nothing to do on your side most of the time.

And some new features that were presented, and we covered some of them in this session, but check the documentation for a list. And an amazing demo app that we are really proud to share the code with you. Search for fox on the developer website, and you should find it.

Now for more information, please check the resources and documentation we have online, and for general inquiries, contact our Evangelist, Allan Schaffer. Some related sessions, Model I/O, great new frameworks introduced this year, and check the What's New in SpriteKit session. They have a really great demo of the scene editors that go much deeper into details. So you definitely want to see that one. And that's about it. Thanks, and enjoy the conference. Thank you. [Applause]