My name is David Hayward and welcome to our first of two discussions today about Core Image. And we'll be talking today about what's new in Core Image on both iOS and OS X. So what is Core Image? Core Image is a fast, easy, flexible framework for doing image processing. And it supports all of our supported devices on both iOS and OS X.

It's also used by several of our key applications such as photos and, on both platforms, and it allows you to get very good performance results and very flexible output. For those of you who may be new to Core Image, I just want to take a few slides to talk about some of the key concepts because those will be relevant for the rest of the discussion today.

So first off, filters: Core Image filters allow you to perform per-pixel operations on an image. In a simple example, you have an original image and you want to apply a sepia tone filter to it. And you'll get a resulting image. Obviously, that's fun but where things start to get interesting is where you combine multiple filters in either chains or complex graphs. And here an example, you can see a very interesting result you can get by just chaining three filters together, sepia tone plus a hue rotation to turn it into a blue tone image plus a contrast to make it more dramatic.

One thing to keep in mind is these intermediate images are actually lightweight objects. So there need not necessarily even be memory associated with these of any significant amount. Another thing that's important to keep in mind is each filter may have one or more kernels associated. So these kernels are the actual algorithm that implements each filter's effect.

And one of the great things about Core Image is we can take these kernels and concatenate them into programs and this allows us at runtime to minimize the amount of intermediate results and with some great compiler technology, both at the image processing and at the low-level compiler level, we're able to get the best possible performance out of a complex graph.

So that's the basics in terms of how it works. These are the four key object types that you need to be familiar with if you want to use Core Image. The first, which we'll be talking about a lot today, is CIKernel. And this represents a program that's written in CI's -- or Core Image's -- Kernel language.

Second object type is a CIFilter and this is an object that has mutable input parameters and those parameters can be images or numbers or vectors or other types. And it also allows you to use one or more kernels to create a new image based on the current state of the output, uh, of the input parameters.

Third key type is a CIImage and this is an immutable object that represents the recipe to produce an image. Just the act of creating an image does not necessarily do any real work. The actual work occurs when you render a CIImage into a CIContext and that's the object through which you render results.

So those are the basics. What I want to talk about today is what's new in Core Image this year and we have a lot to talk about. We have a bunch of new things that are on iOS. For example, we have our most requested feature which is Custom CIKernels. We also like to talk about how you can do Photo Editing Extensions using Core Image and also how we can now support large images on iOS.

We also made some improvements to how the GPU render is used. We also have some API modernization. We have some new built-in filters. We have some new CIDetectors and then lastly, we will talk about some new things that we have on the Mac OS X side, improve RAW support and how to use a second GPU.

So first -- and most interesting, I think -- is Custom CIKernels on iOS. As I mentioned, this has been our top requested feature. Core Image already has 115 great built-in filters on iOS. But now you can easily create your own. So this is a terrific feature for developers. When you're writing CIKernels on iOS, you can use the same CIKernel language that you use today on OS X. There are a few extensions which allow making typical kernels even easier and we'll talk about that in much more detail in our next presentation.

Where can your CIKernels live? Well, they can live in your application. The kernel code can either be a text resource or it can just be an NSString, if you'd like. The kernel is wrapped up in a CIFilter subclass that you provide that applies their kernels to produce an output image. Another great place for your Custom CIKernels to go is inside an App Extension. For example, Photo Editing Extensions can use CIKernels and CIFilter subclasses very effectively. And you can use them to modify either photos or videos.

So again, we'll be talking our next presentation in much more detail about how to use CIKernels on iOS but let me just give you a little teaser now of how simple it is. Here we have, in just basically two lines of code, how to use a Custom CIKernel.

We create an NSString which has some CI -- Core Image source code in it. This is a very simple kernel that takes a pixel value and inverts it. You'll notice it's actually subtracting it not from 1 but from the alpha value. That's the correct way to invert if you've got premultiplied data, which is what Core Image receives.

Once you have the program written then all you need to do is create a CIKernel object from that string and then apply it. You can specify two things: one is the resulting extent of the produced image; and also, the arguments that will be passed to that kernel. In this particular example, there is only a single argument, which is the input image, and so as a result, our arguments down below is just an array with a single image in it.

So to give you a little bit of idea of what that looks like in practice, I have a quick demo. This is a fun example that we wrote. And it's kind of an example of something that you wouldn't necessarily have as a built-in filter. Let's see. But it would be fun for a presentation like this.

So we have an application called Core Image Funhouse and this allows you to explore all the built-in filters and also allows you to see some sample code for how to write Custom CIKernels. So the image starts out as gray. The first thing we need to do is provide an image to start with. And we're going to say that we want the video feed to come in.

And then I'm going to add a filter and you can see, we're seeing a list of all the filters that are part of Core Image. And we created one down here called WWDC 2014 and I hope you can see this so that I can kind of wave in front of the camera.

What we're doing here is actually algorithmically taking the luminance from the video feed and then using that to control the size of a geometrically-generated, rounded rectangle. And we can change the size of that, larger or smaller, or we can change the amount of the rounded radius here. It's actually a little easier to see that it's a video feed when it's smaller but it looks more cool when it's bigger.

So... And we're getting about 30 frames per second on that, which is probably the frame rate of the camera right now. So that's our short example and we'll have that code available for download at some point soon. So again, that's Custom CIKernels and please come to our second session to see all you can learn about that.

The next thing I'd like to talk about briefly is the Photo Editing Extensions on iOS. There are whole talks on that this year at WWDC. I'd like to talk a little bit about how that works in relationship to Core Image. So here's just a little short video run-through of how this works in practice.

What we have is an image; the user went into Edit; they brought up a list of extensions. We picked the Core Image one and this particular Core Image based extension has two sliders: one is the amount of sepia tone (which we can slide, and we're getting very good frame rates to the screen as we do this); and then the second slider is a vignette amount. So it starts out with a large radius and then as you bring the radius smaller, you get more of the vignette effect as you bring it down.

And all of this is happening right now on a display-sized image. Later on, when you hit Save, it's applied on a full-sized image, which is a 12 megapixel image in this case. And it goes back into your library with your edits. So that's how it looks in practice. I'm not going to go into too much detail on how to code this but I'll give you some good advice here.

First off, you can start to create a Photo Editing Extension by going into the templates in Xcode. We'll also provide some sample code as well so that'll be a good starting point. But as I said, I wanted to talk a bit about how you can use Core Image effectively within Photo Editing Extensions.

There's basically three steps. The first step is when your extension is initialized, what you want to do is you want to ask the EditingInput object for a display-sized image. Initially, that is a UIImage object and from that you can create a CGImage and then from that CIImage. That sounds like a couple of steps but it's -- actually those are just lightweight wrappers. Once you've created that CIImage, we're going to store that in a property for our delegate.

The other thing -- it's a good time to do at init time -- is to create your view that you're going to be rendering into. We recommend using a GLKView and also create a CIContext that is associated with that view. And it's good to store that away in the property as well.

Step two is what you do every time the user makes an edit in your extension, so, every time the slider moves. And this is very simple. What you do is you re-call the display-sized CIImage that we created in step one. We apply the filters that correspond to those slider adjustments. So in that previous example, was the sepia tone filter and the vignette effect filter. And then once you've chained those together, you get the output image from that. And then you're going to draw that using the CIContext that we also created in step one.

And Step three is what happens when the user clicks the Done button. And this is slightly different because in this case, you want to apply the effect on the full-sized image. So what we have here is we can ask the EditingInput object for its fullSizeImageURL. From that, we create a CIImage and we apply the filters to this as well. Now, for the most part, this is the same as we did in step two. Some parameters, however, such as radiuses may need to be scaled in accordance to the fact that you're now working on a full-sized image.

Once you have chained together your filters, you ask the output image and then you -- the way this API works is you return a CGImage, so, you can do that very easily with Core Image: you ask a CIContext to create a CGImage. And this will work even on the full-size image.

So that brings me to the next subject I want to talk about today, which is working on large images on iOS. So we've made some great improvements here in addition to the supporting kernels, this is, I think, our second key thing that we've added this year on iOS.

So now you can -- we have full support for images that are larger than the GPU texture limits. And this means that input images can now be larger than 4K and output renders can be larger than 4K. We refer to this as large images but in practice, 4K images are not that large these days. Many of our devices' cameras are bigger than that. So this is actually a really critical feature to support this size image as well.

The way we achieve this automatically is that we have automatic tiling support in Core Image. And this, among other things, leverages some great improvements that were made in ImageIO and they're JPEG to improve how the decoder and encoder works. And also, there's some great features in the Core Image language that allows supporting of large images as well. So let me talk about that last item in a little bit of detail. So the CIKernel language allows your kernels to just work automatically regardless of whether tiling happens or at what size it happens. So this is a great feature that makes writing CIKernels very flexible.

The way this is achieved is by two key extensions that we have in our language and these are available both on OS X and on iOS now. The first is a function called "dest coordinate", or destCoord, and that allows Core Image to support tiled output automatically. It basically allows your kernel to see the dest coordinate in the native images space even though we may only be rendering a given tile at a time. Similarly, there's a function called samplerTransform and that allows Core Image to support tiling of large input images automatically. So this is the two key things about the CIKernel language that we'll talk about in much more detail in our second presentation.

So another great thing about our large image support is how we work together with CGImageRefs and how we get some great improvements on iOS 8 by being lazy. So one thing to keep in mind is if you have a small input CGImage that you create a CIImage from, then this image is fully decoded at the time you call CIImage initWith CGImage. And that's actually usually the right thing to do for small images because you may be using that image multiple times and you want to take the performance impact of decoding the JPEG once, early.

However, for large images, that's not a good strategy because you don't want to require all of that memory to be -- for that JPEG to be compressed unless you know you need it. So if you have a large input CGImage, that image, that JPEG image behind that CGImage is decoded only as needed when you call CIContext render.

So that's a very important detail. Similarly, when you're producing a CGImage as an output of CIImage, when you call CIContext createCGImage, if the output CGImage is small, then the image is fully rendered when CGImage is called. However, if you're producing a large CGImage as an output, such as an example of the photo extensions, the image is only rendered as needed when the CGImage is rendered. This is also important because a very common situation is you pass the CGImage to CGImage DestinationFinalize to encode it back as a JPEG.

So what all this means is that if you have a very large JPEG, you can take that large JPEG, decode it, apply a filter to it and re-encode it back into a JPEG with minimal memory and great performance and this is a huge win for Core Image on iOS. So let's take a quick example. You're applying a sepia tone effect to a 4K by 6K JPEG, so 100 megabytes of image. That on iOS 7 took 17 seconds to decode, apply the filter and re-encode it as a JPEG. On iOS 8, that's 1 second.

And just as important on iOS is the memory high-water mark, because that can really force your application into an unhappy place. And our high-water mark on iOS 7 was 200 megabytes, which makes sense; we have a source image that was fully decompressed and we need to produce a whole new image which is the same size. However because we now have tiling, our high-water mark is now 25 megabytes.

And just to summarize, on iOS 7, we worked on the full image at a time and because it was large, we often had to use a CPU renderer. On iOS 8, we have automatic tiling and as a result, we can use the GPU which is a huge win.

So we've also made some other improvements to how GPU rendering works with Core Image on iOS which are important. So your application sometimes needs to render in the background. Often either when the app is just transitioning to background or when it's fully in a background state. On iOS 7, that is supported. However all background renders used the slower Core Image CPU Rendering path.

On iOS 8, we have an improvement in this regard which is that renders that occur within a short time of switching the background will now use the faster GPU renderer. Now, it is serviced with a lower GPU priority and the advantage to that is that any foreground renderers that happen at that time will not be -- have any performance impact because Core Image will be using a lower priority renderer. So this is another great advantage.

There are some restrictions on this GPU usage. It is not allowed if you use CIContext drawImage :inRect:fromRect because in that case, Core Image needs to render into the client's [inaudible] context. However, any of the other render methods calling createCGImage or render:toCVPixelBuffer or render:toBitmap will all work in this way.

Another great improvement we have is: oftentimes you want to do rendering in the foreground -- when your app is in the foreground -- but do it from a secondary thread in a polite manner. So if your application is showing one thing and then doing something on a secondary thread using Core Image, on iOS 7, that required care in order to avoid -- in order for the secondary thread to avoid causing glitches for the foreground thread.

And of course, the only sure-fire way to avoid that was to use Core Image's slower CPU renderer. On iOS 8, we have a new feature which is the secondary thread can now render into a context that has had this new option specified, which is CIContext PriorityRequestLow. And the idea now is that context renders using that context will not interrupt any foreground higher-priority renders. So this is also great for your application.

So this brings me to some final thoughts on Core Image's CPU rendering. Basically, there were three key reasons why an app would need to use the CPU renderer on iOS 7. For example, the CPU renderer was used when GPU texture limits were exceeded. Well, starting on iOS 8, that's no longer a limit in Core Image so that's not a reason anymore.

Similarly, the application might have needed to render briefly when in the background, that's also been improved in iOS 8. And lastly, if your application wanted to render from a secondary thread when in the foreground, you might have used the CPU renderer and now that is no longer a limitation. So we have some great ways to keep us on Core Image's much faster GPU rendering path.

The next subject I want to talk about this afternoon is about some API modernizations that we made both on OS X and on iOS. These are small conveniences but they add up in total. First off, Core Image filter subclasses on OS X can now use properties instead of ivars.

One thing to be aware of is that Core Image filter subclasses do not need to release the object associated with input ivars or properties. So it's a little bit nonstandard as a class in that regard. By supporting properties, that means that code that used to look like this, where you have outputImage = filter valueForKey: kCIOutputImageKey can now be a little cleaner and just look like outImage = filter.outputImage.

We also have a convenience method if you want to create a filter and also set a bunch of parameters all in one fell swoop. This can be now done by saying filter, filterWithName and then you could specify some parameters at the same time. And in those parameters are a dictionary where you can specify all the inputs in one convenient manner.

There's an even slightly simpler case which is very commonly usable where one of your inputs is an input image and you just want to get the output of a filter. So this means you can apply a filter to an image with a set of parameters without even creating a filter object.

Lastly, one of the common questions we get from developers is, "How do I correctly orient my image so the orientation is correctly upright?" And the standard TIFF specification has a set of 8 possible values that tell how the image should be flipped or rotated and we've provided a code snippet in the past for that, but much easier is that we provided an API for that now in iOS 8 and OS X. So the simplest way of calling it is to say imageByApplyingOrientation and that gives you back a new image.

And again, you're specifying an integer orientation value. As an alternative to doing the same thing, we also have an API that allows you to get back an AfffineTransform that is equivalent to that. And the reason why that's useful is, usually orienting your image upright is only the first of several affines that you may apply to your image. You may also be scaling it to fit or panning it. And so by getting this affine matrix and concatenating with any other affine matrix, you can get a little better performance out of Core Image.

So we've also made some modernizations on OS X with regard to color spaces. The default RGB color space is now sRGB which is great because it matches with the default RGB color space that we have on iOS. And it also matches what most modern applications expect for untagged images.

Similarly, our default working space has also changed on OS X. It is now a linearized version of the Rec.709 chromaticities and again, this matches the default we have for our working space on iOS and has a great performance advantage, which means that in most typical scenarios -- where you have sRGB content going into a filter in its working space and then going back to sRGB output -- no matrix math is needed at all. So this is a great, great advantage.

Next subject I'd like to talk about today is some new built-in Core Image filters. So we have several I'd like to talk about. One is new to iOS 8 is we've added CIAreaHistogram and CIHistogramDisplayFilter. The first filter, CIAreaHistogram takes an input image and the rectangle that you want to generate the histogram of it and it'll produce an output image that's typically 256 by 1 pixels. So that image is useful if you want to render and get the pixel values out of it because that'll give you your histogram data very efficiently.

However, oftentimes you also want to display this histogram to the user. So we have a second filter which is CIHistogramDisplayFilter. And it takes as an input this 256 by 1 pixel image and it produces a pretty graph with red, green and blue graphs in it just like this. It's very easy to use in your application. You just chain together these two filters.

This is another great filter that I'm really pleased with. This is -- we've always had filters for doing Gaussian blurs on an image but we have a new filter called MaskedVariableBlur. And the idea is you want to apply a blur to an image but you want to apply a different amount of blur at different locations. So the way this filter works is you start with an input image and you provide a mask image. In this example, we have the mask is white in the lower left-hand corner, black in the center and then white again in the upper right-hand corner.

And what this means when we combine these two images with the MaskedVariableBlur is we get a resulting image that is defocused at the corners and then gradually transitions to a nice sharp image in the center. This is not just done with blends but it's actually done with variable radius blurs (which is quite a trick).

So there's a couple of different ways you can use this. You can use this to achieve a sort of fake depth of field effect where the top and bottom of your image might be blurry and the center may be sharp. Or you can actually hand-create a masked image with a person in the foreground and then nicely blur the background with a nice bokeh. So I hope to see lots of fun examples of that.

This is another fun one we added which is AccordionFoldTransition and this is something we did for the mail team but we've also provided it as a public filter. You provide two images -- a before and an after -- and a couple of parameters, like how many folds and how many pixels at the bottom are shared. And what this filter looks like in practice is this. And if you actually look carefully, that's the actual entire kernel for this filter. So it's a nice bit of trickery.

Another filter we've added, in prior releases, we've had filters for generating QR codes. We've added a new one for generating code 128 barcodes and it works in a similar fashion. You specify an input message as NSData and in this case, there's an additional parameter which says how many pixels of quiet space you want and it'll produce an image like this. We've also added another one for Aztec codes. Again, the same kind of idea for the API, you just specify the input message and for this particular generator, it has an input correction level which tells how many error correction bits it will have.

Another new filter which is also fun is CIPerspectiveCorrection. And the idea behind this is you have an input image and you specify 4 points and it will create a new image that is cropped and undistorted preserving the original and intended aspect ratio. So this is again very nice for capturing parts of an image and distorting them.

We've added a handful of new blend filters, linear, dodge and burn, pin lights, subtract, divide. Also, just to be aware: we've made a fix to SoftLightBlendMode so it better matches the spec. And then there's a few other new ones we've added that are new on iOS such as GlassDistortion, StretchCrop for anamorphic correction, Droste which is a great demo from our conference show two years ago, and then who knows, if we have some more time, we'll get a few more in. But what that brings us to today is over 115 built-in filters on iOS and of course, that really is an infinite number now that you guys can create your own custom filters. So we're excited to see all sorts of new things.

Another area we've made some improvements in Core Image is CIDetectors. So what is a CIDetector? Well, CIDetector is an abstract class that allows you to help find things within an image. And prior to iOS 8, we had just one type which was TypeFace. But we've added two more. So we now have CIDetectorTypeRectangle and CIDetectorTypeQRCode.

So how does this work? Well, creating a detector is largely the same regardless of what type of detector you are creating. Here we have an example of creating a detector of TypeFace where we say detector, detectorOfTypeFace and we can also specify some options. There are a couple of options that are very useful for all the detectors.

One is, you could say whether you want to have high accuracy or low accuracy which -- depending on your need -- might allow you to trade off performance versus precision. Also, you can tell a detector what the smallest feature to detect is and that also can greatly improve performance.

And of course, now that we've added these new detectors, you can just use DetectorTypeRectangle or DetectorTypeQRCode as well. So just as a reminder, so when you're using the Face detector, there's a couple of options that you want to pass in when you're asking for the actual features in an image.

One is you can specify what the orientation of the image is. That's important because the FaceDetector looks for upright faces. Also you can specify options to say "I want to look for eye blinks or smiles" and that's specified in the same options dictionary. And let me show you a little bit of code about how we can now use this detector to create a sort of augmented reality example here.

And the idea we wanted for this little bit of sample code is we wanted to start with the input image, find the faces in it and then put squares over the image where we find them. And so this is a little clever bit of sample code. First off, for each face that we detect in the features array, we're going to check to see if the eyes were closed or not.

Then we're going to create a CIImage WithColor. And we're going to have a different color based on whether the eyes are closed or not or whether face is smiling or not. Now that API actually returns an infinite image so what we then need to do is to crop that image to the bounds of the feature that was detected.

We then take that cropped image color and we composite over the previous resulting image. And this is also a new API that we've provided. It's basically convenience API that's equivalent to using the Core Image source over compositing filter. And this is what it looks like in practice. Here's a little sample video we shot where we are detecting the faces in real time and then coloring them based on whether the face is smiling or blinking or combinations. And we're getting about 25 frames per second.

We could do something similar also for rectangle features. So the idea behind rectangle features is we understand that in a lot of cases, the first step in looking in an image for something interesting is to look for something like a rectangle. For example, if you're looking for a sign or if you're looking for a business card or if you're looking for a piece of paper, oftentimes looking for the rectangle first is a great place to start.

So we've created a generic rectangle detector object and it takes one option parameter which is the aspect ratio that we want to search for. And again, you can ask the detector to return the features array. Now right now, it just returns one rectangle but that may change in the future.

So here again, we wanted to do a little sample here, a little bit fancier because we want to, instead of just doing the bounding box overlay, we want to make it a little bit prettier. So again, we're looping over all the features in the image. We're creating a CIImage WithColor which is infinite.

But we're going to take that infinite color image and run it through the CIPerspectiveTransform WithExtent filter. And that filter does two things. First of all, you specify an extent -- which in this case we're specifying zero zero one one -- so now effectively, we have a unit square image.

And then the other parameters take that unit square and stretch it to the top-left, top-right, bottom-left, bottom-right coordinates. And then we overlay that on the previous result. And here's what that looks like in practice. So this is the nameplate from my office and we are taking -- running it through the Detector, getting the detected rectangle and then producing this overlay tinted red image.

Lastly, we can do the same thing with QR Codes. The code here is exactly the same. The only difference is that we're using the QRCodeFeature instead. This example, you could have also gotten the message from the QR code but in this case, I'm just going to do an overlay. So all I needed to do was use the coordinates and again as you see in the example, we can detect this QR Code and do an overlay in real time.

So that's the bulk of my conversation there. The last thing I want to talk about is improvements that we've made to RAW support on OS X. So let me talk a little bit about our RAW support. So I'll talk about our history, the fundamentals of RAW image processing, some architectural overview and how you can use this great filter we have called the CIRAWFilter.

So history first, so Apple has been supporting RAW since back in April of 2005. Over those years, we have been continuously adding support for cameras and improving the quality. We have about 350 cameras supported today and that's not including all the DNG possibilities. And one of the improvements we've made in OS X this year is that we support the latest version of the DNG specification so that greatly improves the number of images that we can support.

And the other thing that's wonderful about our support is that it's provided to the entire operating system, which means everything from NSImages to CGImages will automatically support RAW files. System services like Spotlight and Quick Look support, these key applications like Preview, Finder, even Mail support RAW. Our photo applications Aperture, iPhoto and Photos. Also all third-party app can also get this for very little effort.

So what is involved in processing a RAW image? And this is why, you know, this subject is actually very dear to my heart because it involves a lot of very advanced image processing to produce a RAW file. So you start off with the fact that RAW files contain only a minimally processed data from the camera sensor image.

And in fact, the image is actually missing typically 66% of the actual data because at each pixel location, you only have a red or a green or a blue value. And that means to produce a final image, we actually have to make up good values for those missing 60% of your data. And that requires a lot of advanced image processing to produce a beautiful image at the end.

There are several steps in this process. They involve extracting critical metadata from the file, decoding the raw sensor, de-mosaic deconstruction (which is a hugely complex task), lens correction, noise reduction. And then there's a set of operations that are in the color domain, such as mapping scene-referred color values to output-referred, and then adjusting exposure and temperature and tint, and then adding contrast and saturation to taste.

So it's a lot of steps and we've made some significant improvements to several of these in OS X Yosemite. So we've benefitted for lens correction, great new noise reduction (which we'll show in a minute) and also some improvements to color as well. So as I said before, APIs like NSImage and CGImage will get RAW support for free. And that's because our support provides default rendering, which is processed according to all of our parameters and whatever our latest algorithm is.

However, we have this API which is called the CIRAWFilter which gives your application much more control. And it allows you to get a CIImage with extended range, floating point precision and also on that object are easy-to-use controls to control how our RAW imaging results are processed. And it gives you fast interactive performance all in the GPU. So it's some great stuff that you can use in your application.

So this is sort of how it works as a flow diagram. You start out with a file and that can be passed either as a file URL or NSData. And that's passed as an input to create the CIRAWFilter. Also it can be specified on that RAW filter are several of our processing parameters.

Once you've set those correctly, you can get a CIImage output which you can then display on the screen. And by default, it'll look just like our default rendering. However, the great thing about the CIRAWFilter is that once the user has seen those and if your application has controls, you can alter those values, send them back into the CIRAWFilter where it can be re-displayed all in real time.

Another great feature we have on this is we actually have a place where you can insert a custom CIFilter in the middle of our RAW filter processing before we've done anything to change the data from a linear space. So this is very useful if you want to do certain types of image processing.

Now of course, you can also apply filters after the CIRAWFilter but this is a great set of functionality for certain use cases. And lastly, it doesn't have to go to the display. You can also take the CIImage, create a CGImage from that and produce a new CG -- ah, a file on disk from that.

And this is an example of how little code it takes to use this filter. Basically, we start out with a URL. We create a CIFilter filterWithImageURL and that'll return to CIRAWFilter. In this particular example, we want to get from that filter what our default value for the luminance noise reduction was, that returns to us as an object. We can then make slight changes to that, like say for example: you want all of your images to be slightly more noise-reduced. You can take that value, add a bit to it and then set that as a new value.

And then once you're done setting values, you can get an output image. So with just a few lines of code, you can leverage all of our RAW pipeline. So to show this in much more detail, I'm going to pass the stage over to Serhan who will be giving a live demo of this. Thanks.

In this part of our talk, I would like to show you some of the great things that you can also do in your own applications using the CIRAWFilter and OS X's built-in support for RAW camera files. To do that, we created a very basic, simple application that simply puts up an NSOpenGLView which is tied up to a CIRAWFilter, and another NSView which is tied up to the controls of the CIRAWFilter. So let me run that and point it to a RAW image.

Now, by default, when you actually open up a RAW file, we will tap into our own calibration database and make sure that we apply the correct set of calibration settings that are specific to make and model for this RAW file. And some of the settings are for you under lens correction, white balance settings, noise reduction settings (that we will go into more detail in just a second), exposure and boost controls. So there is not much going on with this very good image in the first place. So let me pull up a more challenging image to show the great benefits of using RAW files.

Now, on this image, by default when you load it, you see that parts of the image is close to clipping point especially the sky and the mountainous region. So we're probably losing some color fidelity in this region. What's more interesting is the part of the trees which are underexposed and we're probably not getting the right amount of detail. So let's see if we can actually improve this image.

The first thing that I would like to try is setting the exposure to see how it actually looks like. Want to probably increase the exposure to make sure that I get the detail in the tree part of the image. But as you can quickly see, we're losing all the detail in the highlights. And the opposite is also true. Once you start decreasing the exposure, you're getting back the color in the sky but you're losing all the detail in the low lights.

So there is something that can be done better and the answer to that is CIHighlightsAndShadows filters. Normally, if you were shooting JPEG, you would tie the output of the JPEG decoder to this highlights and shadows filters. But what's interesting when you're shooting RAW is that you can actually insert this filter into the middle of our RAW processing pipeline and take advantage of the linear input space that we're operating in. That means that you will be able to better keep the color fidelity. You'll operate on a linear 16-bit pipeline and at the end, get better results. So let's try that.

The first thing that I want to do is increase the shadows and almost immediately I can see that all the detail in the shadow part is kept, is brought back. Same for the sky. I want to bring it down to make sure that I can see more of the sky colors.

And I can easily do that without overblowing any part of that image. So that is a good example of how you can actually use the CIRAWFilter to make sure that you can double up your own images in the best way that you think is appropriate. Next: noise filter.

Now noise reduction is a very challenging problem and traditionally it is very computationally expensive algorithm. We're very happy to offer you a new noise reduction algorithm, starting in OS X Yosemite, that doesn't compromise on the quality and you can still use it at an interactive 60 frames per second rate. To show you that, we have this very noisy image of the Moscone Center and I want to focus on this part of the image. Just for fun, I'm going to turn off all the noise reduction to see what we are dealing with initially.

So this is the original -- this is how the original image looks like. And using the CIRAW LNR and CNR noise filter settings, I can get it to a state where I feel that is most comfortable for my image. So probably the first thing that I want to do is get rid of all the color noise and I'm using the CNR slider to do that. And look how interactive this process is.

Same for LNR, you have a wide variety of settings that you can play with. You can go with something that is very smooth or something which keeps all the luminance noise. So I want to probably hit somewhere in the middle where I got rid of most of the noise but still kept some.

Another good thing that you can do is brought back some of the fine, high detail back to the image after you clean up all the bad noise. So the detail slider is the one that you would be using for that. And quickly you can get back to this film grain type of look.

Same is true for high frequency contrast and if you choose to do that, you can also play with it again 60 frames per second. So that is the noise filter. Starting with OS X Yosemite, you'll also be able to use this filter for your JPEG images and this is going to be a really nice advancement on top of our offerings.

The last thing that I want to show you today is lens correction. So a lot of the point-and-shoot cameras in the market today are actually relying on digital signal processing techniques to fix some of the compromises that are made in the lenses. What I mean by that: the input image, as you can see, by default is looking correct to us. But actually, the RAW image that is coming in is looking like this.

So whenever that data is available in the file, RAW camera will try to do the right thing and actually correct for this aberration. But for your own application, you may choose to skip this step and actually do your own set of filters or your own lens correction. And it's an easy way to actually go back to the actual RAW sample of the file itself. I'm going to now quickly turn it back to David who's going to talk about usages of the second GPU.

Thank you, Serhan. So as you saw, we have this great new noise reduction and it's a very complex Core Image filter that we developed and it makes great use of the GPU which brings us up to talking about the second GPU. So a year ago, we announced at the WWDC our new Mac Pro which has this great feature of having a second GPU, just waiting for your application to take advantage of it. So let's talk a little about how that can be used. So we had some thoughts about -- for Core Image and for RAWs. When is a good time where you might want to use the second GPU? And a couple of scenarios come to mind.

One is if your application has ability to do speculative renders. For example, you may have a large list of images. The user might be looking at one but he may switch to the previous or the following image at any time. Your application could be speculatively rendering the next or previous image on a second thread.

Similarly, your application may have the ability to do a large batch export and you want to do that in the background and you want to use the GPU but you don't want that background GPU to affect your foreground GPU usage. So these are both great reasons to use the second GPU because it allows you to get the best performance without causing your user interface to stutter for its usage.

So how does one do that? Well, you could do this today on Mavericks. It takes around 80 lines of OpenGL code to tell, to create a CIContext that refers to the second offline GPU. However, we've added a simpler API in Core Image on Yosemite which is CIContext offlineGPUAtIndex and typically you just specify zero. So with one API call, you get a CIContext and that -- when using that, all renders will use the second GPU. So it's very easy. And to show that in action, I'm going to bring Serhan back up to do a quick demo.

Well, in our first demo, we showed that even the most computationally expensive noise filter algorithm can be done at 60 frames per second. I'm going to bring that application back and open up a very noisy image. So our LNR controls can be done at 60 frames per second.

To show you that, we actually wrote a little bit of code to display the frames per second when I'm actually sweeping through all the noise filter settings. And as you can see, I'm getting 60 frames per second all the time. Now let's say that you have a background trait where you are constantly exporting images and for some reason you wanted to do a GPU pipe on your first GPU. To simulate that, we have written a little bit of text application which is using the first GPU.

And when I go back to my own application (which is now also using my first GPU) I can see that the frame rate is actually suffering a little bit. I'm going to run my test shoot one more time to see what type of frame rate I'm getting out of this. And you can quickly see that it has dropped down to 50%. I'm getting 24 frames per second.

So can we do something better than that? And the answer is yes. If we can offload this work to our second GPU using the CIGLOffline context, I will get back to my original performance in my active app. And to show you that, here we go one more time.

I can see that the user controls are once again very smooth and the frame rate that I'm going to get is close to 60. So once again, this is a great way to take advantage of the second GPU if you are constantly doing computationally heavy algorithms in the background. I'm going to hand it back once more to David.

So to summarize what we've talked about today. We've talked about some key concepts to understand about Core Image. We've talked about what's new in Core Image on iOS 8, most notably Custom CIKernels and large image support. We talked about some new things in Core Image on Yosemite, notably some API modernization and some great new noise reduction and RAW support.

We've also talked about how to use the latest CIDetectors and how to work with RAW images in ways that you may have not imagined before. So this is the usual information about who to contact. Allan's a great person to talk to if you have a request for more information.

Related sessions: there's one I really hope you guys can come to, is our second session this afternoon where we're going to be talking about how to write Custom CIKernels on iOS. And also, we have a lab session that follows that so if you have coding questions, please come and we would love to hear your questions or suggestions. So that's all. Thank you so much for coming.