So welcome to session 103 getting started with Core Data. I am Melissa Turner I am one of the engineers who work on Core Data and they drafted me to come up here and tell you all about all the wonderful stuff I do. Now if I had to make a guess, I'd guess that a lot of you have something in common. A lot of you are probably programmers or the unfortunate few that have to manage programmers and get us all going in the same direction.

So I'm going to show you some applications and then we're going to talk about what they have in common. Some of you are pretty familiar or might be familiar with this application, its Yojimbo by Bare Bones software. It's a personal organizer application. Let's use track and store notes, archives of web pages, few other interesting things.

This is iCal, many of you are probably familiar with it, we shipped it on Tiger. Some partially rewritten for Leopard, it does some new and interesting things and we'll get into those later as well. Most of you probably haven't seen this, this is Terrain editor, it's an application developed by one of our customers, it's an OpenGL application that lets you navigate and sort of fly through rendered landscape.

And this is Aperture, Apple's latest edition to our stable of Pro applications. What do all these applications have in common? Well starting with the obvious, they are all Cocoa applications. All of them deal with data, they create it, they store it, they retrieve it and while that data is live, they manage it. They track undo, redo, that kind of thing. And all of them use Core Data.

Hwy do they use Core Data and this is sort of the part you are really here for the why's, the how's. You want to know how we can make your life better and what motivated our customers, the ones we've got started using Core Data. Well they wanted to spend time working on their features, when it comes right down to it they didn't want to reinvent wheels.

Persistence is really critically important but not what anyone buys an application for, they don't go and say oh wow you store your data as, at least more people don't. Most people say this is a really cool accounting application. This is a really cool game. They're interested in the features that they can see and manipulate. Persistence isn't really on their radar. So something that is very easy for programmers to hand off to a framework and let somebody else do it.

Object lifecycle management is the same kind of thing, undo redo. People notice when it's there but they don't particularly care how it got there or how it's implemented. So these are things that are really ideal to hand off to somebody else who is willing to write them so you can concentrate on building the really interesting user interactions that make your application something your customers are really going to want to go out and buy.

And the additional benefit of using somebody else's framework is that well you get automatic access to everything they do and if your using an Apple framework that means you get automatic access to all the platform initiatives. If for example Apple goes off and I know they'd never do this but if they decided to switch chip architectures and say go from something that was big endian to little endian, you wouldn't have to do anything because the underlying technology takes care of that for you. So it's less work you have to do on stuff that is not terribly interesting to your customers. It gives you more time to spend doing whatever it is that you do best.

So today we are going through some of the basics supported, what the pieces are, how they fit together. We're going to talk about building models cause models are essentially the scaffolding around which all of Core Data sort of operates or within all which Core Date operates. We're going to visit some applications and stuff and we are going to visit some applications that use Core Data and sort of talk about how they are using it, why they are using it, what they got out of it and it might give you some ideas for things you hadn't thought about. Either application you hadn't thought about building in Core Data or things that you hadn't thought about putting in your application that are kind of neat and nifty.

We'll talk about Core Data features and a little bit more depth, some of the things you can do sort of what the features you can expect is and we'll go through some of the common questions that people come up with when there first starting to look at using Core Data.

So the pieces, what are they? How do they fit together? Well most people start looking at Core Data because they want a persistence layer and well it's got data in the name so maybe that's what it does. So the foundation, the lowest level of Core Data is the persistence stores, these are the pieces of technology that take care of putting bits on a disc and pulling bits off a disc and putting them into some clarinet form for you.

We provide four stores under the box and SQLite and XML and in memory and the binary store. These all have different characteristics, it's up to you to pick the one that works best for your application. We've tried to give you good enough selection that there's something out there for everyone.

But incase whatever we, the stores we have aren't quite what you need, you maybe need to support a legacy format. We also pride in Leopard and API that allow you to build a custom Core Data Store, first class store just like all the others only its your data format doing what you need it to do.

But as I said we're pushing bits in and out of a file. Something needs to define a structure for those bits to say you know this piece of it is a date. This piece of it is the strength. That's done by the managed object model, the models sort of defines the structure of your date. It says these are the pieces and this is how they interact with each other. The main components in managed object models are entities. Entities have attributes, strings, dates, numbers and they have relationships which are as you might guess, relationships that describe interactions between entities.

The objects that are created from the design that is the managed object model called managed objects. These are instances of the entities and these are the sort of real work horses, these are the things that hold your data. In a sense they are the nouns of Core Data, the songs, the people, the presentations, these are the things you operate on, you work with, this is the important stuff.

But a model and a managed object really isn't sufficient, you need some kind of connection between the two of them so that your application can make use of the information that's in the model. In Core Data this position is filled by the managed object context. It's sort of the central scrutinizer of Core Data.

It watch's all of the managed objects that are registered with it, takes notes whenever a change is made and records information so that undo and redo just work. It also takes care of maintaining relationships, if something changes on one end of a relationship it will go off and do whatever's necessary to update the other end of it. All of this for free, you don't have to worry about it.

And in between the stores and the managed object context because those are the top and bottom levels of the Core Data stack lives a persistence store coordinator. This is sort of the center of the Core Data world, the fulcrum point around which every thing else spins. This is where we register the managed object models, so if you ever need to get to your model you go to persistence store coordinator.

It mediates between the context and the stores so that the stores don't need to know whose using their data, all they know is that the coordinator has asked them for some. In context don't need to know where data is coming from. All they need to know if they can ask the coordinator and they will take care of it for them.

This is a very important piece of the stack, you probably not have to interact with it much, you need to register the model and this is where you add and remove stores. But most of the rest of your interactions are probably going to be with the managed objects and the managed object context.

So how does it all fit together? Well sort of like this, on the one end you have the persistence stores. On the other end you have the managed object context. Your user comes in at the managed context level and says I want to look at all songs in my song library, the context say okay we can do that and it passes the request down to the persistence store coordinator which goes off and hands it to a store and says get me songs.

The store goes off and retrieves everything that matches the description songs and passes it all up back to context which returns those to the user. So that's where all the components are attached to each other, that's how they fit and that whole fetch concept is encapsulated in another object called the fetch request.

A fetch request describes the data you are interested in retrieving from the store, specifies what entity you want and it can specify filtering and sorting rules as well so you can say not, I don't just want songs, I want all Pink Floyd songs. And that's done with a fetch request.

And we have some features on fetch request that will allow you to find tune how you get your data back. We have a head start example that you can have a look at, see some of the parameters that are tweaked and Adam Smith and the session immediately after this one in this room will be talking more in depth about fetch request and how to tune them. So at this point I am going to bring Ben up for a demo and he is going to show you, now that I showed you the picture of how Core Data fits together, he's going to show you what it look like in code.

Good Morning I'm Ben Trumbull. I am the manager of the Core Data team and I'm going to show you a fairly simple foundation tool. So one of the questions we have is a lot of our demo's and work, work with Cocoa bindings and Interface Builder, you see a lot of examples with sort of dynamic UI and stuff like that and I just wanted to show you something that's fairly straight forward, a little app. So it's just a page of code and you can see some warnings there, my code.

And we have a fairly simple model for this app here which basically goes off and every time you run it, it records the process ID and the time stamp in a launch entity. So here I just imported Core Data header and we set up a Core Data stack and the basic stack is set up with just these two methods.

We create model, in this case we initialize it with a contents of the URL for the model we created in the tool and we create a coordinator and we register the model with it and then we add a store and I've just got a URL to a little XML file that's in the current working directory and we clear the context which is alloc inited.

We set the coordinator, now the store is empty when we first run this app. So we want to create a new instance of our launch entity here, which we do. Asking entity description to create a new one and then we ask NS process info which is a foundation to get the current process identifier. We wrap it in a number object and then here we tell the record we created to set this attribute.

Now I am getting a warning because actually didn't declare any new classes. As you can see the total work here is I'm linked against two frameworks. Xcode decided to give me like a man page and some other things that I am not sure I really needed. I have the model and I have my dot M. so what's going on here is in Leopard, Core Data is automatically going to generate accessor methods for anything you need for the entity that you are working with.

So the launch entity has a PID attribute and the launch entity also has a time attribute. So we can set and get these properties even though we actually haven't made a class for them because at run time Core Data will specialize the class of the object that we are working with to match the entity.

So I do the same thing here, I ask NS date to give me a date, then tell managed object context, I'd like to save the changes I've made with this newly inserted record and then the app basically goes off and creates this fetch request that Melissa was telling you about.

I tell it that I am going to fetch launch entities and then I don't send anything else because I am just going to grab all of them, all the ones that have been saved. I ask the context to execute the fetch and then I just log and I call value for key on the array to get all the PIDs.

So basically I have a model and I have the compiled program here. And if I run the tool it just logs the PID and we can see that we created and XML file here as the type of store we created and we got a little XML file, there's some metadata at the beginning. The Core Data is using to track information, you know what version of the launch entity I was working with and then here we can see the object, the PID and the time stamp.

And then if I run it a few times you can see they just accumulate. So this is basically, this is probably the simplest Core Data app you are going to write. And it's broken up into just a few pieces where you set up the stack, you add a store, you create a context and then you start working with your managed objects. I do most.

So Ben talked, I talked to you earlier about models and Ben talked about models. It's really hard to over estimate how important these are to your application. The first is sort of high level view, what is a model? Models a blueprint of your data, it describes what it looks like.

In a more traditional world you might think of it as a schema but that really applies to the external representation. Models sort of the description of the Y versions of your objects and how they interact. A good way to think of it is in the context of Objective-C. a class is to an object as an entity is to a managed object and a model encapsulates a whole bunch of entities and describes how they interact. So the second most important part of your application, most important part of course being the idea, the spark, the thing that is your trying to communicate.

It's influenced by and will influence your interface as you start thinking about modeling and as you start learning it, you sort of start seeing how it might. How your application might be faster or more elegant if you built it this way but the model will also designed by sort of how you want to present, be influenced by how you want opt present you data to the user. There's a few simple principles to follow when you're building a model and we're going to go watch for an example that will sort of help you see how it works and see those principles at play.

We're going to go with the songs example because well it's interesting and that's what I have slides for. But when you're talking about building an application, a song library type of applications maybe call it my tunes, you get a bunch of pieces of data you need to organize.

Here's some of them, just a few random ones that are the most obvious things that you might think of when you're talking about songs. So you've got these things, you need to play with them somehow. You need to conglomerate them together so you can operate on and manipulate with them. So how do you do that? Where do you start? Well at first pass might be something that looks like this. You just stick everything into one big table. Each song has all of the information associated with the song in a single row.

Well it's not bad for first pass but it's not the most efficient thing we can do either. If you look at it you'll notice that there's duplicated data. This is always I mean it increases the size of your application. You got extra strings all over the place, you got extra copies of data and it makes it a little bit more difficult to maintain.

Its not as visible in this song type application but if you envision a contact dictionary, if you change a name in one place you often want those changes propagated else where and if you put the information in just one place you'll only have to edit it in one place and everything else relates off to it. So we've got album information and if I edit it in one row I'll probably going to want it edited everywhere. So whey don't we split that off into a separate table. Take all of this row and move it somewhere else, often to its album table.

Well for those of you that are familiar with normalization and doing relational modeling in databases your probably saying oh yeah now she's going to tell me I have to add a row ID column and add foreign keys back to the other table, no you don't need to do any of that. Core Data takes care of all this for you, all of the key management taken, you can just concentrate on your data. The Row ID just goes away, so there's your album table.

So we had that row and we made it gone. Following the principle of eliminating redundant data we see that well we've got yet more information sitting there duplicated. Things that really are the same piece of information just in multiple places, so we'll take that row away too, but wait a second do I want to just make that a single to one relationship cause you know the way this is recorded, in some places I might have multiple artist for the same song and that make searching kind of a painful. One or the other artist is likely to get lost in the library and last we give each of those artist there own entry and link both of those back to the original table. This is what's called a (inaudible) relationship.

So there goes that row, what else can we do? How can we make this smaller? Well here's some more duplicated data, that's what we are working with isn't it? But when you really think about it our song title, wish you were here, is that the same thing in both cases. I mean if I am talking about an album Delicate Sound of Thunder by a group Pink Floyd, those are really unique identifying pieces of information for those objects. Song titles really aren't unique in the same way.

There basically key information about the song itself but the same key information can be shared across multiple songs. If I edit one, I only want to edit that one not all of them. So think we can leave those in place. So what else have we got done? We've got this column sitting over here song data, now this is an interesting case and this is where we start getting into the art and science of model design.

If you think about your application my tunes, how are you going to present information to the user? You probably are going to have some kind of a table view listing you know song titles, lengths, ratings, that kind of stuff. And that's what the user sees so really that's all the data we really need to load in order to show them the interface. We don't actually need to load the song data because well it's big and it takes time to pull stuff off disc and they are not going to use it.

So we don't really need that song data sitting right there in to that row, we can just move that off into a song table file and leave the sings table to show only the information that's going to be right there in the interface when your user goes looking for it. Systems and basic split out of the model sort of conceptually how it works and what it does, so now I am going to go through and show you how to do this in the Xcode modeling tool. Whoops.

Where is my project, where is my project? There we go. So here we have a sort of vestigial model, shows all the table that I broke out as we walked thought that example. We got song data, bytes, we got artist with group or with name and whether a group or not, we got albums with labels and names. And how did they get the model it was actually pretty simple, you just go up to Xcode, new file, scroll down and we have under the design category data models.

But I've already created this one and cause its boring for you guys to watch models on stage. So just going to add a songs entity since that is truly the center of the world. And I am going to add some attributes. Its going to be done either using the contacts menu or by coming up here, clicking on this little arrow

Give it a title, give it a rating.

Okay so its morning and that means that I am not allowed to type. You can come up here to the inspector view and give all of these types, last plate is going to be a date, blank, well let's make that a decimal, rating is going to be an integer. Untitled is going to se a string. So there we have our basic song information but no connections to anything else that is actually related. So add some relationships while we're at it.

And again we can come up to this corner of the model editor and set up a relationships, we can edit the friends names here and we set the destinations of the relationship, in this case is album and immediately below that we have a pop up list for inverses and this is actually kind of important because it allows Core Data to figure out what the other half of any other given relationship is.

Because when you think about it when you got a relationship it's really bi directional, songs have albums and albums have songs, if I change something about one, if I added a new song to an album I also want the son to know that it now belongs to that album.

So by setting up two relationships is inverses, your really saying this is really the same relationship seen from two differing directions. And that lets Core Data do all the relationship maintenance for you. So we're going to go through and songs belong to albums, but album can have multiple songs so we will mark that as a it's many. Set up the artist relationship.

Songs can have multiple artists. Artists definitely have multiple songs most of the times. And we'll also set up the song have been to everything. And we'll also set up the song, data relationship. But that's one long because any song really only has only one pierce of information.

That's our model, pretty simple. But wait there's more once I've got the model I can just go into build, interface builder and build a little interface for it. Where's my window? I scroll through my library of interesting stuff and I grab the Core Data entity element. And I am just going to drag that out and interface builder is going to notice that I am trying to set up an application, an interface for Core Data entity. What does it know about Core Data entities? Well it knows it's got a model, it's got a project open, project has a model so maybe I want to use one of those and indeed its correct, I do.

I am going to set this up so I can display multiple instance of my song, add some separate widgets so you can see the changes I am making in this view over here. This is what is going to end up looking like. And I am going to select the elements that I want to display. This is going to be an inspector window so I don't need any of the stuff that's in a relationship. There we go, I have an interface. I am going to arrange it for a second because I like having titles up top.

And let's go and edit the table a little bit and move the title over here. But there, that looks like it's not a bad first pass at building a little song inspector. So what next, well to files those, oops I missed something in my model. If I go through and look at all of these, it's important to Core Data that I specify types for all of my data because this allows it to say when you pull these bytes off discs turn it into an object of this type. Well we notice over in this column here that some of my things are undefined. Well I should probably go through and give those a definition so I can collect multiple ones.

We do multi select so it's easier to edit multiple attributes at the same time. What else did I miss? Missed the artist songs relationship, I did not give it indeed correct. Somehow I deleted that relationship so there we should be all done. We should just be able to build it and at this point I got my fingers crossed cause I haven't run it on this system before so we'll see if it works.

There we go.

Just add a couple of songs here.

We can just, oh I forgot to set up a save method. Oh well. Its morning, I'm slow. I could come in here and tell it, actually that's not correct, it automatically set it up for me, so if I quit the application and rerun it, there we got our data.

It's the same thing I entered before, not a line of code required, well that you wrote anyway. This is just one of the standard template applications, if you want to see how a basic Core Data application, a little bit more complicated then the one Ben setup, one that has a user interface looks. You just go into Xcode and create a new project Core Data application. We got a few types. Document based and document based for the Spotlight importer. It's a way to get you started. I have a drink somewhere.

So can we go back to the sides please? So that's sort of the how to get started on modeling, its very easy, we provide integration from between Xcode and interface builder and Core Data so that 9its really to bring up a simple application, get things started and start playing with your data and figuring out what you want it to look like? That was a basic model, we also support version models, which is a way of telling Core Data and that different models are really different generations of the same thing.

Miguel will be talking more about that in his session at I believe 2 o'clock today, also in this room and he'll also be discussing schema migration, which is how to translate data that was written to disc in one format, into a different format, so later version of your application can read it.

So that's according to basics, modeling basics. Now we are going to look at some of the applications that are using Core Data and sort of see what they are doing with it and what they are to give you a since how broad the scope of what Core Data can help you with is, first off is your system events, well system events, well technically its Cocoa application but really what it is, is a demon that runs on your system so sure it lives in the background and waits for people to send Apple events to it, it's AppleScript integrated. That's the API.

Background process just sits there and waits for people to send a request, send, create data, pull data out from the system of events server database. It's very generic, they create models on the fly, they pull data out and return it as generic objects, through the magic of key value coding it all again just works. It's very flexible and very powerful, it doesn't have a defined schema, but it allows people to do useful and interesting things, without a lot of work.

Yojimbo, you saw this earlier, it's a personal organizer written by the folks over at Bare Bones and it's a bunch of types of data and the important thing with Yojimbo is it has to scale. It has the potential to have Gigabytes and gigabytes of data in your data store. They need a technology that was capable of doing that and Core Data is, they use or SQLite store, everything is goes out into the same file, gets brought in, very fast, for a snappy application.

iCal as I mentioned was rewritten for Leopard, they wanted to take advances of Core Data because they were starting to want increased skill ability and they wanted to be able to store more of that, more people, more interesting things hanging off of that. It's also a framework in Leopard it's the iCal framework so that other applications are integrated in integration with the information in iCal have an API that they can use to call into it and retrieve information about events. You see this in Mail where the to do's are shared between iCal and Mail.

So they needed multi process access, which Core Data could give them, and they desperately wanted change tracking so that if you undid, or if you made a bunch of changes to an event and the suddenly realized oops, I'm editing the wrong event, not that I personally would ever have done that, you can just undo back across all of those changes. And even if there's been a save because iCal saves automatically for you quite frequently, you can undo back across saves. Core Data does all of that.

Address book. Again address book has been rewritten in Leopard to use Core Data. Again, they are after increased capability, scalability. They had a slightly different set of requirements, because where iCal was for the first time publishing their framework APIs, Address Book already had a set of framework APIs that they needed to remain binary compatible with.

And they've successfully done that using Core Data. And one of the other requirements is that legacy API that they needed to support needed to support multi threaded access, needed to be able to pass address book objects to people, contacts, groups, they needed to be able to pass that around between threads.

And they're successfully doing that using Core Data in Leopard Xcode, in the spirit of eating your own dog food, also uses Core Data, in a couple of places, actually. We use it in the modeling editor that you saw earlier and we use it in the documentation viewer, which you might not have expected.

Documentation viewer has multiple data sources, they have multiple persistent stores. Each section of documentation, the Carbon stuff, the Java stuff, the Cocoa stuff, is its own file. And they present a unified interface on top of that so it looks like all of the data is coming from a single place.

They get hundreds of thousands of symbols that they need to be able to manage and they need to be able to complex text queries on a return results very, very quickly. They actually have a hybrid storage model. Well all of the meta information, all of the symbol information is actually st0ored in Core Data. The bulky files, they actually store outside on the file system because file systems are really great at storing large elements of infrequently accessed data that you don't need to search on. And as with the rest of Xcode is 64-bit garbage collected.

Terrain Editor, we showed you that one earlier. It's actually an OpenGL application that does real time rendering and allows you to navigate through a virtual landscape. In this case we have a picture of Britain that was downloaded from some information that was retrieved from the British government, I believe.

All of the data, all of the vertices that are used to construct these scenes are stored in Core Data. The author just stores them as data attributes and passes those data blobs directly into the OpenGL rendering engine. His requirement was really it had to be low latency. Rendering engines and the people who use them are very, very fussy.

They don't want to have to sit around and wait and have one scene per second as your application slowly pulls data out of a database. They need it to be very, very fast, they need it to be very, very seamless and Core Data is capable of giving him that. And also it gave him undo, redo, binding support, a bunch of stuff that he didn't want to have to write himself.

And there's other apps on the system that use Core Data as well and other applications out in the world that use Core Data. Some of the other benefits, some of the other reasons people like it, it enables rapid development. You say how quickly I brought up a simple little inspector type application in Core Data. Very, very fast and flexible. It allows you to get it started with minimal code and time. It lets you concentrate on your features.

And at least one of the applications we have out there, Icon Composer, has a custom Core Data store that they're using that actually writes the traditional .ICNS files. So just because you're not using what you think of as a standard database type application doesn't mean there's not a way that Core Data might be able to help you. And I'm going to haul Ben back on stage and he's going to go into some detail on Core Data features.

Thank you, Melissa. So I'm going to give you an overview of some of the features that Core Data is providing. We're not going to go into too deep a detail, but just kind of raise your awareness of things about some of the things you can expect from Core Data. So in particular, one of the higher level parts of Core Data, there are two main pieces, there's the persistence mechanism, which is fairly self explanatory and then the data management, and this is broken up into change tracking, relationship management and undo.

So for the change tracking, the manage object context is tracking all the insertions, updates, deletes, whenever you changing your managed objects, it's recording that and keeping track of what's changed. It's posting notifications for you, so the managed objects themselves are all key value observing compliant and the managed object context at the end of every event will batch up and send a meta notification of what changed recently. And with the SQL store, it's providing incremental saves, so it'll only save the objects that have actually changed.

For the relationship maintenance, there are a number of pieces, and the first is unique references. So in the songs, for example, that Melissa showed you, if you fetch multiple songs and they're all from the same artist, when you traverse the relationship there, you're going to get back the same artist object for each of those songs. So Core Data is going to make sure that you have a single unique object for all the artists working with those songs.

And as Melissa mentioned, Core Data is maintaining the inverses, so if you decide that you made a mistake in your editing, and a song has a different artist, it'll fix up the other end of the relationship for you. And if you start deleting artists, it'll follow the relationships and it'll clear things out for you.

Or in your model, you can specify that you want it to cascade and remove a whole chain of objects and then finally the undo support. This is something that really helps a lot of developers out because it's very easy to do. You wire it up and your done. And bindings, in the App delegate.

So there's no real configuration on your part. It's all based on the model. The manage object context is just rolling back changes to the state of affairs that existed the previous event. And it works across saves. This is something a lot of developers really like. But there's not much really to say about it. There's not much that you need to do about it.

And as I mentioned there's the persistence mechanism, which for the most part is pretty self explanatory. We've got four main stores, the XML Store, which is using NS XML from foundation, the binary store, which is using a keyed archive reference foundation, the SQLite store, which is using SQLite library, and then a custom store, an atomic store, this is the store that you might write.

And then the persistent store coordinator is working with aggregation. So if you do a fetch, we'll find out which stores actually have those objects and give them back to you. And if you do save, will remember where those objects came from and save updates back to the originating store.

And then in Leopard, we support versioning, so we keep track of which entities were used to create the store and whether or not you load a store later, perhaps after one of your customers has done an upgrade, if that store's still compatible with your current version. And we have some tools to do schema changes to migrate to your next version.

And as Melissa mentioned iCal, Address Book and some of our other clients are using multiprocessing. So the SQLite store offers incremental changes and it does file system locking and it supports multiple readers and authors and the conflicts are handled by the context. So we'll talk about that right now.

So we'll detect changes made to the same objects in the data file, whether by multiple threads or by multiple processes. And we have automated recovery options for you. And these are called the merge policies. And if you decide you want to learn more about what exactly is going on, Wikipedia has Optimistic Concurrency Control. But for the most part, on the context you just set what kind of recovery option you want and Core Data will handle if for you.

And as mentioned, we provide a lot of the platform initiatives here. So we're integrated into Cocoa and we support key value coding on all the managed objects. As I demonstrated earlier, we'll even generate the methods for you to match all the accessors that your entities might want to define. And everything's key value observing compliant, so you can get notifications whenever a property changes.

We have the undo feature. And Core Data is built on top of the NS undo manager from foundation. So if you want to customize our undo behavior, you have all the foundation API to do that. And the same is true with the notifications. We're just working with the NS notification center and you have all the foundation API there to customize that behavior. And we're integrated on the other side, sort of the at Kit Cocoa side, with Cocoa bindings.

And in the tools integration side, we have all these modeling tools for you. We have the templates that both Melissa's app and my app were based off of. So there are a number of other templates for you to explore to get you started faster. We're integrated with Cocoa bindings, an interface builder.

And we have wizards. So in addition to the wizard showed you, there's a wizard for the collection view, to do a grid layout, and a bunch of other things. And finally in Leopard we have some integration with Xray and we have built in probes to track performance issues for you.

And high level Mac OS X initiatives. We're fully 32-bit and 64-bit compliant. So you can just get started with your 64-bit efforts. If you have the, obviously universal binaries. We're supporting the new Objective-C properties, so in Objective-C 2 with both dynamic properties and garbage collection. And we also spend some effort on multi core utilization. So a lot of people's machines, most of the shipping systems here are dual or quad core or even 8 core and Core Data is finding some ways to use those up for you.

And something important to keep in mind here, we get a lot of questions about this, Core Data is not an ideal solution if you can't link against foundation. We're integrated into all the Cocoa technologies and if you can't use foundation for some reason then Core Data isn't a great solution for you.

We also have, we want that model information. All of these automatic behaviors are derived from your schema. So if you have an application in which the user can change any aspect of the schema all the time, then Core Data isn't really a great solution because there's some overhead and we want to compile the model and have it relatively static.

So you can make changes to the model, but we're going to want you to reload the data after you do that and stuff like that. So if you need to make changes all the time, Core Data's not a perfect solution for you. And then finally, if it's not a client server database, all of the persistence technologies are backed by files and we currently don't have any plans for client server support at this time.

So some other common questions that are coming up have to do with performance, so here's a little sample that I wrote several different versions of this project and it fetches 500,000 rows in SQLite store. They're all using the same store. And the purple column, the SQLite store, is pretty typical of the kind of code when I see people using SQLite directly themselves.

The model objects here are NS dictionaries, so they have NS dictionaries in a table view, for instance, and they're working with SQLite to pull back the rows and they set all of the columns as keys in the dictionary and the column values as those values in the dictionary.

Now if you refine that and you have a custom object and you run it through Shark a lot, you can get it to be quite a bit faster. So that's what the optimize SQLite column there is in blue. And both of these are raw C tools. There's actually very little. So the dictionaries here are CF dictionaries. So there isn't any overhead here in terms of language initiatives or anything like that. These are C programs.

The optimize SQLite does a lot of work to, for instance, optimize retain counting and a bunch of other issues if you're working with core foundation. And then finally we get to Core Data, which is again about twice as fast as the previous optimizing as SQLite yourself. And the reason for that comes up with the multi core utilization that we're doing.

So this fetches large enough that we realize hey, we've got some extra cores in the machine, we can do something useful with them. And as a point of reference, I wrote a tool that malloc'd the same amount of memory and then freed it. So we're about twice the speed of just mallocing that memory and then releasing it as a bench point here. And we're going to get into this example a little bit more in the next session talking about optimization.

So the basic speed is pretty fast. We're smaller and faster than dictionaries. Dictionaries are very flexible. There are arbitrary hash tables there. And the Core Data is making use of that schema information and providing a much more optimized layout and much faster look ups. The dynamic methods we generate are direct dispatches, so it's very synonymous to ivar access.

And we're providing configurable options on the fetch request for your applications specific needs. And there's an entire example devoted to exploring this attached to the next session, session 105, Optimizing Your Core Data Application. So you can take a look at that. And even if you're new to Cocoa, you can run the app and get an idea of what the little options are and how they change the behavior of the application.

So some common questions we have in terms of customers improving their performance, something that often goes overlooked is fetching performs IO, so if you're working with the SQLite store in particular, we're going to be loading that data as you ask for it. So every time you ask for new data, we have to go to disk and that's kind of slow. So you don't want to have to do that for every single object.

What you're rather do is have one fetch that says hey, I'm going to need all these objects really soon now, can you go off and get them for me? So it's much, much faster to issue a fetch that's for an entire batch of objects. And if your batch is large enough, then we might decide that we can actually spool up some background threads to help you get that done even faster. Since this is IO, you really want to be reading from the database the same way you would from a file.

You're not going to be reading a single character out of unbuffered file one character at a time. You're going to read a bunch in the buffer and then you're going to work with them and then you're going to read the next batch. And that's really the best way to work with Core Data an any of the options we provide.

There are also a bunch of performance tools here that tend to go overlooked. So new on Leopard, we have Xray and DTrace, but I want to bring your attention to Shark, which can be loved more. So there's an all thread state option in the time sampling with Shark that'll track how long it took to get data from disk, which is obviously a big issue for working with a lot of data. And then finally Core Data itself offers some SQL logging. There's a user default and we'll just log what we're sending to the database and give you an idea of how often we're going to the database. It includes timing information, annotations and stuff like that.

Another common question that comes up is how to create unique values. So how to assure that there's only one artist of a given name in the database. And it turns out that this kind of behavior is a little bit intentioned with Core Data providing unique references. Because if you make a mistake, and you said no, these two things were the same at run time and we've already created two unique references, resolving that is somewhat difficult.

So one of the key things to remember is to avoid fetching each of these objects one at a time. So if you're creating a unique artist, you don't want to every time the user enters a new artist, you kind of want to avoid fetching is this artist unique, is this artist unique, is this artist unique, is this artist unique.

If you can, you want to batch them up. But if you can't, you want to leverage some application specific usage pattern there to try and cut back on the amount of fetching you're doing. So one thing you can do is you can cache the keys that you already know to be unique in memory in the NS dictionary and check there first.

On the list we see a fair number of questions about memory management. So Core Data does the Cocoa technology and all the standard Cocoa memory management rules apply. And something to keep particularly in mind is that all of the objects you fetch from Core Data are all auto released. So if you don't know what that means, that's a basic Cocoa term and you'll need to work with that to create auto release spools to manage the scope of how long those objects survive.

So if you're doing a lot of fetching in a loop, you might want to create an auto release spool to go inside that loop so at the end of every loop iteration you pop the spool and all the objects you fetched during that iteration of the loop go away. Or, on Leopard you can use garbage collection and not worry about this.

We also have some good questions about the notifications. As I mentioned we provide a lot of notifications, both key value observing notifications and NS notification center notifications. They are provided at the end of every event. They're also provided whenever you save a context. We don't, however, provide any process notifications for you. There are a lot of technologies on Mac OS X that already do this, and you can leverage any of those technologies.

For instance, you can take the notifications we send and repost them with the NS distributed notification center. And there are kit keys where you can use to observe the file descriptor of the database you're working with and how the other processes notice whenever that changes on disk. Stuff like that. So there are a lot of Mac OS X technologies to handle cross process notifications.

And we also get a lot of questions about multithreading. So one of the things is Core Data wants to use these patterns called thread confinement. And basically, we want you to keep your managed objects on one thread. And if you want to work with them on another thread, we want you to bring over clones into that other thread as a separate scratch pad, a separate working space confined to each thread.

The raw data that we use, that you fetch from disks is actually going to be shared between the managed object context by the coordinator and that shared copy and write. So pulling these managed objects into separate threads isn't as expensive as you might fear. And as I mentioned a couple of times, on Leopard, we're going to try to find ways to utilize all those extra cores on your machine on demand.

And then just a quick note, schema version migration, there's a session at 2 p.m. entirely devoted to this and to the tools. Custom atomic stores we're going to talk about a little bit in the next session after this. And then dynamic properties and our integration with Objective-C 2.

and then there's also transformable attributes, so if any of you have actually played around with Core Data, there's an attribute type which basically allows you to attach an NS value transformer to it and then we just serialize out in NS data, so you can put in any kind of object you want on the object graph side and then Core Data will serialize it into a blob for you. So that way you can set up more custom classes for your attribute types.

So a summary of some of the things we discussed today. We went over some of the basics. Melissa showed you an example of building a model and normalizing song data. We showed you some of the examples of applications on the system that are using Core Data and some of the things you're using Core Data for, and an overview of some of the things you can expect from Core Data and some of the common questions.

So we're going to start Q and A in just a bit, but some partnering requests for you is the documentation's fabulous and there's a lot of it now. So a few years into Core Data here we have accumulated a lot of documentation on different features and performance and a bunch of things like that.

It really helps us out when you file bug reports. So not just, you're welcome to use the mailing list, but if you come across a problem or even a performance issue, if you can go to bugreport.apple.com and let us know. And we also love stack traces and Shark samples, so if you have something that you think is a performance issue and you attach a little Shark sample, that's really going to help us out.

And a sample application is one of the best ways to communicate your problem to us. We click run, we have it in the debugger with the Core Data source, and we see your problem. So if you have basically a way to isolate the bug into a simple app, that would be great. And we get to those bugs first.

It's true. So some overview. There's a Core Data programming guide. There's a persistent document tutorial. We haven't gone too much into detail about some of the integration in the app kit side. There's a subclass of NS document that integrates your Core Data for you. There's a low level Core Data tutorial and then there's all the reference documentation. There are a whole bunch of examples on your system in slash developer slash examples.

And there's some related sessions all here today. So next up will be optimizing your Core Data application. Then there's going to be managing your schema versioning and data migration this afternoon. Somewhat related, Tuesday, is getting started with Cocoa bindings that Malcolm's going to be doing. And our lab, if you want to come in and show us your application or just ask more questions, will be tomorrow at 3:30 in the Mac OS X Lab. Standard who to contact, Matt Formica is the Track Manager here and the Cocoa Evangelist.