Hi. Welcome to the Advanced CloudKit session.

Thank you. My name is Jacob Farkas. I'm a Software Engineer at Apple and one of the designers of the CloudKit API. And my colleagues and I have put a lot of hard work into the CloudKit API. I'm really excited to talk to you guys about it today. So, let's jump right into it.

So, CloudKit is this thing that we've built on top of our iCloud servers. We've built a lot of iCloud services, and what we're doing with CloudKit is exposing those database servers that we use at Apple to all of you developers. So, we're actually using this. This is something that we use in the new iCloud Photos and iCloud Drive feature that we're introducing, and we're building all of that on top of CloudKit.

If all of this is unfamiliar to you, you might want to go back and check out the Introducing CloudKit session. It was given on Tuesday and there should be videos online. So, we're going to jump right into this. What we're going to learn today, we're going to over the CloudKit private database, which is a way for you store private user information in iCloud. We're going to talk about modeling your data in CloudKit.

We're going to talk about advanced record manipulation and different ways of saving records to the server. We're going to go over how to handle notifications reliably, if you miss a push, what to do. And finally, we're going to go over the iCloud Dashboard which is a web-based interface for managing your CloudKit application.

So, let's start off by looking at the CloudKit API really quick. We designed the CloudKit API to be highly asynchronous. Everything has a callback, nothing is synchronous. And we did this because all of these requests are going out over the network. When you get the network involved, anything can happen, you know, the server might not be responding quickly, there might be a bad network connection. We don't want to block the UI and cause a bad user experience.

And to help do this, we've used NSOperation almost everywhere in our API. I say almost everywhere because if you went to the introductory session, you remember seeing what we called the Convenience API. And the Convenience API is our way of helping you get started with CloudKit really quickly and simply. All of these APIs are, you know, single calls that work on one record at a time. So, in this case, we're fetching one record and it's asynchronous. We get that one record back in our completionHandler.

But, as you use CloudKit more, you might find that you need some additional functionality. And that's where the NSOperation-based API comes in. So, what we've got here is the CKFetchRecordsOperation. And this is an NSOperation that does the equivalent of that convenience API we just saw. We give you a lot more functionality when you use the NSOperation-based API. So, you can see here that our initializer takes an array of record IDs. You can fetch a whole batch of records all at once.

We also give you more feedback on what's happening. We've got a completion block for each record, and we also give you progress as we download those records from the server. And finally, these operations give you more knobs and levers to tweak what is returned and what the operation does. In this case, we've got a desiredKeys property that lets you specify what key should come back on the records that you fetch.

So, since all of this is built on NSOperation, let's do a quick overview of what the NSOperation class looks like. What we've got in NSOperation is a completionBlock and a cancel call so that you can manage the life cycle of your operation. We've got a couple of variables that tell you some state about the operation. And NSOperations have dependencies, so you can link two NSOperations together.

Once you have an NSOperation, you're going to want to start that operation, and you do that by adding it to an NSOperationQueue. When you have an NSOperationQueue, you can also manage the life cycle of that operation queue. You can suspend it and resume it, and you can cancel the operations in it.

So, if you go to look at our NSOperation-based API, you might just see this big list of a bunch of different operations. It's kind of overwhelming and confusing. The best way to think about this API is to think about the objects that you want to deal with. If you remember from the introductory talk, CloudKits-based objects are records and zones and subscriptions, and you'll see that up here we have a fetch and a modify operation for all of those items.

One of the really cool things about NSOperation is that is has dependencies. So, you can set a dependency between two operations and the second operation won't fire until the first operation is completed. This is really handy with CloudKit if you want to do something like fetch a record, add a property to it, and save that record back to the server.

You can make the FetchRecordsOperation, make the modify operation at the same time, set up a dependency, and when the fetch completes, you can put that data in the modify operation, and it'll start automatically. These dependencies also work between queues. So, even though CloudKit has its own internal operation queue that you can use to run operations, you're welcome to create your own NSOperation queue, and you can then manage its life cycle. You can stop operations or suspend them or cancel them.

One tip with using NSOperation though is that the NSOperation-based class has a completionBlock on it. This completionBlock ends up firing asynchronously with dependencies. So, if you've set up dependencies and you're using the NSOperation's completion block, they might happen at the same time, and that data you were trying to funnel into the next operation won't get there in time, you know, that operation has already started.

So, what we've done with the CloudKit API is we have CloudKit-specific completion blocks. And if we look at what we had for that fetchRecords operation, we have a fetchRecordsCompletionBlock. You'll see this pattern on all of our NSOperations. And these completion blocks hand back all of the information you needed to know about the operation that just ran. In this case, we've got the errors for that operation, and we've also got the records that were fetched.

And finally, NSOperations can have their own priorities. So, you can set background operations and have them run at a really low priority and keep your UI responsive. So, when we were designing CloudKit, one of the things we noticed is that there's two general classes of applications. There's one class of application that stores a whole bunch of data up on the server, and when you use the application, it's just presenting the view of that data.

It downloads it on demand, displays it to the user, and then, you know, tosses it out because it's, the real copy's on the server. But there's another class of application that has just a little bit of data, but you want that same data on all of your clients.

So, if you remember from the introductory talk, we talked about what we called Big Data, Little Phone. That's that case where we have a lot of data on the server. It's not all going to fit on one phone, and you download it and view it on demand. So, you can see these clients are downloading records, viewing them, and the truth lives up on the server.

However, there's another class of application, where it's a small amount of data. It lives on one client, but you want it on all your clients. An example of this is an application that manages your receipts. So, every time you buy something, you take a picture of the receipt, and you want that information on your phone, in your iPad, in your Mac, and you want them all to be up to date with the same receipts.

So, what we've done to help you solve that is we've got something in CloudKit that does that. If you remember, we have, every application gets a container. In every container is a public and private database. And just as a refresher, that private database is actually one private database for every user.

Inside of those databases, we then subdivide them down into zones. So, both the public and the private database have a default zone in them. But we've also given you what we call custom zones, and these custom zones allow us to give you some extra features that we can't provide in the default zones.

You can create these custom zones and use the new features. Let's go over a couple of them. The first feature is atomic commits. So, CloudKit has relationships between records. And if you start using CloudKit and using these relationships, you're going to build up an object graph, and you're going to realize you want consistency in your data. If, you know, you might have an object graph that you want to commit all at once. And if some of those things don't get committed, the data on the server doesn't make sense.

On the public database, we can't guarantee that because there might be thousands or millions of users hitting the same database. So, there's no way to lock the database while we, you know, commit your very special records. But in the private database, you only have one user. It's the current user's account. And in that case, we can provide you things like atomic commits.

So, with atomic commits, these batch operations in the NSOperation API will succeed or fail as a whole. So, if any record had a problem, you will get back a CKErrorPartialFailure. Inside of that partial failure error, you're going to see a user, the userInfo dictionary is going to have this CKPartialErrorsByItemID key.

And that's going to be a dictionary of record IDs to errors for each of those records. And some of those errors are going to be the real failures that you care about. These are the reasons that the atomic commit failed. But, you know, the rest of the records failed because it was atomic commit, and we need to let you know that they failed, so you're also going to see CKErrorBatchRequestFailed. That's just a way of saying that something else in this batch failed, and, you know, this wasn't the real problem, it was that other record in here.

Another great feature that we give you with custom zones are delta downloads. So, delta downloads are a way for that second class of application to be possible in CloudKit. You can download only the changes that were made in that zone, and you can cache them all locally. So, what does that look like? Well, we've got our Mac here, and we've got our iPhone, and we've got a custom zone.

So, let's step through a delta download really quick. We've got an orange record and a green record over on the Mac here, and we want them on the phone as well. So, we're going to first upload those both into CloudKit in our custom zone, and then our iPhone is going to perform a delta download to get up to date.

So, these delta downloads are based on what we call change tokens. And a change token is a way of tracking the state of the server the last time you talked to it. So, this phone has never talked to the server, and what it's going to do is send up a nil change token.

And that's a way of saying, "I've never talked to you, just send me all the records in the zone." So, the server is the going to take that nil change token, send down an, the orange record and green record. The phone is going to save them into its local database.

And then the server is going to send down a new change token. In this case, it's change token A, and that means that, you know, the records you have are all from state A. If the phone came back again with change tag A, the server will go, "Well, I don't have anything for you, A is good enough. There's nothing to download." But let's say the Mac comes along and it creates that purple record, and it deletes the green record. And it uploads those changes to the server.

The server is going to track the changes, so it'll note that that green record was deleted. It'll note that there's a new purple record. And now, when the iPhone comes back with change tag A, the server goes, "All right, well, we're are at B now. That's farther than A. Here's a delete of the green record and here's a new purple record that happened while, you know, since the last time we talked." Then it sends back that new change token and everyone is up to date.

So, you can use this delta download to implement an offline cache in your application. If you want to do that, there's a couple of steps your app should to take. This is kind of an outline of a basic state machine for every time you talk to the server and you want to do a delta download.

The first thing you're going to want to do in your app is to track the local changes. You're going to want to make a change table, and every time the user makes an edit in your application, you want to write that down. You're going to want to do this because you might be offline when the user makes the changes, and you're going to need to replay all those changes back to the server when you can talk to it.

Then, you're going to need to send all those changes up to the server. You want to do that before you fetch the changes because someone else might have changed the record in the meantime, and you need to resolve these conflicts. So, we'll go over conflict handling in just a little bit, but just keep in mind that that's an important step in this process here.

Finally, you're ready to do the delta download. This is the point where you're going to call CKFetchRecordChangesOperation. The server is going to send you back updates and deletes and modifies and adds of records, and you're going to save those into your local database. Finally, the server is going to send you back a new change token. And that's the change token you want to save so that the next time you talk to the server, you can get only the records that have changed and not everything.

So, one of the other features we give you with custom zones and delta downloads are zone subscriptions. In the case of that state machine I just talked about, you could pull every, you know, 10 minutes or 5 minutes or whatever and hope that there are changes up there. But wouldn't it be great if the server just told you, "Hey, I've got changes. It's time for delta download." Well, we give you that. We give you what we call zone subscriptions. These look like query subscriptions, but they're a little bit different.

What they do is every time something changes in a custom zone, you'll get a push notification. When you see that push notification, you know you should go do a delta download with the server, and you'll get new records. So, a couple of notes on designing custom zones and when you should use them.

These custom zones are meant to compartmentalize your data. Because of that, there's a couple of restrictions on them. The first is that you can't move records between zones. You can pick these records up and copy them and make a new copy in the new zone, but you can't move them. You also can't make any cross-zone delete self relationships. So, you have to think of these zones as self-contained. If you have records that need to go across zones, you might want to rethink your model.

And finally, these zones determine the level of updates. If you're using a zone subscription, you're going to get a push every single time something has changed in that zone. If you have a lot of data on the server and you only care about getting updates for one part of that data, you might want to make that a zone so that you can ignore the, you know, really busy stuff going on over here and just download the things you care about.

So, let's talk about some advanced record operations. When you're using CKRecord, any changes you make to a CKRecord object locally get tracked, and then when we talk to the server, we're only going to send the changes that you made to that CKRecord. This is the default behavior, and it works great most of the time, but you might want some additional control over what we're sending to the server. And we give you that with save policies. So, these save policies, we have three of them. They're CKRecordSaveIfServerUnchanged. We have CKRecordSaveChangedKeys, and we have CKRecordSaveAllKeys.

So, let's look at the differences between those. The biggest difference is what they determine for a, what I'm going to call a locked save. And a locked save is a way of making sure that you don't overwrite data on the server that someone else has already written. When you perform a locked save, it says that if the record has changed since-on the server since the time you fetched it, the server will give you an error saying your record is out of date, you need to resolve this conflict and try again. So, the only thing that performs a locked save here is CKRecord save policy SaveIfServerUnchanged. The other two actually force these changes to the server. So, SaveChangedKeys and SaveAllKeys always overwrite the values on the server.

These policies also determine how much data we're sending to the server. In the case of SaveIfServerUnchanged and SaveChangedKeys, we're only going to send up the, sorry, we're only going to send up the values on the record that have changed. In the case of SaveAllKeys, we're going to send the entire record, all the values, whether you've changed them or not.

So, back to locked updates. We've got this thing called locking, and if you don't use it, here's what could go wrong with an unlocked update. Right now, we've got this contact card. We've got the same name, first name, last name, picture on two clients, and in iCloud. Everything is good right now. But let's say the iPhone changes that contact completely. So it modifies the picture, it modifies the first name, and it modifies the last name.

Well, it's going to perform a SaveChangedKeys to the server, which is going to send up the things that have changed, and that's going to overwrite whatever is on the server. Next, before the Mac gets a chance to download that record, the user edited the first name of that record and just changed it. And that Mac then sends- does a SaveChangedKeys with the server, and it sent up just the property that was modified. And now, we've got a problem here, oops, we've got a problem. This contact record isn't what either the clients meant to have.

So, to work around this, we have locked updates. And locked updates are a way of making sure that the server knows that you're updating the same values that are already on the server. And we do this with a change tag. So, the change tag is a property on the record that necessarily changes anytime a value in the record changes. In this case, we've got change tag A both on the server and locally, and so we know that they both have the same values.

If we go and we add a new property on the local record, when we save that to the server using the default save policy, it's going to send up both the change tag and the property that changed. The servers then, before it does anything, it's going to check the two change tags, and if they're equal, it'll apply the change that you made.

If on the other hand we have another client that, oh, sorry, and then the server is going to send back a new change tag because we did change a property. We return this new record to you with the updated change tag in the completion block. So, if you take the record that we gave you at the end of a save and you do all your subsequent modifications on that, you'll be using the updated change token, and you shouldn't have any conflicts with the server.

Now, let's say we had another client that, you know, hadn't seen that change and he was still at version A. And he decides to add a different property on the record. Well, he's going to save it to the server, he's going to send up the change tag and the modified property, and the server is going to realize that, hey, those two values aren't the same. And the client is going to get back an error saying, you know, you have a conflict.

So, how do we handle these conflicts? Well, if you run into this case of a locked save failing, you're going to get an error that CKErrorServerRecordChanged. And because you guys all went to the introductory talk and you learned about how great error handling is and how important it is, you've got some fantastic error handling here, and it's going to check for CKErrorServerRecordChanged.

When you see that, you know that you're out of date, you're going to need to get the new record from the server. You're going to need to take your local changes, apply them to that record and try to save again. And we've already anticipated that. So, what we're going to do is help you out a little bit, and we're going to return those records to you already because we know that's the next logical thing to do. So, inside of the userInfo dictionary of this error, you're going to find three copies of the record.

You're going to find the record that you attempted to save to the server, the one that encountered the error. You're going to find the original copy of the local record, which is the copy we downloaded from the server before you made any changes. And finally, we're going to give you the server record back.

This is the copy of the record with the most up to date change tag. You're going to want to figure out what makes sense out of all of those copies of values and keys. You're going to apply them all to the server version of the record, and then you're going to retry that save.

So, you might be wondering at this point, you know, what type of save policy should you use? And the simple answer to that is that you should just always use CKRecordSaveIfUnchanged, and it's the default. It's the default for a reason, and that's because it's the safest. If you remember that example of an unlocked update gone bad, you could end up with really mixed-up, corrupt data on the server if you're not using locking.

However, there is always a time and place for using unlocked updates. And the biggest case for these are highly contentious updates. If you're doing anything in the public database, you might have hundreds or thousands or millions of clients accessing it at the same time. And if every one of those clients is trying to save if, save the same record and you're locking on that, most of those clients are going to be hitting locking errors, they're going to hit conflicts, and they're going to be retrying really frequently.

So, if you know this is going to be a really contentious update, you can structure your client to do unlocked updates as long as you know you're always writing consistent properties. On the case of that unlocked update that failed, you know, you could make sure you always write both the first and last name, and you know it'll be consistent.

The other reason to use an unlocked update in the case of SaveAllKeys would be if you want to force something to the server. The client might say, you know, the copy on the server is bad, but I know I have a good copy of the record here. I want to just force that all on top of the server's copy.

There are some catches to using SaveAllKeys, and one of the problems is that any property on the server that doesn't exist in the local record that you're trying to save isn't going to get removed unless you explicitly remove that property on the record. All those words are really confusing, and it's really hard to explain.

So instead, I'm going to try and explain that with the picture here. So, we've got our contact record again. What we're going to do is add a couple of properties. We're going to change the first name, we're going to add a new property, and we're going to delete the hobby. We're now going to do a SaveAllKeys to the server.

What that's going to do is send up all of these properties, even the ones we didn't change locally, and they're going to update the values on the server. But what you'll notice here is that the server had an additional property that we didn't have in our original copy of the record. This hometown property wasn't removed.

We do this to help you with versioning of your app. You might release a version 2 of your app that adds hometowns but version 1 didn't ever know about the hometown. And if version 1 was using SaveAllKeys, it's going to overwrite these properties on the server that it never knew about, and that's kind of a bad thing. You can't have backwards compatible code easily that way.

So, what we do instead is we still allow you to remove that, but you need to explicitly tell us that you want that property deleted on the server. So, even though there's no hometown property on this record, we can still call CKRecordSetValueForKey nil or we can call removeObjectForKey. We're going to remove the hometown, and now when we save it up to the server, that delete will also go up with the save.

Finally, one of the really neat things about CKRecord is that we allow you to have partial records. So, you might have a really big record on the server, and you only care about one or two properties of that. Well, with the desiredKeys property, we allow you to fetch just a certain subset of the properties that are on the server. And we expose this desiredKeys property on any operation that fetches a record, so you're going to see it on CKFetchRecordsOperation, CKQueryOperation, and CKFetchRecordChangesOperation.

And the really neat thing about this is you can then take these partial records and save them back up to the server. You don't need to work with entirely full records. You can, you know, if you want to update just one value on a record, fetch that one value from the record, update it, and do a save of just that record that has that one value. If you're doing this with locking enabled, you know it's safe because if the record is changed on the server since then, you're going to get a conflict.

So now, let's talk a little bit about modeling your data for CloudKit. If we go back to that example of a Receipts app, let's say we have a shoebox that holds all our receipts and that's going to be one object in the cloud, and we're going to have an object for each one of our receipts.

And the initial way you might design this is you create a receipts array inside of your shoebox, and every time you add a new receipt, you're going to add new entry to that array and that entry is going to be a reference to the record you just created. We call these forward references. These are references from a parent to its child.

And the problem with these is that you end up getting bottlenecked on that receipts array. If this was a public database, you might have hundreds of clients trying to save this record at once. And every time a client tries to add a new record, they need to update that array on the shoebox, and even though the clients might be adding different records, different receipts, they're all going to run into conflicts on the shoebox app, or on the shoebox record.

So, we recommend that you try and avoid forward references in your design, and instead, use what we call back references. So, instead of having the shoebox point at the receipt that's in it, have the receipt point at the shoebox that it's in. The great thing about this is it's very scalable. You can have millions of clients adding receipts all at once, and there's no bottleneck. They can create their receipt, point at who owns them, and none of them are going to run into conflicts.

When you need to go get everything that's in a shoebox, rather than fetching that shoebox, getting the array of records in it, and fetching all those records, you can instead just use a query. That query is going to select all of the records that have that shoebox as an owner. One other neat feature we give you with the relationships are cascading deletes.

So, cascading deletes are a way for you to make your object graph get cleaned up automatically, and you do this by marking your references with a CKReferenceActionDeleteSelf, and that means that when the record that you referenced is deleted, you will also be deleted. In this case, the green record has that reference action pointed in the orange record.

When the orange record gets deleted, the green record is going to get automatically deleted by the server. These deletes also cascade. So, if we had a whole tree of objects hang off of that one orange record, those deletes are going to cascade all the way down and clean up our graph for us.

But one thing to keep in mind with these delete self references is that they're not reference counted. That means that if you have multiple references coming out of one record, the first one of those records that's deleted will cause you to get deleted. And finally, these delete self references give you one additional feature.

If you're doing a batch save and you've got a jumble of records in this graph, and they have delete self references between them, CloudKit is going to do an automatic topological sort for you. We're going to upload the records in order so that all the records that are referenced will be up there by the time the record reference in them is uploaded. This is really great in the public database where you don't have atomic commits but you still want to be able to upload data that looks consistent at any point when a client downloads it.

So, while developing CloudKit internally, we've had a couple of our clients ask us, you know, why do I need to use these reference objects? I have to, you know, alloc and omit and it's so much work. You know, I've got a record ID, why can't I just put it in the string and put that in my record? You can, so, you know, you come up with this clever little RecordIDToString, but then you look at the recordID class and you realize it's actually two properties, and you can't just store the description of the record on the server so you get a little smarter and you store the recordID and the name and the zoneID.

But then, you look at zoneID and that's also composite. We need to know what, where zone is and whose account it is. So, you get a little more clever and you make this RecordIDToString and ZoneIDToString, and now you call that everywhere. The problem is now you're not forwards compatible.

If anything ever gets added to references, all of your records on the server are already going to have this hard coded format, and you're going to have to go through a lot of work when you query records. Instead, just use references. It's going to make your code really clean. I mean that shoebox could be a recordID, that shoebox could be a record itself, it could be a reference. All of those will just work as long as you use the CKReference.

A couple last notes on modeling your data in CloudKit. One of them is that CloudKit is a transport mechanism. What we've tried to do here is give you an easy way to access the CloudKit servers, but we're not meant to be a local persistence layer. We want that to be up to you, and you need to figure out the best way to store your objects.

And to that extent, we recommend that you don't subclass CKItems. You should take those items when you receive them from the server and translate them into your own model objects, and you can do that on the way out as well. When you need to save one of your model objects, translate it into a CKRecord and upload all of that to the server.

So, if you remember from the introductory talk, we have these things that we call subscriptions. They're persistent queries that run on the server, and every time something changes that matches that query, you're going to get a push notification. And these push notifications are sent via the Apple Push Notification Service, just the same way that it would work if you built your own server.

But there are some drawbacks with Apple Push Notification Service. One of them is that they can't make any guarantees on delivery of these pushes. They're kind of meant to be these ephemeral, little, you know, here's a push, here's a push. If one of them gets dropped because of a bad network or anything else going on, there's no guarantee. So, in practice, they're really good about delivering these pushes. And one of the reasons is because the server will store pushes for you if you're offline.

So, if you put your phone in Airplane Mode and you get a push, as soon as you come back out of Airplane Mode, the server will have that saved and still deliver the push on to you. But the problem with this is the server only stores one push per client. So, if you received a bunch of pushes while you're offline, you're only going to get the most recent one.

What does that look like? Well, we've got our CloudKit server, we've got the APS server, and we've got the iPhone up here. And we send a push, and we're online, everything goes through, we get our badge, everyone's happy. But then, we get on an airplane. Our phone is in Airplane Mode; we have no network connection. And when that push gets sent, the APS server helpfully stores it for us. If we were going to come back online right now, we would get our subscription push and we'd be happy.

But this is a long plane flight and while we're on the plane, we got a zone update push, that new zone subscription that I was talking about. When that gets to the APS server, it's going to drop that previous subscription push, and now when your phone comes back online, all you're going to receive is the zone update.

The problem with this is that you never heard about that subscription that fired. So, how do we solve this? Well, we've created a notification collection in CloudKit. Every time the server sends a push to the APS server, it also makes a copy of that push in the notification collection.

So, you can see we have that same problem where we're in Airplane Mode, we dropped our subscription push, our phone comes online, and it gets the push, and because this is a well-behaved client, it knows every time it receives a push, it should go check the notification collection to find out about anything it might have missed.

So, how does that work? Well, this notification collection is a lot like the delta downloads I talked about earlier. It's all change token-based. You hand a change token to the server, and the server hands back all the notifications that have changed since then along with an updated change token.

So, since this is our first time talking to the notification collection, we're going to send up a nil change token. We're going to get back both of those pushes, one of them which we missed. And then we're also going to get an updated change token for the current state.

One thing with subscriptions is that you might want to use those to change some UI in your application. For example in our Receipts app, we might want a subscription for any receipt that was over 100 dollars. If that happens, we want to mark that receipt in a different color. So we're going to get this push for a subscription, and now that receipt is blue because it was a big expensive receipt.

But, you know, this is going to happen on all your clients, so you're going to have you iPad showing that receipt in blue, you're going to have your Mac showing that receipt in blue, and once this acknowledges it, you want that UI state to go away on all your clients.

Well, the way we do that is we let you mark a subscription, sorry, we let you mark a notification as red. And when you do that, you're going to get a push for the updated subscription, updated notification on every client. So, in this case, we mark our receipt. We're going to go mark that subscription as red on the server, and now there's a new entry.

And if we switch over to our Mac which is still showing it in blue, it's going to get a push, it's going to check the notification collection, it's going to get that subscription notification, and it can tear down its UI and now all of your clients are in the same state.

So, with the notification collection, keep in mind every time you get a push, you should check the notification collection. You never know what you might have missed. And this isn't just for Airplane Mode or bad networks. This can happen if you get multiple pushes in a row. If there's a lot of changes that happen all at once, the CloudKit server might coalesce them, or the push server might coalesce those pushes.

And of course, because a lot of these are mobile devices, they're iPhones, they're going to be moving in and out of network states. They might be on Wi-Fi, or they might be on a bad cell network. You never know what your network is like. Just assume that there might have been more pushes that you didn't hear about.

So, now we're going to go over the iCloud Dashboard. And the iCloud Dashboard is our web-based interface for managing your application in CloudKit. The iCloud Dashboard lets you view your data. It lets you manage the schema that we're creating for you. It lets you control what's indexed, and it helps you moving from development to production. I'm going to explain all of those in detail in a bit. Let's start with viewing data.

So, if you remember the view of our containers up here, we've got a public database and a private database. And when you're viewing your data in the iCloud Dashboard, all of that data in the public database is of course public. So, in the dashboard, you'll be able to see every record in your public database.

But if you remember, the private database is again one database for each individual user. And in this case, we're only logged in as one user. We're logged in as our developer account. So, all we're going to see in the iCloud Dashboard is the data in the private database for our developer account. This is really important. You know, you might use a different account for testing and a different account for viewing data in the iCloud Dashboard.

If you do that, you won't be able to view your private data in the dashboard. So, what does the dashboard look like? Well, we log in, and the first thing we're going to see here are our different record types. So, you can see we're using the Party app. We've got parties, and over here, we see the different schema for those values that's been created.

When you're developing a CloudKit application, you're talking at the development environment, and the server in the development environment creates a just-in-time schema for you. So, we did this because we wanted you to be able to develop as rapidly as possible. We didn't want you have to go to a, this dashboard and plan out what you were going to do in your application and choose all the data types. That's not as much fun as just writing some code. So, we let you dive right in, you write some code, and the things that you upload to CloudKit as you upload them, those values get locked in into a schema.

So, you can see here that we've got a couple of different values, and we've got their, couple different keys and their values but we notice that we made a mistake when we're developing this app. We uploaded a date value or a date key, but we use a string value. And what we really want there is a date value.

The iCloud Dashboard is going to let us fix that up. So we can go to this and we can delete it. We can remove it from our schema, and now we can recompile our app, use a date value, upload that record again, and when the server sees that record, it will create a new schema entry for the date value using an actual date instead of a string.

So next up, you can view all the records in your public database using the iCloud Dashboard. So, you can see here, we already have a couple of different parties in the public database. What you can also do is create records in the public database. So, you can tap on this Plus button and fill out a new entry. We're going to make a new party for coffee on Friday, and we can save that and you'll see that that actually got saved into the public database. Any clients that are fetching records are going to see that change in the public database.

We can also search for records. This is just like CKQuery. So we can click on that magnifying glass, and we can type any string, and we can filter down to any party that mentions WWDC. Additionally, this gives us all the functionality that we have in CKQuery. We can build compound queries right in the dashboard. So, if you click on the Plus button, let's choose location, and let's filter down to everything that's within 5 kilometers of Moscone Center. I happen to have those memorized. So, that filtered everything down just to the two parties that are in this area.

Finally, in the public database, by default every record can be read by every user, and it can be created by any user. Once you create a record, that record can only be updated or deleted by the user that created it. But we understand that that doesn't provide all the functionality that you might need to make an application in the public database. So, what we've given you are what we call roles. These roles let you choose sets of users that have different permissions for record types in the public database.

So, one example of that might be that I want to restrict it so that only I can create parties in the public database. So, what I'm going to do in order to do that is create a new role. I'm going to call this, you know, PartyAdmin, and in that, I need to choose a record type and give it special permissions. So I'm going to choose parties, and I'm going to give it Create, Read, and Write permissions for those parties.

Now, what I need to do is go to the record type and restrict it so that no one else can create a party. So, I'm going to find my party, my party record type, and you can see up there that parties can be created by anyone who's authenticated. We don't want that, we want only party admins to create this record. So, we're going to uncheck that value.

Now, we need to assign people to that role so that there's actually users out there that can do this. If we go to the user records, we can see everyone who's used this app so far. If they have marked themselves as discoverable, you're going to be able to see their first and last name. If they haven't, you're just going to see that recordID up there. So, I'm going to click on my user, and then I'm going to add myself as a party admin. And now, I'm the only user in the public database that can create a party.

So, one of the last things the iCloud Dashboard helps you do is move from development into production. So, as I talked about earlier, we wanted the development environment to be as quick and easy to use as possible. We wanted you to hit the ground running, we wanted you to just open up Xcode, start typing some code, save your record to the server, and see immediate results.

And that all works great because the server does just-in-time schemas, but that's not very efficient if we're working in a public database that might have millions of users. We need to prepare some things before we go into the environment that all of our customers are going to hit.

So, the way this works is we've got our records, we're developing our app right now, and we just created a record for the very first time, and we uploaded it in the development environment. The server is going to see that and since this is the first time it's seen a party, it's going to automatically create a new party record type.

And then for each of these keys and values, it's going to create an entry in a schema, and it's going to create an index on every one of those values. This index is what lets you query for any value in a record. So, while you're in development, you can just run a query and search on any value for any key in a record.

And let's say we're going ahead and we're developing this app and we decide we want to add an additional property, so we decide we want to have a background property for the color of the party. So, this party's background is blue. All we have to do is upload that record. It's going to send all those other properties to the index, and the server is going to notice, "Oh, there's a new property in here." It's going to create a new schema entry, build a new index, and index that property.

When you're ready to release your app, you're going to want to make sure you've run everything in development first so that you've built up this just-in-time schema. You're then going to use the iCloud Dashboard to move that schema from development into production, and you're going to want to check all of these fields to make sure that they're the right values, make sure that every key that your app will ever use exists in the schema. And when you do that, you're going to move it into production and lock that schema in.

Additionally, all of these indexes take up a decent amount of space because it has to make it so that you can query those records. If you know you're never going to query a value like we know we're never going to query for records with just a blue background, you can drop that index, and this will help free up some space in your app's database or in your user's database.

So, finally, a couple CloudKit tips and tricks. So, we already went over this in the intro talk about error handling. Please handle your errors in CloudKit. This is really important. I know error handling is hard and it's not fun to do, but CloudKit is all based on network communication. And when you're talking over the network, anything can go wrong.

You know, the network can get dropped. Because we have other people talking to the server, we can get conflicts. We can get errors. All kinds of things can happen, and as Paul said in the first talk, this is the difference between a working app and a not working app. It's not the difference between a good app and a great app. If you don't handle errors, your app just isn't going to work right.

In CloudKit also, we've tried to avoid any sort of magic. We don't want to try and handle these errors for you and figure out what might be best. We want to just tell you what happened. We want you to be able to figure out what you need to do next, and we want to give you all the information to do that.

So, keep in mind when you're handling your errors here that your operations can have partial errors. These partial errors when you're using the NSOperation API, you might be sending up a batch of records. And if you're saving them to the public database, it could be the case that just one of those records had a conflict but the rest saved just fine.

If that happens, you're going to get a partial error, and inside of that, you're going to find one error for the record that failed. If this was in a custom zone, you might see that as an atomic update error. So, you might see that one of those records failed, and the rest got the batch error in there.

And finally, we want you to make sure that you retry any server busy errors. It could be the case that the, you know, a bunch of people are going at the servers at the same time and the servers can't handle it and they need clients to back off.

It could also be that your client is misbehaving and hitting the server too frequently, and the server is saying, "Slow down, buddy." This is our way of saying, "You know, we need a little more time. This request was OK, but please try it again". And anytime we give you a server busy error, we're going to hand back a CKErrorRetryAfterKey. This is a number of seconds that we'd like you to wait and retry your request.

So, limits, in the keynote, it was mentioned that CloudKit is free with limits. How do those limits work? Well, anything stored in the public database is counted against your app's quota. We give every app a quota, and that's just for the public database. Anything you put in the private database is counted against the user's account. So, every iCloud account gets 5 gigabytes of free storage, but users might have bought more, or they might have filled up all that space with emails or backups or photos.

So, every user is going to have a different amount of storage in there. And because this is kind of the user space and it's shared space, we want you to, you know, remember to be nice to your users. It's technically free space for you because it's theirs but, you know, don't fill it up with unnecessary stuff.

So again, with the limits, how much do you get for your app container? Well, what we're concerned with here is customers having a great experience, and these limits that we've specified here are really to try and prevent abuse. We don't want to prevent legitimate use. We just don't want anyone abusing CloudKit.

And the numbers that we give you here also scale with the number of users you have. If you go on developer.apple.com, you can get the full breakdown of the different values, but as an example, if you had 10 million users of your app, we're going to give you a petabyte of asset storage, 10 terabytes of database storage, some pretty high transfer limits, and this is all for your public database in your application.

So, finally, a note on efficiency. One thing about CloudKit is, again, it's a transport mechanism. We are only there to talk to the iCloud servers for you. We're not storing any records. We're not caching records. Anytime we give you back a record, it's something that we got from the server. We're trying to be as transparent about that as possible and not do any caching shenanigans here. We're always just giving you what the server gave us.

In terms of efficiency though, you might remember from the save policies that we're only going to save the values that have changed on the records. So, we try and be smart about what we send over the wire. When we're sending up assets, those can be potentially really big blobs of data, and that data is transferred efficiently. So, if the server already has a copy of a file, we won't re-upload it. If we already have that file locally, we don't need to re-download it.

So, in summary, we're really excited about CloudKit here. We've built a lot of really great features and it's something that we actually use at Apple. We've built iCloud Drive, we've built iCloud Photos on top of this, and we want you guys to start using the same technology that we're using at Apple. I'm really excited to see what's going to happen. I can't wait to see some apps that use CloudKit, and good luck with using CloudKit.

So, if you need any, if you need to contact anyone, Dave DeLong is our Evangelist. He's the guy with the bowtie. If you need documentation, it's on developer.apple.com. We had a Introducing CloudKit session earlier in the week. There's a couple more related sessions. Thank you.