Good afternoon, everyone. Welcome to our session, "Continuous Integration with Xcode 6". Thanks for being here this afternoon. My name is Brent Shank. I'm a software engineer on the Xcode team, and today we're going to talk about my favorite topic, which is, of course, continuous integration. So just a show of hands, how many people are using some kind of continuous integration system out there? All right. Awesome! Okay, next question. How many people are using Xcode Server? Okay. Okay. Good. Well, we hope next year, when we ask that question, it's going to be everyone. So if you're not familiar with continuous integration, we're going to teach you all about it today.

We're going to talk to you about our continuous integration product, which we call Xcode Server. We're going to tell you about all the great new things we built into Xcode Server. You probably want to see that slide. So we have lots of great new features in Xcode Server, in Xcode 6, and we're going to give you lots of demos, because it really shows well on the screen. So, first, if you're not familiar with continuous integration, what is it all about? Well, fundamentally it's all about improving software quality, and there are a lot of ways it does that.

A continuous integration system that catches problems quickly and automatically. So you don't have to worry that when you make code changes at the end of the day and you leave and you introduce the build error or warning, you don't have to worry about that not being caught until the next morning, because a system like this is going to catch it for you quickly and automatically, and for me that really enhances collaboration because you don't have to worry about blocking your team members, and let's face it, when you're writing code, you're taking a lot of risks. You know, you're doing interesting and creative things.

You're going to introduce problems and a system like this helps catch those problems quickly. Another key thing is that a system like this helps broaden your test coverage. Because you have your code running on a server, the server can support lots of different testing configurations that would be a big pain to do on a client.

So another great thing about a server product that's building your code is that it can gather histories. So build history and test history for your product over time, and this is actually really interesting. Because at the beginning of a product development cycle, as I'm sure you know, there's a lot of change. There's a lot of churn, and you're probably introducing a lot of issues.

You're introducing warnings and static analysis issues, and you probably don't have a whole lot of tests at the beginning. Towards the end of that cycle, you don't want to have any issues, and you want a lot of tests, and a system like this will help gather that history and show it to you.

So it's a great indication of where you are kind of in your product development cycle, and of course a system like this makes it really easy for you to distribute builds to your team. So hopefully you have testers. I hope everyone does, and if you do, a system like this makes it easy to distribute those kinds of builds.

So terminology. Bear with me here. So there are three terms that we're going to talk about over and over again in relation to Xcode Server. Term number one is a scheme. So I won't quiz anyone, but I'm sure if you've used Xcode for a while, you've seen this term, and it's really simple.

All a scheme is, it's just a recipe for building your project. It contains the targets that you want built, and the target in turn contains the source files and the resources and everything else that make up your product. So why is this important for Xcode Server? It's important because a scheme needs to be shared.

This is not a big deal. Xcode makes it really easy to share schemes, but when you first create a scheme, or when Xcode creates a scheme for you, that scheme isn't shared. We'll show you how to do it. It's not a big deal, just something to keep in mind.

So term number two is bot. And this is our friend the bot, and it's just a thing that we - the word we use to refer to the thing that builds your software on a schedule. And when the bot runs, we say that it integrates. It makes an integration. So these are the three key terms: scheme, bot and integration.

So let's take a look, in a little more depth, at our friend the bot. So what defines a bot? So fundamentally there are two key pieces of information. One is the project you want the bot to build. That's pretty basic. The second is the SCM information, or the source control information. So where does your project live? So if your project lives in a Git repository or a Subversion repository and if Xcode can check out your project, then you can create a bot for your project.

That's really simple. The next part about a bot is the schedule. So when do you want the bot to integrate? There are a lot of possible options here. You can choose to schedule your bot periodically. So, every hour or once a day or even once a week if you want. There's also on-commit bots, and that's a very popular option because it means that when ever anyone commits source control to your repository, the bot will fire off an integration, and of course there's also the manual scheduling too.

So, how to build it. So this is where the shared scheme comes into place. The shared scheme tells the bot how to build your project, and you can also configure three other options. So do you want the bot to run static analysis? Do you want it to run tests, and do you want it to build archives? Finally, a bot wouldn't be a whole lot of good if it didn't tell you when something went wrong. So we have a great system for notifications, and you can set these up to fire before a bot integrates and after a bot integrates so that you and your team know the results of each integration.

So like I said, we call our continuous integration product, Xcode Server, and there are two parts of Xcode Server, actually. There's the Xcode part, the part that lives in the Xcode application, and then there's the part that lives in the OS X Server application. This is important to know.

If you try to set this up with just Xcode, you're not going to get very far. You also need to get OS X Server. It's easy to get. There's a new preview of that available on the developer website this week, and when it gets released to the public, it's also available on the App Store. So there are two parts to it.

Some of its features is that it integrates very easily with the Apple Developer Program, and this is really important because, in order to run any of your code on devices, we need to do all their provisioning and certificate stuff, and that needs to know your Apple Developer information in order to do all of that. That's part of the product. We make it really easy to get started with that, and of course Xcode Server builds any iOS and Mac projects. If Xcode can build it, Xcode Server can build it.

And one of the really great features about Xcode Server is that it can run your tests on multiple devices and simulators. So think about it. Before we had this product, if you wanted to test on a couple iPhones and a couple iPads and a couple iPod Touches, you would have to plug each one of these into your development system separately and run the tests on them, and it was a big pain.

But now you can plug all those devices into an Xcode Server. You can configure your bot to run tests on all those devices, and it's great. You have this really broad test coverage I was talking about before. Of course, Xcode Server can produce IPA and PKG files so you can more easily distribute these products, and there's great integration in Xcode. It looks pretty good too.

So, like I said, OS X Server is the other part of this equation. Setup is really easy. It's got great features, like hosted repositories, and this is where you configure the devices that you want to make available for testing your software. We'll show you a demo of that in just a minute.

So last year in Xcode 5, we introduced Xcode Server for the first time, and we were really, really proud of its feature set. When we took a look at the product and we took a look at what features we wanted to build in this release and future releases, it was pretty clear to us that there were parts of the foundation that we really needed to improve.

So we're happy to say we have a shiny, new foundation for Xcode Server and Xcode 6. It's much better in all ways. It's faster. It's more reliable. It's super scalable, and it's allowed us to add support for all these great features like support for Swift, support for performance testing, a new feature we call issue tracking and probably the most requested feature from the first version was triggers. [Applause] Thank you. Thank you. Just wait. Just wait.

It's like I said, we have a great new foundation. Super reliable, and it's really scalable too. So I decided to do a little test a few weeks ago. I took an old Mac Pro that was sitting in my office. It's a five-year-old Mac Pro. It wasn't doing anything. It was a nice footrest for me, and I set up Xcode Server on Xcode 6 on it, and I decided to add a few bots.

So I started with a small app. I started with the WWDC app that you have today in your devices. I created a bot for that. That went pretty well. Then I decided to just keep going. So I added a bot for Xcode. That went really well. I added a bot for Game Center.

That went pretty well. Again, these are bots all running on this one old server. I added a bot for the Maps project, for UIKit, and that was all going well so I threw WebKit in there too. This is a very old piece of hardware. It's building bots for all these projects, both small, medium and really big, and it's doing just great. So this is something that just wasn't possible with the first version of Xcode Server.

Triggers. Okay, so we heard you. You asked for this loud and clear, and we have a really great story for triggers. So we support triggers, both before an integration starts and after an integration runs, and there are lots of interesting reasons for supporting both of these. There are a number of conditions you can specify that determine when a trigger runs, and there are two types of triggers. There's a shell script trigger and an email trigger.

So what's really cool about the shell script triggers is that you can do anything you want to do. Right? It's just a shell script, and we provide a number of environment variables that you can leverage on those shell scripts, and we'll see a few of those in the demo in just a little bit. The email reports that we provide, you can configure them to show you just the minimum amount of information, which is just like the build errors and warnings, or you can customize those and make them really rich and add all the issue tracking that we do.

So speaking of issue tracking, what is this? Well, what's an issue? An issue is either a build warning, a build error, a static analysis issue or a test failure, and issues are important. You don't want to ship software with issues, and in fact we have a policy, a really good policy, on the Xcode team, which is, if there's a build warning or static analysis issue in the source space, that's a P1, and that's what we work on. It's important. We can't ship with these things, and you shouldn't ship with them either, and by having a great issue tracking service in Xcode Server, it makes it really easy to know when those issues get introduced.

And we can do really cool things, like we can track that issue all throughout its lifecycle. So if the issue still exists, we mark that as an unresolved issue, and then when an issue finally gets resolved, we service that in the reports too, and because Xcode Server tracks all of this history over time, it's really great, easy to see when an issue was introduced and even who introduced it, where we can be pretty smart about that. So there's a lot of stuff there, but to really bring the point home, I think it's better to show it in a demo. So I'd like to bring up Matt number one, who's going to talk about all these things.

All right. Thank you so much, Brent and as you heard, we've got a lot of really great new features in Xcode Server and Xcode 6. But in case you've never used it before, we're going to start from the very beginning, which begins in Server. So we have OS X Server installed on this machine, which you can get from developer.apple.com.

You need the preview to work with Xcode 6 beta that you all got this week, and we've already set it up on this machine. You can see our server is called Bobert, and it's ready to start integrating. So what we're going to do is set up Xcode Server.

So we select Xcode on the left-hand side in the Services list, and the first thing that we ask you to do is choose which version of Xcode you're going to be using. So you do need Xcode installed on your server machine in order for Xcode Server to work, because that's what actually does all the building.

But we give you the choice of which Xcode you use in case you want to try out a new version of Xcode or in case you need to support multiple versions of the iOS SDK or something like that. So we're going to select the Xcode that you all have a copy of right now, and when I choose that, it's going to start setting up our database. It's going to generate client SSL certificates, and it's going to start all of our processes so that we're ready to start doing integrations with Xcode Server, and after a few seconds, that sets up and it's ready to go.

Now, if you're doing Mac development only, that's all you need to do. You can stop right there. All set. Good to go. But because we're going to be testing on devices and everything that we do on iOS needs to be code-signed, we also need a way for Xcode Server to be able to do that code signing, and for that we're going to join an Apple Developer connection team. So to do that, next to the developer team's label there's an Add button. So we'll choose that, and then you sign in with your Apple ID.

Once you've signed in, that's going to give you the option to become a member of your developer team. So when I click Add, what it's actually going to do is add your server as a new person on your Apple Developer team. It gets its own set of certificates, private keys, signing identities, and it's also going to download a copy of all the provisioning profiles that work with the apps that you have registered for app IDs on your developer team. And this can take a few seconds because it does go out and make a bunch of certificates, sign-in requests and then pull those all in. But once it finishes, you'll see that you are now a member of the developer team that you signed up for.

Now, since we're going to test on devices, I do have an iPad Mini right here. You can all see it. It's real, and what I'm going to do is plug it in right into my server machine, and after just a second, in the Devices list that shows up.

So you can see it's ready to use, and it shows which teams it's enabled for. If this device had never been used before, you also get an option right here to choose Use for Development, and that goes out to the portal and adds it to your developer team. So, really easy to do.

From this screen you also have the option of configuring who can use your server. So by default, I'll click this Edit button here. You can see that bots can be created by anyone who can log in, and anyone can view those bots once they've been created, and we want to lock it down just a little bit more. So I'm going to choose only some users can create bots. Administrators is fine for me, but I'll turn view-only access for everyone back on.

So that's pretty good for me. I think that's a good set of access controls for my server. From OS X Server you can also set up hosted repositories. If you'd like Xcode Server to host your repositories for you, you can do that here. You can also set that up directly from within Xcode.

But that's all we need from Server for right now, so I'll close Server, and we'll head back over into Xcode itself. So once you've launched Xcode, the first thing you're going to need to do is set up an account for your server, and that happens in the preferences window on the Accounts tab.

You can click the plus button in the bottom left-hand side and choose Add Server, and you can see my server, Bobert, is going to show up right here. So I'll select Bobert, choose Next and log in with my username and super-secret password, and there we go. All set. One of the new features that we've added in Xcode 6, which we think is pretty useful, is down at the bottom of the Accounts window, we do show you which version of OS X, OS X Server and Xcode your server machine is using.

So if you've got multiple servers and something starts going a little funky, you can look right here and it will tell you what your server machine is running. So that's not a bad place to start when you're trying to diagnose a problem. So that's all set, and we're going to close the accounts window, and we're going to open our project, and it's a project called CoffeeBoard.

So on our team, from time to time we make mistakes. We're only human. I don't make that many mistakes, but some people on our team do, and when that happens, if a mistake happens that causes someone to spend more than half a day fixing a problem, we think retribution is in order.

So we've written an app that keeps track of when that happens and assigns debts of coffee and ice cream to the members of our team, and we call it CoffeeBoard. So that's this app right here. The other Matt's going to show it to you a little bit later, but you can see it's got a lot of tests all set up. It is written in Swift, which is pretty cool. So we're going to create a bot for that project. I'll switch over into the report navigator, and I'll choose from the Product menu, Create Bot.

Now, the first choice you get is which scheme your project is going to integrate with. I'll use the iOS one. You get to choose what the name of your project - what the name of your bot is going to be. So I'll choose CoffeeBoard. You also choose which server your bot gets created on and whether or not it integrates immediately after creation. So I'll leave that set.

Now, one of the issues that some people ran into in the first iteration of Xcode Server was that you would go through the process of creating a bot, it would all look like it was all fine and then your bot would go to run its first integration, and it wouldn't be able to check out your project, which wasn't fun because that wasn't all that easy to fix. So one of the new features we've added in Xcode 6 is the concept of source control preflighting.

So what's going to happen when I click Next is it's going to scan your entire workspace, find all of the source control working copies that it knows about, gather up the credentials that Xcode knows about that goes with those repositories, package them up in a nice, secure, little bundle and send them up to the server so that the server can determine right now if there's going to be a problem when it's time to check out the repository. So I'll click Next. Yeah. It's a cool feature.

So I'll click Next. It tells us that all of the repositories that it found. The CoffeeBoard repository it couldn't authenticate, and that's because I was recently a victim of identity theft so I changed my password. So what I can do now is correct this issue right away. So I'll click the Change button. Enter my username and my new password.

Click Next, and this time it says my credentials are verified. So it knows that when it goes to check out my bot for the first time, it's going to work, which is a really comforting feeling. So now we have an option to choose how frequently we want our bot to run, which is great.

By default it's going to be a periodic bot that runs hourly on the hour, but because we want this to be a little bit more live for our demo, we're going to choose On-Commit so that every time we make a change to the bot, it's going to kick off a new integration.

You also can choose which actions your bot will run, when it integrates and also how frequently it cleans. By default, we always do incremental builds, but because I really like squeaky clean bots, I'm going to choose Always. Now, depending on which scheme you select, if your scheme can build targets that have products that run on iOS or on Mac, you have a choice of where your targets will run for testing. So since we're doing an iOS target, we are going to use iOS. But we do have an option of specifically which devices our test will run on. So I'll drill down into that and choose the iPad that I connected earlier on.

And then you get to the option to configure your triggers, and I'm going to talk more about that in just a minute. But we will show you that by default, we create a trigger that will send an email notification when something goes wrong, and you can choose if you'd like that email to be sent to all of your committers, and you can also build another list of email addresses if you'd like to send it to more people.

You can choose what goes into that email message, whether you want Issue and analysis issue summaries, the Commit messages, if you want to hide those because they might contain sensitive information. Or you can turn both off if you just want people to know that an integration is completed, and you also can choose when that email will get sent, whether it's on success, test failures, build errors or any combination of the three.

So we'll click Create. Our bot will get created, and after a few seconds it will start integrating, and you can see another great new feature we have in Xcode 6 is live progress information right down to the source file that's being compiled by your bot. So you can really keep close tabs on exactly what's happening while it's integrating, which is really cool. So this will take a few seconds to compile, and once that's done it will start running tests on the iPad that we have connected right here.

I know you can't see it, but it's unlocked the iPad. The app is launching, and the tests have run, and you can see we have the results for our integration that has completed. You can see that because multiple people have committed to the repository for this integration, we have a nice breakdown of the commits. And we have these nice little circles which indicate, relative to each committer, how much of that integration they own, so to speak.

It's the number of files that have been changed, and if I click on a committer, I can see which commits were made by those committers, which is pretty cool. But because a single integration isn't that exciting, we're going to switch over to our other machine which has a bot that's been baking for a few days.

So you can see this bot has been running for quite a while. I've gotten taller, if you've noticed. This bot has been running for quite a while, and it's got a much more interesting history, which is a great way for us to show off another new feature that we're really excited about, which is our issue tracking feature.

So if I select integration #4, you can see that it shows we have no errors, one warning, one analysis issue and six tests ran successfully. But these warnings and issues are new in this integration, and you can see that Xcode Server's smart enough to notice that, because it puts a "+1" under each of those circles so that we know that these were just recently introduced, and you can see here in the issue summary it also says New Issues: 1 warning, 1 analysis issue. Well, I came along and fixed that warning.

So in integration #5, you can see that it says Resolved Issues, Warning: Unused variable. And it knows, based on the file location of these issues and the message that the issues generated, that this is exactly the same issue, and we can safely say that this issue has been resolved. So that's a really great way to keep an eye on the health of your source space. Really, really cool.

We also have here logs just like you would see for local projects integrated in Xcode, and you can even drill down into them and see the same kind of view that you would expect from looking at these from a local product, which is really cool. But now I want to show you probably what was the most requested feature from the first version of Xcode Server, which is triggers. Triggers are really cool, and we're really excited to show you what they can do. So we'll start by talking about how we work with them on our team. We have a friend.

His name is Jeeves, and he sits in our messages chat room all day long answering useful questions for us like, what's the weather like? For instance, you can see in Cupertino it's 77 degrees and partly cloudy. Of course, here in the beautiful city of San Francisco it is a good 12 degrees colder, which is to be expected.

But Jeeves is really helpful. We really like having him around, and it's very helpful to our team to be able to get that kind of information on the fly, but we want to supercharge this with information from Xcode Server, and the best way to do that is with triggers. So we'll switch over into Xcode, and I'll put that up so you can see it side-by-side.

I'm going to edit the bot, and most of these settings I'm going to leave the same, because we're not really changing much about the bot, and you'll see as I click Next, it will revalidate those credentials that I set up earlier to make sure nothing is broken in the meantime.

We'll leave all of that information the same, but when we get to the triggers screen, I'll choose Add Trigger and select Run Script. Now, I've already got some triggers written down here. This is a Ruby script. They're shell scripts by default, but you can put a little shebang at the front here and run using whatever interpreter you'd like, and all this does is it hits an HTTP endpoint that we expose on Jeeves, and it uses things like the integration number and the bot name to build a message that will show in our chat room.

So I'll select all of that and paste it into a trigger, and I'm going to do the same for a post-integration trigger, and these are just a little bit different. You can see I actually use the failure count, error count and warning count so that we can provide even more information in our chat room, and for post-integration triggers you get a choice of under what conditions your trigger will run, whether it's only on success, only when something goes wrong or any combination of those. So I've created the triggers. I'll choose Update, and that will kick off a new integration of our bot.

So we'll pull up the status here, and you can see we get a message in our chat room that integration 14 on CoffeeBoard is now starting, and that message ran after the source code checkout happened. So that way if you'd like to, for instance, inject versioning information into your built product, you can do that right from a pre-integration trigger, which we think is really cool.

You can see the integration completed, and we got yet another information message in our chat room, and because we wanted to take it just one little step further, we added another little feature where we can say, "Jeeves - oh, not weather - Jeeves, integrate CoffeeBoard." And you can see it's kicking off another integration in Xcode, which we think is pretty cool.

So that's just one of the many things that you can do with triggers. We're sure you have ideas about how you want to integrate them into your own workflow. We have a lab right after this session. Come see us. Come talk to us, and we can help you make sure that you can make triggers do whatever you would like. Thanks so much. Brent, back to you.

So, of course, the big news this week: Swift support. So how does Xcode Server deal with Swift? Well, it deals with it just like it deals with Objective-C. It does complete feature parity with Objective-C projects. So whether you have a pure Swift project, whether you have a mixed Swift and Objective-C project, Xcode Server can integrate bots of those projects no problem. All the features work the same.

This is true of testing support. So whether you write unit tests in Objective-C or you write them in Swift, all the testing support just works. The same goes for performance testing, and Xcode Server has always had great Assistant Editor integration in Xcode. It still does, and of course that works just as well for Swift code as it does for Objective-C code.

Let's talk about performance testing. This is another great new thing we released this week, and if you haven't heard any of the talks yet, I highly recommend you go back and watch them. There was a session this morning about testing, and performance testing is pretty great, and we've made it really simple.

The great thing about it, in the context of Xcode Server, is that when you write a performance test, you can have Xcode Server run that test on multiple devices. So this is really great. So you may be surprised to find that your code runs a lot slower on one device or on one operating system than you expected it to, and with Server, it makes it really easy to see this.

The key thing about performance testing is that they work great out of the box. You can just write your code in a measure block. You don't have to set a baseline, but if you set a baseline, you get a lot more value out of them, and the thing with baselines is you have to set them on a per test, per device basis. So that looks something like this. We have this report. You can see all the tests here.

You can see I have a couple of different devices that these tests are running on, and in order to get more value out of these, I need to set a baseline for each test, for each device, and this is the UI that does that. We'll see more of that in a minute.

So in case this is new for you, this is what a performance test looks like. It's really simple. This is what it looks like in Objective-C. Any code in the measure block here is going to get executed 10 times. It's that easy, and in Swift we have also a measure block.

It's actually a closure, but it's still called measure block. That's the API. It's the same thing. So to show you all this in action, I'm going to bring up my officemate, Matt #2, who has the loudest keyboard I have ever heard, to show us a demo. Come on, Matt.

Thank you, Brent. So I'm really excited to be here today to share the new Xcode Server and Xcode 6 with you guys, with support for Swift, our brand-new programming language, and the awesome new performance testing features. All right. So we have the iOS simulator open with our CoffeeBoard application. I've got a list here with me, Matt, Brent and some of our friends and our balances. So it looks like I've got a negative balance. I guess I owe some people some coffee.

Here we see a list with exactly what I owe to each person or what they owe to me. I think I remember Brent causing the issue earlier in the week that cost me some time. So I forgot to add that transaction. I can do that right here, and it will get added to the list.

Now, I'm worried that as we keep adding transactions - we have some back-and-forth on our team - I'm worried that as this grows, that condensing these down into a nice list for the UI is going to get more time-consuming. So I want to test the performance of that and make sure it stays good as I keep adding to the application. And the performance testing feature in Xcode 6 makes it really easy to do that. So I'm going to quit the simulator and switch back to Xcode.

So here I've got an XCTestCase subclass with a bunch of tests making sure that the function I have for condensing down these transactions works correctly, but I think we all know there's more to a great application than just working correctly. Customers expect great performance. They want your apps to be fast and get out of their way, and a performance testing feature will help you make sure that that's the case. So I'm going to add a new performance test to our application, and then I'm going to go ahead and commit that to my repository so that Xcode Server can start integrating that.

We'll push that up, and while it's doing that, I'll take a closer look at what this test is actually doing. So it's just like any other unit test. It's just a method on your XCTestCase subclass. It starts with a word test. Takes no arguments. Returns no value. The first thing we do is create a list of transactions that we're going to merge, and then I don't want to waste any time measuring that, so I do it outside the measure block, and that's the next step here. We call this measureBlock function, and pass it at a closure, and whatever code's inside there, we'll run multiple times and measure the time, each iteration, and average those together and report that to you.

So I think our integration has finished completing. So I'm going to go take a look at that now. We have this lovely test report that's new in Xcode 6. It breaks down exactly what happened in each test. I can see our performance test is here on the list, and we've got this new column for time that reports the average time that each iteration of that test took.

It looks like this took .15 seconds on average. That seems pretty good. If I click that time, then I can see a breakdown by device. I've only got the one device plugged in right now, but I could have several, and they would each have their own time measurements.

Now, Xcode's telling me something here. It's telling me that we have no baseline for the time performance metric, and that's because when you first add a performance test, Xcode's not going to make any assumptions about what's good performance and what's not, because every test is unique, and every device is unique, and we don't want to make any presumptions. So we'll report the time for you, but we won't try to failure test based on how it performs. If you have any assertions in your test, we'll fail based on that, but that's it.

But when I keep working on this application, I want to know if this performance gets worse. I want Xcode Server to notify me when that happens, by failing my test, and so to do that I can set a baseline for this test. And to do that I click the time for this particular device, and I can hit the Set Baseline button.

What happens when I click that button is that Xcode updates its project file with entries to say for this particular test and for this particular kind of device, like iPhone 5S, iPhone 4S, iPad, that this is the performance standard, that if we significantly regress and get slower, then this number, that should fail our test.

The interesting thing about this getting added to the project files is that they get committed with source control. So I'm going to go ahead and do that now because I want our bot to be able to use them. You can see these are just property lists. You could edit them by hand if you want, but we've got a nice UI on that test report for managing those baselines.

So I'm going to push that up and let Xcode Server start integrating that, and while it's doing that, we can look at the test report here. We have a lot of filtering options here. Like, we can show only failed tests, which I don't have any of, which is great, and if we want, we can just focus on the performance test, like the one I just added.

If there's a particular test I'm interested in looking at, I can search by name and filter to just tests that match that name. Or if I'm interested in what the code is for a specific test, I can click this little button here and jump right to the code for that test.

Now let's see what's happening with their bot. We can see exactly which test it's running and there it goes, and now let's see what happens when a test runs when we have a baseline set. So now I can see that this time our test ran a little bit worse, a little bit longer than the baseline that I set, but our test still passes because it's within a acceptable range. We understand that there's going to be fluctuations between each test run, and we don't want to fail overzealously.

So let's see what happens when I actually do introduce a significant regression. So I'm going to switch back to the code that handles merging these transactions. Now, what I have here is clearly working. My tests are all passing. The performance tests are succeeding, but I'm worried that later on when I come back to look at this, I'm not really going to understand what this was doing.

I think I have an idea for how I could rewrite this function in a way that makes more sense to me that I'd understand better later. Now, I'm just going to be a little reckless here and just commit it right away because I know Xcode Server has my back, and if I break the application or if make it perform significantly worse, Xcode Server is going to tell me and my entire team.

Now I'll push that up and let that start integrating. Now, I'm really excited about this performance testing feature, but they can make your test workflow slower. So Xcode Server gives you a great way to keep your speedy workflow but still take advantage of these performance tests. Now, they'll run slower because we're running several iterations of what is ideally significant work. Otherwise, it wouldn't need testing. So you can set up one scheme that Xcode Server runs that runs all of your tests, including your performance tests and another one to run locally that just runs your quicker ones. So you get the best of both worlds.

I think I saw Jeeves tell me that my integration was done. Oh no, it looks like I have some failing tests. I guess that new merge function wasn't as good as the old one. Looks like it is performing 137% worse. That's pretty bad. I don't want to ship that when we ship our application, but I'm a little busy to fix it right now. I'm busy giving a WWDC presentation. So I'm going to trust that our notifications let my team know and that someone else will take care of fixing that.

All right. I'm going to go ahead and hide Xcode, and I have one more thing that I want to show you guys. So last year when we introduced Xcode Server, we gave a great way to give high visibility to what all of your bots were doing inside your office, and we called it Big Screen, and we have a new, beautifully updated version of Big Screen that I'd like to show you now.

So this is the new Big Screen. Along the left side, you can see we have a list of bots. These are all the bots configured on our server. We got a little punny on our team, and we can see that below each bot name, we have icons indicating what kinds of issues we might have on those bots. So CoffeeBoard clearly has test failures and a performance regression.

Oh, but it looks like someone started a new integration on there. I hope somebody came along and fixed my mistake. Now, when we focus on each bot, we show the counts of errors, warnings, analysis issues, tests or test failures if some are failing, and we even show the change in those numbers form the previous integration.

Along the bottom you can see the list of committers, and you can see even - if we have issues that we've attributed to them, how many they might have. Looks like CoffeeBoard finished integrating. Our tests are now passing, which is great. It looks like somebody came along and fixed my mistake. It looks like it was the other Matt. So applause for him.

That's the new Big Screen, and that's all I've got to show you. I'll hand it back to Brent. Thank you everyone.

Thanks, Matt. Those are great demos by two great guys, both with the same name. So in summary, we've got a great new foundation in this version of Xcode Server. Got tons of bug fixes and other improvements. Please keep sending us your feedback. We are listening, and we've got great, great new features, like in-flight progress notification.

So you can see exactly what your bot is doing when it's integrating. We have issue tracking. So Xcode Server can tell you when an issue was introduced, if it still exists, who introduced the issue and, finally, when it was resolved. This gives you a great sense of the overall health of your project.

We have a great new test result UI, including support for performance testing. We have a really easy way to set baselines for tests for individual devices, and of course we have awesome support for triggers. So we've got a lab right after this. Here's more information, if you need it. Come to the lab, and maybe you can try to out-trigger us and write something even more awesome than our friend, Jeeves, and so that's it. Thank you. Have a great week.