Good afternoon. My name is Garth Cummings. I'm with Apple's Developer Tech Support Group. I'd like to welcome you to session 508, Customizing Mac OS X with Open Source. And here to kick off the festivities is open source engineer Kevin Van Vechten. So today I'm going to be talking about how to customize Mac OS X using the open source software that Apple provides on its website. And this source is part of Darwin, which I'll describe a little bit in a minute. And the emphasis of this talk is really going to be how to make these sources useful to you to solve problems that you might be interested in solving.

So, as an overview, we're going to talk about what open source is, in case any of you aren't familiar with the concept of open source, and we'll move quickly to talking a little bit about what Darwin is, and Darwin is Apple's open source project. Then we'll go over briefly how to add some common open source projects to your Mac OS X system, but the heart of the talk will be how to customize Mac OS X using the open source that Apple provides.

And the second half of the session will be hands-on, which means if you've downloaded the sample code for the session and have it on your PowerBooks, you can follow along. And what I'll be doing is using some tools that Apple provides in order to build the sources and install them on your systems.

So first, what is open source? Well, open source is a very loose term, but basically what it means is there are programs whose source code is available on the internet that is free for you to download, inspect, modify, and deploy. And then in the spirit of open source is to take your modifications and contribute them back to whatever group published the sources in the first place so that they can benefit from the enhancements and so can all the other users.

And by working collaboratively in this fashion, the products really start to take flight. Much of Mac OS X is open source, specifically everything at the Unix layer. And more information about open source in general can be found at the OpenSource. OpenSource Initiative's website, and that's this URL that you see here, www.opensource.org.

Now, Darwin is Apple's open source initiative. It is the open source portion of Mac OS X. It includes the kernel, the I/O Kit drivers and families, and the BSD commands and libraries. You can see on this diagram that Darwin provides the base of Mac OS X, upon which Carbon and Cocoa and Aqua sit, but those technologies are proprietary to Apple. It's only the Darwin technology that's open source.

So, how does open source help you? Well, one of the biggest examples of it helping you developers is as reference. If you're curious how the file system is implemented or how certain libraries on the system are implemented, you can actually download a lot of these sources and read through them. And, you know, as I'm sure you're all familiar, documentation sometimes doesn't give you the whole picture. And in certain cases, documentation is best done by reading the source and seeing what's actually going on.

It also helps with debugging. It's possible to take a lot of these sources and build a fully symboled version of a library so that when you're doing your step-by-step debugging in GDB, you can actually step into the functions that you're calling and inspect what the variables are inside the library.

Um... It's also useful for customization, which is going to be the emphasis of this talk. If there's a particular default flag that's very annoying in your enterprise setup, or if there's a security fix that you need immediately and Apple doesn't have a software update for it yet, perhaps you could download the sources from Apple's site, apply that patch yourself, reinstall, and be ready to go. Also, it's useful for research. There's a lot of work going on. For example, trusted BSD work is being ported over to Darwin. It includes things like mandatory access controls. And there are other high-performance computing research applications that can use the open source that Apple provides.

So for our hands-on portion of this talk, what we'll be doing as an example is adding Bonjour support to the BSD Unix talk utility. So basically we'll be making an iChat of the 1970s. And to do this, we'll use the Darwin build scripts, which basically set up a lot of the environment that's necessary for building a Darwin project.

For the remainder of the talk, I'm going to be talking about what that environment is, why it's so important, and then when we actually do the hands-on portion, we'll use the scripts that take care of all the nitty-gritty details for us. So adding new software to Mac OS X in general.

There's kind of a basic open source lifecycle, which is pretty common to almost any project that you're going to download and install on Mac OS X. So you can see here that there are five overall steps, the first of which is pretty obvious. You need to download the source. So in this example, it's using the cURL command line utility to download the source code to Lua. Lua is a very lightweight scripting language, which many of you probably have never heard of. It happens to be the scripting language used by World of Warcraft, so you might wanna check that out. But we can download it.

And the next step is to extract it. Usually it's a archive, a single tar.gz or tar.bz format. And once you've extracted it, you have a whole tree of all the sources on your system. The next step is actually a pretty magical step, and that's to configure the sources.

And what you usually do is just change into the directory and run the configure script, and it goes and figures out a lot of the properties of your system, which libraries you have installed, which header files are available, whether it's a big endian or little endian system. As you know, there's Darwin on both PowerPC and Intel, so we'll be talking a little bit about that.

And once everything's configured, you can go ahead and compile. Usually there's a top level make file where you would do make all, and it would go through and compile all of the portions of the project. And once that's done, you could say make install, and it would install it on your system. Well, there are a few things in this basic lifecycle that you actually probably want to do a little bit differently when installing on Mac OS X, so I'm going to talk about that in detail now.

Sometimes in order to get things to work properly on Mac OS X, you need to patch the software. So Linux's driver model, for example, is quite a bit different than I/O Kit. That's some major areas of the software needing to be changed. One of the systems that actually has a large collection of patches and makes it easy to grab software is Fink.

Fink is hosted at SourceForge. It's based on the Debian packages system, which is used in Debian Linux. And they have over 5,000 programs which are ready to go on Mac OS X. And one of the major advantages of Fink is it offers pre-built software. So you can download a package that's already been built, install the binary, and run all of your common open source programs.

Another popular system is Darwin Ports, which is hosted at opendarwin.org. Darwin Ports is similar in nature to FreeBSD Ports. It's a completely different implementation, but the spirit is largely the same. They have over 2,500 programs, which you can install. Unlike Fink, Darwin Ports really emphasizes building these sources from scratch, which is useful sometimes if you're looking to customize them.

But one of the things that both of these products do is make these open source projects work well with Mac OS X. And one of the keys to working well with Mac OS X is to avoid installing things into the system directories. And the main reason for this is Apple uses Software Update. And Software Update doesn't care what you've installed on your system. It wants to install what it wants to install. And it'll happily overwrite any files which you've modified.

So staying out of the slash system directory and out of the slash user directory is a good way to stay out of Software Update's way. Traditionally, user local has been available to you to install into. That's perfectly fine. Apple will never install into there on one of its CD updates or software updates.

But if you're building the Darwin sources themselves, if you're building actual components of the Mac OS X system, there are intermediate files that get placed into user local. So it's really best to stay out of user local as well. If you want to be completely out of Apple's way. And for this reason, Darwin Ports has chosen opt local. And Fink has chosen SW as directories that are completely independent of the base system. And that's where they install all of their libraries and all of their programs.

So if you're wondering how you can move software to something other than slash user or slash user local, usually this is done in the configure phase of the open source software lifecycle. And quite often a configure script will have a dash dash prefix argument, where in this example we're setting the prefix to slash my stuff. And instead of going into user local bin, this project would probably install into slash my stuff bin.

Another good practice is to actually install into an intermediate location before copying it onto the base system. Because when you just do a make install without any other arguments, it'll just install onto your system and you have no idea which libraries and which binaries were installed. However, if you a lot of times specify a destr argument, in this case we're setting it to temp new project, what we can do here is install into this intermediate location, look in the intermediate location to see what will end up being installed, see if there's any conflicts that we're worried about, and once everything's okay and we're sure of that, we can then copy that into the base system.

And it also gives us a chance to keep a record of how we've modified our system. So I think this is a very important thing to keep in mind, is to inspect the results of anything you build before installing it on your system so you have a good idea. of exactly how your system's being modified.

But as I said earlier, the bulk of this talk is about customizing the things that are in Mac OS X themselves using the open source Apple provides. And there is some risk doing this, right? If you modify the kernel, for example, there's a chance that the system won't boot if there's a bug in your modification. So that's a fairly high risk change. You can make lesser risky changes by changing libraries and frameworks, although the exposure of any change to a library framework is fairly large. Any incompatibility introduced there will affect a wide variety of applications or command line tools.

Plugins are a little bit less risky because it will only affect the application that happens to use that plugin, not a wide variety on the system. And then individual command line tools are generally pretty safe to update because they're at the edge of the dependency tree. They're only used by themselves, and it's fairly isolated from impacting the rest of the system, though there are exceptions to that.

If Perl doesn't work on your system, you might find a lot of the various scripts in an unhappy state, so it's best to be judicious about what tools you're modifying. And always make backups because if something doesn't go how you planned, you're going to want to restore from a backup of that tool in order to get your system back into a working state.

So, as I mentioned earlier, software update may overwrite things that you've modified, and there are a few best practices we can do to mitigate this risk. Libraries and frameworks may be incompatible, right? So your modifications could work fine today, but when software update comes out and updates a new application that has a different expectation of how that framework should work, that might be when you notice the incompatibility. So it's a good idea to keep your backup copies of the modified sources around for a long time, because it might work today, but that doesn't mean it'll work tomorrow. And you'll want to restore from backup to see if that was indeed the problem.

One of the practices that I do personally is to download the software update packages from Apple's website at www.apple.com/support and install them manually and not rely on software update to install them. And the reason for this is you can inspect which files the software update's going to install when you're installing manually.

So, using the installer application, you can choose the show files item from the file menu once you're installing the package. And you can see here that we have a long list of files, and the list of files in this instance is all the files in the user directory of the Mac OS X 10.4.1 update.

In our example, you'll recall we're modifying the Unix talk command, and the talk command lives in user bin talk. As you can see, it's not present in this list, so this update's probably going to coexist peacefully with our modification that we're going to do. There's also a command line method for listing all the files in a software update package, and that's that command at the bottom of the slide, the ls bomb, and then the path to the archive.bomb file inside the package. Bomb stands for bill of materials. It's a list of all the files that are included in the package, and it also includes information about what their default permissions should be and ownership.

So to minimize the risk of your modifications interfering with software update, you should also download the sources from Apple, because those are the actual sources that went into Mac OS X. If you were to grab a version of Perl from, say, SourceForge or some other site, there's really no guarantee that those sources are anything like what went into Mac OS X. There could have been a lot of modifications that were done by some third party.

But if you start from Apple's site, which is at this www.opensource.apple.com slash DarwinSource, or there's a mirror of it at opendarwin.org, which is a community-run site that does a lot of Darwin development, and that's at darwinsource.opendarwin.org. These sources are the actual sources that go into Mac OS X, so they're a good starting point for your modifications.

And then once you have those sources, you should really change as little as possible. You should add the specific feature that you're interested in, or the specific bug, and then you can fix and reinstall, because the smaller the changes you make, the more likely it will be compatible for a long time with Apple's software updates.

When building the software, there's a lot of different details which you need to keep in mind, some of which are completely unique to Apple. You've never seen anything like it in any other open source project, except for talks like this and some hints on the mailing list. You'd probably never know that it needs to be done. There are some obvious things that should be checked, though. For example, which system version and kernel version you're building on.

This will affect the outcome of the build because the tools that are available on each system of Mac OS X are going to vary. The system calls that are available from the kernel are going to vary. And if something like a configure script is doing auto detection of what features are on your system, if you're building on, say, a Panther system, but you want to install on a Tiger system, there's a good chance that it's not going to work because that configure script is going to pick up all the wrong features. It's going to be configuring itself for a Panther system, and that's not what you want. So one of the good things to do is to stay on the same general system version as what you want to install on.

The developer tools version is equally important. There are also a lot of environment variables, which I'll go over, that need to be set, and they will dramatically affect the outcome of the build. And there are also headers and libraries, which you probably won't be able to find by default on your system, and I'll tell you a little bit about how to get those.

So the system in kernel version can be inspected by using the SW_version tool. That'll print out the product version of Mac OS X, which in this case is 10.4 or 10.4.1 or 10.3.9. There's also a build version. In this case, it's 8A428. That is the build of Mac OS X that is on the Tiger DVDs. This build version is really the most precise number that you can find to identify which system software is running on your system. And the build number will change with every software update, with every major CD release. And it's really the most exact number that's available to you.

The uName tool, in this example I'm using uName-A, will print version information about the kernel that you're running. For Tiger, the kernel is 8.0.0, and this is a good way to figure out which kernel is on the system in case you might have modified the kernel and it doesn't match the overall system version.

As you know, Apple has shipped many different versions of the developer tools. There's GCC 295.2, which I believe was the original GCC version for Mac OS X. And as time has gone on, they've introduced GCC 3.1, GCC 3.3, GCC 3.5, which is an internal build that was used for certain parts of Tiger, specifically anything to do with 64-bit.

You don't need that. It actually has been rolled into GCC 4.0. But one of the things to keep in mind is different parts of the system happen to be built with different compiler versions. So GCC 4.0 won't build much of the open source that's available for Tiger. In fact, you need to go back to GCC 3.3 in order to build a lot of the Tiger sources that are available. You can look and see which version of GCC you're using by default by typing gcc-v. And in this example, you see it is using GCC4 by default, which is consistent with Tiger.

If you want to change which version is used by default, there's the GCC Select tool. And GCC Select will change the global default. In this example, we're using sudo to make this global change, and we're selecting GCC version 3.3. And 3.3 is what is needed to build most of the Tiger open source.

The environment variables are used by the Makefiles and Xcode projects that compile most of Apple's open source. And there are four very fundamental environment variables that almost always need to be set. The first one is source root. Source root should be set to the path of the sources which you are compiling.

So if you've downloaded a tar archive and extracted it onto your disk, whatever directory that extracted tar archive created, the full path to that should be specified in the source root variable. This lets the make file access files relative to the variable to find the sources that are supposed to be built.

Another directory that needs to be specified is the object root. And the object root is a purely temporary directory. It should be empty before you begin your build. And the object root is where all of the .o files and other intermediate files are going to be placed while this project is being built.

The following two environment variables, the symroot and the destroot, are where the actual products of your build are going to be placed. Both of them, again, should be a full path to an empty directory to start with. And the symroot is where all of the fully debugging-enabled versions will go, and that has all the symbols that are needed to do source-level debugging for the project using GDB. So you can actually do line-by-line debugging using whatever libraries and tools are installed into the symroot. And then the destroot is where the finished product goes, and that is basically what you'll end up installing on your production system once you have made the change.

There are some other very important environment variables as well, such as the Mac OS X deployment target. This variable interacts with the availability macros header file. And basically the purpose of it is to determine which APIs are safe to use on the version of Mac OS X which you're targeting. So if you're targeting version 10.4, which we will be today, then it will give you warnings if you're using any APIs that have been deprecated in Tiger. Perhaps they were perfectly fine to use in Panther, but they're no longer fully supported or won't be supported in future releases. So you'll get a warning for that.

It also conditionally defines a lot of symbols so that, for example, if you're on a Tiger machine, but compiling for Panther and something new is introduced in Tiger, you can use it to define a new version of Mac OS X. And you'll get a warning because you're using something that won't exist back on that system that you're targeting.

No matter which version of Tiger you're installing for, so whether it's 10.4 or 10.4.1 or any future software update, this deployment target should just be set to 10.4. It really only corresponds to the major release number. So for Panther, it should be 10.3. For Tiger, it should be 10.4. It doesn't need to be set for any of the software updates. It's really just a major number here. The same goes for the RC release variable.

This is something that's used by various make files in open source projects to see if they should configure themselves slightly differently, whether they're building for Panther or for Tiger. So you should set them to Panther or Tiger, depending on what you're interested in installing on. I guess in the future, we'll be setting that to Leopard at some point.

The uname release variable, again, is a major version number. So for Darwin, that was 7.0, and that was the kernel version in Panther. For Tiger, it will be 8.0, since that's the major kernel version in Tiger. And this is actually a very important variable to set. What it does is it affects the output of that uname-a tool.

And the reason you'd want to do this, especially if you're cross-compiling from Panther to Tiger or Tiger to Panther, is a lot of scripts will create directories that have the name Darwin 7 embedded in them, or Darwin 8 embedded in them. GCC is an example of a project that does this. And so by setting this environment variable, you're basically overriding the default value on the system and telling it instead it should always return this 8.0 for the duration of the build, and that way you'll get the directories created that you expect.

There's also the RC_ARCHES variable. This is the architectures which you are compiling for. As you probably know, Darwin has been available on x86 for some time. Now Mac OS X will be using the Intel architecture as well. You're going to want to specify both PowerPC and i386 in this variable. And by virtue of doing this, the open source projects will build universal binaries and universal libraries that will run on either of these architectures.

And the last of these really important environment variables is the RC project name. And this is the name of the project that you're building. So, for example, if you're downloading GCC sources, then you should set this variable to GCC. Or if you're downloading CCTools sources, you should set the environment variable to CCTools. For some projects, they will produce a completely different output depending on what this environment variable is set to. And that's known as a build alias.

So build aliases are really where there's one tar archive, it's the same sources, but there are completely different build results. And that, again, is based on the value of this RC project name environment variable. Here are some of the more popular examples of this. There's the Directory Services Password Server plugin. That's a source code project that you can download. But it has an alternate identity.

It has the DS Password Server framework. Now, the same tar.gz archive will produce... So both of these things. If the environment variable is set to Password Server plugin, then it's going to create a plugin. If it's set to the framework, it'll create a framework. And both of them operate together.

And the reason they're in the same set of sources is really because there's an interdependency there. The revision lock, right? They have a private protocol that the plugins talk to the framework. So the same set of sources produce both of the results. But if you failed to set this environment variable... There'd really be no way for you to build the framework.

I mean, it's absolutely essential. Another example is cctools and cctools-o-files or libc and libc-debug. Setting the environment variable to libc-debug is the only way that you can get the debug version of libc to be built. If you were to just build the libc sources without this environment variable set, it wouldn't build the debug version at all. Thank you.

So this includes a lot of the variables I was talking about, as well as some others that are a little more archaic, and the list goes on and on. And really, it's too tedious to specify them all yourself, and what I really recommend is using the Darwin build scripts, which I'll describe in a few minutes, to take care of all of this hard work for you. It'll also keep things up to date. We'll be modifying which variables are set by these scripts as time goes on to make sure they're in sync with what these projects are expecting.

And the last point I had about building is headers and libraries, which might not be available on your Tiger TBDs. There are a lot of intermediate, files that aren't available on the Tiger DVD, which you do need to build. And the reason they're not on the DVD is because they're simply not needed at runtime, and there's no reason to install them on every user's system. Some examples of this are the kernel linker static library.

This is something that's needed by I/O Kit in the user space, and so it's also something that's needed by the kernel. It's produced by the cctools-ofiles project, and again, the only way to actually get this would be if you were to build cctools-ofiles, pay attention to the fact that there's a build alias, and set the environment variable, and you'll get this file.

But while you were building that, you'd probably first get a warning that there wasn't a streams.h header file, because again, this is a header file that's never needed in the end user systems. It's only needed while you're compiling portions of Mac OS X itself. The streams.h header file comes from the libstreams project, and so basically you can see how there's a lot of dependencies which you'll need to chase.

You'll need to get all of the headers that are needed to build these Darwin projects and all the libraries, and it really boils down to starting somewhere at the beginning, installing a few of the headers, installing a few of the libraries, going to the next level. But the Darwin build scripts take care of a lot of this for you, which is another good reason to use the Darwin build scripts.

So, Darwin Build. Darwin Build is hosted at darwinsource.opendarwin.org/darwinbuild. It's a specialized tool for building the Darwin source code. It emulates Apple's internal build environment, so it sets all the environment variables, the source root, the obj root, the sim root, the dest root, the deployment target, the uname release, all of them.

And it cleanly handles private headers and libraries by installing them into a directory called the build root. It'll grab all the pieces it needs to build something, put them in there, produce the build, and then copy out the sim root and the dest root into a well-known location so you can inspect them and then install them on your system.

The architecture of the star and build script has three main parts. There's the property lists, which are a normal XMLP list format. You can edit them using the property list editor application. And it contains all the project information. It contains the names of all the projects in a particular release of Mac OS X. It contains all the versions of them, the URLs where they can be downloaded from, which projects, which header files, which libraries are needed before you can successfully build each of these projects. the environment variables that need to be set, everything.

There's the Darwin XRef tool, which will load in this property list and has the ability to search it quickly to figure out exactly what environment's needed. And then there's the Darwin build script itself, which uses all the information available from Darwin XRef to actually carry out the action to build the software and give you the end result.

So here's a little flow chart of what happens. The first step is you download all the tools and you take a property list for a build. So since we're talking about Tiger here, we'll be talking about build 8A428, and we load that into the Darwin XRef tool. And I'll go over all the details in the hands-on portion of the talk. Once that's done, we can use Darwin Build to extract all the information and actually issue the commands.

What happens at this point is the sources are copied from the source root into the build root. So in our example, we were building the talk sources. It'll grab those sources, copy them into the build root, along with any other intermediate files that may be needed to build. And all the temporary build results will go into the obj root.

Once that's done, it's going to produce all of the symboled versions in the symroot and the final versions in the dustroot. And when the build is over with, both the dustroot and the symroot will be copied out to a well-known location where they're ready for you to use. So let's move to the hands-on portion of the talk. And that's going to be the demo.

So hopefully you've had a chance to download the sample code from the website. If not, it should still be useful to follow along as I type the commands. I'm not sure what the tar archive was named when you downloaded it. In this example, I have WWDC508 sample code. Maybe it's the same name, maybe it's not.

And what we're gonna do is change into the temp directory. It really doesn't matter where you extract this. If you wanna use the temp directory, you can follow along exactly with the commands I'll be typing, but anywhere will work. And what I'm going to do is extract using the tar command.

And it's going to extract a lot of files. I'm going to clear the screen here and change into that WWDC 508 sample code directory. Now when I list the directory, this is the overall organization of the Darwin build scripts. There's the Darwin build route, which is where all the building actually takes place. And what happens is a mini copy of Mac OS X gets put into the build route, including the GCC compiler, the lib system library, any of the other tools that are needed. And that's where all of those intermediate libraries and headers are put.

So when you're using Darwin build, you don't have to worry about it polluting your system with header files that wouldn't ordinarily be there. Everything it downloads and installs will go into the build route directory. So it's really sandboxed away from your system. It's not going to mess it up.

The logs directory contains a log of each build that you do. So all the output from the make file and all of the environment variables that were set, everything is logged and put in a log file in this directory. The sources directory is where all of the sources are cached.

So whenever you build a project, it'll go download the sources from Open Darwin or from Apple's site, and it'll store them in this directory. If you want to make a modification to the sources, you can change into here. You can start editing the source files, and that's what will be used when you do a build. The bin directory contains the actual scripts that will be running, including the Darwin build script and the Darwin XRef tool.

The headers directory contains the finished product of the header files of each project that you build. The roots directory contains the finished product that's the dest root of each product that you build. Symbols directory will contain the symboled output of each project that you build. That's what you can do the line by line debugging with in GDB. And the plists directory contains the property lists of each project that we're building.

So, the first step is to load in some of our property lists using the Darwin XRef tool. And we can use this, or we can do this by using the load index command. And inside the plists directory, there's an 8A428 plist, and that describes all of the open source projects that are in Tiger.

I think I had already used this demo machine before, so. I'll skip ahead a step. Usually when doing a lot of Darwin building, you're actually gonna need to be doing this as root. The reason for that is most projects, when they're being built, will change the ownerships and groups of the files in the destroot.

And on Unix, as you know, the only way that you can change a file to a user other than yourself is if you are the super user. So usually fairly early on, you will need to use sudo to give yourself root access, and that needs to be done before you use the Darwin build script. I'm gonna set my prompt back down so it doesn't get in the way. Okay, so revisiting the command that we just attempted, we'll use the Darwin XRef tool to load the index, and the index we're loading is the 8a428 plist.

Oh, it's missing an S. Okay, finally success. So we can see there are 377 projects that are open source in Tiger. All of them were loaded into our index. There's another feature that the Darwin XRef tool has, and that is to query the files that are in there.

So for build 8A428, if we're interested in knowing which version of the kernel was in that build, we can ask for the version of XNU. That's the project name that produces the kernel. You can see here that it was XNU-792. That's what's in Tiger. So this is a good tool that you can use interactively to figure out which projects belong to which release of Mac OS X.

Another function is the ability to look up which files are produced by each of these projects. And we can load the database from a text file. So we'll use the load files command, and that's also stored in the plist directories. There's a... Files-8a428 Text File. And what it's doing here is it's actually loading in all the contents of the text file, and that has project names and all the files that they produce.

And when it's done with that, we can actually do lookups and we can ask, you know, here's a file that's on the disk. Which source project did it come from? So if we use the Darwin XRef tool, You can say find file, and in this example, since we're going to be adding Bonjour support to talk, we're probably interested which project talk comes from.

So we can see here that there's talk, it was found several places on build 8a428, but the one we're interested in is user bin talk, and that comes from the network commands dash 245 project. Since internet connectivity may or may not be working for some of you here, I've gone ahead and included these sources already with the sample code download.

But if you didn't have the network commands 245 sources, DarwinBuild is actually smart enough to go out to the network and find them for you. So if we change into the sources directory, you can see here that we do indeed have network commands 245. We can go ahead and extract these sources.

And all of the Darwin sources have a tar archive containing a top level directory and only one top level directory. And that directory has the same name as the tar archive, just without the .targz suffix. So network commands-245.tar.gz will produce network commands-245. And that's the full sources of this Network Commands project, which as you can see, has a lot of other things in addition to the talk utility.

We can go ahead and inspect this diff file. The diff is a standard patch file, which I've included, and this is what will actually add in the Bonjour support to network commands. So you can see here, I've actually gone through and disabled a lot of these other commands from being built in the interest of keeping this demo short.

And what we've added in are, A few things like a new browse.c file. This is what contains the Bunger browsing for the talk command. We can see it actually adds in a few new flags, like yes, we are using the DNS service discovery, which is the technical term for Bonjour.

And scrolling down, there's a lot of Bonjour-specific code that's being added in to the network commands project. So to apply this patch to our network command sources, we can use the patch command. And based on the style of the patch, we're going to use patch level zero and pipe in the network commands diff.

And that'll actually go through and change all of those source files in the network commands project to do all the modifications. You can see here, it actually patched a lot of files and created some new ones, including the new browse.c file for the talk command. Another part of our, another part of our, our, our, our, our, Demo will be adding in a PAM plugin.

PAM is the Plugable Authentication Modules interface. And what this is going to do is it's actually, every time you log in, it'll register with Bonjour that you are logged in. And that's the heart of the demo is talk. We'll be able to browse all of those registrations, see who's logged in on your machine, and you'll be able to pick and choose the logins from a list.

So if we change back to the directory we were in, and again, this is that top level directory that has the general organization of DarwinBuild. There's one more thing we need to do for this demo. It's only done for the sake of expediency here, and that is we'll load in a WWDC property list that has some special values for this demo, just so you don't have to go out to the network, and you can build these projects a little bit quicker.

So basically this is a copy of that 8a428p list, the Tiger property list, but it has a few changes. So, what we can do now is we can actually go ahead and build the network commands project. And we can do this using the Darwin build tool. And Darwin build takes a build argument, So if you're gonna be building for say 8A428, or in this case, we're gonna be building for the WWDC build, which is just a little hack for efficiency, you can specify it here.

If you want Darwin build to download all of those intermediate libraries and header files for you, you can give it the chroot option, and that tells it to do everything entirely within the build root and make that a self-contained environment. And then all you need to do is give it the name of the project that you want to build. In this case, network commands.

What it's doing now is it's actually copying things from the build root and the various sources directory, I'm sorry, from the various sources directories into the build root. - And once it's done that, it goes through and it actually did a whole build right there of the talk command, and you can see these are all the files that were produced by the build. Everything scrolled by rather fast, but it keeps a log for us in that logs directory. So let me list the logs directory. So inside the logs directory, there's a subdirectory for each different project name.

You can see here you have network commands dash 245 dot log tilde one, and network commands dash 245 dot log tilde two. What the number after the tilde means is the attempt of each build, and that'll just be incremented by one with each attempt. So every time you task it to build network commands 245, it's going to add one to that number. And this basically lets you compare the log from the last build to the current build, see what might have changed, see if you fixed the bug that you were hoping to fix.

The reason why there's a tilde one in there is that was already there when the sample code was provided to you, and the build that we did just now produced tilde two. So if we actually look at the contents of that file... Then we can see what happened when we asked for the network commands to be built. You can see here a little header that kind of describes what's going on. So it's building network commands 245, attempt number two on the current date.

has a list of some important information, including which version of the compiler tools is used. So inside our build route, it knows that we need to use GCC 3.3, and it set it up appropriately. It also indicates which version of ccTools was used that contains the GNU assembler. It'll go on and list out all of the environment variables that were set.

You can see that the environment variables, which appear to be in alphabetical order, contain things like Full path for the dest root, a full path for the object root, the right deployment target, and all the other variables which I had talked about earlier. After setting up the environment, it'll actually issue the command to make.

You can see the right way to do this would be to say make install, and then take a lot of those same environment variables and actually pass them as arguments. Some of the make files will use the environment variables, some of the make files actually expect it to be arguments, so to be safe, it's best to do both. To both set the environment variable and pass it as an argument to the make command. And this is where the actual build log takes place. All of the following commands are things that were output after the make had started.

So, now we can look in the roots directory. And in the roots directory, we also have a, - This is the sub directory, that's network commands. And you can see that there's a network commands dash 245 dot root tilde two. And that is the finished product of what was built at the same time we captured that log that we were just looking at.

And inside here, there's the user directory. Change this to a find command. We can actually list everything that's in that directory. And you can see there's user bin talk. There's our PAM module, which we'll be installing to register our logins with Funger. And then there's a man page for each of them.

In order to give you an example of how you can use this open source for line-by-line debugging, what I'm going to do now is do a build for the - So, we're going to do build wwdc in the chroot, and we're going to be building ncurses. This is going to copy files in to the build route and start building end curses.

End curses will actually take a little while to build, so I'm going to cancel out of this and use the pre-built version that was supplied in the sample code. If you want, you can let this finish, or you can just cancel it at the same time and we can use the pre-built version.

So I'm canceling it. So assuming that build succeeded, it would put the finished product into roots. There's end curses. anchors 15.root tilde one is the finished product. But in this case, we want to use the symboled versions, right? We want to actually do some line-by-line debugging using GDB.

Now, one of the things that Apple sets up when it builds all of its Mac OS X sources is to set the source cache directory, and that's where all these sources go. So what we'll do is we'll make a SIM link, and we're gonna link into our build root, So we have temp, WWDC, underbar 508, underbar sample code. That was that sample code directory that we extracted. In the build route, there is a source cache directory.

And we're gonna make a link at the root level source cache. So when we're done, we'll see that. Source cache points to the one that's in our build root. What we can do at this point is actually use GDB. And if we look in the symbols directory, there's a network commands subdirectory, as you might expect. There's a network commands dash 245 dot sim tilde two, or if you haven't built it, you can do tilde one, which was the pre-installed version.

And then there's the talk command. That's a fully symboled version of talk right here. So we've started talk in GDB. There's an environment variable which you can use that overrides the default search path for libraries on the system. And what we can do is we can point the library path to the symboled version of end curses.

So if we set our environment,

We can actually set that to temp-wwc-508-sample-code: symbols-wwc-508-sample-code: Ncurses 15.SIM.TILDA1 Sorry, there's a top level directory that has just the directory name. So, ncursors/ncursors-15.sim-1. And then in this directory, if we were to have browsed around beforehand, you would see that there's a lib directory followed by another lib directory followed by, or actually we don't need to specify the library itself, we just need to give it the path. So it'll find the NCRISIS library in here.

Now we can set a breakpoint at main, which will be the main function of talk, and you can see that's in talk.c. And we can also set a breakpoint at init screen, which is the initialized function for the endcurses library, which does all the screen drawing in the terminal.

Now I can run and we'll do the --list argument. So what we're gonna say is we wanna list out all of the Bonjour registrations that are available to talk with. You can see here we've stopped at line one of the main program, or of the main function in the talk program.

We can use next to step line by line. When we continue, we should break at the init screen function inside the ncursors library. And sure enough, we have a break at init screen, although I don't see the source there. I might have given you the wrong directory here.

- Can you put out a GDB for a second? Uh-huh. So we had a mistake in our path. Okay, so starting from the top quickly, if you're still on GDB, we'll catch up with you there. So we're gonna do a GDB of symbols, network commands, networkcommands.sim, tilde two,

To the same thing we did before, what we were missing was this directory here, the /user. And when we set that environment, now we should be able to set a breakpoint at main and set a breakpoint at init screen.

We can run again with the --list argument. We have our breakpoint at main, and if we continue, We have our breakpoint out in its screen. So this is actually source code that's in that NCurses library, which is the library that's used by Talk. It's a standard system library, so you can use this approach when you're debugging your own applications.

And you can see that we can actually step inside the dynamic library. We can do line-by-line debugging in here. We can print the values of the variables. So this is actually pretty useful stuff. Now that we've demoed the debugging, let's see the whole thing in action and see how we can actually install this modified version of Talk onto our Mac OS X system and see what it does.

So, like I was saying earlier, it's really important to make backups. So we're gonna be copying the userbin talk utility out of the dest root and into the base system. Before we do that, we want to set a copy aside so we can restore from it once we're done. So let's copy userbin-talk to userbin-talk.original.

Now we can copy from our roots directory, which has the finished product of network commands, Network commands -245.root tilde two is our finished product. And user bin talk can be copied into the real user bin talk. We also have our PAM module, which we want to install. There wasn't anything there originally, so there's no backup copy to make. We're just installing something new.

And that is stored in user share. No, it's not user share. These are lib, pam, pammdnsregister.so, So we're gonna copy that into the real user libpam directory. Now, there's two configuration files that we'll need to edit in order to activate this. Kevin Van Vechten One is the /etc/pam.d directory login file.

So this is the list of all the PAM modules that should be activated when you do a login. And we're gonna copy that aside so we have a backup copy. We're gonna copy that to login.original. And then we're going to edit it. Actually, we can use VI to edit it.

You can open a new line by typing lowercase o. This is going to be a session plugin. It's just optional. It's not required for logging in successfully. And the plugin is pammdnsregister.so. You can hit the escape key to get out of edit mode. The colon key to do a command.

We're gonna write this file. You can do that with write, or some way with W exclamation. So we've written the file back out to disk, and then colon Q will quit VI. The second thing we need to do is start the talk server on our system. And in Tiger, there's a new facility for doing this. It's called LaunchD. There's the launch control program.

We'll want to actually load in a new daemon on the system, which is the talk daemon. And basically how that's done is you load and unload property lists into LaunchD, which will then activate the write system service. And that property list is already on your system. It's in system library.

Launch demons. And the one we're loading is ntalk.plist. So now we're running talk on our system, and we also have a PAM module which will register us with Bonjour every time we log in. So let's try a new login. I just did that by creating a new terminal window.

will make the font size big enough for everyone to see. There's a DNS SD command, which if we give it the -b argument, that means we want to browse Bunger registrations. We're looking for ntalk, oops.

I'm sorry, it's an uppercase B. And you can see here that there is actually one registration that was listed out on the local domain. That's ntalk.tcp. And the text is Apple, that's my username right now, at the local host, and I am in TTY, P2 is my terminal name.

So if we go back to where our talk command is, if we actually run talk with --list, there you see a list. Here's my registration. Hopefully if you guys are on the network, you can see each other appearing here. If I create more login sessions, more people will be added to the list. When using the arrow keys, you can select one from the list. We'll send a talk request to that person. I don't know why it just quit. Anyway, that's basically how you would go about using the Darwin build scripts to modify some built-in component of Mac OS X.

Real quick, I'll go through the steps of restoring your system. Is the demo screen back on? Okay, if we simply move this /etc/pam.d/login original file back to /etc/pam.d/login, I'll restore your original PAM settings. We can move userbin_talk.original back to userbin_talk. You can remove user lib pam_mdnsregister.so, and we can unload the talk launch daemon.

and talk.plist. And your system should be back to its original state. Now, I think we can move ahead to the question and answer period. Anyone have any questions in general about using Apple's open source, building projects for Darwin? Please step up to a microphone if you have a question.