I was asked to come up as the technology manager for Mac OS X. My primary responsibility is actually working with all the Core OS developers, including those doing file system plug-ins, network extensions, drivers, anything else in that area. Plus, I also come over some additional areas. I was asked to come up here to give a short two-minute talk. However, Brett, the main speaker, gave me some conditions. I could not reuse my Kentucky Fried Chicken Mach kernel joke. I couldn't do any magic tricks. I couldn't do any bad jokes like the CoreOS Core OS. So I have another minute and a half.

Now, I do actually want to stress that I'm glad to see we actually have a fairly full room for this session. Apple views with Mac OS X the importance of having a strong foundation for the core, for Mac OS X having a strong foundation. As a result, we've spent the last, what, three, four years now actually working at building up a good, strong foundation. If you went to the Bertrand session, you saw the one slide he had about the collapsing building. And if you know anything about earthquakes and so forth, one of the major causes for building collapses is actually a weak or poor foundation. We don't want our buildings collapsing.

So as a result, we're putting a strong foundation, but at the same time, we want to make it flexible so that you can actually enhance it with your products. If you notice this year, we actually have several sessions dedicated to developing down at the Core OS. Unlike last year, we had like maybe some overview sessions, this year we have several sessions, including several sessions dedicated to the I/O Kit.

So we really believe in the importance of you working down at that area. So to cover the Core OS and to give you an overview of what's down there, what you can develop on, and where you should attend the rest of the week, I'd like to now introduce the director of the Core OS engineering team at Apple, Brett Halle. Brett?

I appreciate you being good this year. Good afternoon. So, let's start with what is Darwin? We've talked about the Core OS, we've talked about Darwin. So I want to stress early on here that Darwin and the Mac OS X Core OS are synonymous terms. The Darwin is the Core OS of Mac OS X.

So throughout the sessions that you'll hear this week, you'll hear these terms used interchangeably. And I think it's important to note that these are equivalent terms. You'll hear Darwin used usually in reference to the fact that the OS is open sourced. You can grab the source to the operating system.

You can get the source today off of the our open CVS repositories at www.apple.com/darwin and you can download and access the source to the DP4 release that you've got today. So if you want to actually see what's part of the OS, what it is that makes up the system, it's there and available. So it's really important, again, to stress that you will see these terms used interchangeably.

So now that we know that, how in the world do you describe what the core S is? I tried to explain to my mom, mother-in-law, what an OS was. It's real easy to talk about Aqua and the Finder and things. They're very visual things. But the OS isn't very visual. And in fact, if we're really, really doing our job well, you really don't even know we're there.

The whole point is that we're supposed to make the system so stable, so robust, and just so cool that it just does the right thing. But as a result of that, we're very, very invisible. So I was scratching my head over the past couple of weeks trying to describe what type of metaphors you can use to describe an OS.

And it's actually very difficult. Certainly, it's a strong foundation. It also provides plumbing for the system. It basically is responsible for all that infrastructure stuff that you would expect as part of the system, but you don't ever see. Most of these things in your normal life you don't ever worry about or see, but it's really ultimately what makes for a good building or a good system.

A good environment. So what I do want to get into though is what the technology is that makes up this system. We'll try and get away from the metaphors a little bit and focus on exactly how this system is built. One thing that's very unusual and interesting about the Core OS is that we've taken a very unique approach to the architecture.

Instead of just using kind of one piece of code or one piece of technology that we developed, we actually went out and gathered from multiple best of class technologies in the industry. For example, we use Mach, we use BSD cores, there's a number of other pieces of technology that we've gone and actually gathered together to build the Core operating system.

It's also based on proven technology. A lot of this stuff has had literally thousands of man years of investment in development and testing. So this isn't just stuff we came up with overnight. There's been enormous amounts of mileage on this technology. Things like BSD, of course, runs the Yahoo websites, and they certainly take an enormous amount of stress and hit.

I think even more interesting is there's an enormous amount of untapped potential in this core. Everybody has heard a lot about all the excitement with the UI, with our application toolkits. And one of the things that we really haven't stressed a lot is all that stuff that's under the covers, that's down in the core of the system. And I think, frankly, this is where, as we move forward for Mac OS X, where a lot of the untapped potential exists for applications.

So in terms of the services that the Core OS provides, certainly it is responsible for the preemption and memory protection model for the system. It actually provides multiple address spaces. So each application that runs on the system is in its own separate address space. So that if you have an app that tends to go run wild, write all over memory, it'll ultimately hit the edge. The kernel itself will actually trap that condition and kill the application. You won't actually be in a situation where your application can bring down other apps on the system.

The Core OS supports the application environments. Our key role in the system is to make sure that the application environments, Carbon, Cocoa, and Classic, run as efficiently and effectively as they can, that we support ways for those environments to communicate with the outside world and between themselves. Our job is to make sure those application environments are just top-notch. The Core OS is also processor independent.

For those of you who were just at the open source session, we demoed the Darwin, the open source component that's available out on the net, actually running on Intel. Now, this is not the Core OS for Mac OS X, not all of Mac OS X, but just the Darwin piece, the open source piece of Mac OS X is actually up and running on Intel.

We've done that, put some effort into that to actually make sure that our OS architecture is as flexible and as well-designed as we can because we want to make sure that the OS is as flexible as possible. The Core OS is a very important component of the future, and as the future progresses and as processor technologies evolve, that we have a very flexible architecture.

It's also high performance. That's the marketing term they insisted I put that in there, but it really is important that we consider performance a very key part of the OS, and we'll talk a bit more about that in terms of how we actually support that in the system. Flexibility is also key. Because we support multiple application environments, we deal with things such as processor independence and an unknown quantity of new hardware and things that come into play, flexibility as part of the architecture is critical.

And you'll find in a number of the subsystems that that plays a very key role in how we've approached certain problems. And of course, scalable. This same core that we use for our client operating system is the core that we use for our server operating system. So it is pretty critical that we have some scalability potential in the technology that exists here.

Of those of you who have used the Mac OS X server product, which is based on a slightly older code base, but we'll be syncing up in as we move forward, that, you know, we perform incredibly well for things like as a web server and for stream, as a quick time streaming server. I mean, it's a scalability in the technology such as networking and file systems and I/O that really make that work.

And of course, I think it's really important to know, because we're an OS, that buzzword compliancy is a real critical part of it, certainly if in networking. So you'll hear a lot of these today. So to look at the Core OS kind of at the grand perspective in the architecture, you'll see, of course I think it's kind of the proper scale that has been applied here in terms of importance, that where we sit in the system is below the core services.

And the core services are things such as Core Foundation, Quartz, all the services necessary to support the graphical user interface. The application services themselves that support Classic and Carbon and Cocoa, those are all above us. We have a very layered system for Mac OS X, and I think that's pretty important in terms of how you look at this.

This is not just a random collection of shared libraries. This actually follows more of the conventional onion skin, if you will, model. Within Darwin or the Core OS, there's a number of key technologies. The Mach microkernel, I/O Kit, and the drivers that are part of that. The BSD kernel, our file system support networking, including Apple Talk and IP. And we'll go into each of these in some detail.

First, let's talk a little bit about the Mac microkernel and how it plays into the system. One needs to consider Mac really as an OS foundation. It's responsible, its role in the system is to manage the processor and the memory resources. It deals with scheduling, handles all the preemptiveness in the system.

It deals with the threading of the system. I think it's very important to note that for Mac, the threading granularity is done at the thread level. This is particularly important because as you write applications that are multi-threaded, the threading services and the scheduling is handled by the current microkernel itself.

It also is responsible for memory protection and memory management, including handling things like backing store for VM. All of that is managed by Mac. One other aspect about processor scheduling that I think is important to mention for Mac, it's very critical that we've considered. From an architectural standpoint, being able to deal with multiple processors as well. Mac is responsible for, on a thread by thread basis, being able to schedule threads across multiple processors.

Mach is also, because of the level that exists at, the way of communicating between address spaces or between tasks is through IPC or RPC. And this is a very key part of Mock's architecture. And if you get into more detail around Mach, you'll see what a critical role the fact that it's a message-based architecture is. Mach is policy neutral. I think it's particularly important to note.

Down at that level, all it's really doing is managing the processor and memory. Beyond that, it doesn't worry about I/O, it doesn't worry about users, it doesn't know anything about file systems. Basically, it doesn't provide all those high-level OS services. Its role in the system is just to control the processor and memory. For Mac OS X, the technology we used for Mach was basically Mach 3.0 plus a bunch of stuff.

We based it on work that was done in the past. We've done a lot of work on the OSF research institute. And in fact, we did some investing, have been investing in the research institute for a number of years, even a couple of years before Next was acquired for evolving the Mach 3.0 work. Plus, obviously, in the past few years, we've done quite a bit of work on this ourselves. That, as I mentioned, inter-task communication is done through messaging. And you'll find that as part of Mach, there's this untyped IPC and RPC model that is critical to its design.

Architectural support for SMP in real-time is another key feature of 3.0. SMP is something that was really designed into Mach from day one. And it's something that we've really made sure that we've continued to move forward. And you'll see, as I mentioned, that scheduling is done on thread-by-thread basis across processors. In addition, real-time is a serious consideration for us.

There's a lot of work yet to be done here, because real-time, as it's been done for Mach 3.0, from the Research Institute in CMU and others, has been focused mostly towards things like weapon systems and stuff like that. Since we aren't really considering using Mac OS X for nuclear deployment, our focus is very much more on how we can take some of that real-time technology and evolve it towards multimedia and other real-time needs, audio, things like that. So, I think that's the first thing that we're going to do.

The pager and the mechanisms for dealing with VM are also part of Mach. So the external pager is another key feature of 3.0, as well as a strong modular architecture. For those of you who have followed Mach, Mach 3.0 was originally designed in the research community to be a true microkernel, in the sense that it was all that was responsible down at the kernel level and everything, including all the rest of the OS services, were up in user space. That is not a model that we've used for Mac OS X. It is not a very efficient way of approaching things.

But that modularity that's been designed into Mach is something that we've really tried to leverage. So you'll see that for Mac OS X, that the OS architecture and its use of Mach more closely matches a monolithic architecture with the way BSD and Mach and iOS system and the file systems all tightly integrate with Mach in the same kernel address space. We do that for efficiency. And it's a very key part of our design.

But we have kept the modular architecture of Mach such that the interfaces between those components are very clean. If you're interested in finding more about the Mac kernel, there's a session on Thursday bright and early in the morning over in the Civic Center, and I encourage you to go to that.

So, one other aspect that I wanted to get into here is that we've really factored performance as a key part of the OS architecture. You'll see as you look into the Darwin source code, or as you start playing with some of the tools, that we've actually plumbed the OS for performance. There's lots of counters and tools and entry points specifically for gathering performance data and information.

And this is particularly helpful for us in terms of how we tune the system, how we evolve it, and probably, I think, more importantly, is how you, the developer, can get involved in making sure that your apps are using the OS in the most efficient way possible. There are some really cool tools available on the system, and they're shipped with DP4. And to show us a couple of these tools, I'd like Joe Sokol, who's the manager of the Core OS Performance Group, to join us and show us some cool stuff.

Okay, am I live? Yep. All right. Yeah, so could we put the two screens up? Anyway, so I'm going to show you a couple of tools that are not real fancy. They're command-line-based tools. We chose to do them this way so that they'd have as little impact on the system as possible while you're actually using the tool to kind of view what's going on. And they're used for the first one we're going to look at is top.

And, you know, it's kind of a traditional Unix tool for looking at what's going on in the system. And what I thought I'd do is just show you three different ways you could use this tool, at least the one that we've done, to get some insight as to what's going on in the system and also to help you track things like potential memory leaks or other resource leaks.

So just to get started here, I'm going to show you how to use the tool to actually capture a picture of all of the... resource consumption, system calls, things like that that are used during the launch of an application. It's kind of an interesting view into what goes on kind of system-wide, right, outside of your own application space.

So this particular option for top It's just the minus A flag, and basically it's going to accumulate all these different counters. So there are counters in here for page faults, page ends, BSD system calls, Mach system calls, pretty much context switches, pretty good view as to what's going on, the amount of CPU time, percentage of CPU time, networking counters, and disk counters for both numbers of IOs and the total amount of data being read off the disk.

Joe, I hate to interrupt. Just for clarity, you might talk about this. Okay, sure. What's on your screen? So this guy over here, the screen over here on your right, the audience right, is where the particular tool is actually running, and you can see all the different columns there that we're about to hit.

And then over here is just going to be on your left will be the actual application. Point size on the terminal. Excuse me? Pump up the point size on the terminal, please. It's not going to fit width-wise, so sorry. So over here we're going to just launch the TextEdit app.

As that launched, we just captured a whole bunch of information. Then you could stop at this point and basically go take a look at what you did to the system. Let's say it was your app that you were working on. You could see not only what your app did directly, but indirectly it may cause some of the other system services to fire, like the Window Manager or Pasteboard Server or some other service that is done as a separate process.

Then another way to use the tool, OK, with the minus D option basically gives you similar sets of counters, but this is now done as a delta for each sample period. So this is a good way to see what's going on in the system when you just grab a window and just kind of shove it around so that you can discover what system services become active when you're doing something like that, or if you're to go up to the menu bar and sort of drag across there, you can see that mainly it's going to be the app and the window manager doing something like this, which is kind of what you'd expect. There's also a little bit of networking traffic that's showing up there because we're actually telemedded in remotely to the other systems, so again, to minimize the kind of intrusion of the tool on the system that you're actually trying to take a look at.

These first two uses of the tool are really more toward finding out performance-related things. How busy is the system when you're launching the app? What type of resources are you using? What other services are you getting involved? These are more hints for you to be able to go back and play with and use these tools in terms of gaining visibility into what's going on.

The final interesting way to use Top is as a method for watching for memory leaks and also for port leaks. Ports are the basic mechanism and naming convention in the Mac OS X. The most common use of the framework is to use the for establishing your IPCs and RPCs. Most of you probably won't ever really program to that level directly, but an awful lot of the services that you use in these frameworks indirectly cause ports to be allocated, messages to be sent around. If you're forgetting to release those particular uses of the framework, particular functions or whatever that require you to close things down, this particular view of Top can help you see that something wrong is going on. Over here, let's start this over.

And again, we'll just go ahead and launch that app. And you can see that there's a delta columns that are showing up like on the port usage. So the window manager picked up 11 ports due to the launch of this particular app. Some of the other guys, TextEdit itself as it was launching, once we actually captured and actually started to account for the stuff that's going on, picked up an additional 37 ports.

Over on the right-hand side, you have all of the information about your memory usage. So there's--again, distinguish between-- The virtual size of your applications and the actual resident size, which is an important concept. We provide a fair number of different views into that. The interesting one, all the way over to the right, is again a delta, which if that were to continue to grow over time, might be a good indication that you have a memory leak that you'd want to do something about.

When we quit the app, then we should see most of these things disappear. Window Manager went back to his baseline in terms of the number of ports that he's using. So when we quit that app, everything nicely released. Looks like the dock and pasteboard server is still hanging on to a port that was probably set up to communicate with that app.

All right, so that's about it for TOP. So those are three interesting views. And then one other tool right now, which is a tool called SC Usage, which will give us a little bit more of an in-depth view into exactly what's going on. This is kind of-- TOP is kind of a nice tool for looking at things in a general way and seeing everything that's going on in the system.

But we can't provide too much detail on every single app or every single process that's running all at once. So SC Usage is a tool that allows us to get a little bit more visibility into what's going on. And let's just pick one of the running apps, like the Dock.

take a look at that. Of course, these command line tools all have man pages in the system and they have little helps that are associated with them. But if we go over here to the dock and just start kind of playing around with it, you get an idea. Here's all the different system calls that are being executed, the number of system calls that are being executed during that little bit of activity that I did in terms of making the dock active.

The tool also shows you how much time is being spent outside of your application, so both in system idle and in system busy. Some of that time that's accumulating there would have been like in the window manager or other services outside of your app. The tool will show you how many page faults were taken for that particular application during that, again, the time that we're sampling.

It also shows down at the bottom that we actually have three threads that make up this particular tool, or make up the dock, and that we're going The relative priorities. These would be the scheduling priorities in the system. And also, the cumulative wait times. So this is like a really well-behaved application since we're not doing anything with it right now. All three threads are just waiting for something to do. They're not polling.

They're not waking up to kind of see what's going on in the system. So a good target for everyone is to write an app that when you're not using it, when it's just sitting there, basically does nothing, has no impact on the system. And with this SC usage tool, you can actually see that occurring. These tools are going to be gone over in a little more depth in session number 195, which is Thursday at 4:00. So I think that's enough to whet your appetite for right now. We're going to do one more tool a little bit later on. Thank you, Joe.

Okay. Thank you. Thank you. Thank you. I think it's particularly important to note that as you're writing applications for Mac OS X, that there are some very important tools available in the system to give you some idea of what the impact of your app is and ways that you can tune it.

Since under Mac OS X you don't have the issue, for example, of having to pre-allocate the amount of memory for a given application, things like the fact that you can actually have memory leaks that just continue to grow and grow and grow and grow are something that you actually need to watch for and test for before you release your applications.

So, let's spend a moment now and go over I/O Kit and our driver story. I/O Kit is a framework, in the true sense, for easy development of device drivers. It provides the mechanisms for plug and play, as you would expect from Apple, but it also provides some other core services that are necessary to support devices.

Things such as power management down at the actual architectural level, dynamic device management, the ability for things to truly come and go in the system, hot plug ability and such. And a very modular and extensible environment, because we don't even know what the next device is that might come down the chute. So we really need to be prepared for a very flexible environment to be able to support these things.

It provides the abstractions for most of the device classes in the system. Certainly all the standard things, things like mass storage and SCSI and ATA and stuff. I/O Kit does not provide abstractions for some very high-level devices, things like scanners and printers. Those are actually done within other groups, specifically as part of the graphics environment. I believe there are a number of sessions that go over those elsewhere. The focus for I/O Kit is primarily in devices that actually represent tangible hardware that we touch directly.

Key features of I/O Kit are, again, it is a framework in the sense that, one, it is object-oriented, and I think that's important to note that part of its design architecture was to really provide this like in App Kit or in Cocoa, to be able to have a framework to leverage off of. It deals with a lot of the standard mundane things that you would expect that need to be done for I/O.

It provides a lot of the common kernel services that you needed to work and talk with the device and to talk and communicate with the rest of the system. Protocol specifics, however, are divorced from the core framework of I/O Kit and are dealt with in something called families. Families provide the different kinds of protocol that you would expect in very different kinds of devices.

Certainly, SCSI and audio are very opposite ends of the spectrum in terms of the ways that you would use them. One's a bus and one is, you know, has issues of dealing with volume and there's a whole bunch of issues that are very, very different between these two classes of devices.

And the family portion of I/O Kit really deals with providing mechanisms to be able to deal with those protocol differences. The actual device specifics themselves are captured in drivers. And this is the part that most people will be dealing with as they port their piece of hardware to Mac OS X. Drivers are those pieces that actually got in touch with the hardware directly.

All of this, by the way, for the most part, lives in the kernel itself. And a driver, drivers and families and things like that tend to, are all very low level OS services. There are mechanisms that I/O Kit provides and the families provide for being able to communicate up to user space. Your applications in Mac OS X cannot talk to hardware directly. You must go through I/O Kit or through a driver or some other abstraction to be able to deal with that.

In DP4, it's important to note that we, actually a little while ago, released an early DDK. It's part of what we call the Kernel Development Kit. There's actually a series of services that are part of the KDK, and that was something that shipped about two or three months ago. It is also part of DP4, so on your disk you should see quite a bit of information on that. And it's, of course, part of Darwin and Open Source, including things like documentation.

Documentation for IOCit is done through something called HeaderDoc. And if you were at the Open Source session, there was a little bit of information on that. But it's what we used to embed documentation and services within the actual source code itself and produce a very, very cool format for being able to browse API information. Also, as part of DP4, you'll see some of the early parts of some of the power management work that we've been doing.

There's a lot of stuff we've been doing. Kind of back at the lab that isn't part of DP4 yet. That stuff is coming, but as part of DP4, things like shutting down the display, spinning down the disks, things like that are part of something you expect. For those of you who are interested in more on power management, this is something that will be coming post-DP4.

There are a lot of families that are part of I/O Kit today. And I believe we have something on the order of eight or nine different sessions this week to go over a number of the I/O Kit technologies, things like USB and FireWire. I encourage you to actually go to the I/O Kit overview tomorrow afternoon. All the detailed sessions will be described and more information about I/O Kit. We could spend the entire session just going over this subject alone.

It provides all the OS personality, if you will, APIs and services for the system. BSD handles things like file systems and networking in terms of it being really part of the BSD environment in terms of how that is exposed. It provides the basic process model, the concept of users and security within the system.

It also provides the security in terms of from both the user by user basis in terms of how it's managed also through the file system. BSD really is responsible for much of the very high-level view that everybody sees in the system. There's a number of pieces that compose the BSD component, if you will. One is the system framework.

This is basically the set of libraries that actually expose the system APIs. In addition, as part of the system framework, there's other OS personality services that are provided, various other libraries, things like the math library and the standard C libraries and things like P threads. That's all part of the system framework.

As you're interested more in the system framework, we'll be talking more about that at a BSD session later this week. There's also the BSD user environment, which is the command line and shells and all the bazillion tools and things that are part of that environment. This is a particularly set of powerful facilities that are available.

Brett Halle, Joe Sokol This is a particularly set of powerful facilities that exist in the system. These are tools and services that are available to you as the developer. As we'll talk about later, this is not something that we will be exposing to the end user, of course. This is not exactly the best user experience.

There's an enormous amount of power here, both as you're developing your applications, just as the tools that we've shown you, for example, and as ways to debug them and manage them and actually extend your applications in pretty new and exciting ways. Brett Halle, Joe Sokol If you're interested in more information on BSD, both at the kernel level and at the BSD command line environment level, please join us on Friday, 3:30. We'll be going into a lot more detail on that.

File Systems. File systems is something that everybody here will be dealing with in one form or another. And of course, it's not just one file system, but a set of file systems. And this is a very powerful part of the BSD kernel environment. It provides all the access to disk and even network file system services.

It's important to note that the file systems architecture that we have is part of the current BSD kernel environment. So you'll find as you're working with file systems, there's a lot of dependency with that environment. Again, how the process model works and how the APIs and things interact with each other.

It's based on the BSD VFS architecture, but we've extended it in order to be able to take advantage of certainly things like HFS+ capabilities and things that we think are pretty Apple unique. The VFS design is a very layered architecture. It has the ability to take file systems and stack them to provide a number of unique capabilities.

It also can support numerous file system types. In fact, in DP4 we support HFS, HFS+, UFS, ISO 9660, NFS. There's also an early version of an AFP, Apple Share IP, file system as part of DP4. The file system, in terms of its development environment, we package up as part of our KDK, our kernel development kit. There's a whole series of services that are part of doing development within the kernel in terms of ways to extend that. We'll talk more about that a little later in this session.

In DP4, I think it's important to note, for those of you who have been following Rhapsody and Mac OS X, that for DP4, HFS+ is our primary file system format. There's a number of APIs that have been extended to be able to support HFS+ effectively and to even extend them to use some of the real raw power that exists in HFS+. Things like long file names and some of the other services that are part of HFS+. As I mentioned already, our server team has already added some support in DP4 for the Apple share IP. You should actually look into that, but it is, to be noted, a pretty early drop of this code. This is something that we wanted people to start working with, but it is headed to some issues.

This is client, yes, absolutely, as someone kindly pointed out. Also, there is within the file system APIs pretty universal support for Unicode names, specifically UTF-8. For those of you that are dealing with this and want more information on this, be sure to go to the session. This is obviously of particular importance to worldwide deployment.

One other area in DP4 where there's been some interesting development is permissions on removable media. Because of being based on BSD and the BSD file system architecture, there's this concept of BSD file system security, the concept of a user and groups and the ability for read and write protection on a file-by-file basis. Removable media brings some interesting problems to the space, which is when you get the disk from somebody, what permissions and users are actually associated with that particular removable device?

There's been a lot of interesting work done here for DP4, and I believe that will be discussed also a bit in the file system session. This is something that we think is obviously unique to a client system that isn't typically dealt with on a server environment, for example.

Another area of key development for DP4 is the unified buffer cache. Again, this is kind of an OS-y, holy grail item, but it's been very, very important to how we deal with things like unify the way we deal with map files versus standard read and write access to files.

In typical OS architecture, the map files are dealt with by the VM system and the read and write is dealt with purely by the file system. There are situations where these two views of files, particularly if two people are accessing them in two different ways at the same time, where they can actually become out of sync. There are lots of clever ways that OSs have approached dealing with this particular issue. We think we've done a particularly excellent job in dealing with this for Mac OS X.

If you're interested in file systems and how they fit into Mac OS X, please join us Wednesday at 2:00 p.m. We'll go over UBC and a number of the other aspects and how to write a file system for Mac OS X. I'd like Joe to join us one more time and share with us one other performance tool, particularly one that relates directly to the file system usage. Okay.

Okay, so one more tool to show today. This one, again, in terms of a strategy that you might apply, you might have run top initially, kind of checked out that a particular process seemed to be doing a lot of system calls or seemed to be taking an awful long time to do something, but maybe wasn't really banking a lot of CPU time, right? So you might suspect that instead of chewing up a lot of CPU, you're just waiting for IOs to occur, and you'd probably be right.

So we have a tool called FS Usage that gives you a lot more visibility into what's going on in terms of access to the file system and also other types of system calls that tend to cause you to wait for things to occur. And the tool also will track, will show you where your page faults are occurring that actually cause disk IOs, where you actually have to go to the disk to page in as opposed to maybe just finding it in your cache.

And again, those are instances in time that could cause your application to look choppy if there's a lot of paging activity going on at some point, or if you have a lot of file system activity that's concentrated in a particular area could cause a lot of time to elapse where the app is waiting for that IO to complete. So what we will do is, go ahead and bring up our friend the TextEdit app again.

We're going to go ahead and take a look at text edit. And so he's just sitting there doing nothing. But if we go over here and let's go ahead and see about opening up some kind of a document. So what's going on there? That's probably-- that's not your app doing all those opens and LStats, which are the equivalent of in Git Adders and the Mac OS 9 world. That's a lot of the frameworks actually going behind your back and doing a lot of work for you. So just that simple act of bringing that panel, you can see we caused a whole bunch of activity to occur.

Now, what we're showing you here over on the left The Core OS X is going to be the actual time of day clock out to millisecond accuracy. Then you get the name of the system call that's being executed at that point. Then some additional information. We provide the file handle, file descriptor in Unix parlance that is being used or being returned. Then the I/O size.

Again, that's an important thing to take a look at within your own app when it actually starts making those requests because lots of small, discrete I/Os aren't nearly as good as nice, big, large, fat I/Os that get a lot of work done all within a single call. We also provide up to the last 28 bytes of the path name that's being actually referenced. That will be the last 28 bytes so that it tries to give you a sense of what the path name is. as much uniqueness as possible.

Then over here in this column is the amount of elapsed time that occurred to actually execute that particular call. If it has a W tagged on it-- I didn't see any in this particular session yet at this point-- that would be indicative that you actually waited for that I/O to occur. So most of these are very fast, down in the sub-millisecond range. This is in microseconds here. So these are like in 30, 40 microsecond kinds of ranges.

And then this would be the name of the task that you're actually-- or the application that you're looking at. FS usage, you can look at multiple applications at once. So you could be watching this along with some of the system services if you wanted to see how they were interacting with each other. Let's just try one more thing over here. Let's see. How about documents? Nothing there. I guess this is all going to be binary stuff. Excuse me? The Print Setup Panel. Print Setup? Print Setup. What's that?

Okay, so just bringing up the print setup panel causes it to go find a whole bunch of stuff on the system for you. Anyway, you get the picture. The other...there's one with a...so here's one that took a little bit longer where we're actually syncing something out to the disk. There's actual IO that occurred there. What this tool is useful for is, one, again, gives you a lot of visibility into what's going on in the system.

You get to see some of the things that go on when you make those very high-level toolbox kinds of calls may actually trigger a whole bunch of stuff behind your back. It's important to know that, that if you're having trouble kind of determining where the performance is going, some of it might be that.

This is also a useful tool for discovering where the IO is occurring and where you're going to be actually causing perhaps jitters or just kind of breakups in the UI in terms of having to wait for that IO as opposed to just pure CPU usage. Again, it's a tool that's meant more for gaining some visibility into the system.

There are other tools that will be talked about in that session number 195 that will help you drill down into your application and actually discover where in the application these calls are being made, which would help you to either decide how best to use them or perhaps manage to eliminate them if they're causing too much overhead. Thank you, Joe.

It should be noted that our engineering management team in particular loves running these tools. This is a way that we actually kind of get a handle for each build exactly where we think there might be performance issues or things in the system. Certainly, this is something we've found to be very useful in being able to detect things like apps that pull or where we have leaks or where people are doing things in a very inefficient way. I think you as a developer should know that these tools exist and there's a lot of these cool tools in the system to use. Moving on to networking.

It should be noted that networking is also a component of the BSD kernel environment. In fact, it's based on the BSD 4.4 TCP/IP stack. In fact, over the past year, we've spent some time and synced that stack with FreeBSD 3.2. There's a lot of very, very cool stuff that's been going on in the BSD community in terms of evolution of the IP stack. It should also be noted that the APIs for dealing with networking in Mac OS X are socket APIs. There is an OT library specifically for IP that can be used for Carbon applications, but the low-level services and APIs for networking are sockets.

Key features of networking for Mac OS X are IPv4 and AppleTalk multi-homing, routing services, things like firewall and NAT are all core services within the networking services. There's a lot of very powerful raw services that exist as part of Mac OS X, a lot of which aren't directly used. Right now, we're exposed in DP4 from a user perspective, things that may be used more in a server environment than they might be used in a client environment. But these are all part of the core services that exist.

Networking is also part of our Kernel Development Kit environment. We'll talk a little bit more about the specifics of how you do development in this area, but it's important that we have three core technologies that are part of the KDK: I/O, file systems, and networking. The mechanisms for being able to get services in and out of the kernel are based on this kit.

One key area where we've done quite a bit of extension of the basic networking services are kernel extensions for networking. Basically, the networking group has taken the basic SOCKETS architecture and stacks and provided mechanisms for extending those stacks in very important and interesting ways. More ways than I can get into in this particular session, so this is where I strongly encourage you to go to their session to see more. Another key area of support is that for DP4, that the IP environment for both classic and the rest of the OS are shared.

If those of you that use DP3 or use previous versions releases of Mac OS X, you actually had to set up multiple IP addresses for the classic environment and the base system. That is no longer the case. They actually share the same network. Also, as I'd already mentioned, there's an OT support library as part of Carbon.

In fact, if you have played it all yet with DP4, I noticed during lunch a number of you had installed it on portables. Or you saw at the keynote, IE5 is actually part of the release DP4 product. That all sits on top of the open transport layers as part of Carbon, which is part of our networking services.

That has really made porting of applications to Carbon to be very, very easy. In fact, the performance has indications that we easily meet the performance of Mac OS 9 for these OT interfaces. In addition for DP4, we have PPP now as part of the system. There's an early UI and control panel for setup and connectivity. We'd certainly be interested in your input on this.

There's also some extensions as part of the OT services for being able to use the OT services. We've also got a lot of support for the PPP connections, and this is something that we'll be doing more work on as we move forward. Also, DHCP client and server services are part of DP4. You can actually set up your DP4 system to auto configure with DHCP and even provide those services in a server environment. That has actually been very important in terms of being able to provide ad hoc networks where you get just two systems that connect together.

You don't have DNS services, things like that. There's a well-defined standard for that. There's some really cool stuff that's coming as we proceed forward in the networking domain. Support for mobility is, I think, one of the most important ones that is on the plate. Specifically, being able to change address or configuration or things like that without reboot. Our intention is to take this even beyond what has been done in Mac OS 9.

This is going to take us a while, but this is something that we're putting a lot of effort into. As part of that are also some native Mac OS X specific network setup APIs. This has evolved from some of the existing APIs that you may be familiar with in Mac OS 9.

That's a key part of our mobility story. IPv6 and IPsec is another key area of technology. This is certainly looking very forward in terms of our networking technology. During the networking session, I think Justin and his team will be going over where we're going with that and how you can actually get your hands on some of this stuff.

Whoa, looks like this one didn't get set correctly. There is a networking session, an overview session. Justin? Tuesday at 4:30 in this room. If you're interested in networking, please come join us.

Kernel extensions are the kernel plug-in model that we use. They're basically called, as you'll hear them referenced in other sessions, KEXTs. And they're bundles that contain a property list and the binary actually for use in the kernel.

There is a service called the KEXT manager which actually auto-loads and unloads these services as needed. This is particularly used in I/O for when a driver is needed on demand. You plug in a USB or a FireWire device or you have some new device that is detected at boot time.

It's the KEXT manager which auto-loads the necessary kernel extensions to be able to support that. There are a number of manual tools as well, so as you're developing kernel extensions, that you can actually manually load and unload these things and protect, actually work with them and test them.

Each kernel module actually has two key entry points that are common to all of them: an initialized, which is called "unload," and a finalized, called "just before unload." All other APIs are very domain-specific. So for I/O Kit, there's a whole set of interfaces that are driven around the family interfaces between the driver and the specific family.

For the kernel extensions, there's a whole series of interfaces that you need depending on where it is you're plugging into the network stack. And file system is part of the VFS plug-in model. So again, for details on how these things work, you need to go to the specific sessions.

There is also on the KDK, which is again part of your disk and on the website, a Hello Kernel tutorial. So if you want to actually get familiar with how these things work, you can actually play around with it. Special things for developing inside the kernel. At this particular point, you need to use our tools.

And mainly the issue here is because of the way the ABIs and things work within the kernel. They're very specific. They don't have all the dynamicism that exists in user space. We have very specific ABIs and unfortunately need to use our tools. C++ in particular is part of I/O Kit. Until we resolve some of the ABI issues with some of the other tool vendors, this ends up being a problem.

Developing inside the kernel, you need to do two-machine debugging. Obviously, you're inside the kernel space. You're in a very protected context. The only way to do debugging at this is with two machines. There are very limited programming language features here within the kernel. As I mentioned, I/O Kit uses C++ as the way that the frameworks are implemented. But it's a very restricted usage of C++, something that has been actually termed as embedded C++. But there are other programming language limitations that exist within the kernel. You are in a protected environment.

There is no direct user interaction from within the kernel. There are some interfaces and tools for being able to do logging and to be able to pass up things from the kernel to the user space so that you can get interesting interaction out. But primarily, you're going to be stuck doing debugging.

Resources within the kernel are highly costly. Certainly, allocating things in interrupt time is a no-no. Other, they're very restricted usage situations for being able to allocate resources. And as a general rule, anything that lives in the kernel needs to be very, very, very conscious about the memory resources that you use, because generally they tend to be wired resources. You're actually taking real memory away from the rest of the system.

and failures in the kernel environment are fatal. This is why we don't want people as a general rule to be writing kernel extensions, because it's at this environment where you really are in charge of the machine again, and this is a real opportunity here to pull the rug out from under all those apps. A key goal of Mac OS X is to provide a robust environment, and we need to make sure that code and things that exist and run inside the kernel are as robust as possible. If you're writing code within the kernel, you have that responsibility as well.

So I want to close with some final things to think about. One, I think I mentioned already that the Unix experience is our intention for Mac OS X is this is to be hidden. This is not the goal of Mac OS X is to be the next cool Unix box, although it is very cool. This is really an optional environment for power users and developers, you folks, but it's not something we want to expose to mom or the kids. This is not part of the new user experience for Mac OS X.

So it's very important as you're developing applications and using all this really cool stuff that you are very conscious not to expose that to the people that are running your software. They shouldn't have to manipulate and edit configuration files. They should never have to deal with the command line. Those things don't exist.

And in fact, our intention is that the terminal application for user, the user system will actually not be included. This is something that would be an optional install, not default install. So don't depend on it. Another key thing to think about is using shared memory is dangerous. One of the things that we found is people have implemented apps on Mac OS X that you're used to Mac OS where you have access to the entire address space of the system. And so you take shortcuts about how you communicate between separate applications or shortcuts about how it is that you interact with multiple pieces of the system.

Be very conscious of the fact that as part of the way of keeping the system robust, is to use some of the protection that exists there. If you go around and circumvent, and we do have services for being able to use shared memory, every time you do that, you're lessening the potential stability of the system in your application. So be aware of it and just take that into consideration.

Always use the highest level of abstraction possible. We've talked today about a lot of really cool technology within the OS. But as a general rule, you shouldn't start down at the bottom in terms of the APIs and services that you use. You instead should use the application services that exist as part of Cocoa, as part of Carbon.

That allows you the greatest amount of compatibility and the greatest amount of portability. There's going to be situations where you need to use these services, and that's okay. But realize that you're doing it with some cost. You may be affecting the portability and compatibility of your applications. Carbon is a great example.

open source is a powerful tool. As I mentioned, all of this technology is open. You can get the source to it today. You get on the Darwin website. You can look at it. You can build the system. It's actually possible to build a version of the kernel that you can replace with your existing DP4. There's only a couple of pieces that aren't included, and those tend to be drivers, for example, that we don't own the rights to, things like Adaptec drivers, things like that.

But all the other key technology that's part of the OS is available as open source. That means that you can find out about how things work by diving into it and looking, that you can actually get involved, and we want you to get involved in helping make this a better product. We've already gotten quite a lot of changes and enhancements and things from the Darwin community become part of that community. There's mailing lists and a lot of other things that exist so that you can get involved here.

This is a real unique situation that we have in having, you know, basically one of the primary OSs, consumer OSs, also open source, and this is something you should take advantage of. In each of the sessions, you'll actually hear some suggestions about ways you can get involved, and if you go onto our website, you'll also see that there are some interesting opportunities where if you want to get involved more and contribute, you can.

Last thing I want to leave you with is that there's an enormous amount of potential in this technology. Everybody's busily and should be busy right now worrying about how do I get my app onto Mac OS X. The next thing you might want to think about is underneath Cocoa and Carbon, there's some incredible power here.

And this is where I think the next killer app is hiding. The Mac OS X and the Core OS and all this cool technology is where we're going to be able to pull ahead of the competition in a major way. We have barely begun to tap some of the potential here. So something for you to think about.

I want to review real quickly the roadmap. There's a number of sessions that you can get involved in. Open source, we already, unfortunately already had. We talked a little bit about the open source story and how you can get involved. There's a lot of this stuff on the web. The IOCAT overview on Tuesday. Networking overview, we got it right here. Tuesday at 4.30. File systems, Wednesday at 2.00.

There's a birds of a feather session. I believe we don't know exactly where, what room it's in, but it's Wednesday in the birds of a feather sessions. In the evening. And this can be a lot more informal communication and ways that you can get involved. The Mach kernel is on Thursday at 9 a.m. The BSD kernel and user environment is Friday at 3.30. And there's also a feedback forum Friday afternoon. And if you've got questions, feedback that you can provide, I would strongly encourage your participation there.