Hi, my name is Chris Emura and I'm the File System Engineering Manager within Apple's CoreOS organization. Today we'll be talking about what's new with the File System for Tiger. Over the next 40 minutes or so, we'll briefly cover the SMP KPI work, continue on with Extended Attributes, Access Control Lists, File Notification Mechanisms, and the Finder's very own Copy Engine.

I carved out a little bit of time for what I call "Oldies but Goodies." These are basically topics which may not necessarily be new for Tiger, but are definitely worth covering in some detail. And then finally, the very 15 minutes or so, the very last 15 minutes, we'll leave a good amount of time for Q&A.

Quick word about the intended audience. The vast majority of today's presentation content is targeted toward application developers. That said, those of you who are File System or Kernel developers, as well as IT professionals, will still get a lot out of the presentation. And I will be sure to highlight or call out other labs and sessions more closely aligned to your needs.

Okay, let's just jump into the SMP KPI changes. I realize this wasn't hyped up very much in the media, but this easily represents one of the most significant changes in OS X history. The essence of the SMP KPI work starts with two major benefits. The first is the capability for the OS to scale as the hardware scales.

Underscoring all of this was a big change in the kernel where we moved away from two giant locks to a much finer grained locking model. The benefits from any kind of SMP work are typically expressed via performance as the number of CPUs goes up, but there's also positive impact at the single processor level as well, involving better concurrency.

Some examples include the ability to do concurrent stats, reads and writes to a single file, as well as greatly improved lookups, much faster lookups. In general, as an application developer, you get a lot of these performance and scalability benefits for free without having to do anything to your code.

The second benefit is this sustainable KPI, or Kernel Programming Interface, which isolates kernel developers from implementation details. As many of you already know, binary compatibility and performance optimizations on our part are very difficult when dependencies are made or built up around accessing internal data structures, private variables, and betting the farm on an implementation specific. The KPI prevents that, or isolates you from those changes, and gives us binary compatibility moving forward.

For those of you who are kernel or file system developers, I do want to call out the KPI feedback at apple.com mailing alias. This is something that's been active for some time now. And again, you can use it to provide feedback, suggestions, start some dialogue. I also want to call out this lab tomorrow at 3:30 PM. For those of you who are, again, File System developers, this is dedicated for you. It's going to be at 3 o'clock tomorrow in the Performance and Tools Lab, which is right behind, I believe, this wall here.

Before I move on, and because I won't have a chance to broadcast this at the lab tomorrow, I did want to call out two specifics involving future plans in the SMP KPI area. Just to be clear, this slide, or this particular slide here, is targeted toward the handful of File System developers in the room today, and not you application developers.

The first is that we do plan to drop Funnel Support in Shibli, or Leopard. Is it Leopard? Yeah, Shibli, their next major release. This means that all of you will have to protect your data structures on your own and not depend on the funnel. From our own experience at Apple, and even at other companies which underwent a similar type of transition, if this is new territory for you, you'll probably want to pay special attention to the potential new races caused or created in the Rename, Unlink, and Create paths.

The second change here involves the FNode cache semantic. This is a direct I/O of sorts for those of you familiar with that paradigm. We do plan to expand this semantic to the per file descriptor level as well as maintain the more global per VNode semantic. Those of you who write File System will just have to pay attention to the I/O flag when you read and write VNOPs.

All right, let's move right along to Extended Attributes, or EAs for short. EAs simply refer to arbitrary bits of data which do not live in the actual file itself. As application developers, I'm sure there's a ton of EA uses which we haven't even seen yet. But examples today include things like a place to put hints, checksums, keywords, and so on, especially in the cases where the file format you're working with makes no such allowances for such additional content.

Quick overview here. Extended Attributes or EAs are going to be supported on all file systems and network file system protocols, starting with Tiger. In cases where the file system or network file system protocol supports EAs natively, that's fantastic. We'll go ahead and use that method. In cases where the file system does not support EAs natively, we will synthesize support at the kernel VFS level and use Apple Double Files as the backing store. Apple Double Files are, of course, those corresponding dot underbar files. We'll be using a very simple reverse DNS naming convention for each EA tag. That's definitely not something that's new as far as a convention for file naming and so on.

I didn't want to call out two current limitations. The first is an approximate 4K per EA size limitation. You should note that HFS is definitely capable of completely removing this limitation. We just didn't get around to it. We'll definitely be able to move this in the future without incurring a volume format change. Chris Emura The second limitation has to do with the EA API itself. Currently, it's very much a get-and-set type API, even though, again, we have the implementation robustness to handle a much more streams-oriented API or access workload.

Before I jump into the actual source code walkthrough, I wanted to give you a lay of the land of sorts with regard to the various layers or frameworks. All of you are already familiar with Carbon and Cocoa, but the vast majority of today's presentation as far as the source code walkthrough will be taking place at the BSD API level. The one exception will be the Finder copy engine, which of course takes place at the Carbon level.

Okay, moving right into the ListXAdder call as far as the EA API. ListXAdder, of course, lists the EAs for a given file. And if you look at the sample code here, the flow is actually pretty simple. We have the first call to get the size of the list.

We go ahead and then allocate a buffer. And then, of course, the second call to get the list itself. This particular call is wrapped in this LSX.C sample code. Incredibly simple, incredibly distilled down, very easy to read. It's a command line utility of sorts you can use to experiment with EAs.

The Get Ex Adder call, of course, gets a particular EA on a given file. Similar flow to the list call in terms of the two passes, if you will. The first to get the size, and the second to get the, in this case, the EA itself. This is going to be wrapped in that same sample code.

Moving on to the Set X Add or Call. This of course sets an EA for a given file. This comes in its own wrapper as far as this setx.c sample code. Again, very simple and distilled down. You can use to experiment with. I didn't want to call out the last argument. And here it's zeroed out.

But the last argument to the Set X Add or Call, it's an options field. And for the Set Call, you can actually specify setting an EA for a symlink or a symlink target. You're able to differentiate between those two. And we've also provided a replace semantic where you get an error if the EA already exists and also a create semantic, which is very similar to the open O create.

The Remove X Adder call rounds off the EA API family and of course simply removes an EA or named EA from a given file. It's worth noting that all of these calls have got an equivalent identical version that takes a file descriptor instead of a path. And that option I mentioned about differentiating between a symlink with a symlink target is a common option for all of them. This Remove X Adder call here is wrapped in its own rmx.c sample code. It's the same theme, it's like a utility you can use to play around or experiment with EAs.

Okay, a quick few words about preservation. Preservation is something that all of you need to be mindful, even if you do not use EAs directly. I typically worry most about those of you who are doing your own copies, moves, archives, backups, or safe saves. By safe save, I simply refer to a mechanism whereby prior to modifying a file, you go ahead and make a complete copy, you work on the copy version, close it off, and then do an atomic rename or exchange data so that you get these all-or-nothing type semantics. But in general, when it comes to preservation, we have an internal mantra we use that always boils down to three things. The first, of course, is data. This is no surprise to anyone. Basically, what's in the file, the data fork.

For those of you who are familiar with that paradigm. The second one involves EAs, or attributes in general, and EAs, Extended Attributes. You should note that things like the finder info and resource fork will show up in the EA namespace, so you are able to use the EA API to preserve those. The third thing here is the security information.

In our context for today, this is typically going to be the ACLs or the access control lists. I'll be talking about that in a little bit more detail shortly. But again, if you need to focus on these three things when it comes to preservation, you need to be mindful of the EAs. preservation, you'll be in good shape.

I wanted to bring up industry fragmentation a little as far as it relates to preservation. And by industry fragmentation, I just mean that if you look throughout the industry on various platforms, and even versions within those platforms, EA support can be spotty, right, as far as what's supported, what's not. On top of that, you also have size limitations, which definitely are not consistent throughout the industry, and sometimes have very dubious error paths, if you will, as far as losing things that kind of fall off or get truncated.

So definitely something to be aware of there. Chris Emura On the topic of industry fragmentation, we have tried some things internally that have been exported via things like the tar, the tar binary on our system, where we do synthesize dot underbar files to help deal with some of this industry fragmentation.

But it's by no means a panacea or, you know, a complete solution. So just something to be mindful about. Of course, tools, applications, and utilities on our platform have been retrofitted or modified to be EA-aware and, of course, preserve everything. This, of course, starts with the Finder, as far as Finder file operations, copy and move, and so on.

It goes all the way down to the core commands, especially the big ones like CP, move, tar, and rsync. I believe that's the first time in history this has happened, so-- I mean, out of the box anyway. So a big round of applause for those people that have done that work.

On the topic of preservation, I did want to call out two calls. The first one is this rename call as far as semantics that we've worked hard to preserve here. Rename is typically used a lot in that safe-save context I mentioned earlier. To demonstrate this semantic, we start with two files, foo and bar, and both these files have been initialized with the data in an EA that clearly reflects their initial state or their origins.

We go ahead and do a rename of foo to bar. And you see that the rename semantic is very much intact in terms of all that has really changed is the name. I mean, the file foo is now called bar. The data and the EI are exactly where you'd expect them to be.

The second call I wanted to highlight here was Exchange Data as far as those semantics. And again, we start off with the exact two same files, foo and bar, same initial state with regard to the data in the EA that clearly reflects their origins. We go ahead and do an exchange data on Foo and Bar.

And here you see that the exchange data semantic has also been very much intact. As far as all that has changed is the data fork, if you will. It's been swapped or exchanged. Everything else, as far as the name and the EA, and the ACLs for that matter. But the ACLs just follow the EA as far as the semantics go.

The exchange data semantic here has been preserved. I know this might take a little while to get used to or digest, but you do have those utilities to play around with this. And as long as they've been called out, this is something that you can predigest in your own time, I suppose.

Okay, moving on to access control lists, or ACLs for short. In a nutshell, ACLs are a permissions model typically associated with, but definitely not limited to, files and directories. As an application developer, they give you much finer granularity around what you, your application, and your users can and cannot access.

They do help enable much more effective collaboration, workflow, and sharing. Fortunately, the ACL paradigm is not new to the industry, so you'll find a good use of, or a good group of users and professionals that will be able to use this functionality right out of the box without any kind of initial shock.

Let's go over some quick terminology that will be used throughout today's presentation, actually even in some of the documentation and source. Access Control List or ACLs, which are made up of Access Control Entries or ACES. The old school POSIX or UNIX UID, User ID, and Group ID. You want to spend a little time here as far as GUIDs and UUIDs.

For the purposes of today's presentation, as well as I've noticed in the header files, documentation, source code, and things like that, in that context, as far as ACLs is concerned, these two terms are used interchangeably. So don't get too hung up with that as far as some of the swapping that goes on, especially in a few of the header files I've seen. Of course, we have the Windows Security ID, just the model used in the Windows network and security protocols.

Okay, let's start off with where we are today, or where we were as far as the old school historic Unix permissions model. The UID, GID, and this Mode T, or these permission bits. Pretty straightforward. You've got, you know, read, write, and execute bits for the owner, yourself, a group you belong to, and then other or the rest of the world. Read, write, and execute there.

You also have this sticky bit semantic, which a lot of you are familiar with in terms of temp directories, drop boxes, and such. You also have this set UID and set GID, most commonly used for running executables as someone else. A bit of a no-no. Also have this 16 group limit, this historic 16 group limit, and there are definitely no nested groups or groups within groups.

So you compare that with the much finer granularity of ACLs, and here I've just enumerated the permissions for a file and their directory counterparts. You should note that there's definitely a allow or deny tag that's associated with each one of these permissions, so the granularity is quite fine without being overly complex or complicated. If you look at any one of these permissions here, there's nothing here that seems very difficult to understand or grasp.

Let's do a little overview here. ACLs, again, are made up of ACES, or Access Control Entries. Here I've cut and pasted a little bit from the retrofitted ls command, showing the ACL or the access control list for the file foo. You'll see them enumerated there as far as the seven ACES.

If you look closely at this, you'll see that we do support inheritance, but by inheritance I want to call out the fact that we only support static inheritance at this time. Static inheritance simply refers to the fact that an object or a file or directory inherits its ACE from its immediate parent at create time.

Now you contrast that with what a lot of you think about when you think about inheritance in general with regard to you have a huge directory hierarchy, and you want to slap an ACE on at the very top of the root and have it somehow trickle down. That can be done manually, but as far as static inheritance, again, static inheritance is the case where an object inherits its ACE from its immediate parent at create time.

This last bullet here is my attempt to give you a simple mental model for how the kauth or kernel authorization subsystem makes its decisions, how the ACLs model kind of works. We always start with this tuple, this identity, operation, and object tuple. The identity is basically you or your GUID.

The operation is what you're trying to do, whether it be a read, write, delete, and so on. And the object in this case, in this context, is typically going to be a filer directory. So you start off with that tuple. We go ahead and scan the ACEs in order, as far as you look at that list above, the one through seven. Scan those ACEs in order, try to make a decision, and of course, along the way, we go ahead and compare identities and resolve groups as needed.

If after all of that we still cannot make a decision or don't have enough information to make a decision, we do fall back to the POSIX Mode T, or those old school Unix permissions I mentioned earlier. So that in a nutshell is the mental model as far as the ACLs decisions are concerned.

Continuing on with the overview, ACLs are enabled by default on the server product only. And I put only in quotes here just because there is a FS ACL cuddle command that ships on even the non-server version. I don't think it's currently documented. That'll probably change. But I have a slide. The next slide will show a quick usage there so you can get started and start experimenting with ACLs on your non-server version.

The supported file systems and network file system protocols include HFS, CIFS, and AFP. And note that because we are so functionally close to the Windows ACLs model, there's great potential with regard to the CIFS client and integration there. For those of you who are IT professionals or just want to learn more about ACLs in general, there's a very dedicated session tomorrow at 2:00, session 617, so be sure to attend that.

Here's that command line blurb I mentioned. Here we go ahead and use the fsaclcuddle command to enable ACLs on a volume, the dash p or dash path to specify the mount point, dash e to enable. Although it might not be in the documentation, if you do go ahead and enable ACLs on your volume just to play around, you may want to then unmount and mount that volume, or just reboot just to make sure there's no caching effects in the upper layers.

The change mod command here has been retrofitted with this +a or -b, and it's been retrofitted with the change mod command. So if you want to use the change mod command to enable ACLs on your volume, you can do that. Chris Emura Here's that command line blurb I mentioned. Here we go ahead and use the fsaclcuddle command to enable ACLs on a volume, the dash p or dash path to specify the mount point, dash e to enable.

Although it might not be in the documentation, if you do go ahead and enable ACLs on your volume just to play around, you may want to then unmount and mount that volume, or just reboot just to make sure there's no caching effects in the upper layers. The change mod command here has been retrofitted with this +a or -b, and it's been retrofitted higher level GUI administrative tools.

Let's go ahead and walk through the Access Control List, or the ACL API, the user-land version anyway. I wrote a very quick ACL API fragment.c sample project. All this sample project does is create a file foo in the current working directory with what I consider read-only permissions. And I know that sounds incredibly straightforward, but there is quite a bit of detail that I want to walk through just because, again, the documentation has been in a state of flux.

Chris Emura First thing we do here is make the path conf call to see if ACLs are even supported on the directory or the file system we're working with. Pass in this extended security non-portable flag. For those of you who work with network file system clients, there's also a PC auth opaque NP or non-portable flag you can pass in to see if authorization is occurring locally or on the server. Again, that's probably more useful for those of you who deal with network file system clients.

We go ahead and get our GUID from the UID. This is probably one of the most frequently asked questions from the last WWDC as to how do I get or how do I generate a UID. These MBR, these member functions, are finally documented in Tiger, so take a look at those. There's some good detail in the man pages. So we go ahead and generate our UID and pass it on into this ACL read-only example function. This is the function that does all of the heavy lifting. So let's go ahead and dive into that.

ACL init, go ahead and initialize our ACL structure. I'm not sure why I passed in 32 there. But basically 32 simply refers to the number of aces or slots in that particular structure or that particular ace. There's a corresponding ACL free call that you should be using to clean up after yourself.

Create Entry: Create an actual ACE. The actual call I wanted to highlight was the non-portable version of this call. If you remember the mental model I mentioned earlier where we process the ACEs in order, you can see that order obviously matters for both deny and allow permissions. So this non-portable version of the call does allow you to specify a position as a third argument. Definitely a useful feature.

Allow or deny, we go ahead and set allow in this case, and then go ahead and associate our UUID with that particular ACE. The classic get, clear, add, and set flow. This is basically where we just add the read-only permissions that I've defined to that particular ACE. Those read-only permissions are defined a lot earlier in the file. And here you'll note that there's an interesting parallel with that preservation mantra I mentioned earlier, where we fixate on three things, the data, the attributes, and the security information. So that's a consistent theme there.

Finally, get to see the light at the end of the tunnel with regard to associating with the actual creation of the file. Here we use this extended non-portable open, the openx_np. There's a whole class of calls like this as far as being extended and non-portable that take a file sec t as an additional argument. That file security object is what houses our ACL, and it's what we pass in. So basically here, if the open call succeeds, you'll be left with a file foo with those read-only permissions I defined earlier.

Okay, moving on to file notification. As application developers, it doesn't take much imagination to see how important file or directory chain notification can be for your deployments. And again, although it's not new for Tiger, I didn't want to call out KQs as being the de facto standard for userland file notification.

Those of you who have seen KQs or have used it on other platforms realize that it's far more efficient than disjoint implementations of SELECT and PULL. It's great for directory update notification. In fact, our very own Finder uses KQs, and it's very evident when you look at highly visible active windows like the desktop, for example.

You go ahead and create some files in a terminal or an SSH session and see them appear instantly and very efficiently in the desktop. Chris Emura That said, there still are some current limitations. The first is that KQs aren't currently supported in all volume formats, although if you've been following the industry, you do realize that there is precedent to uplift KQ support into the kernel VFS layer such that all file systems can benefit.

Local only here refers to the single client or single server multiple client type configuration where an update caused by another client may not show up on yours. No payload actually means a lot of different things, but here I just want to call out one case where when you do register for an event, you don't get any hint or payload that tells you what has changed.

So, in the case where you register against a particular directory and you're just waiting for events there, you'll get an event that tells you that the directory has changed, but you won't have a payload or hint to let you know what file or directory within that directory has changed.

Finally, there is what I refer to as the Recursive Directory Hierarchy Challenge. This is the case where you as a developer want to monitor an entire directory hierarchy and then be notified whenever anything in that hierarchy changes. Of course, with a single KQ registered against the root immediately underneath that particular parent, a change that happens much lower in the hierarchy will go unnoticed.

Because it is a de facto standard for user land file notification, we did provide some really simple sample code. Again, this is another completely standalone command line utility that can be run against a file or directory and then prints the standard out whenever something has happened. Chris Emura We start by allocating a KQ structure here and then go ahead and use the EV set macro to initialize that data structure. I did want to call out the OEventOnly flag, which for some reason isn't highlighted here, that flag over there in the open call.

Chris Emura A very useful flag to use, especially for those of you who have always been hit by the I cannot eject the volume cleanly type scenario where you try to eject or unmount a volume and you get a message telling you that it's in use. Chris Emura This particular flag here is useful in that case where your reference will not prevent the volume. from being unmounted cleanly.

As I mentioned earlier in the ACLs portion, the KAUTH subsystem is a general framework or kernel framework for authorization decision making. I really want to highlight the second point. It was not initially designed for file or directory notification. That said, if you look at how cleanly it's been architected as far as we finally have a case where the redundant authorization code has been uplifted and stripped out of the individual file systems and then centralized so everything effectively goes through the subsystem. So you take a look at that particular architecture, that design, and you combine that with the fact that we do allow third-party developers to write keks or plug-ins, and it becomes very tempting to use this as a file notification mechanism.

Should you go down this path, the basic design involves the registration of authorization events you're interested in, that's your scopes, and a pretty simple voting mechanism. Actually, the main takeaway from this slide here is that your plugin or your text is permitted to block, and this is shorthand for it's very easy for you to bring the entire system down. So definitely be aware of that. You want to take care if you're in a commonly called authorization path. It's the hot paths I mentioned there, basically something that's getting called over and over again.

And you want to be extra paranoid if your text calls outside of itself, because you definitely increase your chances for deadlock there. For those of you who really want to do this kernel programming, go down this particular path, we do have two documents, or two reference bits of reference material.

I believe the first one, I think they're both on the DTS website, you can do a search for kauth. There's some documentation as far as a white paper, and then there's also this sample text you can use to get started and experiment a little bit with the kauth subsystem.

Okay, let's go ahead and move up the stack a level into the Carbon File Manager domain. If you remember all of those steps you had to take to preserve data, EAs, and ACLs, and it's a little bit too hairy for you, you can save yourself a lot of trouble and finally use the same copy engine the Finder does.

When you get a chance, take a look at this FS file operation sample project, which again is a basic, simple command line wrapper that demonstrates this functionality. I've just called out the async version of this copy call, but keep in mind that there's also a move variant and a simpler synchronous version of both calls. This API supports both the POSIX paths as well as FS refs. It supports a limited form pre-flight and is integrated with the CF run loop. So again, just go ahead and check out the FS file operation sample project there to get started.

Okay, oldies but goodies are winding down here. These are basically issues again that may not necessarily be new for Tiger, but are definitely worth covering in more detail, especially since documentation can sometimes be scant or non-existent. The first thing I wanted to call out was this UID or this User ID 99.

This is actually, this starts with the Ignore Ownership on this volume checkbox that you'll see on the finder's Get Info panel for any non-network volume. That setting is the default for drives which were not present at first boot after an install. If you're not sure, you can go ahead and take a look at the var db volinfo.database to see if your drive is actually in there.

This corresponds to this mount ignore ownership bit in the mount T structure. You can programmatically check for this by calling statfs and look at the F flags field, I believe. The mount HFS, the lower level mount HFS mount command itself does take a -u and -g option. We can provide a user and/or group. And that capability definitely adds a wrinkle with regard to expected semantics. So what I wanted to do was walk through some of the possible configurations there.

This is the default case where you're basically doing nothing. You're not setting any bit or setting any flag in the finder get info panel, and you're certainly not passing anything in as far as the mount command is concerned. There you get pretty much what you'd expect. The UIDs and GIDs are preserved, meaning that what you read is what's actually on disk, and what's written is pretty much who you are. Chris Emura They raise this wildcard, this UID 99, as far as preexisting files and directories with this UID.

Those will be considered owned by the current user for authorization purposes, so you'll have free reign over those files. Again, this is the default case. For volumes mounted with that Mount Ignorant Bit set, where you basically go in the Finder window and you check that bit, but you still haven't passed anything in with regard to the Mount HFS command, here your UID reported will be 99, regardless of what's on disk. And what's written will be 99, regardless of what your UID is. Items will be considered owned. Files and directories will be considered owned by the current user for authorization purposes. Basically means you have free reign over that particular volume.

Chris Emura So far, so good. Here's where things get a little bit more tricky as far as volumes mounted with that Mount Ignorant Ownership Bit set. And you're also passing in an explicit owner at mount time via that Mount HFS command. Chris Emura There, the UID and GID reported will be whatever you passed in. What's written will still be 99, and files and directories will be considered owned by the provided owner for authorization purposes.

And then rounding off as far as the fourth case, in the case where mounting ownership bit is not set, but you are passing in an explicit user group at mount time. This is actually kind of similar to the first default case where UIDs and GIDs are preserved for the most part, meaning that what you read is what's on disk and what you write is pretty much who you are. The difference here is that items with the UID 99, as far as files and directories on disk with pre-existing UIDs of 99, will be considered owned by the provided user for authorization purposes.

Okay, moving on to some miscellaneous topics. The F-Sync one especially. This has definitely got a lot of mileage on the external boards. The first, this F-Sync one involves widespread public misconceptions around F-Sync on OS X not working correctly or resulting in some kind of data loss or corruption. Nothing could be further from the truth. F-Sync on OS X and just about every other Unix platform has always flushed data from the host or from the system down to the drive on which the file resides.

The critical detail to be aware of here is that while the data has made it to the drive, it may be in the drive cache and not the actual platters, what we consider stable storage. To complicate matters much further, the drive or the storage subsystem may also reorder the writes such that any given writes in say A, B, C order followed by an F-Sync and then more writes in D, E, F order followed by a fault or a power outage, crash, whatever.

You may see cases where, you know, writes A and D have made it to disk but B and C have not. This behavior is clearly unacceptable for a whole bunch of users as far as the general file system, databases or any applications that require this unstable storage semantic and, you know, no write reordering. So we've introduced this F-Sync F-control to ask the drive to flush its cache.

Of course, history continues to demonstrate that there are certain drives or storage solutions which Apple does not qualify that basically ignore this command anyway and you're left with the same problem. Of course, history continues to demonstrate that there are certain drives or storage solutions which Apple does not qualify that basically ignore this command anyway and you're left with the same problem.

HFSx or Case Sensitive HFS. This last point here is something you kind of mentioned in the feedback forum as far as case insensitive environments. As application developers, my plea to you here, or the takeaway I want you to focus on here is, you know, please test your applications on these case sensitive environments.

We've had incidents both internally and externally that have resulted in breakage just because an application developer, for whatever reason, was hard coding these lookups in their code. And they got away with it for a long time as far as running on case insensitive HFS. But again, if you can just test all your applications in your environment, make sure there's at least one matrix configuration for this case insensitive configuration.

Okay, tools. Tools to help you debug file system issues. The first one is FS Usage. Pretty straightforward. This is actually a pretty simple command. Pretty powerful, too, in terms of dumping out all file system activity on your given platform or on your given system. I personally use the "-w option a lot to get a much wider printout of the path and just more information in general. There's also this dyld image suffix variable you can use. You can set it to default and just get the Carbon File Manager calls in line in context with regard to the output. The man page has got all that detail, so be sure to reference that.

The Sample Call or the Sample Utility is definitely not file system specific, but we use this quite a bit internally to help debug cases where the system is hung, at least user-land-wise it's hung. You get the spinning beach ball of death and so on. The Sample Utility can be pointed against a particular process and you'll get a number of stack traces to give you a hint as to where things are stuck.

LSOF List of Open Files. This is definitely not specific to OS X. It's imported on a number of Unix platforms. This is most useful in cases where you're trying to debug a situation. You've got a lingering reference on a volume that cannot be cleanly ejected or unmounted. LSOF is great for dumping that out as far as being run as root. One thing you want to be aware of in Tiger, which is nice, is the fact that LSOF has been enhanced to print out MMAP references. That's definitely a welcome change and will help you catch those type of references as well, which previously went unnoticed on Panther.

KTrace, basic syscall level tracing, very similar to PAR on iRix or Trust on Solaris. Basically gives you the syscall, a little bit of payload information and return value. It can give you a feel for where things are going wrong. And then finally you've got general profiling tools like Shark. I like Shark just because it does cross the user land kernel boundary.

It gives you a little bit more holistic view of what's going on with your app. It's also quite good at digesting huge amounts of data and showing them in nice condensed GUI form. So you can kind of pinpoint areas you want to drill down on and figure out what's going wrong.

I'm just going to do a quick demo here to demonstrate those command line utilities I mentioned earlier and also kind of hit home with the preservation mantra I mentioned earlier. I'll go ahead and switch to the demo.

So this is a representation of what you have on your DVDs or on the DTS website as far as those utilities I mentioned earlier. They have been Xcodified if you want to go ahead and use that, but I'm just going to quickly do a make here.

So, you know, I'm just going to create a file here, touch A as far as, and then do a set X, what should we call it, com.apple. Set an EA for this particular file. Pretty simple semantics there, simple usage there. List those extended attributes there. You can remove them as well, I guess. Remove X.

So we removed it there, unless nothing's there. Let's go ahead and set it again real fast. Wanted to quickly show that if you look at things like the copy command-- actually, the only way we should do is actually set an ace for this particular file as well. So change mode, plus a-- actually, who am I? Let's go ahead and just add an A quickly to that file there. Change mode, plus A, it's local, allow read, something really simple. Of course that ls-le again on A will show it there. So now we have a file A with both an ACL and some extended attributes.

Just to show that the copy command does work as advertised. Oh, you know what we should do is actually use the preserve. I forget about this all the time. This is the default for the finder, but not for the copy command. It's just preserve, or preserve permissions and stuff. A.copy.

So just make sure that the actual EAs are preserved as well as the... More interesting thing for a lot of you is the Finder info, or the Finder making sure that it does the same thing here. As far as the A, I'm going to go ahead and select that. Do a duplicate as far as the Finder copy.

That's kind of difficult to read there, but if we do an LSX of A star, you'll see that the A space copy is also preserved. That's the byproduct of the Finder copy there. Just to show you guys what I said earlier about the Finder info and resource forks also showing up in the EA namespace. I'll go ahead and start with that file A. I'll go ahead and create an alias. We'll get a resource fork loaded up there.

Also, that's the easiest way to do this. We get info, kind of tag that file with a color, which will get some Finder info information in there. It's red now. Again, if we do a LSX of A star here, you will see that that file A, as far as that little flag I added, that color flag, got a Finder info entry. Then the alias itself, of course, lives in the resource fork. All of that stuff is preserved. If you do a copy, So, cp minus p.

Another copy. You'll see that that's preserved as well as far as even that flag, that red flag I added there earlier. Also show that the ACLs have been preserved as well. So you see here that the ACEs for those particular files and their copies are all preserved.

That's pretty much it as far as just showing you how these utilities basically work. You can play around with this and experiment with scalability, performance, and all those things. Pretty straightforward stuff. Right? All right. I think we're about done. We're not quite done yet. It's more information really quickly.

Go to this WWDC site here. Again, do a search on this particular session or the kauth if you wanted to dive into that a bit. Those two related sessions I mentioned earlier, the ACL session tomorrow at 2:00, and then right after that in the Performance and Tools Lab, the session or the lab targeted towards FOSTER developers. But if anyone, feel free to attend and interact with my group. Some contact information.