Welcome to Creating I/O Kit Drivers for Multiple Architectures and OS Versions. My name is Ethan Bold, I'm an engineer on the I/O Kit team, and my co-worker Jay Towslee is going to come up on stage in a little bit and join me. So today we're here to talk about some things you need to know, and some tools that you need to build your drivers to run on as many versions and architectures of OS X as you need to. For example, you might be releasing a driver that you want to support as far back as OS X 10.4 for 32-bit PowerPC and 32-bit Intel architectures; but that you also need to support on 10.6 Snow Leopard for your 64-bit Intel arcdhitecture.

So one of the things that you would do to do that, would be to... to go into X code and define a, let's see, for your 32-bit PowerPC architecture you would target the 10.4 SDK for your Intel 32-bit architec2ture. You target the 10.4u SDK and for your 64-bit Intel architecture you target the 10.6 SDK.

Now an SDK is a software development kit, and basically that's a snapshot of all the API's that we publish with a given major release of OS X. We release a new SDK with every release of OS X, every major release, and each new SDK is likely to contain deprecated API's, are likely to replace API's, and to add new API's. And since you'll be developing drivers against different SDK's, you need to know some of the big differences in those SDK's and how to handle them. So let's get started.

First off this is not an introduction to I/O Kit. I'd recommend you check out the I/O Kit fundamentals talk from WWDC 2008. I think that's available for download on-line. It's a good intro level talk. Also this is not an iPhone talk. We're talking about kernel extensions, and kernel extensions are not supported in the iPhone SDK. So we are talking about kernel architectures, different architectures of user processes, and K64 and 64-bit address dspaces. So by kernel architectures I mean 32-bit PowerPC, 32-bit Intel, and 64-bit Intel.

User process possible architectures include 32-bit PowerPC, 64-bit PowerPC, 32-bit Intel, and 64-bit Intel.d And we're gonna talk about K64, we're gonna talk about some of the differences between K32 and 10.5 and earlier, and K64 in Snow Leopard, and then Jay's going to come up on stage and give a demo of how to implement some of this in Xcode.

So let's talk about building drivers for multiple kernel architectures, and let's start with a little history lesson. In the beginning from 10.0 to 10.4, PowerPC was the only... 32-bit PowerPC was the only supported kernel architecture. So for you as a developer, things were a little easier because you only had one architecture to develop for.

And then in 10.4, 10.4.4, and 10.4.7 we introduced some new Intel hardware with a 32-bit Intel kernel architecture. Starting from that point, if you wanted to release drivers that targeted both kernel architectures, you needed to release a FAT driver with binaries for both PowerPC and i386. So coming in in Snow Leopard, we have new 64-bit kernel support and we have dropped 32-bit PowerPC support in the Snow Leopard kernel. So with each of these new architectures came changes that we had to make and accommodate in the API.

So we're going to talk about some of the API changes that accompanied these architecture changes. And first we're going to talk about... before we get into the actual changes between OS X SDK's, we're going to talk about tools you can use to write code against multiple SDK's and a single source space.

So first off we're going to talk about AvailabilityMacros.h. This is a header file, you can include. It's defined in 10.5 and 10.6 and it defines a few constants - two important ones that we're going to talk about today are MAC_OS_X_VERSION_MIN_REQUIRED, and MAC_OS_X_VERSION_MAX_ALLOWED. These are pre-processor constants you can use to conditionally compile your code. MIN_REQUIRED let's you say, run this code against... when we're building against 10.4 SDK's and higher.

Like I said, this is only available in 10.5 and 10.6. We do have an alternative that you can include, I'm sorry, we have a header in our AppleSamplePCI project. It's a project associated with this session, and you can copy it and include it into your target to provide similar constants when you're building against a 10.4 SDK.

And the one thing I haven't mentioned is that MAC_OS_X VERSION_MIN_REQUIRED is specified by your Mac OS X deployment target. That's a setting that Jay's gonna talk about in your Xcode build settings. So here's an example of how you might use MAC_OS_X VERSION_MIN_REQUIRED to conditionally compile some code against 10.4 and against later OS's.

So in this top block of code... First I should say that this code implements a virtual method from the I/OUserClientClass. So if you're implementing an IOUserClientClass, prior to... let's see, in 10.4 and earlier you would have implemented the method IOUserClient get target and method for index, to provide user client access to processes attaching to your driver. And from... that API was deprecated and replaced in 10.5 and later by the method IOUserClient external method.

So this code at compile time says if we're building for an SDK that's 10.4 or earlier, we're going to use the old deprecated get target method for index. Otherwise for building against a 10.5 or 10.6 or later, we're going to implement IOUserClient external method instead. So like I said, MAC_OS_X VERSION_MIN_REQUIRED is one of the big tools we can use to deal with SDK and OS version differences, so let's move on and look at a few actual differences.

First up... when we added i386 support in 10.4.7 we also added Rosetta. Rosetta lets you run PowerPC, 32-bit PowerPC b2ased applications in user space over your driver sitting at 32-bit Intel kernel. The big consequence of this is that if you communicate and share data between your driver and the kernel, and this PowerPC process and user space, is that PowerPC and Intel have different bit orders.

They represent bytes differently internally, and this is something that you need to be aware of, and we're going to later on talk about some ways that you might cope with that. Next ... in the 10.4u SDK for the Intel architecture, we introduced a class called IODMACommand. This replaced IOMemoryCursor for any DMA that you might be doing in your text, and... yeah, just need to know about it if you're working with any pre 10.4 code that does DMA.

Let's see, so moving on to the 10.5 SDK, one of the big pushes in the 10.5 SDK was to add support for 64-bit addressing within the kernel, which will help your driver when you adopt these API's - will help your driver communicate with 64-bit processes in user space. But to be clear, this was not K64 yet.

K64 arrived in 10.6. This changes the support 64-bit addressing, so the two big API changes that we're going to look at now are IOUserClient, and IOMemoryDescriptor. IOUserClient is a class that you can implement to share data between your driver and user space. And we replaced the, we deprecated the method "getTarget" and method for index because it did not support 64-bit data types, and replaced it with the method "externalMethod". And with IOMemoryDescriptor we deprecated several API's for similar reasons, and replaced them.

Let's look at examples of each of those. So this is the same code sample that we just discussed. On the top we're running, on the top we're building against an SDK 10.4 or earlier. And on the bottom we're building against an SDK for 10.5 or later. So it's as simple as that.

We've got an API that we know isn't supported after 10.4, and we're easily able to split it up like this such that we can build against a 10.4 SDK to the architectures that we want to support our driver on, which would be 32-bit PowerPC 2and 32-bit Intel, and the example I gave earlier. And we can support K64 in 10.6 as well. So we have for IOMemoryDescriptor, we have this enormous chart which we usually do a quiz on after the talk - so pay attention.

And again these are just changes we made in the 10.4 SDK and earlier, versus the 10.5 SDK and later. And here's another little code snippet showing how we use Mac OS X version min required to conditionally compile for the 10.4 SDK versus the 10.5 SDK. There's one other interesting little tidbit I have to point out about this code, which is that we're not only checking if MAC_OS_X VERSION_MIN_REQUIRED is less than or equal to Mac OS X version 10.4, but we are also checking if we are building for a 32-bit PowerPC architecture; and that's because... in the 10.4 build frame, in 10.4 when we released Intel, we maintained two different SDK's. So while we only released one SDK for 10.6 and one SDK for 10.5 there are two SDK's out there for 10.4.

PowerPC builds against the 10.4 SDK, and Intel, 32-bit Intel, builds against the 10.4u SDK. So this code is using a method that's defined in the 10.4u SDK on the ?Elxsi interview statement, that's Intel code. And on the top that is using the previous method to implement that. OK, so we just talked about some kernel architectures in OS X, and some of the changes that you need to be aware of, and SDK changes that you need to be aware of in your drivers to build against multiple architectures and OS versions. So let's jump ahead and look at user process architectures. And really we're talking about how your driver communicates with user processes.

So your drivers in the kernel, executing in the architecture of the hardware, or executing in the kernel's architecture... processes are in their own address spaces, and executing in their own architectures, and sometimes you need to talk. So that problem arises when... like I mentioned before, when you are a 32-bit Intel or a 64-bit Intel process running in the kernel, and you need to communicate with a 32-bit PowerPC process in user space.

We're going to talk about that in just a second. First let's run through the history of user space architectures on OS X. First everything was PowerPC, it was 32-bit PowerPC. In 10.4 we introduced 64-bit PowerPC user process architectures, but we're not going to talk about that a lot for two reasons: one it was never very widely supported by OS X, and 2 - Rosetta does not support 64-bit PowerPC processes. So from now on when I talk about PowerPC user space processes, I'm generally talking about 32-bit user space PowerPC processes. So with our adoption of Intel we got 32-bit Intel processes and 64-bit Intel pdrocesses. That's where we are today.

So... let's first talk about whether you need to do any work to make sure that the data you are sharing between your driver and user space is correct. If you don't communicate any data with user space at all, then you're fine. There's nothing for you to do. But don't leave yet because we're still going to talk about K64 in a little bit. If your device communicates with user space through an I/O Kit family, then there is probably nothing special for you to do in your driver, because the family will take care of byte swapping.

Otherwise... otherwise you may have to worry about byte swapping and making sure your data is accessible to user processes that you're talking to. So the two easiest, most common bridges to share data between user space and the I/O Kit kernel in your driver are the I/O Registry, and the IOUserClient.

The IORegistry is... it's pretty lightweight and it's very easy to use. It's just a few lines of coding user space, and just a few lines of code in your driver to share data back and forth. The up side is that you get free byte ordering, byte ordering switching if you would need to switch byte ordering for your user process. The downside is that it's not suitable for large quantities of data.

If you have lots of data to share between user space and kernel space in your driver, you should look into using an IOUserClient instead. It provides much more flexibility and control, and it let's you define arbitrary entry points from user space into the kernel so that you can define calls, and your user process can pass arbitrary chunks of data to you from user space, and get arbitrary chunks of data back. Unlike IORegistry, you do need to be aware of the endianness of the process that you're talking to.

And in just a second we're going to talk about how you can do that. So let's start with a code snippet of some IORegistry code. Like I said, it's fairly simple to share data between a user space process and your I/O Kit driver. This sample on top shows this single line of code from user land that passes a dictionary of key value pairs into your driver, and as long as your driver implements the IORegistry method, IORegistryEntryMethodSetProperties, your driver can then get that dictionary and investigate all of the strings and numbers and arrays that it might contain.

So next up is a considerably scarier example of using the IORegistry. So you don't need to read this code very closely, this is mostly up here... to provide an example of the fact that it is unsafe to modify a collection in the IORegistry. If you have code in your driver that does a "get" on a property, and the IORegistry gets a dictionary or an array, and then modifies that dictionary or an array that is unsafe code which could lead to a kernel panic, due to the fact that those collection modifications are not atomic and if another thread should come along and try and read that property, either in user space or kernel space, you are at risk for a kernel panic.

So that's just something to be aware of. We also have some boot-r's that you can set via NVRAM that will kernel panic the second that you actually modify a collection, and that's great for tracking down this kind of bugs. If you're ever tracking down a very sporadic crash in your driver, I'd recommend you dig up that boot-r again, make sure you're not modifying any collections live.

So that was a brief look at the IORegistry. Here is the code that you would use in a user space process to share data with your driver through an IOUserClient. Now if you'll recall, a couple of times we've looked at a slide with example code, showing how you would conditionally compile your IOUserClient implementation for 10.4 versus 10.5 or 10.6.

And just like IOUserClient changed between 10.4 and 10.5, the UserClient methods to send data to an IOUserClient also changed. So this just demonstrates the same idea that we looked at in the kernel of conditionally compiling your code for a 10.4 SDK... when necessary, or a 10.5 or 10.6 SDK otherwise. So here's the slide I've been alluding to.

This is where we actually can check in our IOUserClient code whether the user process that we're communicating with is the same endianness of us, and if not make sure that we byte swap any data that we share with that user process. So... so this top block of code, this first block of code is run in the AppleSamplePCI User Client. Again, this code is all from the AppleSamplePCI project, sample code you can download.

This is from the init routine. So if you're not familiar with an IOUserClient, each IOUserClient attached to your driver is attached to exactly one process in user space. If you had five or six or a dozen user processes, user process clients of your driver, each one of them would have its own I/O User Client in the kernel; its own instantiation of this class. So this is in the init routine of AppleSamplePCI's IOUserClient implementation.

The first thing it does is it checks for the property User Client CrossEndian key. If CrossEndian key exists, that means that the client is of an opposite endianness, and we've have to swap data before sharing data with that client. So what we do is we set a CrossEndian compatible key to true, and we remember that this is a CrossEndian client by setting a local member variable, fCrossEndian to true.

And then later on, this is a snippet from when we're publishing data up to our user space client, how we just check that local member variable, and it's set. Then we're going to run OSSwapInt64 on this 64-bit data type, to individually swap the bytes to make it so that this data is readable by the process that we're targeting.

So I also want to point out that this code detects that it is attached to a CrossEndian client and it says OK that's great, I'm compatible. But you don't have to do that. You can, like this code could have just as well de-allocated all of its, all the resources that it had previously allocated, and returned null... and that user process simply wouldn't get a user client into your driver.

So you don't have to support CrossEndian clients. Alright, that's everything I have for managing different user space architectures. We're talk for just a little bit about K64. K64 is the new 64-bit Intel kedrnel in Snow Leopard, and we're going to talk about some of the big differences between K64 and K32, which was the 32-bit kernel - kernel architecture in 10.5 and earlier.

So the first thing to know about K64 is that pointers are twice as wide as they used to be. K64 has a 64-bit address space, so to represent that we have to use 8 bytes to represent a pointer instead of as in K32 when we only needed 4 bytes to represent that pointer.

One consequence of that is that if you have any code that assumes, like on our prior 32-bit kernels, that integers and pointers were the same size because they used to be. Integers used to be 4 bytes, integers were 4 bytes and pointers were 4 bytes. However in Snow Leopard, integers are still 4 bytes but pointers are 8 bytes.

So one consequence of that might be that you have code that stores a pointer into an integer data type, which... which probably would have worked just fine in a 32-bit kernel,2 but in a 64-bit kernel that 8 byte pointer stored into a 4 byte integer would truncate the top 4 bytes of that, and you would be left with a meaningless pointer. So if you're ever debugging an issue like this in GDB, one thing you can look for is that in Snow Leopard, in 10.6, the top few bytes of each address... of every pointer you have, should begin with 0x7ffff... and maybe a few more f's.

So that's a good safety check if you're ever investigating a pointer, or looking at a pointer and it doesn't look like that, it's wrong. Another thing to be aware of in K64 is structure alignment. Since longs and pointers have doubled in size from 4 bytes to 8 bytes... if you're passing any structures of integers and longs and pointers from a 64-bit process, or a 64-bit driver,d to a 32-bit process, then you need to just be aware that the compiler may very well have laid out that structure differently.

And I should also point out that if you're interested in learning more about K64, there's a couple of really good references that are much more detailed than what I'm talking about today. The first is the K64 talk at WWDC in 2008. It's a really, really information-packed talk. The second is there is a document published by Developer Relations called the, I think it's the K64 Transition Guide. And we have a link to that at the end of this talk. So I'm sorry, I was talking about structure alignment. The compiler, usually the compiler likes to align data types along boundaries equal to their size.

So... so an 8 byte pointer is likely to be aligned at a multiple of 8 bytes in the structure. So because of that, the compiler's likely to insert a lot of empty space. If your structures aren't optimally arranged, if it needs to put a 4 byte next to a 4 byte pointer, it could very likely just drop 4 bytes of empty space in between.

So not only is that a waste of space, but it might not be the same struct format that maybe your file formats were expecting, or other clients were expecting. We should also mention that... that with 64-bit addressing and K64, we started paying a lot more attention to address data types.

A lot of the data types we were using in the kernel and in user space were really only suitable for 32-bit addresses, so you'll see a lot of change, a lot of code moving to use these new bigger data types that store 64-bit addresses like mach_vm_address_t. And of course we already talked about some of the API changes in the 10.5 SDK that you need to pay attention to, as you're moving on to 10.6 and to support K64.

So... I've got a quick code snippet to show some of the different addressing data types that I just talked about. Up top this is some kernel code that used to use the data type IOVirtualAddress. It still does, when building against the SDK for 10.4 and earlier, and that is replaced by the... the more 64-bit friendly address data type mach_vm_address_t.

And here in user land, we have a similar sort of thing. This is a user space process and it's checking the seed... to find constant LP64, if it's building against a... a 64-bit architecture, in this struct, the middle of the struct that it's defining that will use the 64-bit address data type mach_vm_address_t.

Otherwise it will use the 32-bit address data type vm_address_t. And that's my very, very quick summary of K64. I'm really not doing it justice, so I recommend you check out that K64 talk from WWDC 2008. And with that I'm going to hand things over to Jay Tasley. Jay is going to walk through some Xcode settings.

Okay before we get started, quick sort of summary of the K64 development tools that you're going to need to look for. In the move to Snow Leopard we've changed GCC to 4.2 as the system compiler. There's a lot of calls that might have been deprecated in K32, or the PowerPC kernel. Not only are they deprecated, they're not there in K64. So deprecation is for real.

So we cleaned up a bunch of things, and you're going to need to watch for that. In terms of moving your KEXT over, the simplest piece of advice I can give you is enable... let the compiler do a lot of work for you. Enable all the warnings, ironically Wall doesn't actually enable all the warnings, so you might also want to enable Wshorten-64-to-32, which is the one that will probably cause you the most trouble; particularly if you're dealing with an old code base.

And finally, and Ethan's been alluding to it, architecture specific SDK's and OS version specific SDK's are something I'm going to show you how to use to target in your code the changes. And finally, if you're doing new development, all KEXTs are both 32 and 64 bit what we call universal. Three-way FAT if you're going to support back to PowerPC code on even as early as 10.3.9, 10.4. And certainly two-way FAT if you're going forward from 10.6 on. So with that, I'm going to switch over to the demos.

Do I need to... I switched my selector up here and I'm not getting video. Sorry.

What did I do wrong? This one...

Victim of my own success. Okay, this is the general settings for the project, the AppleSamplePCI project, and you're looking at setting a base SDK of 10.6 and this is pretty much the normal deal. Here also setting overall the gcc warning flags, those flags that I mentioned earlier.

It's also important to note that this is probably the kind of place where you're gonna see some architecture specific settings. For those of you that haven't played in Xcode in here in a while, remember that that's down in the little gear setting where I can add basically any new setting I want; specifically if I'm going to add to an SDK, I can do this... oops, sorry.

And basically choose, based on the architecture type, which SDK I might want to use. And theoretically you could take this example all the way back to 10.3.9. The reality is there is one piece of code that is not in the 10.3.9 SDK, which is a collections piece, which deals with that live registry piece that Ethan was talking about earlier.

But it certainly compiles and runs 10.4, 10.5, and 10.6. So looking at the settings for an actual KEXT itself, this is the setting for the KEXT that we're going to run on all 10.5 and all 10.6 systems. So we talked earlier about shedding both the SDK, so specifically we're setting the SDK's for 10.5 for Intel 32-bit and PowerPC 32-bit. We're also setting the compiler versions appropriately, and we're also setting the deployment targets appropriately.

So you need to get all three of those in alignment to make the availability macros do what you want them to do. Get that wrong, and it's a world of hurt to figure out which one you didn't do correctly, because numbers will not make sense - not that I've ever done that or anything. Anyway here's also where you're going to choose, if you're doing that new development this buys you a lot of things, just to say standard 32-bit, 64-bit Intel or 32-bit, 64-bit Universal.

Now if we look at the specific sub-KEXT that we'll talk about in a couple of minutes in this project, this is a KEXT that is built primarily to support 10.4. This is directly targeted with the base SDK, and you notice I've got the PowerPC SDK specified here, that's actually not necessary - you could take that out because it'll just pick up the 10.4 basic.

And you'll also notice that we changed the compiler version. So that is going to, when I build this, the flags that are set by this SDK are what's going to conditionally compile base in the availability macros. Anyway... so let's zoom back out so we can see a little bit more of what we're doing.

The KEXT itself...

... loads and it should be pointed out that this is a K64 system. And Ethan talked earlier about the addresses and the types of addressing you're going to see, and it looks like we got it a little bit backwards. So ignore that 7, it should actually be all f's and an 8. Sorry about that.

That says we didn't edit well enough, apologize. Anyhow, this is the magic number that you're going to see. If you're not seeing in your debugging on a K64 system, if you're not seeing this in your pointers, you've probably got a truncation issue and that's the time to go back, if you haven't already, and enable that shorten 64 to 32 and look for 2where that's biting you.

So now that the KEXT is loaded, let's try a couple of sample clients against it. So we'll start with PowerPC. PowerPC support, or Rosetta support, in the developer seed is enabled as an option so if you try this when you get home and it doesn't, or if you've tried it here at the show, it's because you'll need to reinstall or go into the installer and specify the option Rosetta setting.

But you'll see that Rosetta has loaded, and you can tell that Rosetta is loaded if we look over here, we can see that we have the CrossEndian flag set - this is my kernel log by the way - and you can also see where I've actually been able to use that set properties piece that Ethan was talking about earlier, and I've set a value in the registry here of 24.

For those of you that were paying attention to the source code sample Ethan showed a little earlier, remember that we had 8, 7, 6, 5, 4, 3, 2, 1. You'll notice that in this PowerPC version, we've swapped those bytes because this would not be 8, 7, 6, 5, 4, 3, 2, 1 if we hadn't handled that using the Rosetta support.

So just to prove that we have everything working, let's quickly go through and check 386. You'll notice that you have slightly different buffers, and that's because as Ethan alluded to, every user client gets its own instantiation of the different user methods, and the buffers are allocated local to the process. You'll notice that we're still getting our correct answer here, and you'll notice over here that we no longer see the CrossEndian flag, but we do still see that our set properties got the 24.

And finally... well almost finally. You'll notice we get a much larger buffer address, which makes sense because this is now a 64-bit process. And again, the same things. It's not CrossEndian, and the 24 gets set.$ So everything's working as we expect, again the 8, 7, 6, 5, 4, 3, 2, 1 is coming through as we expect.

And just to prove that... Rosetta is not 64-bit

... that's what you're going to get. So that's kind of the way it works. So what's important also is that we, you had to know how the magic works that I can run two different KEXTs and sub-KEXT types to get all this to work.

So I'm going to switch back to the project itself... and talk about a couple of things. Here are two targets. The source code files for the 2 targets, for AppleSamplePCI and AppleSamplePCI 10.4, are identical. They're ifdef like we talked about earlier, but this actual source code that is compiled is the same source code.

We're just targeting with the SDK selection, and the deployment targets to get the proper conditional compilation. What we do to make all of this work so that the right KEXT tries to load on the right system, is two things: 1 - we copy the sample PCI 10.4 KEXT into the plug in directory of the one that we would normally load on 10.5 and 10.6. And so the kernel only grabs that at boot time. You can't play with this by doing KEXT load, that doesn't work because it doesn't search into the actual plug in directory with KEXT load.

So you'll have to actually install it in the system library extensions to test this function. Having said that, what we do do is the individual KEXT plist... will use the bundle library property to load the right thing on the right system. So for those of you that are familiar with all these numbers, you know that if it's 8.0 that means it is going to be a Tiger system. And so this is only going to try to load on a Tiger system, or later.

However, you'll note that on the one that we've got built for 10.5 and 10.6 support, that has for all architectures, it's going to try and load on a 10.5 system - that's the 9.0 kernel - and this is the first example you may have seen of using the bundle library specific, architecture specific flags which the talk earlier today about KEXT management, was using.

And this is where, if it's a 64-bit kernel, we're going to then only load it if it's 64-bit and Snow Leopard or later. That's how all that works. And just so you don't think we're yanking your chain, I'm switching over to a x86 32-bit system. So this is a normal 10.5, 10.6 system.

In this particular case it happens to be Snow Leopard. We're loading that same kernel, or that same KEXT, and...

... see that we get the same results. So the same code, conditionally compiled, loaded onto 2 different systems. If I had a 10.4 system, I'd be able to show that too.

Anyway, back to our slides. Summary - what is it we'd like you to take home? Well there's new API's and in particular if you're going to support 64-bit, which by the way you have to support even if you don't have user land interaction, if you expect your device or service or "fill in the blank" to function in a K64 world, you at least need to recompile, resolve the warnings, resolve any deprecated API's.

You need to have a 64-bit and a 32-bit Intel KEXT, or slices, in your blob for it to load and work on a K64 system. You're going to target appropriate SDK's in your code in order to conditionally compile in the right stuff, and availability macros is your friend here. You want to match the SDK and deployment targets. You need to do both things, otherwise it won't work correctly and it's not a lot of fun. It's kind of embarrassing to actually be caught out doing that, again not from personal experience.

And finally you're going to need to test and ship, which we all know is on the list of things to do. So with that, a couple of more pieces of information, Craig Keithley is your evangelist, contact. Documentation - we've talked about today. The AppleSamplePCI, which is available today on the developer website. The SimpleUserClient, which has a lot more detail about IOUserClient information. The 64-bit Transition Guide and Universal Binary Programming Guidelines, those are all important.