Good morning. I'm glad to see that everybody made it here on Friday, very early morning session after the beer bash. My name is Nicole Jacque, and I'm with the Directory Server and Sharing Technologies group at Apple. And the first thing that I want to just point out is that I did this last year. How many people came to my session last year? Oh, good.

Lots of return--returnees. I must not have done too bad of a job last year. So last year, it turned out to be pretty popular that I did session notes for you guys and put them up on the attendees site. And I've done the same thing this year. And last year, there were some problems finding them on the site, so I just thought I'd point out where they are. So if you log into the attendees site, go to this session. They're actually called, I believe, release notes, but they're actually just session notes for this session.

So the idea here is instead of you guys having to try and scribble as fast as you can or type all the commands and various things that I have up on the slide, I've tried to provide as much of this as I can in the notes for you, so you should be able to just refer to the notes afterwards.

Okay, so I take it you like this, the notes. Okay. Just checking. And then I thought I'd also point out that last year's notes are actually still available. They actually got moved out to the regular attendees' site--or sorry, the regular developer site. You can get to them by navigating to OS X Server Networking, or you can follow this link here if you can actually remember it.

And the reason I point out that last year's notes are available is I'm going to try to not cover the same set of things that I covered last year because you can go watch my session from last year on the ADC iTunes site. So this year, what we're going to cover is we're going to first talk about extending Directory Services with a particular emphasis on what's new in Leopard and what's actually going to be particularly useful for people like you, people that are system administrators, people that are trying to solve particular problems with Directory Services. I'm going to talk about some new troubleshooting information that you're going to want to know about Leopard.

And then I'm also going to take you through five of what we think are probably the most common troubleshooting scenarios that we see with Leopard so that you get five hopefully relevant troubleshooting scenarios. Nicole Jacque And then I'm also going to take you through five of what we think are probably the most common troubleshooting scenarios that we see with Leopard so that you get five hopefully relevant troubleshooting scenarios that you can use to solve particular problems.

Nicole Jacque And then I'm also going to take you through five of what we think are probably the most common troubleshooting scenarios that we see with Leopard so that you get five hopefully relevant troubleshooting scenarios that you can use to solve particular problems. relevant troubleshooting scenarios that you'll get to go through here.

So we're going to start out by talking about extending directory services. And when we talk about this, mostly I'm going to be using as my example of the directory service that we're going to be extending as Active Directory. And I use this just because for a large number of you, that is what you are going to be talking about. But everything that I'm talking about also applies to Open Directory as your directory that you're going to extend for certain things, like your own web server or some other third-party LDAP server. So even when I'm saying Active Directory, this extends to other directory services as well.

So if we're going to talk about extending directory service, the first thing we should think about is, well, why would we need to do this? And the things that people usually want to accomplish with extending directory service is they want to provide some client management for OS X users logging in on clients that are logging in with accounts from some other directory service like AD, where we don't have the built-in managed client functionality that we have with Open Directory.

The other thing that people want to do is they want to use OS X Server, and they want to provide extra services to these users from the other directory service. And sometimes these services actually require per-user configuration that actually needs to live in the user record. So you need a way to do this.

And so we've always had a couple of solutions. And the first one was just schema extension. You need a place to store some information, so let's add some attributes, let's add some object classes, and then we can go ahead and store the information there. The second method that people have started using is something called the Magic Triangle or the Golden Triangle.

And these are two things that have been out there and they've been around for a while. How many people are using one or both of these--Schema Extension or Magic Triangle? So a lot of you are pretty familiar with this already, even if you're not using it. So we're not going to talk real in-depth about these because there's actually lots of you using it already. There's good documentation out there on this.

What we're going to spend most of our time talking about here is the new way of extending Directory Services, which is augmented records. But we're going to start by just going quickly through Schema Extension and Magic Triangle so that you can then understand what augmented records do for you.

So we're going to start with Schema Extension. This was the first method of Directory Services extending that people did. They went ahead and they went and added extra object classes that Apple uses, put them in Active Directory or their other Directory server, and then they could just go ahead and store all their information in Active Directory. And this is pretty great because all of your Directory data is stored in one place. You can continue to leverage your existing Directory service.

So if you've got Active Directory and you've got all of your domain controllers set up and they're replicating all over the place, you can just leverage all of that. You don't have to duplicate anything. All of your data is still stored in one place. When you back that up, you're backing up all of your data together.

So that's pretty cool, and the nice thing is that you don't actually lose the ability to use things like Workgroup Manager and all of the Mac-like ways of managing this. Once you've extended the schema, you can use Workgroup Manager to set your managed client preferences and so on.

And it's not too hard to do if you have a set of scripts so that you don't have to type every single OID number, which are numbers about this long. And we actually have these scripts, and if you talk to your SE for your account or you talk to Apple Professional Services, they can provide you with these scripts, or Apple Professional Services can actually come out to your site and do it for you. So literally, it comes down to almost just a double-click in terms of actual work required to do this.

So if this is so great, well, why doesn't everybody do it? Well, the first problem is that you're modifying the schema. And this is a forest-wide change. And so you need enterprise admin rights in Active Directory to do this. Which means not only do you need the actual enterprise admin to do it, but you also need their agreement to do it.

And sometimes this is more of a political issue than a technical issue. Because for some reason, your general Active Directory admin may not want to just say, "Oh, I've got these scripts. Will you just run them for me on your domain controller?" They don't seem to like that idea.

And so they want to look at them. And so this can take some time. A lot of you have pretty lengthy approval processes for making anything that's a forest-wide change. And in some cases, the answer is, for just about anything, no. We're not going to mess with the schema in Active Directory.

Also, in Windows 2000, the original version of Active Directory, this was a little scarier because you couldn't back out any of your schema changes. You can actually do that now, but that misconception still exists out there that once you put in a schema change in Active Directory, it's permanent. You can't get out of it.

The other problem here is that even once you have the schema extended, to actually store this information in Active Directory, you still need admin rights to Active Directory. And in a lot of cases, just the way in which your IT departments are structured, your Mac guys are over here, your Windows guys and your Active Directory guys are over here, and your Mac guys don't have admin rights to Active Directory, and so the Mac guys who would normally be the people putting in the managed preferences and stuff like that, they don't have admin rights to Active Directory to do this and may not be able to get them. So again, it comes down to essentially maybe more of a political problem than a technical issue. So to get around this, people came up with a strategy called the Magic Triangle.

And what the magic triangle is, essentially, is a setup where you have a Mac server, a Mac client, and Active Directory, and they're bound together such that You have a server which is bound first to AD and then to itself as an Open Directory master. And what you want to do here is you want to actually bind to AD first, then create your Open Directory master. The Open Directory process will recognize that, "Oh, hey, you're bound to AD. We've got a KDC over there. We don't need to bring up our own KDC." So that is normal to not see the KDC running.

And then after you bind to AD, you're going to want to kerberize your services for Active Directory, so you would run dsconfigad-enablesso. So you've got the server bound to AD and bound to the client, and then you have the client that is bound to AD and to Open Directory, and so you get this lovely little triangle shape.

Once you have this set up with your clients bound to Active Directory and Open Directory and your Open Directory server set up, the actual management is pretty simple. You just create Open Directory groups, you put some managed client preferences on them, and then you can add Active Directory users or Active Directory groups into this Open Directory group. And then when the user logs in on one of these clients, as long as they're bound to both Directory services, they'll get whatever's been applied to the Open Directory group.

So the great thing about the Magic Triangle is you don't need any help from your Active Directory admin. As long as you can bind machines to Active Directory, they don't need to do anything special for you. So your AD admins really love this configuration. The other thing is that it gives you the majority of what everybody wants here. You can manage your client with per computer or computer lists or computer groups, and you can do group-based management for users. And most of the time, for client management stuff, you don't want to manage each user individually, so that's really not an issue.

The problem here is that if you do need to do that, or you have some service that you want to run, like, for example, the new collaboration, you can run the collaboration services in Leopard, iCal Server, or the Wiki Server. You actually can't run those with AD users because there's some extra information that needs to be in that user record that they're not going to have. And you can't edit because you're not adding anything back into Active Directory. So we need a solution that solves this problem. And what we came up with for Leopard was this idea of augmented records. And really, augmented records is not some totally new thing.

It's basically here as some way to give you a way to create pieces of information that need to be stored in a record without having to write back to this other directory server. So primarily, the two services that you think of with this are iCal Server and Wiki Server. And this is really Magic Triangle Plus. Now, you may have heard somebody--by which I mean Joel--call this the Cylinder of Destiny.

We're not going to use silly things like that because, you know, if we were going to go Magic Triangle into another dimension, it should be like a pyramid or something, but I don't know. So the Magic Triangle Plus And what it lets you do is essentially tag some record from some other directory service--Active Directory or another Open Directory server or something like that--and add some information to it. But you're not writing it back into the other directory server. You're storing it on your Open Directory server. It's kind of like you're adding like a sticky note to something. You're not actually changing something.

You're just saying, "Oh, I got some more information about this." And then the magic happens in the search node, which actually takes the information from the other service, like Active Directory, and the information from your Open Directory server, and brings the two together, makes it look like one big record, as long as you go through the search node. And the services that are querying for it, they don't even know that augmented records are in play. They just query the search node and magically all the information is there.

The new bit about this is that originally when we shipped Leopard, the only way that you got augments through the GUI without doing things manually yourself was using the workgroup config. And we've actually expanded that as of 10.5.3, which shipped a few weeks ago. We now support augmented records in Workgroup Manager in Advanced Server Mode.

So we're giving you a little more flexibility about how you can use these. There are a few caveats here. One is if you have an augmented value for a record, you can't search on that because that's not actually in the directory that it's in. And searching for everything through the search node for this was pretty expensive, so right now we do not support search on augmented values.

It only works with the search node. So if you have some application or some service that is not using the search node for some reason, if it's going and talking directly to the other server, it's not going to see the augmented records. And so if you're writing applications where you might want to let people take advantage of augmented records, make sure that you're going through the search node in Directory Services. The other thing here is that you cannot use this to replace an existing value.

So if there's already a value for some record, say somebody already has their phone number in the directory, you can't give them a different phone number. You can only give them, for example, a new phone number. You can't overwrite. The idea here is we don't want to replace existing values, just give you an opportunity to put more data in.

So let's just take a quick look at what this looks like conceptually. If we have an Active Directory server and we have our Open Directory server, and in Active Directory we have a few user records, and we have Scott and Henry and Nicole, and I want to use some services on an Open Directory server.

So we go ahead, we can create this augmented record either using Server Preferences or Workgroup Manager, and we add some information for the augmented, so that's going to be used for iCal and Wikiserver. And this actually gets created as an augment record in Open Directory. We haven't changed anything back in Active Directory.

So now what happens when things actually use Directory Service, we query for Nicole and we query the AD node. What we're going to get back is just the information that's in AD, no augmented information. If we search the LDAP node, the Open Directory node, for Nicole, we're going to get no such user. Because, we're going to get a new user. And this is because there's not actually a user record for me in Open Directory, there's just an augment information. So it's an augment record, not a user record, without the actual user record, it doesn't actually mean anything.

Then the magic all happens if you go through the search node. Directory Services says, "Oh, hey, I know I've got augmented records here. I better go check and see if there's any extra information." And what it will actually return is a record for me with all of my information from both nodes. So my Active Directory information, my Wiki server, my iCal Server, all my augmented information.

In the GUI, what this looks like, if you're in workgroup mode, you actually use server preferences. It's called importing a user. That basically just means create an augment record for the user. You can bring them in. You can add some information for them. And now, as of 10.5.3, you can do this also in advanced mode. And you use Work Group Manager. There's a new menu item for Create Augmented User Record. So you can search for your user, or you can search for a group full of users that you want to augment.

And the best practices here are basically the same as what you have for Magic Triangle. Because this is not some totally new config, this is Magic Triangle Plus. So you're going to bind to AD first, then you're going to create your OD Master. You won't have a KDC running on the OD Master. You still want to Kerberize your services.

One thing to look out for here is, especially if you've already got this Magic Triangle set up, and then you go and you create these augment records, until you restart Directory Service on these clients, if they've been bound already and then you just go create the augment records, until you restart Directory Service, the clients will not see the augment information. So after that first time when you create the augment records, you're going to need a reboot of clients or you just need to restart Directory Service, which you can do with pseudo-kill-all Directory Service.

So now let's talk a little bit more in depth here, because we've done some nice pretty pictures and everything. How does this all actually work? Well, what we have is there's actually a record called Augment Configuration that lives in the config container in Open Directory. And it contains information about the Augment node, which is the node that is going to store the Augment information, in other words, Open Directory. The Augmented node, which is the other node, so Active Directory or some other foreign server.

And then the Augmented attributes. So for every record type--and you can use Augmented records, by the way, for records other than users. Right now, mostly that's what people are using it for, but there's no reason it's restricted to this. But you actually need to, in here, configure which attributes you can augment for particular record types.

And what this actually looks like is, not terribly surprisingly, it's an XML plist. And you can see in this, you can see the directories that we have for the Augment Directory Node Name. That's the directory that we are storing the OD information, the augments. The Augmented Directory Node Name is the directory that we are augmenting, in this case Active Directory. And in this case, and by default, the only attribute that we have set up for you to augment is the services locator attribute on the user records.

Once you have this set up, you have augmented records in play. Directory Services knows to look for augmented records. And you can go ahead and create augment records. These are stored in your LDAP Node, your Open Directory Node, under the Augments container. And they are stored in a record name format of record_data. And you can type: record_name. So if it's a record for me, it would be users: Nicole. So if you were to use DSCL and take a look inside your Augments container, you might see a bunch of records like this.

If you read one of these records, what you see is not all of the information from Active Directory, because we're not trying to copy all of this information, just the information that we want to add to the record. And the main reason that people are augmenting things is for the iCal and Wiki Server. So what they actually require is this attribute called Services Locator, which looks like this.

That's the main thing that people are currently using Augments for. We're going to talk about some other ways you can use Augments. But what Augments gives you the ability to do is you have this centralized directory server and you might have a couple different departments and each department wants their own Wiki server and they want their own iCal server. So they can go ahead, they can set these up, they can pull in users from AD, they can augment them, they don't need any help from the AD administrator. Everything's great.

But I want to give you a little bit more complicated of an example of how you might use augmented records. So let's say that one of your departments, say maybe your art department, where you have users that they need a lot of disk storage. And in Active Directory, You probably have some home directory already given to all of your users that your AD admins have assigned. It's stored somewhere centrally, but a lot of times it's not real big.

Maybe they only give them a couple hundred megabytes. And this art department goes out, buys a nice brand-new Promise RAID, connects it to their XServe, and says, "I want to use this for my users, and I want them to get it for their home directory." Well... How do you do that without, for example, having to go back and edit Active Directory? Hmm. Well, you can use augmented records to do this, and here's how.

First thing you need to do--and by the way, all of these steps are in the session notes for you, so you don't have to worry too much about trying to scribble too much down here. First thing you need to do and the first thing you're going to say is, "Wait a minute. A few slides ago, you said we can't actually replace a value that's already there." Well, that's true, but in the AD plugin, you can actually disable reading the home directory path from Active Directory.

So if we're not reading it, it's not there. So you just disable the use UNC path from Active Directory. You can do it either in the GUI or with DSConfigAD. So now there's essentially, as far as Directory Services is concerned, there's no home directory. So we can augment it.

So the next thing you want to do is set up your Magic Triangle config, same way you normally would. And because we're going to do home directories here the way we've always done them with OS X, you need a mount record in Open Directory for the SharePoint that's going to have the home directories. So create your mount record.

Then go ahead, and I'm going to show you the manual way to do this. Now, if you were going to do this for lots of users, you'd want to, of course, script this, but once you know where all the pieces are, it's not so hard to do. So you want to go ahead, augment a user in Workgroup Manager. This will actually create your augment config record, and it will also create an augmented record for the user that you're working with.

And then because we're going to be augmenting attributes that aren't normally in the default config for what you can augment, we're going to go ahead, we're going to edit the augment config record in the augment container, and we're going to say, okay, for the user's record type, in addition to services locator, we want to be able to augment home directory and NFS home directory, which if you guys are all familiar with directory services, those are the standard attributes that store home directory information. So we do that.

And then you can go ahead and in Inspector Mode, or you could do it with DSCL or whatever tool you like, you're going to actually go ahead and just add these two attributes to the augment record. So you're going to add Apple User Home URL, which is the home directory attribute, and home directory, which is actually the NFS home directory attribute.

And these are going to contain the standard information that you have for OS X home directories. So you'll have the sort of XML-looking, like, home directory path that tells you where it is on the network. And then for NFS home directory, you'll have the Unix file path to where the home directory is once it's been mounted on the client.

So once you've done all of this, you need to restart Directory Service, and then you can go ahead and on the client, log in, and it will see all of this information as the user's home directory. And it'll mount it in whatever fashion, like network home directories or mobile accounts or so on that you want.

So this is just to give you a slightly more complex example of how you might put this into play in your own infrastructure to solve particular problems. You could also use this if you have your own custom apps, for example, that you don't want to write stuff back to Active Directory. You could store it locally or just to solve any other problems with Directory Service that might have this kind of a problem.

So now that we've talked about extending Directory Services, we're going to talk about how you troubleshoot Directory Services because sometimes things don't quite go the way you expect. You might have experienced that. And the first thing, because we're going to focus here mostly on stuff new in Leopard, but we always, always, always--we've got to talk about Directory Service debugging, right? We've got to talk about how you get that debug log. And the basic stuff here has not changed. You still get debugging turned on by sending the USR1 signal to Directory Services. You can do a pseudo kill-all USR1 Directory Service. That turns it on until the next restart of Directory Service.

And sometimes that works out, but if you've got Directory Service, for example, crashing a lot or you're going to be doing a lot of rebooting and stuff like that, it's kind of tedious to have to keep going in and turning on Directory Service. Or maybe you want debugging on right away from boot if it's a boot time problem. So then you can actually create a flag file which is .dslogdebugatstart.

And this has to live in library preferences Directory Service. You can use the touch command to create this. This will turn Directory Service debugging on as soon as Directory Service comes up. And as always, the logs file is still in library logs, directory service, directoryservice.debug.log. And we actually roll these logs so you don't have to worry about these logs growing to be gigabytes and gigabytes in size. We'll just roll them after a couple of megs and so you'll see directoryservice.debug.log.1, .2 and so on. And you don't have to worry about leaving this on for an extended period of time. You won't end up filling up your hard drive.

But what is new here? is that we have added debugging levels to Directory Service debugging because we actually put a lot of information in the debug log. And sometimes it's too much and sometimes it's not enough. So now we're gonna let you actually kind of control that a little bit. So the levels are 0 through 7, with 0 being off.

The default when you turn on Directory Service debugging is set to 5. This is all stored in the DirectoryServiceDebug.plist file, which is in Library Preferences Directory Service. So you could edit that file yourself, or you could use the defaults command to read and write it. You will need to restart Directory Services after you change this in order for it to take effect, though.

So this is all important if you're going to be doing some troubleshooting or if you're going to be setting up and filing a bug report. One of the first things I'm sure you all know, anybody that's ever talked to me for troubleshooting help, what's the first thing I ask you for? It's a debug log.

The next thing that I want to talk about is something that a lot of people have sort of noticed on their own, We never really had talked about it last year, so I want to cover it this year. We've changed some of the Kerberos config files in Leopard. And Kerberos, as we all know, is really important for Leopard, for OS X in general. Everything we are trying to move authentication-wise to Kerberos. So in Directory Services, it's really important.

And if you had, for example, a Tiger machine and it was bound to Open Directory and Active Directory in this magic triangle configuration, if you looked at your edu.mit.kerberos file, which is in library preferences, you'd see something like this. You'd have all this Realm information and then you have a bunch of other sections and--well, Everything was just all in this one file. And then you upgrade to Leopard. You're looking to maybe solve a problem or you're just poking around to see what's new. You take a look at this file.

I'm kind of missing something here. Where'd all those realms go? The only realm that you'll see now in the edu.mit.kubaros file is the Open Directory realm. But we're still--if you're using the Active Directory plugin, we're still using Kerberos. We must know about this. Where do we put it? Well, the answer is we actually have written a plug-in for Kerberos that lets us store all this information in the local Directory Service store. So if you look in the config container of your local Directory Service, the DSLocal node, you'll see a bunch of records that are Kerberos, colon, and then your Realm name. So all the Realm-specific stuff has been moved here.

And if we actually look at one of these, the information here is basically the same stuff that we used to store in edu.mit.kirbros. We've got your KPassword server and your Kerberos server. So if you're actually looking for any information, you want to make sure that the configuration is correct and so on, this is now where you need to look for that information.

The other thing that's important here is if you have a problem and you want to talk to us about it, you want to file a bug or something like that, well, That's all information that we're going to need now. So when you're filing a bug report, which you can do bugreporter.apple.com, or if you're talking to AppleCare, Make sure that if you have a problem that you suspect is Kerberos/Directory Service related, make sure you give us not only your edu.mit.kerberos file like you always used to do, but also just do a DSCL read-all of the local node config container and give us all that information because that's actually where your Kerberos information is now.

And as always, just a reminder, if you have a Directory Service problem, pretty much no matter what it is, if you can give us a debug log, that will save us having to come back to you and say, "Please give us a debug log." So I thought I'd just remind you. I know you all would, of course, the first thing you would do is just get us a debug log, but just in case.

So, having talked about that, some general Directory Service debugging, we're going to talk more specifically about the Active Directory plugin. How many people here use the Active Directory plugin? Okay, so pretty popular. We've actually made a bunch of architectural changes in Leopard to the AD plugin. One of the things we've done is we've added domain and policy caching.

This is to try to prevent some delays when you go on and off your network. Before, we would have to sit and try and query and find out this information when you tried to go on and off the network, and if the servers weren't there, we could wait for a long time. So we've just cached this information now.

Another thing that we've done is there's a lot of people that have pretty large domains and forests with lots and lots of domains. And previously what we did is when we bound or every time we tried to redo our configuration, we'd go out and we'd query the entire forest.

Even though you might only be talking to one domain ever, we'd go look everywhere in the entire forest. And that could generate a lot of traffic if you were binding a lot of machines and it could maybe be going out across some slow links because maybe you had some domains that were only for some isolated places. You just didn't want that extra traffic out there. So we've gone to more of an on-demand configuration. We don't put in configuration and query servers unless we're actually going to be using that domain for something.

We've added dynamic DNS registration. So when you bind to AD, we actually will update the dynamic DNS on your AD server. The one catch here is that currently we only do forward DNS for you here. The pointer record will not be created. And then finally, like I said, we have taken advantage of this new Kerberos plugin, and we store all of our Kerberos information in DSLocal now for the AD plugin.

We have a few new dsconfig.ad options. We added packet signing and encryption. This would be the packet sign and packet encrypt. You can set this to allow, disable, or require. By default, it's set to allow, which means that if your server is configured to allow you to do it, packet signing and encryption will be on.

One thing to keep in mind here, however, for troubleshooting purposes, is that if you have to do something that requires a TCP dump, either to send to us or you just want to look at it, it's probably not going to be real useful with packet signing and encryption on.

You're not going to be able to read anything. You're just going to be able to see where the packets are going. So for troubleshooting purposes, a lot of times it may be useful to actually disable this temporarily so that you can get a packet trace and see what's going on.

Another thing that we've added is we've heard that a lot of people needed to be able to change the computer trust account password periodically, either because you had auditors come in and say, "You've got to do this," or because in some cases, your Active Directory admins were using this the last time that a password was set for a computer account as a sign of whether or not that computer account was still active. And so some of you had, like, scripts on your AD server that were going through and disabling all of your Mac accounts periodically because they hadn't changed their password recently. So we've added the pass interval option to dsconfig-ad.

It takes the number of days that you would like to have it change the password after. By default, this is set to 14 days. Zero means off, or anything under one will go to zero. And this is something you want to set at bind time. Otherwise, your default is going to be 14 days.

And finally, we also have a new option to help out those of you with maybe very large, complex domains. And in some cases, these may be domains that kind of came together kind of after the fact. And the admin over here and the admin over there weren't really talking to each other before. And so you've got users with the same short name in these different domains.

Well, that doesn't really cause you a problem in Windows because in Windows you log in specifically against a particular domain. But in OS X, because we look at all the domains that you're connected to, this would cause collisions. So you could either try to get people to change their names or try to only look at a particular domain. Sometimes that would work for you. But if you needed to allow people to log in against any domain, sometimes these users that had another user with the same account in some other domains, they couldn't log in because we just need a unique, short user name.

So what we've done to give you a solution to this problem is we've created an option called the namespace option. And you can either use namespace domain or namespace forest. And if you use domain, you'll get the usernames the way they are now, just the short username. If you use the forest option, you will get all of your usernames in the format of domain name backslash username. And so now all of your usernames will be unique.

And then finally, one thing that I want to talk about is we've changed some of the config files that are used for the AD plugin. In Tiger, this was one single plist file. Why do you care about these files? Well, one thing is for troubleshooting purposes, you might want to look at them, find out information about how it's configured or send them to us. But also, if you're writing scripts to try and automate certain things, sometimes there's pieces of information that you're looking for.

Some of these pieces of information are in these plists, so it's useful to know what's where. So in Tiger, we had just this Active Directory dot plist. In Leopard, this became four files. And so we still have the Active Directory plist, but we now also have Active Directory domain cache, Active Directory domain policies, and Active Directory dynamic data.

And what's in these files In Active Directory PList, most of the same information that we had before. So it's all of your plug-in config options. So if you set things either in the GUI or with DSConfigAD, you've got all of that information is stored here. You also have all of your attribute mappings. So for example, the mapping that we're going to use this particular OID number to be this particular attribute in Active Directory. The computer account information is also stored in here. So the computer account name, the computer account password, so the trust account information is here.

The MCX template that we actually insert into every user account, things like the default doc and stuff like that, is stored in here. And we also store the next password change date, so the next time that the AD plugin's going to change the password. And one thing that a lot of people don't think about here is that this file is actually something you can edit.

So if you want to change your attribute mappings, you can do that. If you want to change the MCX template, you can do that. One thing to also keep in mind here, though, is for imaging machines, you do not want this file to be in a state where the machine is bound when you image your machines. Because otherwise, every single machine that you image is going to think it's the same computer account.

And while in Leopard, since we have, by default, your password's going to change in 14 days, the first time the first computer changes its password, every other computer in your domain that you've bound from this image is going to fall off of Active Directory because its password just got changed on it.

So that's probably a less than optimal sort of experience. So make sure that when you're imaging machines, image them. Go ahead, set all the options and stuff that you want in AD before you image. But actually, image with the machine unbound, and then use a post-flight script or something to actually do the binding.

The Active Directory domain cache is just a cache file. It's got all the information about each domain that you have in your forest, your NetBIOS name, your DNS name, your LDAP partition. And the domain policies files, same thing. It's just the domain policy cache for each domain in the forest.

And finally, the dynamic data file is useful primarily for trying to figure out what is the last thing that the Active Directory plugin has done as far as what servers has it used, what servers has it configured. So for troubleshooting purposes, this is someplace that you might want to look if you're trying to figure out what server is the AD plugin talking to.

So to give you just a quick example of a few things that are actually in these files, in that Active Directory dot plist file, One reason you might want to go here is the pass interval and namespace options that we talked about earlier. If you do a dsconfigad-show, we actually don't report what settings these are set to with that command at the moment. So if you want to know what those are set to, you can look in the plist.

Another thing that you can do here is you can check that password change date if you want to see when the computer's going to change its password. Or for purposes of testing and troubleshooting and so on, if you just want to try to make the password change happen soon, well, right now, you can't supply a number under one day. And if you don't want to wait for a whole day, you can just go in there, change that password change date to something much sooner.

And then one thing you'll want to do here is after you make this change, anytime you edit the Active Directory.plist file, you want to read the name of the file. So you want to restart Directory Services, but not such that it's going to rewrite the config file on you, because you just changed it. So you want to basically do a kill nine on Directory Services.

And then, like I was talking before, an example of what you might use the dynamic data plist for is you might want to know what the last used server is. You want to know what was the last global catalog server that you talked to, or for a particular domain, which domain controllers am I actually using? Sometimes you might want to know, well, am I using the one that's closest to me, or am I using one that's halfway across the world, or so on.

So now that we've talked about Directory Services troubleshooting in general and Active Directory plugin troubleshooting in general, I want to give you five of what we think are some of the most common scenarios that you guys might run into when you're running Leopard with Directory Service. And these are all applicable to the Golden Triangle, Magic Triangle, Augmented Records setups that we've been talking about.

So the first one is an issue that it's not entirely Directory Service related. It's really more of a Kerberos client sort of an issue. But you hit it when you're using the Active Directory plugin mostly. So we're going to talk about it here. And this is issues that you might hit if you're doing multi-domains in a multi-domain forest with Active Directory and you're trying to do Kerberos cross-realm.

So the configuration would be your multi-domain forest. You've got a user in some domain trying to access Kerberized resources in some other domain. So for example, I'm in the engineering domain. I want to access some resources in the marketing domain. And the problem that you might see is you might get some sort of message like, "Can't find KDC." Or you might try to, for example, connect to the file server, and instead of just happily popping up your file share like you expect and everything working the way Kerberos normally does, instead the authentication window comes up and you've fallen back to something other than Kerberos.

So before we talk about the solutions here, there's two main categories that this tends to fall into. Either you have a unified or a disjoint AD forest. So how do you tell which one you've got? If you've got a unified forest, then you have your DNS domain matches your Kerberos realm. So if your DNS is foo.apple.com, your Kerberos realm is foo.apple.com. Pretty simple. And this is really the category that most people's Active Directory domains fall into.

If you've got a disjoint Active Directory for us, though, then you're in a situation where things are a little more complicated. Because your DNS domain might be foo.apple.com, and your Kerberos realm is something else like bar.apple.com. Or, in a lot of cases, you have some shortened version of your Kerberos realm as your DNS name, so you might have apple.com, and then your Kerberos realm might be foo.apple.com. So the two don't match.

One thing to look for here is that disjoint Active Directory for us are--may not be supported by Microsoft after--or even including in 2008 Server. So Microsoft may actually require you just for their own purposes in terms of getting support for them, nothing to do with Macs or anything like that, may be telling you to migrate away from this kind of a configuration. For most people, it was put in place as a NT4 migration. AID, but Microsoft may be telling people that they have to actually migrate away from this.

So if you've got a unified Active Directory forest, the first thing you want to do is, well, let's just go look at your Kerberos records. So if you remember, we said all your Kerberos records are stored in that DSLocal config node on your local client. And what you may find is that for the particular domain that you're trying to contact for resources, you may find that there actually is no config information for it.

The reason for this is since we've gone to an on-demand configuration style for the AD plugin, if we haven't already talked to that domain, we don't have a config for it. So you could work around this by IDing some user so that we actually talk to that domain.

But generally speaking, you know, you don't want to tell your users, "Okay, yeah, if you want to connect to that file server over there, go launch terminal, type ID some--" That generally--people don't like that. So what you want to do instead is after you've bound, you want to edit edu.mit.kubros, and what you're going to do is by default, we have DNS fallback set to no. And that actually needs to be set to no during our bind process.

But you're going to want to set this to DNS fallback equals yes. What this will do is it will let the Kerberos client actually go out, search DNS for Kerberos, and then it will find all the information for this other domain, even though we haven't contacted it for directory service and we haven't created Kerberos records for it.

It'll just find that information for you, and you won't need directory service to actually write out a particular config file for it. Now if you have an edit in this file, if you don't want this to get overwritten the next time that Active Directory plugin goes and reconfigures itself, also make sure you delete the pound lines at the beginning so that it does not get regenerated on you.

Okay, the second troubleshooting scenario we're going to talk about is a slow binding issue with the 2003 R2 Active Directory schema. So how many people here use 2003 R2 on your Active Directory domain? How many people don't yet but are going to upgrade to it at some point in the future? Okay, these are for you. So the configuration, as I said, is Active Directory 2003 with the R2 schema.

And what you'll see as symptoms is, especially if you have a large domain, lots of domain controllers, you may get some pretty slow binding. It will bind eventually, maybe, but it could be pretty slow. And you may also have your AD administrator come to you and say, "You know, I'm getting these really expensive LDAP queries. I'm seeing some high CPU utilization on the server. It kind of looks like it's coming from the Macs. What's going on?" Well, what's happening is there's an attribute that is in the R2 schema that is an attribute that we actually use. It's called MAC address.

And what we use MAC address for is, well, to help us identify computers. Mostly we do it for MCX settings, but when we bind, we check to see if a computer with that particular MAC address already exists in Active Directory. If it is, well, we just join that account. If it's not, we're going to create a new one.

Now, if you haven't extended your schema, you don't have this, and so we can't edit it. We just move on. And in that case, so if you have Windows 2000 or 2003 Server, we just query for it. The AD Server says, "I don't even have that attribute. Go away." So it's a real simple, quick query. It fails fast, basically.

If you have the 2003 R2 schema, though, we actually have a case where Microsoft has added it to the schema, but has not added it to the computer object class. So we can never actually populate it for a computer account. So we're never actually going to have any records with it, but it exists in the schema.

So what this means is every time that you bind an Active Directory plugin client, we're going to be searching for this attribute, and Active Directory is going to search every single record in your Active Directory. So we're going to have a few records in your Active Directory forest. And for those of you that have a few million records, it's going to take a little while.

And so this causes problems during binding because we're sitting and waiting for these servers and sometimes they take so long we think they've just disappeared off the face of the earth and we go try and find another server and try and use that. So the solution here is actually pretty simple. You just want to index MAC address so that when we actually do this query it can say, "Oh, no, I don't have that." You know, it doesn't have to go search every single record.

This is a pretty simple thing. It's just a checkbox in the schema snap-in. And Microsoft has actually realized that maybe this is a good idea. And they've actually done this in 2008 Server for you. So it is something that you only have to deal with in 2003 R2.

But you can see, well, Obviously, it's something that, you know, going forward, you would want to have anyway since Microsoft is going to do it for 2008. So everybody that has this or is going to upgrade to the R2 schema, go back, tell your AD admins to index this.

The third scenario that we're going to talk about is where you have users And you've got Magic Triangle, you've set up your OD Master. Your users can authenticate okay, but every time you try and edit Open Directory, either with Workgroup Manager or a directory app, the authentication works, the authorization fails, and it says, "You don't have access to write to this." So even diradmin won't be able to write things.

But if you use the built-in default root account for Open Directory, this will still work. So how do you fix this? I saw a lot of hands when I asked how many people have been to one of my sessions before. So what's the first thing that you should do when you're troubleshooting a Directory Service problem? I have a very intelligent audience.

Actually, I like that animation. Let's see that again. That's fun. You guys are all going to be seasick when you walk out of here, you know. But if it helps you remember to check your DNS, it's worth it. So yeah, usually the cause of this is that you're missing either forward or reverse DNS. And a common culprit for this is you bound with the AD plugin, you saw that it created a DNS record for you in AD, which it did, but it didn't create that reverse pointer record.

And if you don't have that record, all manner of things Kerberos related and other things are going to fail. So actually for servers that are--whether you're bound to Active Directory, Open Directory, whatever, my recommendation is do not rely on dynamic DNS registration. Go manually put in that registration so you know it's there.

The other thing that can cause this is you might be missing the root DSC file in /etc/open/ldap, in which case you can actually just populate that by doing slap-config/populate-root-dsc. You do, in fact, actually probably need to be root, so either put a pseudo in front of that or run it as root.

Another problem that I want to talk about is just some general issues you may run into with augmented records. And so you may have services that are looking for some information that's in an augmented record and they're not working right. They maybe give you some weird information--you know, weird errors because they're not finding the information that they expect. So what you should do in this case is verify that Directory Services is even seeing the augment configuration at all. So remember, first of all, it's only--you're going to see this only in the search node.

So use DSCL, go through the search node, try to read the record using the search node as opposed to the, you know, Active Directory node or the LDAP node and so on. Also keep in mind, if this is a machine that was already bound to OD before you created that initial augment config record, it's not going to see the augments until you reboot it.

So those are the two things that usually catch people, but once you've done that, then you can go ahead and you can actually make sure that your Augment Config record is created correctly or that your actual Augment records are correct. But these are the two things that usually catch people.

And finally... How many people here try to automate their Active Directory binding process? Post-flight scripts or so on? Okay. So you use DSConfigAD to do your binding, and there's two common problems that people hit with this. And they have the same symptom, which is DSConfigAD acts like it's binding okay. You don't get any errors. Everything looks good.

And then nothing works. Like you try to look up users in Active Directory or authenticate and nothing's working. So what has gone wrong? Well, usually it's one of two things. The first thing is that the AD plugin is not actually enabled. Turns out DSConfigAD can give you all of your binding and everything like that without the AD plugin being turned on. So it goes ahead, binds you, but then the AD plugin is still off. So Directory Service is not actually using it for anything.

So I wanted to give you some little code snippets or script snippets to help you do this. These are actually all in the notes that are on the attendees site for you. So you don't have to try and type these all out. But the first thing is this information is just in the Directory Service plist. So you just need to set the Active Directory key to be active. Then you need to restart Directory Service so that it takes effect. And just for human readability purposes, you may want to convert this back to XML1 format instead of binary.

But that's purely optional. And the second issue is just that Active Directory may not have been added to your search path. So the way to do this is you can use DSCL and you need to add your Active Directory search node to the CSP search path. And keep in mind, if you're allowing authentication from all domains, this will be Active Directory all domains.

If you're not doing that and you're specifying each individual domain, you will need to do this for each of the domains that you're going to specify. So it'll be like Active Directory/yourdomain. And then you also may need to create the CSP search path on search policy. So you can just do these two commands. That will take care of adding things to your search path.

So just to wrap up, we talked about extending directory services. We have augmented records in Leopard to help you solve some of these problems that we could not fix with just Magic Triangle and that let you do them without having to go through schema extension and get all of your AD admins buy-in. And just to remind you, we now support this in advanced server mode with Workgroup Manager as of 10.5.3.

And for troubleshooting, remember we have variable debug levels, so you may need to adjust the logging level up or down. Your Kerberos config information is now stored in DSLocal, so that's where you need to look, and that's what the information you would want to send to us if you're filing a bug report.

DNS, DNS, DNS. I could say it all day, it's that important. And finally, if you're having Kerberos issues, you can adjust a lot of these things by actually editing edu.mit.kerberos. For more information, you can talk to Mark. I will actually be in the Directory Service Lab immediately following this session.