Ladies and gentlemen, please welcome our first presenter, Senior Networking Engineer, Laurent Dumont. Good morning, everybody, and welcome to the Core Networking Update. So this morning, we'll talk about the goal of this session for the Core Networking Update. We want to do an overview of Mac OS X networking and the different components that are involved in networking for Mac OS X. And, of course, we'll also talk about what is new in Tiger, because Tiger is a big theme for this WWDC. And also, second part of the session, we'll have a networking API overview. We'll go through the different level of APIs that we have in Mac OS X for-- which are related to networking and try to see, you know, which API belong for you as an application or a developer, which API to use best fit, you know, what you're doing. So we'll look at that.

So first of all, I'd like to remind some of the ongoing goals that we have for the core networking in Mac OS X. So one of the first goal is ease of use. That's probably a trademark of Apple and the Macintosh. So we're trying to make sure that the networking in Mac OS X is really easy to use. So that's one of our first goals. Second goal, which goes with the first goal, is mobility.

There is a big unfaithfulness for us about having our user mobiles. I see here quite a few people running with PowerBooks. And that's a pretty big focus for us, to make sure that your mobile networking experience is very smooth and powerful, but everything works together. So that's one of our big goals.

The other big goal we have, especially considering that we're Unix-based and we're getting more into the server, is the performance. So we're trying to make sure that we can squeeze every little bit of performance out of the Macintosh and the system through the networking stacks and all the networking components in Mac OS X. And one other ongoing goal that we have in Mac OS X is a security.

You've heard that there is a few security updates, and you've probably seen that in your software updates. And one of the things we're trying to see in the networking part, especially, is that we're based on standards that are pretty much done in another time where things were more friendly. Computers were on the same network and trusted each other. And that kind of idea changed a little bit in the last few years. So security is a big focus. We're trying to make sure that our stack is all current with the fixes and all the, you know, All the problems that are found are fixed as rapidly as we can. So that's another goal. Extensibility, we'll talk about some new things that we've done this year for Tiger to extend the networking in different ways for you guys to be able to create your own KEXTs and those kind of things. So we'll go in detail into that. Another, of course, ongoing goal is standards. We're very, very close to standards. We're trying to make sure that we follow all the standards and we're a good citizen in terms of a networking world where different nodes are coming from different vendors and different, you know, vision of the world. We're trying to make sure that we not only follow standards, but we also give some room to interpretation when some usage things are closer to the realities and closer to the standards. So that's one big focus for us also in the core networking, making sure that we follow all the standards.

So here we got a little graphic trying to explain, remind you, probably for most of you, how networking in Mac OS X is structured. So if we look at the lower part, the kernel part, one of the big components of the Mac OS X networking is in the kernel, just as a reminder, the downwind kernel, which is open source.

So you guys can check out the sources and check all the code there, is made pretty much of the BSD part of the kernel, which has a networking part on your left, which is a little bit highlighted, and the file system. Also in the Darwin kernel, we have the Mac kernel part, which provides the basic OS services, scheduling, memory, all those kind of neat stuff. Underneath that, we have the I/O Kit layer, which as driver developers know, is what provides the interface with your driver, hardware driver, for a networking card or something like that. So if we focus a little bit inside that part, which is the networking, we're based on a BSD stack. So we have, of course, you probably know, the socket layer on top of that, which mirrors a BSD socket from UserLand. Underneath that, we got TCP, UDP, IPv6, IPv4, IPsec. All those are part of the kernel. And this is part of what you'll find in a standard of BSD-like system. And we have a little difference here, which somehow mirrors the I/O Kit layer for driver, which is the data link interface layer, which is something specific that we have for you to hook up pseudo drivers or drivers in the stack. So that's how the system is structured. On top of that, in user land, we have the BSD socket which provide us, you know, basically the POSIX-like and also the Unix-like socket interface that most of our other layers are based on the layers using networking. And on top of BSD socket, we get core services which we'll go into more detail about the for that level, which are based CF networks and all those guys which are based on the BSD sockets or client of the BSD sockets. And of course, on top of that, we have probably your application which may talk directly to the core services through CF network or some other layer or directly to BSD sockets or maybe a combination of both or could also talk directly to the kernel through some other means that we'll talk about during this session. So yeah, this is a little bit of a highlight about what we're going to be talking about here in this session. We're focusing on the networking part of the kernel and the BSD socket and the other layers on top of that.

So I think I pretty much talked about things in this slide, but I always want to remind everybody that our architecture is based on the free BSD, so we did a port a few years ago. And so we're pretty much BSD 4.4 plus a bunch of other things. But this is a core of our networking stack here. So what does it mean for you? It means that because we're very close to a BSD, means that it's easy to port Unix-like applications. So if you have a tool or an application which is coming from FreeBSD or another flavor of BSD, it's really simple to get that to work on Mac OS X. So that's a big plus. The fact that we're based on FreeBSD also means that we have a very robust architecture in terms of the stack, and it's well proven. It's been there for a long time.

It's been improved. It's been tested in very different ways. So that's a very good solution for us. Also, it means that because it's open source, the code is available. So for you developers, you can go and check out the Darwin project, and if you have some wondering how we do this or what's going on, you can always check the code. And we actually had a few cases where people were coming and say, hey, I'll look in this code source, in the source code there for Darwin. And I found this problem in this area. And so we looked at it and said, oh, wow, yeah, that's a problem. And we fixed it. So we do that. So feel free.

This is all open source. Everything in the kernel for the networking is there. So because we have this, we have the TCP/IP stack. And with the work of Kame a few years ago, we integrated IPv6 and IPsec in the stack also. So we also have a full range of PPP elements there, which is part of what we have in Mac OS X. We got PPP, PPPoE. We got a bunch of other things. We got a VPN solution, which is in the kernel also, which is based on PPP, L2TP, and also leveraging from IPSec. We also have a firewall, IP firewall, sorry, and the NAT, which are part of what we got from FreeBSD. Now we got IPFW2 also.

We'll talk about that. And the NAT, which gives you things like the internet sharing in user land. It's based on the NAT that we have in the kernel here. And some of the things that we have and we added from Apple also are things like dynamic configuration. We made, in the past few years, a lot of work to make sure that this free BSD stack that we got is-- we were talking about mobility a few minutes ago-- is more aware of mobile environments. So it means that we have events and a lot of things that we added to make sure that things in user land get notified when we change our address or when there's a DNS change or all those kind of things. So all that is work that we added around this part in the core of the networking to make sure that those events telling you the interface came up or all those kind of things bubble up to user land and to some tools that will make sure that the configuration gets readjusted in function of what's going on in terms of you changing your airport or plugging in your internet cable or going home and all those kind of things are done through some basic mechanism we added in that global stack that we got from PSD. And of course, I'm sure you're all aware that we have rendezvous and the DNS service discovery is one of the things that we have also in our networking architecture which is central to a lot of the services that are used by IAPs and a lot of things like that.

So now that we've, in broad terms, described what's going on in terms of the networking architecture in Mac OS X, I'd like to talk a little bit about what is new in Tiger. So since last year, a year ago, we're here and all the focus was on Panther. And we introduced a lot of interesting things in Panther. But this is a new year. And now Tiger is a focus. And on Tiger, we got a lot of exciting new changes also. We have some very core changes this year in Tiger. We got some kernel changes. You may have heard from Steve's slides or Bertrand's presentations that we're doing something about fine-grain locking.

And we're going to go in more detail about what this means and what those changes are. Another thing that changes that is related to that is we now finally have some kernel programming interfaces that are more modern and give a lot of new functionality. So we'll talk about that. We also have some good changes in IPsec. And we also have VLAN changes that were not there when we talked last year. We also have a bunch of userland changes, some IP configuration changes, high level preference API, which is new in Tiger. So we'll talk a little bit about that. Other changes, IPFW2, which is an upgrade of our firewall solution, if you want, both from UserLand and the kernel, and also site-to-site VPN. So-- So kernel fine-grain locking, OK. So what do we need? Well, to have a little recap here, we said we're using the BSD side of the networking, the stack, and everything. BSD, historically, as one-- in the kernel, you get one thing is going on at a time. So what we did previously in Mac OS X is that we had the mechanism which we called Funnel, which means that we were trying-- because we have a lot of machines that are SMP, so we got XSERVs, but even your G5 and everything, most of them have two processors nowadays. So in order to not be completely locked into the kernel where one thing can go at a time and everything is waiting for it, we used to have this mechanism we called funnel, which basically let us do on one processor some networking operation, while the other processor might be doing user-led operation or file system operation.

So we had this kind of split personality in the kernel where only two things could go at a time, but two different things, we couldn't have two networking operation going on. So the problem with that is this model, we've lived with it for a while, and it's been working pretty well, but it's get some fundamental issue with it. And one of them is not really scalable. When you're a server, like a NIC serve, and you're trying to do just net traffic, and everything is going through your cache, you don't do any file system operation. And basically, you have one CPU which is really loaded, and the other one is kind of idle. So because everything is going for the networking, and so there's this big funnel which blocks everything. So what we did in Tiger is that we changed the underpinning of all this by, instead of having those two big locks, having finer grain locking at different level in the networking stack and in the rest of the kernel so that we can have simultaneous operation going on for different operation, both in the file system and in the networking. So another thing that we added here is, And that was part of our previous model issues, is that we didn't have reference counting for objects. So if you added multicast addresses, let's say, we didn't really keep track of who added multicast addresses. We removed it. And in some corner cases, we might have some issue where we lost reference, and so that multicast address was leaked. So we had some problem with some objects that might leak or might not be in the right situation.

So we're fixing that with a fine-grain locking in the kernel and by having new ref counting for most of the objects that we're using in the networking. And one other thing about this is that it's transparent to application. It's really only in the kernel that we're changing. The socket layer and all the programming model for applications that are in user land doesn't change. However, this is a new model for kernel extensions, because we're changing so many things in the way the stack is structured that a kernel extension will have to change. But we'll talk about that. So the biggest reason for going with finer grain locking, symmetric multiprocessing is getting a big thing. There are arrays of XSERVs and everything. We're trying to get maximum power out of those machines. And to do that, we had to do some changes.

So the SMP improvements that we've done, this is those finer grain locking, you know, networking and file systems will be better because we get some parallelism. We optimize the data layout in the kernel by having, you know, those locks by finer objects instead of having one big lock that gets everything, reference counting, and also provide, because we're changing all this, at the same time, we're providing new stable kernel interface that go beyond I/O Kit. So as a kernel extension writer, you'll be able to use some more stable interfaces.

So I've been talking about all this, but anyways, just as a reminder, this is what kernel locking is for the BSD sign in Pantor. So we see on the left, so networking stack in the file system, and both of them are just funneled by those big locks where everything coming in or out is all stopped by this lock.

So you need to acquire that lock before you can do any networking operation or file system operation. So now if we zoom in into the networking stack in Tiger, we see at the socket layer we get some smaller locks here. At the protocol layer, same thing. And the interface layer is the same thing. So what it means is that if, let's say, one application is doing a socket operation on, let's say, socket A on TCP, at the same time, we'll be able to perform another operation in another socket without contending for that one lock and serializing all those operations. So we're getting this more parallelism I was talking about, so that's going to be a big win. This is our new model. It's still an evolving model, but pretty much this is the direction we're going on, where we're going to get finer locks and finer entities that get locked.

So we can have more parallelism on different sockets. And they're doing the same thing for the file system. And there's a session this afternoon that will explain all that. So the other big change that we have in the kernel part this year, which is related to the change for the final grain locking, is the kernel programming interfaces. For about three, four years, I think now, we've been telling people who are writing kernel extension, network kernel extension, next year we're going to break you. We're changing all this. Well, this is a year. So there is a full session about this this afternoon. But what happened is that because we had to change all those kernel structures for locking, for ref counting and everything, it was the right time to provide a new API for you guys who are writing kernel extensions that kind of isolates from the implementation of the networking stacks and of the internal of what we're doing. That way we have more way to improve it or change it or fix bug. We had a lot of problems in the past few years. We're trying to fix bugs. And we needed a new field in one of the structure in the kernel. And we've never been able to do the right fix because some kernel extension, which was linked against a kernel, was using that field or that structure. And by adding a new field, we were just breaking them.

And so that's not a sustainable model in the long run, and where to scramble and find ways to get around that. With this new model that we're introducing for the final grain locking and the new kernel programming interface, you will not be able to get through the structure directly. There won't be an intimate knowledge of the kernel implementation. That way, you know, we can change your live accessors and those kind of things. That way we can change the implementation without disturbing the text itself.

That's the issue with the NKEs today, is that they have an intimate knowledge of the kernel implementation. Basically, they need to be relinked each time we're changing something. And there is some confusion with those interfaces and some missing functionality also. So we're addressing that. There's new KPIs. The kernel structures are opaque now. And there is accessor functions to do all those things that you used to do by directly calling into or directly accessing those structures. One other thing, we just talked about the new locking model, the locking and the reference counting are implicit, so when you're doing one operation, with your NKE on a socket or something like that, will take care of locking that socket for you and ref counting it. But, you know, we're not doing the whole job. You will now be responsible for doing your own locking of your own structures. Because before you were protected by those funnels where nobody else could get into your structure when you were in there. Now, because of the simultaneous operation we can get with the finer grain locking, It's really possible that multiple thread will be-- or your KEXT globals will be accessed by multiple threads for all we know. So you need to-- and we'll provide primitive for that-- you need to lock your own structure also. So one of the other things with those kernel programming interfaces is that we're providing a more consistent behavior across API. So there was some confusion.

We're trying to address that. And the drawback of all this, as I said, for years we said we're going to break you. The Panther NKs that you have right now, the kernel extensions, well, they will need to be reworked to work on Tiger, because there is no way by doing the extensive changes we've done in the kernel, we can provide compatibility with those NKs that were basically fetching stuff out of structures and linking directly. So there's no way for that.

So just a little overview here about the networking KPI levels that we introduced this year. In dark blue here, we got socket KPI, socket filter, IP filter, plumber functions, interface filter, and interface. And so there is a full session this afternoon at 3:30 talking about those new networking KPIs. So if you're interested in this, I highly recommend to go and see all the glory details about what's going on here. But for people who are just new to networking here, just want to make sure that you understand that the NKs are really a last resort solution as a networking developer to do some things that you cannot do in user land. It's good for doing things like content inspection. You're trying to see what's going on in all the packets coming in from one interface or on one socket. To do pseudo interfaces, it's very specific things.

Packet filtering, or you're trying to do a network file system, those are good cases where you can use NKEs. But try first to see if you cannot do that in user land. Because working in user land is way easier than working in the kernel. In the kernel, you can panic. And now we introduce a new thing. You can bedlock. And those things are not really easy to debug. For your user, if your program doesn't work very well, panic will give a bad experience.

So the performance advantage of working in the kernel for most things is really negligible. So unless you have really a good reason, do what you have to do in UserLand. We provide and we'll go through a bunch of other ways to access APIs and access some of the kernel functionality from UserLand. So as much as you can, it's recommended that you use that. So yeah, programming in the kernel is dangerous. And go see the session this afternoon at 3:30 to get all about--we'll go through all those APIs and see what they're doing and exactly what you have to do with that.

So some of the other changes that we have for Tiger this year, IPSec improvements. So the biggest thing on that side is that we have a completely new AES crypto engine in the kernel. So you may know that the Apple VPN solution we introduced last year in Penter is based on IPSec and is using L2AP. And it's also using AES. So what we did is we did some work here, and we got some things that really give us some good optimization. And we're getting twice the performance throughput with a VPN now with this new crypto engine. So it's a good thing that we have in Tiger here. We have a lot of bug fixes, security fixes.

We're talking about security. We've been monitoring all the Ike changes and all the Ike security things pretty closely. And so Raccoon is our Ike implementation that we have in Mac OS X. And we've been doing some improvements there too. And also, one of the benefits of the finer grain locking we just talked about is that we're providing more parallelism in IPsec. IPsec is a very heavy computational engine, because you do a lot of computation to do the crypto. So having some parallelism there and not having the whole networking side being stuck while we're doing some crypto on one packet is a good thing. So with the new model, it's going to help having a better parallelism for IPsec and better throughput for your applications using that. And we have more improvement to come in Tiger for IPsec. But we cannot really talk about all those yet. But we're working on it really, really actively.

So VPN improvements. So the VPN was introduced last year for Panther. And for Tiger, we were introducing a side-to-side VPN. It's based on the IPsec tunnel mode, something that was requested by some organization where they want to have a VPN between two XSERVs in between here and somewhere else in the world, and they want to have the side-to-side VPN, not just a VPN where you connect to one server as a client. Both sites are connected to the VPN. So we're introducing this in Tiger. We also have one new thing here, which is VPN support for something that we call split DNS. So a little word of explanation here is that at Apple, we're using our own VPN solution to connect from here. I see people, my manager, connected through VPN to the Apple network, checking his email, I'm sure. And so he's using our technology here to get to the Apple campus, to the Apple network. And what happened is that right now we have only one DNS. So every DNS request that you're doing, even if you're trying to go to yahoo.com, it's going to go through the Apple DNS to get that request. With the split DNS, what it does is that instead of having all the requests going through the Apple DNS, if we see it's not an Apple-related, it's a VPN-related request, we're going to go to whatever your provider is, your SBC.com DNS server, instead of going all the way through the Apple DNS. That means that there is less traffic on your VPN for traffic, which doesn't really belong to the secure network you're trying to access. So that's one thing that we're introducing in the VPN support for Tiger. Also, something that was really asked for is support for user certificates. So we're using AAPTLS for that. So now the VPN will support certificates.

Another thing that is pretty new is VLAN, so 802.1Q tagging support. Right now it's for server only. So you're probably familiar with what the VLAN does, but what we did there is that we have the support for the VLAN. It's integrated in the network preferences. You can also do that through the command line. But you can create your VLAN interface and manage it from the network preferences or the command line as you want. At this point, it's for Ethernet drivers supporting the hardware VLAN. So one of the big things is the tagging for the VLAN is done through the hardware. So there is almost no overhead there. And right now, the new G5 XSERV supports that on their on both built-in cards, or you can have add-on cards also that, uh, that support, um, uh, VLAN tagging. So this is for--for the server side of the VLAN where--where you can--you can, you know, basically, um, deal with multiple tags and--and multiple VLANs at the same time. Um, the future direction for servers and what we're--we're looking--looking at, and it's a link aggregation.

to be something that is on our path. So the IEEE 802.3ad link aggregation is something that we're really considering. And also failover which is another feature for those guys for Let's see. 802.1x, so that's another thing which is present. I mean, we introduced that in Mac OS X. I think it was after-- yeah, it's been in some of the Panther updates.

It's essential for wireless LAN and for the security. So it's also used on wired LAN for Ethernet. So we have a wide range of authentication methods that we support for 802.1x. And you can read them at TLS, TTLS, Leap, MD5, PEEP. So a lot of things there for 802.1x. And some of the new things we're introducing in Tiger is the WPA enterprise support. So that's something that was requested. And also EAPTLS, and we have some improved certificate support in there. That's new in Tiger.

IP configuration, we talked in one of the first slides about our ease of use mobility goals, that are ongoing goals for the core networking in Apple for Mac OS X. And the IP configuration is something where we added a lot of things. So we have something that you might know of which is called configd which is a central daemon which take care and listen to all those events I was talking about, coming from the kernel or coming from user lens, changes from configuration, and basically reconfigure the stack kind of dynamically and figure out what's going on. So the IP configuration is a big part of what we're doing here to provide that mobility side of things. And some of the new things that we're introducing in Tiger is that we changed quite a bit the way the DHCP works as a client. And when we use the remaining list time, you know, if there's no DHCP server around, so we're going to be a little bit smarter about trying to figure out, hey, can we reuse this address? We also use ARP, you know, we do an ARP probing to make sure that, okay, if the router is here, we can, you know, kind of shave off a couple of seconds and make your PowerBook, you know, come up faster when you wake it up and be on the network. Those kind of things.

We're also doing-- a lot of people have multiple interfaces and multiple addresses. So now in Tigers, there's a concurrent ARPs that are being sent out on all the interfaces. So we are not waiting sequentially to get all the information for all the interfaces. So there is more parallelism going there.

And we also have support for dynamic proxy configurations, which is known by the-- this is something that was, um, asked for by a bunch of people for the.pac support. And we talked about the split DNS. Now we got the split DNS that we're using for the VPN, but we also have this split DNS mechanism that can be used on a bunch of other ways also. It's part of the base of the system.

Another thing that we introduced in Tiger at a different level of configuration, more higher level, this is more geared toward people that are ISPs and things like that that need to configure a machine for you because you're just getting a new DSL, something like that. So until now, those people had to do a bunch of things to figure out what was your configuration, services.

Now we're introducing the high-level preference API. It's kind of an aggregate of code that were in different places all over the system. So things like the network preference panel or the network config framework, you had to dig in and get some of the information there, some of the information from the admin framework which is private. And things like the more SCF from Quinn that give you some information about how to get this. But now with this new API, we'll let you manipulate a lot of different entities. Network location, you can have location home, work, whatever. Network services, you can manipulate with this API. So your internet seen as an interface can be manipulated, created, deleted, changed as an entity here by some high level primitives. And also the network entities, your configuration for V4, for configuration for DNS, for IPv6, all those kind of things can be seen instead of parsing the XML or going through different frameworks to get through those. Now you get some higher level way of doing that.

Um, yeah, I talk in the highlight about the changes that we have for IPFW2. So IPFW2 is one thing that we ported and put into our core is, you know, it gives us a bunch of new things. So it's faster, it's more complete, and more features. So, you know, it's a lot of things there. more complete because it's got rule sets and so you can have a different rule sets for your firewalls that you can turn on and off instantly so depending on your location depending on a bunch of other you know sets that sets you decide you can turn on and off those sets and have and have different behavior for your firewall on the fly so that's something that is much easier to do with IPFW2. So yeah, you can have address sets and lists, and so you can have your address list and build that and use that and create your rule sets that way, so much more flexibility.

It's also have a big feature, it's got keep alive for dynamic rules, so it will do a much better job getting whatever the content is that gets filtered by the firewall, will get keep alive on those connection and everything, so we'll know what's going on. So that's something we get from IPFW too. It's also backward compatible with IPFW, with the previous one. But if you're doing, you know, an application which is a firewall and you're based on IPFW, what we will ask you for Tiger is to use a new rule set because a new rule set is more rich and we're still compatible with the old one, but we're asking you to move on and use a new rule set so we can, you know, at some point deprecate the old one.

Following this, what you'll see in the control panel, in the preference panel, is that there's a bunch of changes that have been done here and that are kind of leveraging on the fact that we have now IPFW2. One other thing we added in Tiger is UDP filtering. So now you can do UDP filtering from the UI in the control panel. We get better logging. That was one of the problems of the previous firewall, is that it was spewing all its log to the main log file. Now, IPFW has its own log file, so we have more control about this.

And also, we added a mode in there, which is a stealth mode, which basically lets your computer be invisible, if you want, on the network, if you choose to do so. And what it does is, even if right now somebody's trying to, let's say, to do a connect to a TCP connect and sending a SYN trying to get to your port 22, to your SSH, and SSH is off, if you don't have the stealth mode, what's going to happen is that we're going to send a reset saying, no, there's no service there. Turning on stealth mode, we're just going to ignore that we're going to log that request, but we're not going to send anything back. So if that port is not in use, we won't even say we're here. So if you scan or something like that, it may help. But the drawback is somebody's trying to really connect to you. They won't know if it's because you're not there or because you're just playing silent. But it's a stealth mode.

OK. So one of the things, there's a lot of changes in Rendezvous, also this year in Tiger. I'm not going to talk into detail about those because there's a full Rendezvous update session on Friday at 10:30 AM. So there's a lot of things that are going to get announced, or have been announced, actually. So new in Tiger, the text record API.

It's available from Java, and I think we announced that Rendezvous now, there's much more than that. I think we announced that everything is on the different platforms. You get access to the same API and everything. So I encourage you to go see the Rendezvous session to get all the detail about what's new in Tiger for Rendezvous.

Uh, I'd like--I'd like to make a little ploy here and just trying to, um, to get, you know, people to be aware of IPv6. I've been, you know, I've been at Apple for a while, and I've been working on IPv6 for, pfft, I don't know, many years now, trying to get, you know, IPv6, you know, around and things.

And--and I think it's-- I think it's time for developers to kind of try to be aware of IPv6 and try to make sure, you know, we've-- we've been saying IPv6 is coming and everything. everything. But it's really ultimately-- we've done a lot of things inside the OS to make sure that we're ready for IPv6. But until some of you guys come up with a killer app that is using IPv6, it's not really going to fly very high. So this is my few slides trying to encourage you to go with IPv6. So what's going on with IPv6 in Mac OS X? Every single interface on Mac OS X has a link local address. So since Jaguar now, we've been having IPv6 turned on in Mac OS X. And one of the things you probably notice is those FE80 addresses that you get on every interface. So those are great. That's a great integration with Rendezvous, because every single machine on that network, if they're Mac OS X will have a link local address so they can communicate without any configuration using those addresses. So it's a good way to leverage with Rendezvous because you don't really care about the address and the IPv6 addresses are, you know, bigger and like hard to type and everything but with something like Rendezvous, you just don't care. You just use names and you use higher level form of, you know, services so you don't really what the underlying address is really 128-bit or not. So that's a great integration, is that the ease of configuration, it's a stateless configuration. So you have a router that's gonna pick it up. You don't--you're using a link local on that link. So that's really, really easy for you to use. You know, there's nothing, you know, to be worried about too much. And as I said, Mac OS X, since Jaguar has it, and every release, we're adding new-- level of support for it. So we get support in the BSD socket, of course. We get the configuration since Pinter is-- you can configure in the network configuration panel. We can configure IPv6 or make it auto-configured manually. All those kind of things are already there.

We've got a bunch of services that are IPv6 you may not even know, but Apple shares IPv6-aware. You can do over FireWire. That's IPv6. A lot of things in the system are already IPv6 ready. And all the frameworks that we'll talk about later when we'll overview the APIs are all ready for IPv6. CF Network has been doing a lot of work to make sure IPv6 is present in there and that works with it. The DNS, all those kind of things are all IPv6 aware. So I encourage you for those points. It's the future. It may not be something that we need in the next two weeks, but slowly it's getting some ground there. IRD is asking for it. Internet 2, it's mandated there. DoD has made it something that they require also. And it's pretty big in Japan. So the point here is your application should be IPv6 aware. You may not be able to or want to take all the good stuff from IPv6 at this point, but at least be IPv6 aware. And for doing this, one of the things that you need to do is just make sure that your model, your application is address independent.

And we'll see pretty much how easy it is. Of course, it's lying with slides. But a little example here is showing you how taking a BSD kind of application where we're just doing here, what we're doing is we're doing a connect. And get us by name from www.apple.com. And then we're trying to connect to that server here.

So this is like standard BSD, normal Unix code. However, there's a couple of things that are wrong here. All those things that are highlighted in red-- I'm not sure if you can see. If I'm not in front of the-- All those things are highlighted in red here are really address dependent. They're V4 dependent.

Why? Because the structs are cater. Well, it's dependent on the size of an IPv4 address. Something like afinet, of course, it's IPv4. Ghetto's by name, well, ghetto's by name is the old legacy way to get the resolver. It's address dependent. And all those things where we do a size of that structure, which is V4. All this is going to break if you're trying to get with a v6 address. So don't do that. So instead of doing this, to do the same thing, to be able to have something which is address independent, in this version here, we're doing the same thing. We're just trying to create a socket. And actually, here, we're just trying to resolve and get a socket and connect to it in a completely address independent. So if you have a v6 address, it will work with v6. If you just have a v4 address, it doesn't change anything, which is probably the case still for the few weeks to come, it will work with IPv6. So here, what's going on is that instead of saying pfinet or finet, we're saying unspec, which means I don't care. It could be afinet, afinet6. I'm ready for it. And so we are using also get-adder-info, which is a much richer API than get-us-by-name. And here we're saying, OK, we want www.apple.com and HTTP, and we want the hints back and the results back. And here, depending of what we're getting back in this for loop here, we don't even look at the fact that it's an IPv4 and IPv6. just, OK, can I connect with what I get in the result from my DNS query? If I can, I don't care which protocol it is. I just do a connect with that. So I know this is not really a real life application and everything, but from a developer point of view, and most of you here are developers, the fact that it's v4 or v6 doesn't really matter once you get to your application because you're just trying to deal with TCP or UDP, whatever you're doing. By just restructuring your code a little bit and making aware of, OK, the address I'm getting may be more than four bytes because it could be an IPv6 address, and it's going to be 128 bits. By taking some simple steps, you can make sure that your application is v6. And what it means here is this code, the day you turn off IPv4 in Japan, in a device that has those little thermometers that just do IPv6, they have no v4 stack at all. You try to connect to that device here, you'll connect. You'll try first in v4. Or if the DNS doesn't give you a v4 address back, it will give you back a v6 address, a link local address, probably. And so you'll connect to that address here, and your code will work no matter what. If it's v4 or v6, you don't care. So that's just a little-- thing to try to get you guys aware of IPv6 and how it doesn't add a lot to do that. And it's easy to just change your code to get to IPv6.

That was my little IPv6 talk. So now what we're going to do here in this session is we talked about what's new in Tiger for the core networking in Mac OS X. Now we'll go back a little bit and do a review of the different level of API we have available for you guys and which level, you know, may make more sense than another to do your application and to get the most of the system.

So we're going to this kind of diagram here showing at the bottom here is a kernel where we just put the networking stack because we don't really care about the rest. So in the kernel we talked about the fact that you can do this KPI and there's a full session about this. But for most people on user space, we think about the BSD socket because we're Unix based and BSD socket is like the model there. But there is a lot of more things that are based on the Socket API, or also things that we see here on the side, like the DNS service discovery, or the stem configuration, or lookupd, which does the name resolution, that you can use. And they have API provided. So we'll go through those.

So as a reminder here, so the BSD socket that we talked about, the DNS service discovery, the system configuration when you can do network setup, network reachability, the connection dial-up, and also at a higher level, the framework networking APIs, which go with CF sockets, CF networks, the proxy SSL, and the network diagnostics, foundation URL, and ultimately at the top of the stack here, WebKit. So we'll go and have a little talk about every single of those.

BSD Socket, so that's a core programming interface. Everything in user land is pretty much based on that. That's a main, raw interface and the native interface of the system. What it provides, it's fundamental networking API. I highly recommend reading the Stevens book if you want to go through all the details of the Socket API. It's a very rich API, a lot of things in there. It can be complicated, but this is how you get the most out of the system. You do that when you want performance and total control. If you're doing a server application and you don't really care about all nice UI sync and you just want to have raw power, I would recommend using sockets. Because sockets are the closest you can get to the networking. So performance, total control. You still can use things like select or KQs for doing asynchronous operation. One of the things that people tend to think that socket is like, oh, OK, well, I'm doing a socket operation. I'm doing a read. I'm blocked. And my thread is blocked until data come back. Well, that's one way of using sockets. But you can also use it in an asynchronous way. And there is some way to do that. But you have a lot of control, a lot of things in the socket. And there's plenty of books explaining what to do. But it can be a little bit complicated. Also, from the socket layer, what we do is we provide low level access to some of the-- I was talking earlier when we were talking about kernel extensions, network kernel extension, that we would like you to avoid to all possible, as much as possible, to go in the kernel and get your own kernel extension because it's messy and it's dangerous and everything. So we provide low level access to a bunch of stuff from the socket API, things like IPsec or we got PFNDRV, which is a way to get directly to the interface. I think I have a slide on that. And that's where you want to use socket. Also, a good way to use socket, BSD compatibility and portability. We said before we're based on free BSD. Pretty much any tool that is using, you know, some BSDs in terms of the networking is going to compile and be working pretty much without any touching it on Mac OS X because of the BSD sockets. And they're using sockets. And we respect that paradigm here. So BSD socket, really low level, but gives you full control. That's where you want to use it. So the resolver library. So what it does, it does name to address resolution, of course. That's the thing it's known for. goes on the network to your DNS server and get all those A, quad A, whatever, all those records that you're getting and getting that. It does also service location and a bunch of other stuff, actually, for the resolver library. Yeah, the service and you can get things from etc host and all those kind of things.

So it goes hand in hand with the socket API. So this is kind of the same level. It's using the socket API, but it's pretty low level kind of access to the network and the configuration. As we said before, it's got full IPv6 support. So you can do query for quad A. Everything is ready for IPv6. We did a lot of work last year to get that done. It's used by nine, which is, of course, a big standard. And we do have the split DNS support. So that means that as a developer here, you can have several resolvers. You can make it have several resolvers. So for one part of the things, you can say, hey, I want to resolve use this resolver to get the information because I'm on some internal network. And I know that in my closed LAN here, I want to use this resolver that's going to resolve the address of my coworker two cubes down the line here. And that DNS is not public out there. So I want to use that resolver. So the query can be dispatched to the appropriate server, depending on the type of the query. So that's one of the things that we get in the split DNS support here. Another level of APIs, which was kind of on the side here, is the network reachability API. So what it's used for is that you can check the name or the address or an address pair and basically be notified when the reachability is changed. So you're on the road on your PowerBook. You don't get any network access because you're far from Starbucks. And now you're getting reach. And now your network reachability changed because you have connectivity and everything. And so this API will tell your application, your mail application, or some higher level application, hey, now is a good time to send some data because that host that you're trying to reach is there at this point. I can see it. I can go through it. So yeah, it's unified a set of disparate information that are kind of piecemeal information all over the place. it gives you a connection status also. So it ends get both synchronous and asynchronous mode to use.

So DNS service discovery, so multicast DNS to discover services. I'm sure you're all aware with those Rendezvous functionalities. So I'm not going to go into detail on this. But you get the Rendezvous API where you can name and register your services. You can browse for services. And you can resolve the service to address. So for more things about Rendezvous, go to the Rendezvous session Friday.

PFN DRV socket, I'll go through it. So from user space, you have some kind of very low level access. So we're using it in the 802.1x implementation, where basically we're going directly to the interface through this level without being encumbered by being in the kernel and being a kernel extension. If you're trying to do, let's say, a stack in user space or something like that, you can use that too for something like DECnet or IPX or everything. It's a good way. You're getting really direct access to the interface while being in user space. There's some other solution to do that also that we're not going to talk too much. The diverged sockets, which are used by the NATs. So that's another way to get packets from a different level in user space.

And yeah, it's preferred over the kernel extension. So we talked about all the things kind of in blue here. And now we'll talk really briefly on the things that are built on top of the socket and all the services that are probably for a lot of developers the right solution to use because they provide some higher level. They hide some of the complexity of the implementation underneath of the socket. And sometimes you just want to get in a URL you don't really care about how it gets there and managing all the networking side of it. So we have this. So the higher level frameworks here, it's pretty much the big glue which is in between an application like Safari and the raw BSD socket, which are what we're based on.

So building the stack from the bottom here, the first level is core foundation. So core foundation is really close to socket. And core foundation, this is just an overview of what it does here. But basically, the big thing is it's based on the CF run loop, and the CF run loop, if your application is using the run loop, you know that, it basically provides a basic asynchronous mechanism where all the network operation, instead of dealing with the socket directly, are kind of integrated in the run loop. So yeah, it underlies the NS run loop, and the Carbon events are also based on the CF network run loop. So the CF socket, it's still very low level, and it basically connects the socket to the run loop. So you can create your own sockets and then put them in the run loop or do some operations. Still, you have a lot of control in the socket at this point, but kind of integrate it with a CF run loop model.

So it literally handles any socket. The CF stream, that's a basic stream abstraction here. It signals the client via the run loop when the bytes come in. So again, if you're trying to be still close to the socket and kind of have some control, but you want to take advantage of some of the run loop functionality, CF streams might be a good way to do that. And for things that are using stream, of course, like TCP. Or yeah, also client side sockets are a good way to use the CF stream. Building on top of that, we have CF network. So CF network, what it does is it provides some protocol implementation. Here, you don't really care about dealing with your own socket or your own protocol or something. All you want is HTTP service to your application or FTP or using SSL, things like that. Or even things like Rendezvous are using the CF network or doing some DNS host resolution. You get access to that through the CF network framework. Give you full control still. All the protocol details are exposed of HTTP. You really have to understand what's going on and what kind of requests you're going to create or what kind of response you might get when you're using the CF network. So it's still very, very core, but you don't need to have all the detail and do your own implementation of HTTP, if you want, or FTP. So you control each read and write, and you also control the threading policy, which might be important depending on what you're doing in your application. That's a good level when you want to have control, but not too much going into the detail. CF Network is a good layer to base your application on.

So if we continue building up here on what's going on between BSD socket and Safari, we see the foundation URL API. So that provides more higher level things here. So full feature set of URL loading. So there, you really don't care. It just gives the URL, and you get the result back. You don't care what's going on on the network side. All the layers underneath you are going to take care of that. makes most of the choices for you. You don't have to give a lot of details, a lot of options, everything. It's going to do the right thing. Asynchronous, it's based on callbacks, API. So for somebody who just want to get a file, get a URL, that's probably the right answer here. And also, one of the good things about this foundation in SURL is that it's extensible by subclassing. So you can subclass one of the primitive here and get your own version of it, depending on what you're doing. Gives advanced features, authentication, caching, and cookie storage and management also at this level.

And when we get on top of that, ultimately, pretty much Safari, you get to the WebKit. So the WebKit is the last layer we introduced last year. And it gives very little control. But we do everything for users. So it's really, really far away from the networking as the core we were talking about earlier in the session. But you get pluggable architecture, new document type, a lot of things. And this is really so far away from what I'm doing. So I'm not going to go into too much detail. If you want to know more about the CF network and all those layers here, I highly recommend probably looking at the CD at this point, because that was yesterday, to go to Becky's session, the modern networking using CF networks. That will talk about all those layers on top of BSD in much, much greater detail than I can do here, because we're running out of time.

After this, I'd just like a goodbye slide here for some of the deprecated APIs that we have in Mac OS X. First thing, the NKE PDF, the old programming information about what to do for network kernel extension, it's gone. We talked about we're changing all that. Go to the session this afternoon. We'll go into more detail. But don't use that anymore. Don't use the old KPI there, the old network NKE APIs. They're gone. They won't. Open transport API, well, you know, CF network or the sockets provide more flexibility. They've been there for a while. We're deprecating them. So if you can, you know, stop using it in your new application and all those kind of things. Apple Talk API, same thing. We've kept, you know, Apple Talk clamping, you know, around in Mac OS X for a long while. It's still there, but the Apple Talk APIs are deprecated now. We think we have a good solution now with Rendezvous for service discoveries. There's no more need for doing any new things on Apple Talk. So Apple Talk APIs are getting deprecated.

And with that, I'd like to point you to some more information, who to contact. Craig Kisly is our I/O technology evangelist, so use him as a contact point. And also, there is a Mac Network Programming mailing list that you can access. And there is a bunch of things, reference documentation here, all on the ADC website. So I'm not going to go into detail on that slide, but pretty much you go to the ADC networking and you'll see all that.