I actually have the privilege of introducing one of the more emotional and energetic sessions of the Developers Conference this year. And actually that makes a lot of sense because it's the energies of developers like yourselves that actually made what is the internet actually happen. This session is actually going to go through and talk a bit about Mac OS networking. This is networking in Mac OS 9 and Mac OS X. So with that, I'd like to introduce Justin Walker, who is the manager of the Mac OS X networking team.

Thank you. I've never been associated with high energy, so I'm not sure what Tom meant by that. Anyway, So let's start. What we're going to do today is talk about... Networking in our two systems, Mac OS X is what I'll cover, and then we'll have Lefty come up and give you his trademarked walkthrough of the current state of Mac OS 9 networking.

So to begin with, we'll set the stage by looking at a diagram that shows you where we're going to be focusing. This is a diagram of Mac OS X, which you've probably seen a couple of times before. And the important part for us is the networking area, which is in the kernel of Mac OS X.

An important point to note, which we've emphasized before and will continue to emphasize, is that the kernel and the Core OS of Mac OS X is Darwin. And Darwin is a public source operating system that we want developer involvement in. I'll do my best. Maybe we'll get somebody to crank up the volume.

Anyway, the Darwin OS is a publicly available OS, and we've got a website set up for it, which you will have heard about if you were here on Monday afternoon. In networking, just to review, we have... Network stack that's based on BSD, and in particular, it's been upgraded within the last year to a relatively recent version of FreeBSD. And we're providing full multi-homing and routing capabilities for both AppleTalk and for IPv4. And as part of the upgrade to the FreeBSD code, we imported mechanisms to support firewall and NAT.

I want to go over what we've done in the past year and then talk a little bit about what's coming up beyond DP4. Among other things, we've introduced kernel extensions, PPP support, the new stack, and some capabilities that I've already mentioned. And coming up, we'll talk a little bit about IPv6, IPsec, and something we're calling mobility for want of a better term right now.

So we've shipped DP1 through DP3, which was the release that contained the new networking stacks. And these products included improved Carbon OT support focused on performance and some new features. In February, we shipped a kernel development kit, which included support for network kernel extensions and documentation for developing them.

In DP4, which you've now got a copy of, we have a number of improvements I want to discuss. Shared addresses for IP between the classic environment and Mac OS X. Some more improvements in OT in Carbon. And a little bird told us we had to have PPP, so that's in there. We've got full DHCP support both in client and server, including ad hoc networking. And we've been continuing to make progress on a socketization of the AppleTalk code.

NKEs are fully supported in DP4, and actually they're a significant part of the operation of DP4 today. Our support for shared IP with Classic is done with NKE, and PPP is also implemented with NKEs. The development mechanisms in DP4 for kernel extensions supersede the KDK in a number of areas, so the header files and documentation have been upgraded to reflect that.

And coming up in the future, we want to improve support in this area. Particularly, we want to look at support for new device families, for example, for Token Ring, and improve the way we handle Socket and Datalink MKEs. The bottom line here is feedback from you as developers is an important aspect of how we plan and schedule our work.

For PPP, we're supporting serial connections in this release, specifically the internal and external modems. We use chat scripts. Those of you familiar with BSD and Linux style PPP will know what those are. They're, I guess, a poor man's CCL. Dean Reese will be talking about modem family support and the networking device session that comes up Friday morning. And you'll want to attend that if this is in your area. For API support, we have a socket API that is OT-like in the sense that it directly mirrors the OT PPP control APIs from Mac OS 9. And those APIs are implemented in our Carbon OT stack.

Configuration. The dialer app that is in the system demos directory on DP4 is handling the UI for configuration. And in addition, if you're a glutton for punishment, you can fill in the traditional BSD flat files. But the intent here is that you either configure through the dialer API or through another mechanism that I'll be discussing in a little bit.

Also, our aim over the long haul is to support PPP over whatever the developer and customer community require. Particular areas that we've heard requests for Ethernet and DSL. And the support will be integrated in with the rest of PPP. So for example, the dialer will be able to recognize the kinds of devices that are in the system. The devices that support PPP and the same APIs that are used for serial will work for PPP over Ethernet and other mechanisms.

Let's talk some about classic networking. By popular demand, we've eliminated the need to provide multiple IP addresses for each platform, one for classic and others for Mac OS X.

The classic implementation will gather all of the configuration parameters that are important for OT from Mac OS X and install them in the configuration database. The control panel is read-only, so you can see what your parameters are, but you cannot modify them.

As I'll show in a minute, the reliance here is on OT for routing. Any packet transmission is handled fully by OT as far as the IP stack is concerned. In Mac OS X, the implementation splits the flow of packets below the 10 stacks so that they are not directly involved in packet traffic.

So let me take a look at the operation of this mechanism. We've got the classic application with the OT stack, relatively unchanged. It opens up a special socket and binds to a chosen network device. and installs an NKE, which will be used to handle outbound traffic from OT.

The different colors indicate the direction of flow. So a packet going out from OT will be written into the socket. Socket NKE will look at the packet and determine whether it is destined for the wire or gets essentially looped back to the TEN stack. Likewise, when network traffic comes in through the device, it's intercepted by an NKE.

Based on information within a packet, a decision is made whether it goes upstream to the 10 stack or whether it's handed off to the classic stack via the special socket. The same thing happens for AppleTalk, except that in the case of AppleTalk, there's no combination of addresses. Each stack is fully separate and independent.

So let's take a look at how TCP and UDP traffic is handled. The OT stack has been modified to handle port sharing, which means that we treat both stacks as using the same port address space for both TCP and UDP. When there's an OT bind call, it comes into OT, it is directed down through the socket and then over to the IP stack to do a bind within the 10 stack.

And if that succeeds, the success is propagated back up the stack to OT and the normal OT operations continue. So, for example, if you wanted to run a web server within OT and there was not one running on 10, you can fire it up and it will take over port 80 and all traffic to port 80 will be diverted to classic. If the port reserve fails, then the bind will fail and the operation will not succeed.

Moving to Carbon, we've done a lot of work on performance and we're now at a position where when we run the same tests on Carbon on 10 and on Mac OS 9 on the same platform, we get similar performance. As far as we can measure, it's on a par.

The support for control APIs is there based on the OT PPP APIs from Mac OS 9. One point that we want to make for developers using OT in Carbon is that you can't make assumptions about thread behavior the way you could in Mac OS 9. You have to use atomic operations and make sure you're using the proper enter, leave notification.

OT on 9 uses cooperative threading, cooperative task operations, whereas on Mac OS X, threads are preemptive.

In AppleTalk, we've updated Carbon and OT to handle a number of new mechanisms. In Server Provider APIs, we support ZIP, we support DDP endpoints, and we support the NBP Mapper APIs. Apple has decided that Apple Talk is a legacy protocol and we're not going to put any more effort into it. So in particular, that means no PAP or ATP support.

And as always, we listen. So if you have concerns about this, you need to talk to Developer Relations and Marketing so they can give us the proper direction. Finally, we've fixed a bunch of bugs and it's very stable. If you saw the demos of Internet Explorer using Carbon, that was all done over the Carbon OT.

We've got DHCP support in DP4. It's full client and server, so you can act as either. It's RFC compliant with all of the musts of 2131 implemented and most of the shoulds. In addition, the DHCP engine will handle ad hoc or zero-conf networking. If the DHCP server you're trying to contact doesn't respond, then we will auto-configure an address for you in the range that's been dictated by the ITF. And the DHCP engine will continue to probe for DHCP servers. And in the case that it eventually finds one, that auto-configured address will remain in you.

So existing connections and existing programs will continue to run, but the system can now function based on its DHCP address as well. In addition, we'll be looking at more integration with the zero-conf work that's going on in the IETF. And I think Lefty will have some more to say about dynamic DNS support and the direction that we're heading there.

Beyond DP4, we're looking at something we're calling mobility, just for having a simple phrase to capture it. What we're looking at is mechanisms that we can put in place to simplify configuration change and management. We want to be able to change the address without a reboot. We want to be able to put the system to sleep, wake it up, have it deal with changing networking environments around it.

And with net configuration change in general, as might be forced by, say, a user running a configuration panel. We're bringing in native support for the Mac Network Setup APIs. I'll talk a little bit more about that later. And we're implementing IPv6 and IPsec. And we've got two sessions tomorrow to discuss that in more detail.

So let's look at what this mobility stuff is. We want to be able to live in a dynamic environment. Things like the Energy Star requirements from EPA, power books, other portable mechanisms can move around or go to sleep, wake up, and find the environment changed. We want to provide the ability to make those transitions smooth and adapt automatically to the new environment as it's found by the system.

So let's look at a really colorful diagram starting from the bottom

This allows us to have a common view of configuration information and to interlock access to that information so that multiple applications could be modifying this information or watching for modification at the same time. Another component, which I indicated earlier, was kernel event delivery. These are events that will come up from network devices and network stacks as things change.

And finally, that central engine is the configuration daemon. It's a daemon that runs as long as the system is up. We're building a plug-in architecture so we can extend it dynamically. It detects changes from devices by waiting for events. It can detect changes based on action from applications like the control panel. And system components like PPP can notify it when something changes. For example, a link comes up or a link goes down.

And pieces of data that are kept and affected by these events include default routes, IP addresses, DNS information, and so on. We have APIs that allow an application to register to get notifications when certain kinds of events occur. And we have APIs that allow the applications that modify this data to notify the rest of the system when a change has been made.

So to go back just briefly to put it into a picture again, events come up from below, are cached in the ConfigDemons cache, and trigger notifications that can go to interested applications. And we have a mechanism with the Network Setup APIs to let control panels and similar applications modify and display current configuration information.

In the future, we're going to work towards a more integrated system of configuration management. The aim is to get seamless changes supported. For example, you can use airport in a live environment and track base station change and the like. One of the interesting areas for a developer is how to handle a change like this.

There are issues of security. Do you want to believe exactly what the information you receive from the network tells you? Or do you have to provide a filter of some kind and prompt a user for information? It's a complicated problem. I have no idea how complex it is, but it doesn't seem to lend itself to an easy solution.

Another issue we're going to look at moving forward is the IETF's mobile IP support, both for IPv4 and for IPv6. And again, I'm soliciting input. Talk to me, talk to the rest of our team, talk to the developer relations guys and marketing. Give us your input as to what you think is needed in this arena.

IP is kind of the ubiquitous protocol these days, and there are good reasons to think about changing. The IETF has a new generation of IP, and it's called IPv6. Support in Mac OS X is on the way, and we'll be talking more about that tomorrow, both for IPSec and for IPv6.

Tomorrow, we'll tell you more about how to get access to a package which includes both protocol sets, tools, and libraries, and headers, as well as a kernel that supports it, so that you can begin to experiment for those of you who want to get down and dirty. The implementation we use comes from Kame. It's a project based on a consortium of Japanese manufacturers who are very interested in propagating IPv6.

They've done a whole lot of work on these protocols, and they're currently the de facto implementation for all of the BSD projects, and we're adapting it to use for Mac OS X. Check the developer website under Mac OS X for details tomorrow. And now with that, I'll hand it over to Lefty to talk about Mac OS 9.

Thanks, Justin. Hi, everybody. Has it been a year already? Good grief. So, I'm David Schlesinger. I'm the manager of the Mac OS 9 networking team and I'm going to talk a little about what we did last year. We're going to do the thing I guess that Justin was referring to when he talked about trademarking the talk. We do it every year. We go through the list of what developers have requested and where we stand on those requests. And then I'm going to give you some idea of some of the things that we're looking at working on now.

So, what did we do last year? We came out with two sort of general releases of OpenTransport. We had an OpenTransport 2.5.2, and we had various flavors of OpenTransport 2.6. And if you've got Software Update active, you've probably gotten each and every one of them. So, 2.5.2 shipped with Mac OS 9. And one of the big changes that we made there was that finally, after having been out for several years, we upgraded our core streams and TCP implementations.

The streams and TCP that had been in Open Transport prior to OS 9 actually dated all the way back to the original version of Open Transport. And it had become more and more difficult to maintain, and we were missing out on a lot of features that the folks over at Mentat had been putting in.

And basically, we took all of the old stuff out and put all new stuff in. And that was a very major change. It went a lot more smoothly than we thought it might. So, we were pretty happy with that. We have learned a lot about DHCP. We thought DHCP was pretty straightforward. We were wrong. Boy. There are lots of varying interpretations of little points in DHCP, it seems. There are lots of corner cases that we hadn't quite thought of. But I think we may have found every single one.

But I'm hoping. PPC-over-IP. Now this is something that I'm very excited about. Apple events have been around for a while now, but they're limited to Apple Talk Networks, which really places a real strict limit on the context in which you can use them. What we did in OS 9 is we expanded the PPC protocol to work over a substrate of IP.

So, instead of only being limited to doing Apple scripting, remote Apple scripting, in the context of your Apple Talk Network, you can now do it across the internet. You can effectively run an Apple script or send Apple events that will affect a machine all the way across the country or around the world. And this gives a lot of power and a lot of flexibility, particularly to people like systems administrators, ISPs, all kinds of applications for this. So, that's very exciting.

And I think we're going to be doing a lot more of this in addition. More DHCP fixes. We found out that DHCP...

Network Setup 1.1. We introduced Network Setup in Mac OS 8.5, and this was a planned upgrade. We had mostly bug fixes in there. We reduced open transport from six libraries down to two, thereby freeing up valuable real estates in your extensions folder.

And over 250 bug fixes. So all kinds of things are working better now. It's a lot more stable. We've got a lot of performance improvements for some kind of peculiar cases we ran into. So lots of good stuff went into 2.5 too. 2.6 is essentially what we call a dot release. It's got a bunch of bug fixes. And specifically, it shipped with Mac OS 901, 902, 903, 904. And it may keep shipping for a while yet. I don't know. 32 bugs fixed, all told. I sat down and counted them. And guess what? They're mostly DHCP fixes.

But it's working better than ever. We're happy with that. And if you have any DHCP problems, be sure to see me right after the presentation. Okay, so here's the trademarked part. I don't know how many of you were here three years ago when I first did this, but this is a list that Richard Ford, who was then our marketing guy, compiled, I guess, about four years ago.

And if you've been paying attention, you'll notice that the size of the type has probably tripled since the first time I used it. We've got 51 items on this list right here, and if that seems like a lot, you weren't here three years ago. Because we had, I think, 175.

So, every year, I go through and I tell people what on this list we've been working on, and as we get them done, I take them off, and the list gets shorter and shorter, and the typeface gets bigger and bigger, and eventually I'm going to have, like, one item in 48-point type.

At any rate, here are the things that we managed to do over the last year. We've got better admin utilities, network assistant particularly, Apple Events over IP, I mentioned, IP personal file sharing, and again, both of these things, PPC over IP, Apple Events over IP, IP personal file sharing, are all in line with the strategy that we've been following over the past several years, AppleTalk was a fine idea. It was a good thing. It's out of date.

Nobody's using it, really, except in very limited contexts, and it's not where things are going in networking, clearly. So, in order to preserve the valuable aspects that we put into AppleTalk way back when, and there are many of those, we have been transferring some of the most important uses of AppleTalk files, file sharing, Apple Events, and what have you, so that they can operate in an internet environment, and PPC over IP and IP file sharing are just two examples of that, but this is really a core strategy for us. Okay, what's left? Still got a bunch of things there.

We have limited engineering bandwidth, and as I said, this list is four years old. And I thought it was like a really good time at this point to sort of step back and say, well, you know, here are a bunch of things, and we haven't gotten any of these in the last four years. In my best estimate, what are the ones that probably are not worth the effort for us right now? And this is pretty much what I came up with.

Nobody screamed. Good. Okay, great. Anyway, Apple Talk Multihoming, Apple Talk Routing. We've had these in the past. We had a product called Apple Internet Router once upon a time, which I actually supported. I don't see a lot of call for that. I certainly have not had anybody ask me for ADSP optimizations in I don't know how long.

ATM enhancements, same kind of thing. I don't see us doing a lot of built-in kind of file system specific clients. I don't see a com toolbox replacement on the horizon, I'm afraid. IPX, IPX over PPP, ISDN, multi-link PPP, more of the same. So these are things that we basically decided are not going to remain on our list anymore, so I've taken the liberty of excising them. And here's what we have left, and here's what we're working on right now. Obviously, right at the top of the list, bug fixes. We never have enough bug fixes.

Yes?

I'm glad to have that feedback. Remember, this is our take on things from where we're standing, and if something I say doesn't make any sense, let me know it doesn't make any sense, and I'm happy to rethink it. Bug fixes. DNSSEC is something that we're looking at.

Justin referred to dynamic DNS a little earlier, but dynamic DNS and DNSSEC really interlink. You don't want to be dynamically updating names of networked entities out there, unless you've got some kind of security associated with it. So I don't see us doing the one without the other. IP multi-homing is something that people have been requesting for I don't know how long. We are actively working on it right now. We have prototype implementations, various places that are doing various things, and we're making a lot of progress in that space. IP second, IP v6.

Justin has referred to previously, and I'm going to go into those in just a little more detail here, but I want to save the really good stuff for tomorrow when we've got sessions dedicated to both of those subjects. We are working on a unified control panel for networking, so that rather than having to deal with seven different ones, you've only got to deal with one. And we've demonstrated various incarnations of that as time has gone on.

Plug and play IP auto configuration is sort of a long way of saying, "I'm using IP v6." And again, tomorrow in the IP v6 session, I'm going to go into a lot of detail about what that's all about and why it's important and why you should care. And finally, RTP, PPP compression. This isn't just RTP. It's IP, UDP, TCP, RTP header compression for serial links, which is a real performance enhancement for applications like streaming media and what have you. have you.

So, these are the things that we're actively engaged in these days. I wanted to break this out in a little more readable format, but this is roughly our order of priorities on this thing, and there's a reason for that. IP multihoming and IPsec, it turns out, are prerequisites for doing IPv6. So, in order to get that in, I have to get these in first. Now, these are features that have value in their own right, but they head the list because I can't do what I really want to do until I do those things first.

I think we really ought to get familiar with it right now. All of the proliferation of wireless devices has created a situation where all of you folks with airport cards are beaming all kinds of information all over the place. We had a little experiment a while back when we first got our set of base stations and airport cards over at Apple. We sat out in the parking lot with one to see if we could see what was going on. You bet.

So, we're very careful about using VPNs and limiting who can use which base stations, but security over networking is very important, and that's the point you should grasp here. People spend a lot of money on firewalls, but once you put a wireless card in your laptop, your firewall is not doing you a lot of good at that point. So, this is very important. We're going to talk more about this tomorrow. Multicast DNS, this is interesting.

What you have to supply as the developer is the actual key management logic. If you want to have Ike, then you've got to implement it on top of what we give you. You've got to deliver the actual protocol arbitration. We deliver just the core. We're going to have more details on that at the IPsec session, and that is 3.30 tomorrow next door.

Multicast DNS I already talked a little bit about. It's an extension of standard DNS that's being worked on in large part by the IETF being led by one of our engineers here, Stuart Cheshire. Again, it's intended to support ad hoc networks. It's solving a similar problem to dynamic DNS, but not in and of itself. Effectively, we're putting a multicast DNS kind of micro-server into each Macintosh, and you'll register names with this server in an analogous way to the way that you register NBP names now. You'll do it through a mapper object. So the analogy is pretty straightforward there.

Okay, we're looking at making Open Transport callable from MP threads. Now, it turns out that this is one of those things that also has prerequisites. The thing that we basically needed to do to make all of this work was get rid of Open Transport reliance on Apple's shared library manager, which was a fine idea way back when once upon a time, but it's been supplanted by CFM. It's essentially unsupportable. We're hoping that very few people use it, and if you are using it right now, we hope you'll stop soon.

The reason that we want to do this is basically to lay the groundwork so that we can put, as an alternative to the standard OT APIs, a sockets interface on top of OpenTransport. There's no sense in my doing that unless it's semantically accurate when compared with a Unix machine running the same APIs. So I need to have some kind of real robust threading in there in order to do that, and that's what the goal of this essentially is.

IPv6. Next Generation Internet. There are a lot of reasons that I'm not going to get into in great detail today because I've got a big long presentation on this for tomorrow. A lot of reasons why you should be interested in IPv6. There, people talk about the address space being exhausted.

That's not true. It's not getting exhausted. It's just been badly misallocated all along. The big problem that we face right now is a combinatorial routing problem. Every time more nodes get added to the Internet, that's more things that more routers have to remember. The real problem is that IPv4 addresses carry no inherent routing information in them.

The router has to remember everything just based on the IP address. IPv6, just to give you a quick preview, gives you the capability of constructing addresses in such a way that they actually carry routing information in them. There are other points of IPv6 that simplify and speed up all kinds of aspects of using the Internet. We're going to talk in detail about that tomorrow at 5:00. I hope you'll come to that if you've got an interest in. I hope you do. Again, async AppleTalk initialization. We've substantially reduced boot and wait times and about time too.

Okay, I want to talk very quickly about where we're going on remote access. The main feature that we brought out with remote access in the last year was whereas we had previously shipped only a client version, we're now shipping full client server. This is basically the same remote access product that went out in a box once upon a time that people paid good money for. It's now part of the regular operating system distribution.

The plans that we've got in the upcoming year are mostly around PPP multi-homing, and that's in line with our work on multi-homing for the rest of open transport. As I mentioned earlier, we're working on the advanced header compression that's specified in, what is it, RFC 2349, I guess, and elsewhere. We're working on support for the extensible authentication protocol, which is being more widely used and has been requested by an organization. We're working on the advanced header compression that's specified in, what is it, RFC 2349, I guess, and elsewhere.

We are working on giving you finer control over the framing that's applied to the byte stream coming through remote access, which will simplify implementation of alternative links for people who are doing things like PPPoE and what have you. And again, we're looking at adding IPv6 support in there as well.

Okay, so time to talk about endangered species, as I do every year. There are some things I would like to have go away. I don't think anybody much uses them. I'm here to ask you to let me know if I'm grossly wrong about these things. I'm pretty sure that Mac IP is not being very heavily used. We've been carrying it around for a lot of years. It's tunneling IP through Apple Talk, and it's just not a wonderful idea. It places all kinds of limitations and restrictions on things. It would simplify our lives considerably if that could just go away.

Nobody's told me it can't, but if you have a different opinion, you should definitely let us know. This is an old one you've heard before. Prefs files, they're still going away. I know that all of you have been working very, very hard on getting onto the network setup database, APIs, and that's what you should be doing.

I am proposing... Actually getting rid of the prefs files this time around. And again, if somebody has a major objection to that, they should definitely let us know. Because I don't want to cause problems, but I do want to simplify matters. In remote access, it's about time that we said goodbye to the Apple Remote Access Protocol. I don't see any real need... Really? I would have done it last year.

Okay, well, that's for sure then. Okay, so we've got some additional resources that relate to the various points that we've mentioned here. There's the Mac OS IPv6 and IPsec package, which you'll be able to find under the Mac OS page on developer.apple.com. The Project Kame homepage is at www.kame.net. The IPv6 forum is at www.ipv6.org. Mentat's homepage, if you're interested in learning more about the core code that goes into open transport, is at mentat.com. And the IPv6 conformance testing information is at tahi.org.

We have related sessions. We've got this one right now, which is certainly related, but you've probably missed most of it by now. Tomorrow, we've got IPsec in Mac OS at 3.30 in the room next door, followed immediately by IPv6 in Mac OS. And both of those, I think, are going to be well worth attending. These are largely going to take the form of technology introductions and overviews.

So if you're not familiar with IPsec or you're not familiar with IPv6 and you want to get more information, these are really good sessions for you to be attending. There's going to be a lot of interesting and informative stuff there. Also, Dean Reese is going to be having the I/O Kit Networking Hardware Seminar at 10.30 in the morning on Friday in this room. And finally, I really strongly encourage everybody who has an opinion to share to come to the NNC Feedback Forum.

There used to be previous years we had one for 9 and one for 10, but we've unified those as well this year. So if you want to tell us what a great job we're doing or tell us what a bad job we're doing or something in between, come to room J1 Thursday, 10.30 a.m. for the NNC Feedback Forum.

and contact information, I'm Lefty. Justin Walker manages the Mac OS networking group. Tom Weier is our guy in Worldwide Developer Relations. They're all our email addresses. We love to hear from people as long as it's not too abusive. Or even if it is, if it's like well-thought-out or creative. and now I'd like to bring Justin and the Mac OS X networking teams and the Mac OS 9 networking teams all up on stage and we can do our Q&A. Thanks, folks.