Hi. Hey I'm actually surprised there's this many of you here cause it's Friday and it's late. But welcome. I think it's been a great developer conference this year. I want to talk to you today about what we're doing with device drivers for class compliant devices on the Mac OS platform. So the built in drivers for Macs are actually pretty good, but they're pretty good in a limited way.

There's stereo, you can do stereo input and output. The performance of them is really good. But we found, and we know from talking to people who are doing pro audio that they want to be able to expand the built in capabilities of the platform. And there's a list of things here that we know that you care about if you're gonna do a device. So how can you actually extend the built in capabilities of a Mac? Well, you've really got three ways, you can use USB, Firewire, or PCI express or Express card, or a variance thereof.

What I want to do is talk about standards space devices, so devices that are based on industry standards published by recognized industry bodies. And there's a number of advantages for developing your devices in these ways. But we do recognize that there are good and valid reasons why you may want to (inaudible) your own protocols.

The main advantages generally translate into time to market things. So if you're going to buy standards based silicon rather than doing an FPGA, you don't have to do all of the design language stuff that you're gonna have to do to (inaudible) your own solution. If you're buying a standards based solution, you're not gonna have to write firmware for that device. You may have to customize the firmware, but you're not gonna have to write the firmware from scratch for the device, and that's actually you know, a six months to a year thing on most devices.

The other thing is if you're using the driver in the operating system, you're kind of insulated from changes, and the types of changes I mean are things like the Power PC to Intel transition, the thirty two bit to sixty four bit transition. We've seen developers take upwards to six months to really (inaudible) when we go through a major architecture change.

If you're using the built in drivers, you're gonna work from day one. So as an example, AVC fireware devices that we're shipping on Mac OS, when we went to Intel they worked on the first Intel machines that we shipped. So there's a number of really good reasons why you want to go for standards based devices, but as I say, I understand why you may not want to go to it.

So these really translate to a time to market and a cost issue. And ultimately if you're not having to do all this engineering work, you know, just to get your transport working, you can spend more time on the product differentiation features for your device, and that's really important.

So the two things that I'm gonna talk about today, or that actually the team are gonna talk about today, are USB and firewire. We're not gonna talk about PCI or Express card, primarily because there really are no standards in those spaces to develop devices under. So for USB it kind of splits between USB audio class device one point oh and USB audio class two point oh spec compliant devices. And for firewire we're primarily talking about music subunit and audio subunit.

So what I want to do is kind of run through the features that we've put into Leopard, to sort of answer a question that some of you may be answering. We will actually ship the same drivers under Tiger. So you know, what we're talking about today will be on both Tiger and Leopard, although these features have been primarily developed for Leopard. And we're gonna you know, talk about USB, and we're gonna talk about firewire.

The goal is that you'll be aware of the new features that are available in the driver set on Leopard, and we want to provide you with a set of tips and techniques for ensuring your device works well on the platform, and getting the most out of the class drivers.

So you know, we want to point you at tools that we have for making sure your, making sure that your device descriptors are correct, and we want to you know, point you at resources that you can use in order to work more closely with Apple on your standards based device.

Also want to talk about specifically to firewire how you can customize the behavior of an ABC music subunit device. We've got a couple of things that enable your device to work well with our driver, and enable you to tune the performance of your device with the driver. And the last thing I want to cover is some work that we've been doing with the firewire team for networking ABC devices together. And I'm really excited about that.

So I want to bring up Tori (assumed spelling) Walker. Tori is the device driver engineer who's responsible for the Apple USB audio driver.

Apple USB audio is the name of the driver, it's a kernel extension, and it supports full speed, meaning USB one point oh, one point one devices using USB audio class one point oh descriptors. One point one is to transport, one point oh is to spec. The round trip latency is under ten milliseconds, for most devices we've measured it's generally about nine point five milliseconds. Kind of depends on the hardware there, but that latency is measured from analog to analog. That goes up through core audio and back down. And that's a pretty good latency I think. There are also a few newly resolved issues I wanted to touch on.

For one, we now properly handle short refresh intervals for feedback and synch endpoints. What we had before is if you were using a feedback endpoint to report your sample rate, if the polling rate was so, was too low, then we had some trouble with queuing. That's been fixed, so you can go down now basically as low as you want. Also we now properly handle high refresh intervals. And when I say greater than a hundred twenty eight milliseconds, since they have to be a power of two, and the maximum is two fifty six, what I'm really saying is now two fifty six millisecond polling intervals work.

Also we now actually respect the lock delay value. Now as far as I know, most devices I've seen do not record a lock delay value. So it didn't really make that much of a difference before. But we have seen devices where we try to start streaming to the device and it's not ready, and we get choppy audio for the first very, very short period of time. Well now if you specified lock delay properly, we will obey it, and we won't start streaming until your device is ready.

Finally, we handle the volume controls with large ranges properly. Previously if you had a volume control range, it started at low, the low of negative one twenty eight DB up to about one twenty eight DB, with the resolution of about one DB, you might see some funkiness being reported with the DB value that's reported, and audio MIDI setup, or any audio application where it shows a control value versus a DB value. That also has been fixed.

So, with that being said, let's move on to two. It's bigger, it's better, why would you want to use USB two point oh audio? Well, the main reason is you've got a fatter pipe to deal with. That means more channels, higher sample rates, run faster, jump higher, be the envy of all of your friends. Plus any system that you deal with these days, regardless of what platforms you're developing for, generally have a USB port. So it's a good bet to go ahead and get into that.

So, what driver is it? It's the same driver. Apple USB audio is gonna support USB audio class one point oh and two point oh. The round trip latency is still under ten milliseconds, and now we have support for high speed endpoint polling. Now in the USB one point oh audio specification, what you basically had to deal with was every millisecond you're expected to send or receive some data.

With the USB two point audio spec you can have data transactions as many as eight times per millisecond, or once every one hundred twenty five microseconds. Now for those of you who are really on top of it, you think well if I can poll that often, is there some potential for me to perhaps have a lower latency than USB one point oh audio devices? And what I want to say is a very firm maybe.

And the reason I say that is because before the queuing was within one millisecond, so we had one millisecond of jitter for input, and one millisecond of jitter for output. But now if we move that down to one hundred twenty five microseconds of jitter for input and output, I could potentially shave as much as one point seventy five milliseconds of round trip latency off that for class compliant devices, from analog to analog, up through core audio, and back down.

And that's a pretty savings. However, in order to develop a device that will provide that for you, I need you, sorry, a driver to provide that for you, I need you to develop a device that actually polls that often. So if you're interested in developing an ultra low latency solution high speed audio device, let us know, get it to us, and we'll see what we can do to drag that latency down even farther for you.

So what are the big changes in USB audio two point oh? The biggest thing is probably the clock model. There's a new clock model and latency at the feature unit level that have been added so that the timing relationships and synchronicity are detailed. We can actually go in and if you have a latency control, poll every latency control in the audio path and figure out what the latency is from terminal to terminal. Also sample rates are no longer listed explicitly in the format descriptors. They are discovered by querying the clock entities that are published in the USB two point oh audio.

This is a big change, because if you want to have any sample rates that are available for the user, we have to be able to go to your clock source, look at your sample frequency control, and read all of the different sample ranges that you support. If you do not do this, then your device may show up in system profiler, or something like that, but it won't show up in the sound preference pane or to any audio applications, because you never told us what sample rates you run at.

So this is a big, it's a big change, and it's a good area for you to study if you're going to develop these types of devices. Also, the notion of feedback endpoints has been changed. The feedback is formatted differently, and also the feedback is per transaction, as opposed to per millisecond. So for example, if you were running at one hundred ninety two kilohertz, and you wanted to report that is your nominal and actual sample rate, then if you're polling twice per millisecond, you actually report ninety six twice, instead of reporting one ninety two twice.

Also some descriptors have been streamlined. They've changed, make sure that you take a look at the one point oh version versus the two point oh version, and don't mix those descriptors. And also, and this is a big point, high speed devices with USB one point oh audio class descriptors are explicitly disallowed, explicitly disallowed.

I've seen devices, mostly, I think mostly cameras, high speed cameras that will have an audio interface on them, and they have USB one point oh audio class compliant descriptors. This, and they say that it works on Mac OS ten, I want to say that this is not by design, this is by luck.

The reality of the situation is is if we ever have a problem with a high speed device using a USB one point oh audio class descriptor, we may have to just explicitly disable that support. And if you want your device to work properly on Mac OS ten, you need to use the correct audio class. A USB two point oh audio class is not only good for high speed devices, but it's also good for full speed devices. So you don't necessarily have to have some new fancy audio chip out there to develop these.

All you have to do is use the proper spec and we'll respect it. Okay. Let's talk about the clock model. This is one of the biggest changes to the USB two point oh audio spec. Here's a big pretty graph on the screen, I want you to take a moment to admire it.

On the left side, well, stage left. All right, there's an input terminal on one side, and there's an output terminal on the other. Ignore the fact that the input terminal is also labeled OT, that's a mistake. What I want you to see is that there is for, there's an audio path, and then there's a clock path, and the clock path is joined to a few of the new USB two point oh audio entities. You've got the clock selector and you've got some clock sources. Now this is a very, very simple clock model.

What you know here is that this single audio path is going to have the same clock source, either the top one or the bottom one, depending on how the selector is configured. Now if you want to take this and kind of just imagine, just use your imagination. Make another one of these and connect it to the same clock path, then now you know that perhaps you've got two audio signal paths, and they're both synchronized.

This is something that you could not do in the USB one point oh audio spec. If for example you always have to run your sample rates off the same crystal, and your output sample rate changes, your input needs to change as well, but there was no way to say that explicitly. Well now, in USB two point oh audio, you can do it.

Now the big control that you're going to want to make sure that you implement for your clock entitles is the sample rate control. As I said earlier, you're not going to be able to publish any sample rates for the user to use unless you implement this. You cannot specify your sample rates inside the format descriptors any more.

So take a look at this clock source here, it has a sample rate control. The current value which we've just supplied here is gonna be one ninety two kilohertz, and then you'll see three range entities. Now the range subzero entity is just the number of ranges that are actually reported by the control, so in this case it's two.

Range one you'll see is ninety six K for the minimum attribute, ninety six K for the maximum attribute, and a resolution of zero. What this essentially means is it's a discrete sample rate. The other range is the same, it's one ninety two K, and it's also a discrete sample rate.

So when the driver looks into this clock source, it sees this sample rate control, and then what it says is okay, anything that's plugged into this clock source is gonna have two sample rates that support it. It has ninety six K and one ninety two K, and that's it.

So this is what you need to support. There are also other controls of interest that you may want to implement, such as the clock validity control. We don't want to try to choose a clock source for you that does not have a valid clock running. And there are other controls of interest as well that you can read about in the USB two point oh audio specification.

All right. So I want to make my USB two point oh audio device, how do I do it? There are three big ways you can do it. There are ready to roll implementations that are already under development by several companies. What you're gonna do in this situation is make sure that you change the vendor ID and the product ID to be unique for your company and your device.

Because in some cases we may need to distinguish your device from other devices, and if you don't change this, then your device may look like any other number of devices that are on the market. And if there's a problem that is specific to your device, we can't address that without affecting a wide, a variety of devices.

Also you've got programmable audio chips. You're gonna supply your own descriptor, and then you're gonna manage your own audio centric resources. If you want help building a descriptor, we are here for you. We would like for your device to be class compliant, and we would like for your descriptor to work well with Apple USB audio.

If there's, for some reason it doesn't work the way you expect, it could be because the descriptor is wrong, it could be because there's a bug in the driver. So contact us early and often, send us a version of your device, and we'll do our best to get it working on Mac OS ten at no extra cost to you.

And finally, FPGA. This is gonna be the most intense involved and difficult way to design your own USB two point oh audio device. Not only are you going to supply your own device descriptor, but you're gonna do your own device debugging as well. If you're going to take, undertake this massive project, please let us know early. Get us early versions of your hardware so we can help you out with it along the process, and make sure that it works well on Mac OS ten. Okay. Now on to James Brown's favorite portion, the gotchas.

The volume range is invalid. Sometimes I see this on, and the first few are descriptor problems. But sometimes I see the volume range is invalid. The valued hexadecimal value eight zero zero zero, which is meant to connote negative infinity, is only valid as a current attribute. You cannot use it as your minimum control value.

Now if you use this, we won't choke, we won't fall over. We're just gonna bump it up to eight zero zero one. However, if you're doing specific math with your resolution and you expect the step size to be X, it'll be off a little bit because I've had to change the step size, and your math doesn't work out any more. So be careful about this, and there's an extra paragraph in the USB two point oh audio spec that says this is only valid as the current attribute, so don't use it as your minimum.

All right, maximum packet size is incorrect. Let's say that you're streaming two channels, sixteen bit at forty four one. That means your average packet is gonna be, you've got the two channels, and sixteen over eight is two, and then forty four average samples. So four times forty four is a hundred seventy six. Damn, you guys are good. All right. So you're moving between one seventy six nine times, and then the tenth time you'll have one eighty, because you've got that one extra sample. So what if your maximum packet size is one seventy six?

Then when you try to send that one eighty for input, or I try to send that one eighty for output, we're gonna generate an overrun. Now an overrun was more or less benign on Power PC systems, but on Intel systems it's bad, very, very bad. And you don't want to have bad behavior by your device.

You don't want to have audio artifacts, you don't want to have all the traffic that's plugged into that particular USB port slowing down or choking because of these overruns. And also you don't want to report a maximum package size of zero, because honestly, why bother? That's like a no fat decaf sugar free you know, mocha.

All right. So another thing I'm gonna point out here is I'm gonna say the lock delay is not specified. Now that's gonna sound funny to you because earlier I said now we respect lock delay. Well hey, now that we respect it, we want you to publish it, all right? If your device takes a while to get its sample clock cooking before it's ready to stream audio, let us know about that and we will hold off asking for, or sending data to your device until you're ready.

Next, audio class descriptor versions are mixed. So some people wanted to jump the gun a little bit and have some USB two point audio support, but mostly use USB one point oh audio descriptors. This also can lead to unexpected behavior. Remember, if you're using a full speed device, you can use two point oh audio spec, or the one point oh audio spec. Either is fine, but just you know, keep them separate. If you're using a high speed device, you must use two point oh.

Finally, and this is not a descriptor problem, this is actually just kind of a device problem, especially for input. Your packet to packet variance exceeds one sample frame. The USB audio one point oh spec said that you could go up one sample or down one sample for your average.

So say you were doing, streaming in forty four one, then you could go forty four or you could go forty five, or you could go forty three. Well for the USB two point oh spec you can go up one. So you can go forty four, you can do forty five only.

Now I'm not gonna check that, but it is important for you not to dip down to forty, and then try to compensate for it later by jumping up to forty eight, because we have a very, very sensitive, time sensitive measurement that we're doing here to provide low latency, and if the data's not there when we expect it, you could get artifacts or worse.

All right. Few final thoughts before I get off the stage. In Leopard the same driver supports USB audio one point oh and USB audio two point oh classes. And not only that, it's on that Leopard beta disk that you guys got already. So you could start developing your devices against this right now. As Nick said earlier, we're gonna try to get the same support into a software update for Tiger before we wave goodbye to it, but you can go ahead and start developing now using the developer DVD that you've received.

Class compliance does not compromise your features, and it does not cost your resources. Remember, if you develop a class compliant audio device, you have relative immunity to platform changes and software updates. And you can still write your own driver, just provide a higher match score and we will gracefully back away and let you handle your own device. But if something happens where your driver breaks, hopefully your driver can just be disabled, and then your customers can still have access to the basic, and maybe even advanced features of your device.

Also, we can help support you for your product. If you send us your descriptor, we'll be happy to hand parse it and let you know whether or not it's good to go. Send us your device, we'll do device testing, as well as sleep, wake, and hot plugs to make sure that it behaves the way we expect it to behave And finally, if you read the audio spec, and not just the audio spec, but the USB spec as well, so I wanted to point out a couple of sections. USB two point oh, this is not the audio spec, but actually USB two point oh spec. Section five two oh four on isochronous devices, this is a very good read, also section nine six six on endpoints is a very good read.

And finally I'll call out the USB audio class, section four seven two, the class specific audio control descriptor. This is basically where all the new clock entities are located that you're going to need to implement for your device. Thank you for your time, and I'll turn it back over to Nick.

Good job Tori. Thank you. Hey thanks Tori. Okay, so I want to talk a little bit about (inaudible) devices, and specifically about AVC, music and audio subunit place devices.

Why do you use file after audio? Well, it's a big pipe, so you get a lot of bandwidth, so you can get a lot of channels down there. We've actually found that we can get lower latency with firewire than we can with USB, and we have a contest now that we've actually got the latency down to something reasonable. They're both under ten milliseconds.

Firewire is just edging ahead of USB, and Tori's always coming up with schemes to make USB lower. But we understand, and we've heard you, you know. You've come to us and said class drivers, how am I gonna develop a standards based device? Your driver's too high lane, see I can do my own.

What's really interesting though is that sometimes we measure your drivers, and sometimes chip companies will come to us and tell us they've got a very low latency driver for their hardware. We measure those against ours. We know that on Mac OS, with the exception of (inaudible), we have the lowest latency driver now. We're within about one and a half milliseconds round trip (inaudible), and we're gonna try and get closer. (inaudible) pretty cost effective to implement.

That may seem odd, cause you can get a USB one point oh chip for about a buck, but you can't get a USB two chip for a buck. The firewire silicon that we're seeing for audio is very cost competitive with firewire silicon that we're seeing for USB two point oh. That may change if volumes go up on the USB two point oh side, but you know, that's the truth of it today.

And the thing that I find really compelling about firewire as a solution is peer to peer networking. And we've spent a long while trying to get the firewire driver where we want it in terms of performance, and we know the way we want it is the way you want it.

And Matt and Neil (inaudible) and the rest of the team have been working really, really hard over the last year and a half to get the latency to something that we consider to be good. And we think we're there, we'd like to hear from you. This gives us a chance now to look at the thing that has always been interesting about firewire, and actually is gonna come up and talk a little bit about networking and some code that we're gonna release (inaudible) SDK so that you can start to experiment with this.

So I want to talk about the new features in Leopard. And I've listed the things that I think are the most compelling. What the (inaudible) driver team have been working on probably for the last year and a half is rewriting the streaming engine. In the past we took an interrupt to do our work about every millisecond.

Interrupts in Mac OS ten not good. We've rerun everything on new DCLs to be timer based, and that's enabled us to do a whole bunch of stuff that improves performance, reduces CPU overhead, improves the latency, so I want to talk about that stuff. We've also put extended stream format support in, and I want to talk about that. We've improved MIDI, we recognized that it was kind of weird, you know, cause we had low latency on firewire for streaming audio (inaudible) speed driver.

But the MIDI latency was double, why was that? So we fixed that, and we've actually got really good latency now with MIDI data. And then there's been some changes in the user client which are of great interest if you want to customize the behavior of your firewire device.

The new straining engines, it's been a long and hard (inaudible) but we really (inaudible). It's been rewritten, we removed the timer. So those of you who know about this and care will realize that that CPU spiking's gone. CPU spiking was never really a problem, other than we think the CPU sampling in the performance tools was beating against our interrupt. So they sample about every ten milliseconds. We had an interrupt firing every millisecond, every so often they coincided. If we were in an interrupt handler and you were trying to sample the CPU, it looked like the CPU was pinned.

So if you looked at performance monitor, there'd be big spikes where the CPU appeared to be spiked at a hundred percent. We don't think that was ever actually a problem, because users weren't using data, so your audio wasn't glitching. But what we do recognize is that the CPU overhead previously on firewire was way higher than we'd like it to be. And we're measuring it now at about a third of what it used to be, so this is really great, particularly if you want to start using multiple devices.

The other thing we've done is we now support blocking mode transmission. And this is important for certain firewire chips. We now have great compatibility with all of the vendors of firewire silicon out of there. We've been working with them all, they're all great guys. Doesn't matter which you choose, all of the silicon vendors out there have good solutions, and they work really hard to make this a good business for you guys.

The big thing is latency improvements. The sub nine milliseconds is on bridge co, we've seen silicon from other vendors where the latency's a lot lower than that because the chip housed audio more efficiently. You should go review who you're working with, and see who has the right solution for you.

The thing about rewriting the streaming engine is we got better timing, and with better timing we could start to have smaller safety offsets. And this also allows us to put data in exactly the right place, which helps us both in terms of startup latency, and also in terms of making sure that devices are sample locked. And this is very important again, if you want to do multiple devices.

Obviously the rewrite as I've said, gave us a whole bunch of performance improvements. And the big thing here is if you profile it, you can see that less CPU is being used. And you know, we heard you when you came to this you know, those, I see some of you have there, the people out there who are working on firewire devices. We do listen to you, and we've been working really hard to make sure that we fix this. The extended stream format support's actually very, very useful. It allows devices to basically tell the driver accurately what their IO and streaming capabilities are.

And the big thing about this is previously if you wanted to stream (inaudible) data and MBLA data, MBLA is firewire speak for linear PCM data, you had to burn an iso (inaudible) stream for each, and we're limited in terms of the number of iso (inaudible) streams that we have available. So obviously, you know, having to blow two is not desirable. So this is big news if you want to mix this bit of data with PCM data.

The MIDI support, again, this was an area that both developers and customers talked to us about. So we've definitely improved the latency and the jitter, and we've been using tools that the (inaudible) team provided us with to make sure that we've got the best possible latency and the lowest possible jitter with MIDI on firewire devices. And we're actually pretty pleased with this, and we think you will be too when you start playing with this card.

The big thing, I think, are the changes in the user client. We did a lot of work in the user client for various reasons. But there's two APIs that are of particular interest, FWA set plug in path, and FWA open service. If you don't know about the user client, it's basically a back door into the firewire driver, and why would this be interesting?

Well you may want to produce a device that uses Apple's driver for streaming data to and from your device. But you want to actually control the behavior of that device outside of what's specified in music and audio subunit. So for example, you may have a signal processing device that you know, allows, affects processing on audio streams. You would be able to send the parameters for your DSP routines down as AVC down to specific commands, and in order to do that you would need to open a connection to the user client.

So set plug in path is the first thing I want to talk about. Basically this allows you to load a help plug in. And reasons that you might want to load a help plug in are that you're using somebody's silicon in the firmware, and they come up with a bunch of (inaudible) sources that are more or less meaningless to your user, the computer generated (inaudible) sources. You may want to give them human readable clock source names, so you can basically in your help plug in interface, intercept messages and massage them to provide a good user interface to (inaudible).

So you can correctly name your clock sources. Another thing you might want to do is intercept volume and mute commands, so that if you have a custom control panel, you can update volume sliders and mute controls in your control panel to reflect the actual settings on the device. So this is a really useful user client API if you're developing any kind of control panel for your application, sorry for your hardware.

Open with service, is a new API that allows you to open a connection to the driver the correct way. Previous ways that you may have been doing it probably won't work in the future, so we recommend that if you are opening a connection to the driver, you do it this way. Once you've built your connection as I said, you can send the specific ABC commands to your device, and change parameters on the device.

And when you're using open with service, there is a bit of a gotcha. You need to make sure that the service T that you pass in is the Apple remote audio device from the registry. You can traverse the registry in the way described here to get at this.

Okay, so that's about new features in Leopard. For those of you who are developing ABC firewire devices, if you only take one thing away from this talk, I want you to listen to the next part, okay? We've provided a facility for a vendor specific override kext. And what this does is it provide information to the driver that allows you to tune the driver's behavior to the performance of your device. We strongly recommend that if you're shipping ABC firewire devices today, sorry ABC firewire audio devices today, you look at implementing an override kext.

We're undecided as to whether we want to bundle all of these up and ship them in the OS. It seems like it's a good idea, but we don't want to be in the position where we're writing these for your devices. You guys understand how your devices work, so you know, it would be good for you to consider implementing one of these.

The types of things that you can provide, kind of latency in terms of safety offsets for input and output for each sample rate on your device. You can also do some performance tuning for G fours, because we understand that on less capable processor you may need to fuss with your safety offsets in order to make your device work well.

So what I want to talk through very briefly is how do you implement one of these. And it's pretty straight forward. We've provided some sample code, the hot code on the sample codes where there's a URL to get it right here. I encourage you to take a look at this and try building one.

So we're gonna make one for you know, we're gonna assume that your company name is My Great Company, and the vendor ID is given there, and you have a firewire device which we've called My Great Firewire device. We had other names in here, but we didn't want people to sue us.

So first step is take a look at the header file. And you're gonna want to make sure that you update this for your company. A note is that this definition is also used in the P list file, so you're gonna need to update to there. Second step in the source code you're gonna want to change the class definition.

You also want to publish the devices that are actually supported by this override kext. In the P list file you're gonna want to update the bundle executable and bundle identifier to your vendor specific names. And for each supported device you want to add device matching and latency information in the IO kit personality section.

Also you're gonna want to specify your input and output latencies and samples for your device. You can also add G four specific settings, and you may want to do this, you know, you'll probably want to test on a variety of computers. And I, in thinking about this, and this didn't really occur to me, but we probably want to explicitly say what the quote default things are, because obviously things are gonna get faster in the future, and there may actually be a time where, when we want to determine whether we'll be running on a G five or an Intel core duo, or whatever things are coming in the future.

Now you know, the important thing is you know, we recognize that you know, some of our developers are shipping multiple different firewire devices. You can do a single kext, you can call out your product specifically in the IO kit personality section so that you know, per company you only need one of these override kext, and it can be updated on your website as you ship new devices.

You also obviously need to set up your bundle name in the P list strings file. Make sure you mess with your target settings in your project, and then just clean old build and it'll just work. If you have any issues with this, or you have questions or comments, and we're very interested in comments, the people who are responsible for writing this are Hawk and Matt, come up and talk to us at the end. Or you can reach us via email, and we've got some contacts at the end of the session. But like I say, if this is the only thing you take away from this session, and you're a firewire audio developer, I'll be a happy bunny.

Last thing I want to talk about is networking ABC audio devices together. So we recognize that the audio interface market is kind of a weird and wacky place. And really there's a ton of boxes on a rope (inaudible) expression on both USB and firewire. And this works well.

I mean you know, the vast majority of people just want to get audio into their computer, and get audio out of their computer once they've done something with it, and listen to it on monitor speakers. But we know that in a studio setting this isn't an ideal situation.

And particularly having the computer as the center of a star hub for audio devices is undesirable, because we know that you know, if you're coming in fire high speed serial port, and you've got to do a kernel to use a space transition, do some processing and use a space, and go back down through the kernel to your device, you're gonna have latency. It's just the way it is. We know the you know, devices when they're talking to each other have much lower latency. So what we want to do is figure out well how can we do a studio?

And what we want to talk about is a scenario for example where you may have a source of audio, you come into a A two D converter, that converts to digital, sticks it on the firewire bus, sends it to another device for processing, and then sends the processed audio to the computer. So we have a real peer to peer network, and the network could be a lot more complex than this, and all devices could talk to other devices. And we want to talk about how we can get there.

So I want to bring up Ashley, who's been doing some work with the ABC video services library the firewire team have produced. And he's gonna talk about some sample code that we want to release through the firewire SDK so that you guys can play with this and give us your comments. So welcome Ashley.

Thanks Nick. As those of you who are familiar with the ABC specifications are aware, ABC is built around a peer to peer network. The way the current firewire driver is implemented, it wants to be the center of the universe. So when a device is discovered, it detects it, captures it, starts talking to it, and basically means nobody else can use it.

So if we stop the driver from doing its matching, we can let other apps in to talk to the devices. And one of these is networking. Why are we doing this in firewire? Well we're doing this on firewire because USB is host dependent. Everything has to go through the host processor. So everything's there, it's nowhere to go peer to peer.

Firewire however, is host diagnostic, so we can have all the multiple devices on the bus, and let them be networked together, and the audio does not have to go through the Mac on each hop. So what I'll like to show you now is a technology demo of the work we've been doing on the networking.

So the software we've written is in two parts. There's a, let me just maximize this, daemon, which does all the device matching, handles all the communications with the devices, and is the central thing of the app. So we have three Apple (inaudible) sitting here, and the daemon has gone out, looked up a registry, found all the devices, and created a connection, grabbed the device descriptors, and worked out what plug formats are supported, how many streams, how many, what are all the sample rates, and all the details of the device.

So we can now, through the power of Leopard, switch over to our GUI app. So what we have here is some graphical representations of each device. So we have three ensembles here which are Fred, Barney, and Wilma. So through our inspector you can see here that you can give your device a new nickname, you can allow the device to come up in (inaudible) as though we hadn't stopped the device matching in the current driver, and you can get the GUIds up.

If the vendor has written a plug in for the device that we support, a plug in architecture, you can have your own user interface in here, and allow you to do your vendor specific user interface and commands. So that's great. Now we need to get some audio. So what we can do is drag out an audio stream, might help if I start the audio.

( background music ) And now we've connected the audio stripe from Fred to Wilma, doesn't go anywhere near the Mac, it's straight on the bus only. What we can also do is stream to multiple devices. So we've just overlayed a second connection onto this existing stream, and allowed the second device access to the stream.

Now if one of these devices was a DSP box, so in this example we'll use Barney as a DSP box, we can actually stream that audio through the DSP box, and it, we now have a low latency pipeline, DSP pipeline. We don't have any latency at each hop through the Mac.

Now there are some limitations to this. Current devices only support talking in their current native stream format. So if you have devices which do not support this stream format, of the same stream format, they won't talk to each other. So we have come around, up with a little user space solution where we can create from Fred to Wilma a little streaming connection which comes up back into the Mac, and goes through, and produces a plug format change. We can also use it to change the stream mapping. So let me just, so we have audio going into these devices on channels four and five.

We can drop that off and the audio's gone away, because we're no longer mapping those channels. And if we had audio we could map them around to other channels. So for those of you that can see the meters on the front, you'll notice that it's now going into channels one and two rather than, or zero one rather than four and five. So this is a little demo of a small set of the features that we currently support. And I just need to, get out of there. Okay, can I have the slides back please?

So what did we just see? Well we just saw a streaming of audio directly between the devices, without it going anywhere near the Mac, except in the remapper case. So we can create a low latency DSP pipeline with this. There is no latency introduced at each hop by the audio coming into and going back out of the Mac. And we can flexibly route audio, so you can go from one device to multiple devices, one device to one device. And you can have any number of streams established on the bus.

So in the DSP pipeline case what we saw was audio going into the input device, onto the bus, into the DSP processing box, being processed, and being put back out onto the bus, and then out the output device, and into the speakers. So I showed you two of the parts of this, there is a command line client as well.

So the daemon sits on top of IO kit of course, to do device matching, the ABC video services which provides all the ABC device descriptor parsing, all the communications with the device, and the Apple firewire audio driver which provides the core audio integration. And of course (inaudible) so that we can get lovely Cocoa distributed objects.

The GUI client sits on top of Cocoa so we can get the distributed objects, and the Cocoa code base, and all the graphics in that demo are done with core animation. So all the fancy animations come for free. And the command line client of course sits on top of the foundation.

So how did it work for the device discovery? Well IO kit uses space matching, looks for ABC unit, subunits, and once we find one, we get a notification, and the code base reads the device descriptors. Once it's read the device descriptors, worked out what the device supports and created its little internal representation, it notifies any attached clients and daemons that there's this new device available, and it pops up the nice pretty picture.

When a device is removed, the, also receives a message from IO kit, and the device, the daemon automatically tears down all the connections established for that device, so you don't get any screeching or other missing audio features. And once it's done that, it notifies the clients and daemons that it's gone.

So the (inaudible) streams that are used for carrying the audio over the bus have to be allocated, there's only a certain amount of bandwidth. We're using only new Leopard isochronous resource manager API to do that allocation, and it's the daemon that is responsible for allocating and assigning all those streams to the devices.

So what does it work with? Well it only works with ABC standards compliant devices. If it's not a standards compliant device, we can't discover it and talk to it. And at this stage it requires that the input and output plugs have the same stream format. If it's different we can't talk to it.

While the spec actually supports the devices listening in on certain channels, no device manufacturer has currently actually implemented that part of the spec. A future version of the ABC specification that we're currently working on clarifies this point more, and will hopefully allow us to do this in the future. In the meantime we have the interim solution of the user space remapper, which we'll be releasing the code to.

So we'd like to get some of this stuff in an upcoming firewire SDK, so the source code for the demo app will be included so you can play with it. A new little tool, a format switcher which allows us to automatically stop and start the device matching in the driver, and allows us to run other user space applications. After running this tool you must restart the computer so that IO kit updates its kext cache.

In ABC video services we're planning to get a couple of additions. So ABC info is a new command line tool that reads device descriptors. This is the same code used in the demo. It creates a P list representation of all the data its read, and it allows you to verify your device descriptors. So if it crashes or produces an empty dictionary, it's encountered an error in the descriptor, and obviously you've got something to fix.

There's a new universal transmitter, which is used by the (inaudible) mapper. It allows you to transmit any data on an isochronous stream. It uses callbacks to populate its outgoing packets. So anybody can really use it. And we include the sourcecode for the IRE mapper which uses the existing universal receiver, and the universal transmitter to do user space audio streaming.

So what we would like from you after all this. Well we'd like you to use standards compliant firmware. If it's standards compliant, we can talk to it Implement the ABC music plugs in the specification. That will allow us to stream different stream formats from any device to your device, and specify which channels we'd like you to use on what plugs and external outputs. And experiment with this demo code when it comes out.

Use it to network your devices, tell us about our bugs, play with it, expand it, tell us what you want. Write plug ins for your devices so that we can get you know, more support for your vendor specific commands. It also functions as a great test point while you're building your vendor specific commands, because it will not kernel panic your system when something goes wrong, it will just crash the user space.

So I'd like to hand it back to Nick now for a summary.

- Cool.

Thanks (inaudible). So I'd really like to recognize the work of the firewire team, and particularly Andy Annowitz (assumed spelling) for the ABC video services library, and also the work that Ashley's been doing over the last five or six months on this app. Because we think this is a really interesting direction, and we hope if you're doing firewire devices you will too. Okay, so let's cover what we've covered.

Okay, so we ran through USB audio class two point oh topics. And if you're developing a USB two point oh audio device, please stay for the Q and A, or come up at the end and introduce yourself. We want to meet you, we want to help you in terms of getting your script to write, and making sure that our driver works well with your device. We talked about (inaudible) features, new features in Leopard. Rewritten streaming engines are big news. The override kext for tuning the parameters of your devices and networking devices together, which we see as an important technology demo.

What should you do? Well, the biggest thing is work with Apple to make sure that you have correct device descriptors. We've worked with almost every developer of standards based devices before they ship their device to ensure that we can accurately parse their device descriptors, both on USB and firewire.

We want to continue doing that, we're not looking out for free hardware, we're happy to buy hardware from you. But what we do want to make sure is that your device is well supported on our platform. For firewire devices, please if you take one thing, write an override kext, very important. That allows us to get the best possible performance through core audio to your device.

Take a look at the networking thing. We're really interested in comments from developers. We know that some of the IO box developers are really interested in developing firewire signal processing products that complement their IO products really well. You know, we'd be happy to help you do that. What I'd love to see is a whole ecosystem of interconnectable firewire devices from multiple vendors.

There's been attempts at networking, both on firewire and on other transports in the past. They've always degenerated because one company wants to control it, and they want to control information, or they want to license it, or they want to get a revenue from it. We don't care, we'll give you the code. It's free, go implement it.

The other thing is if you have a user client, look at the (inaudible) as a client routine. Make sure that you're setting up connections properly, and make sure if appropriate, if you have UI apps that you provide a help log in to give the best possible UI to your users. And then you know, again, if you're working on a device, let us know. Like I say, you know, our team's happy to buy hardware from you, we're not looking for freebies, we just want to get the best possible experience for your users.

Where should you go for more information? Well there's two points of contact that are huge, and I strongly encourage you if you're not already doing this, to build a relationship with the two people on this slide. Craig Keathley's a great guy. I don't know if he's here, but I know he's at the conference. Craig is the IO technologies evangelist, he looks after USB and firewire devices.

Definitely make sure Craig is aware of your plans. He can hook you up with the right people at Apple, because the right people's not always us. You might not be doing a class compliant device, but you should be talking to the USB and the firewire team. The other person you should be talking to, particularly if you have application software running on Mac OS ten is Craig Linson.

Craig is, two Craigs. So Keathley and Linson. Craig is the music and audio technology evangelist, I strongly encourage you to build a relationship with Craig. And you know, if you're working on software products, give him copies of the products. If you're a hardware developer let Craig know about your product range, send him URLs to your website, just keep Craig aware of your product releases. If you keep these two guys up to date, they can help connect you with other people in Apple. And then again, you know, the (inaudible) for the override kext is in the attendee part of the WWDC portion of Apple's website.