Thank you for coming. This is a session on Audio Processing and Sequencing Services. And just before I bring Chris Rogers up, who's going to be doing most of the conversation, I just thought I'd welcome you. My name's Bill Stewart. I'm the Engineering Manager of the Quario team. If you missed the talk yesterday, we talked just very briefly about the audio hardware layer that we have in Mac OS X.

And all of this sort of fits in the general frame of what we call Core Audio. And aside from the audio hardware stuff, yesterday we talked about the default output unit as a way of very easily having your application get sound out the current output devices selected by the control panel.

So what we're going to be doing today is to talk about audio units, and Chris will go through that in a moment. And the default output unit is one of them. So if you were at the talk yesterday and a little bit confused about where all that fits in, you should be quite clear by the end of this talk, we hope. So without any further ado, I'll bring Chris Rogers up from the Quario team. Thank you. Good afternoon. I'm pleased to be here to speak with you a little bit about some of the higher level services available for audio on Mac OS X.

Overview of the talk is, first thing I'm going to talk about is Audio Unit Architecture. Audio Units are components for processing audio or just dealing with audio in some general manner. Then I'm going to talk about the Audio Toolbox, hooking Audio Units together and processing graphs. Lastly, I'll be speaking about some timing and scheduling services that are available using the Music Sequencing Services. Let's take a look at where these higher level services sit in our audio stack. Audio Units are one level above the core Audio How APIs which Jeff spoke about yesterday.

Then above that we have what are known as the AU Graph Services, which are a set of APIs for connecting these Audio Units together. And the Music Sequencing Services we'll see can talk to AU Graphs and Audio Units. and Core MIDI is off to the side there. And later on, Doug will be going over that in quite some detail and actually showing how there are some relationships between the MIDI services and some of the Audio Unit technology that we provide.

So audio units are components, components that are handled by the component manager that QuickTime makes wide use of. Like all components, individual components are identified by a four character type, subtype, and identifier field, and the component manager can query the list of available components of a given type.

Audio units are the ones we're interested in this case, so you can call find next component to go through the list of available components. Then once you've gotten to the one that you're interested in opening, you can call open a component, and to close it, you call close component. The component manager has some pretty good documentation, so you can refer to our developer documentation about the component manager for that.

The Audio Unit component, in general, it generates, receives, processes, translates, or otherwise manipulates streams of audio, as I have in my slide. And they're good high-level building blocks. They can be connected together to create networks of audio processing. And you can also use them singly and just, for instance, instantiate a software synth and wire it up so this output goes directly to the hardware.

We provide a set of Audio Units that ships with OS X and we'll be adding more in the future, in future releases. And developers are free to create their own Audio Units. And we'll be providing an SDK to make that a lot easier than it would be otherwise.

So what kinds of Audio Units are there? There are Audio Units that just provide audio data. So they're sources. They have no inputs to them. An example of that type of Audio Unit would be what's known as a Music Device. It's a software synthesizer. And Apple ships a DLS2 sound font compliant software synth in OS X. Also there are Audio Units which interface with the hardware.

They provide a higher level abstraction of hardware I/O devices. So they talk to the audio HAL. And one of the most common types of Audio Units are in the category of DSP processors. Reverbs, filters, mixers, this type of thing. So these type of Audio Units typically, they take input and process it somehow and output the audio stream. Another kind of Audio Units is the Audio Processing Unit.

This is a kind of Audio Processing Unit that's used to process audio. It's a kind of audio processing unit that's used to process audio. It's a kind of audio processing unit that's used to process audio. It's a kind of audio processing unit that's used to process audio. It's a kind of audio processing unit that's used to process audio. It's a kind of audio processing unit that's used to process audio.

It's a kind of audio processing unit that's used to process audio. And one of the main types of Audio Unit is Format Converter. Converting between 16 bits, floating point, interleaving stereo streams into discrete mono streams, sample rate converters, and there are quite a few other types as well. And then another class is what I call an adapter class of Audio Unit.

And this is a little bit strange because it doesn't actually process the audio in any way using DSP. It also doesn't change the format of the audio stream. But But instead, it somehow does some other kind of impedance matching between the input and output. For instance, it may do some kind of internal buffering so that the input buffer size is different from the output buffer size. Or it may lay on top of a thread scheduling scheme so that everything before that Audio Unit in the chain would get deferred to another thread. And this way you could potentially take advantage of multiprocessing. on multiprocessor machines. It's a very interesting application.

I guess we're going to try running a little demo here. I should probably explain a little bit what I've done. Oh, this is your demo. Okay. So if we can have the machine up. I can't see anything. Oh, neither can you. I can see this. Okay, let me just get this to run.

So, So what I'm actually running here is a Java program. The Java program is quite simple. This is very interesting, trying to do it this way. This is just demonstrating how to use the component description and the component services from Java. It's the same as you would do in C.

As we've been talking about in the sessions, you have both a Java and a C API to this. So I thought it might be interesting just to show you some simple Java code that is quite clear about the concept of what we're doing. We create a component description. This describes to the component manager the type of components we're looking for. The type is a K audio unit type.

And then what I'm going to do here is I've just told it to look for the first component of this type. And I'll use the AU component, which is actually in this package in Java. And this will find the first one. Then I'm going to print out some information about it. And then I'm going to keep finding the next component from the one that I've already found.

And until I don't find any more, and then my program finishes. So this will give you an idea of the audio units that we ship with the current version of this system. So there's delay, some filters, there's a matrix reverb, interleavers, deinterleavers, a mixer unit. There's the DLS music device.

You can see it's type, subtype, manufacturer. The audio device output, the default output unit, which is actually an extension of the basic unit that will talk to any device. The default output device will track a default output that the user might set. And various others. Thanks. I'll go back to the slides.

And of course I can't see my screen. Okay, just before the demo, I went over some of the categories of Audio Units. This is just a little simple diagram showing that an Audio Unit is basically a box that can have an arbitrary number of inputs, arbitrary number of outputs.

I think this diagram is showing that DSPFX would have both inputs and outputs with some kind of processing going on. Software synth music device may only have outputs because it's actually synthesizing sound. It's creating it from scratch. A default output unit, which Doug talked about yesterday, it would just take inputs and it would send that audio to a hardware device.

Um, we'll talk about audio unit properties. Properties are very, very fundamental to the way that audio units work, and a very general and extensible way to pass information to and from audio units. Audio units themselves, they implement a very small set of, um, functions, which you, which can be called to deal with getting audio in and out, and setting parameters, and so on.

But we created this property mechanism to allow audio units to be extensible, so when we come up with some new feature that audio units need to implement, we can always, um, use the, uh, the property APIs and pass in data via void star pointer, and, uh, the length of the data is indicated by a, a byte size parameter.

The, we define several different properties today, and the type of property is identified by an audio unit property ID. It's just a 32-bit integer. Uh, both the MIDI APIs and the AudioHAL APIs also support this concept of properties. It's almost identical type of mechanism in the API between the three.

Secondary to the property identifier, identifying which property, we have addressing within the Audio Unit to specify which part of the Audio Unit is to deal with the property. So there's a scope, which can be global, input, output, or group. Global means that the property applies to the Audio Unit as a whole.

Input scope means the property applies to a particular input. Output is pretty clear. And group is a particular type of scope that is relevant for software synthesizers, music devices. The Audio Unit element is an index specifying which particular input, output, or group we're identifying. So if we want to talk to input number two, then this argument would be two. Or actually, it would probably be two. Depending on how we're numbering, starting at one or zero.

The IDs are defined in AudioUnitProperties.h. It's important to understand what the data formats of the properties are. We're just passing information as a void star pointer and a length parameter. You have to understand the structure of the data. Those data formats are defined also in the header file. We have a couple properties defined right now. Additional properties will be added as needed. Developers are free to implement their own private properties for their own AudioUnits. There's a range of reserved property IDs for that. All AudioUnits should support the general system properties.

So yesterday at Jeff's talk, somebody was interested in knowing about the performance capabilities of our system. And somebody asked about, well, how many audio units can you connect together? And is there going to be a problem? The KAudioUnitPropertyFastDispatch property is a way of getting access to function pointers which call directly into the Audio Unit. This bypasses the ComponentManagerDispatchOverhead that is typically incurred.

Basically, you're just dealing with function call overhead when you're doing your processing. It's no different from any other kind of DSP you might be doing in your own code. So really, the time, the bandwidth limiting factor is the actual DSP that is occurring. So the answer to the question of how many units can you string together is really a function of how much processing are you doing among all these Audio Units. It's not a limitation of the Audio Units themselves. This particular property can be used as an optimization, but you're not required to use it.

Here's the second example of a property. And that's the The K Music Device Property Sound Bank, FSSpec. This is a way of telling the software synthesizer which sample bank it should use. So you identify the sample bank with an FSSpec pointer and you pass that information in by calling AudioUnitSetProperty.

The first argument is the actual component instance. The second argument is the property ID, the Sound Bank FSSpec ID. Then it's global scope because it applies to the synth as a whole. The element doesn't really matter in the case of global scope. Second to last argument is pointer to the file spec and then the size of the data is the size of the FSSpec. So that shows you how you can communicate property information to an AudioUnit with AudioUnitSetProperty. And all properties are dealt with in a similar way. Of course the specifics of what the property information is and what it does will vary.

Parameters are less for configuring attributes of Audio Units and more for changing real-time control points. And we have the Audio Unit Get parameter and Audio Unit Set parameter functions for this. It allows real-time control of processing algorithms. The parameters are identified with Audio Unit Parameter ID and the value is a 32-bit floating point number. And these parameters are very similar to MIDI continuous controllers, but they're much more resolute. They're 32-bit floating point numbers as I pointed out and it's much, much higher resolution than a 7-bit MIDI value.

The Set Parameter function includes a timestamp allowing the Audio Unit to schedule the parameter change for a very specific time within a buffer of audio. So the timestamp is in units of sample frames on the next buffer of audio to be processed. An Audio Unit supplies its list of supported parameters with the kAudioUnitPropertyParameterList.

Individual parameters can be queried with the ParameterInfo property. Information such as the name of the parameter, what units the parameter uses, whether that be Hertz, Cents, Decibels, whatever. Min, Max, and Default value for that parameter. And some flags as well. The information provided by this property could be used by a client to put up some kind of user interface representing this particular parameter.

So if the parameter represents volume in decibels or something like that from maybe, I don't know, minus 150 decibels up to 0 dB or who knows, the control is actually able to display the units and the minimum and maximum value appropriately. Audio Units are in one of several states and it's important to understand the state transitions that Audio Units go through. When an Audio Unit is first opened with Open a Component, it's considered to be in the open state. Once the component is closed, then it doesn't even exist anymore.

So that's kind of like the base level constructor/destructor, if you're thinking in object-oriented terms. But there's an additional level of construction and destruction in Audio Units. That's the initialized and uninitialized state. And we have the Audio Unit initialized and Audio Unit uninitialized functions for that. Once the Audio Unit is initialized, you can actually start it running with Audio Unit start. And that applies specifically to I/O Audio Units such as the Default Output Unit to actually start the hardware through the Audio HAL.

So, let's go into a little more detail about what's the difference between just opening the component in the first place and initializing it. It seems like it's the same thing. Well, when you open the component, you have a component instance and you can make calls to the component, but at this time, the component is not expected to allocate any significant amount of resources, such as memory.

But you are able to make property calls to the Audio Unit to configure it in some particular way. For example, you might want to configure its sample rate or bit depth or maybe you're interested in setting up some other aspect of the Audio Unit before you actually initialize it.

Then once you call Audio Unit Initialize and you've made your property calls to configure it just like you want, then the appropriate allocations can occur optimizing for the particular configuration. So if you're configuring a reverb for a certain number of delays or something like that, it's able to allocate exactly the amount of memory it needs.

This is when audio is actually being processed. It's actually going through from input to output. Audio Unit or the hardware is running, whatever the case may be. The Audio Output Unit implements the Audio Unit Start and Audio Unit Stop functions. If you have a graph of Audio Units connected together, this is where The Audio Actually Starts Being Processed or Stops Being Processed. The Audio Output Unit generally represents the terminal node in a connection. And as Doug talked about a bit yesterday, the default Audio Unit, it starts the hardware device that the Audio HAL is connected to.

Here's another API to Audio Unit Reset. It basically is used when an Audio Unit is initialized and it may even be running. But if you want to clear the state of the Audio Unit, it's typically used with DSP Audio Units. So for delay effects, you would clear the delay line. For reverbs, you would clear out the reverb memory. For filters, you clear the filter state.

And this is important. If the graph is actively processing audio, but you perhaps reposition the playhead to a different part in the audio stream, and you don't want reverb tail from is the Director of the Java Processing Unit. He will be speaking at the end of the session.

Audio Unit I/O Management Audio Units process audio or deal with audio using a pull I/O model. And what I mean by that is that there's a function, a particular function you call to read the audio output stream from the Audio Unit. And when you do that, the Audio Unit has to go and fetch its audio input. So it typically does that by calling the same function on the output of another Audio Unit.

And then that Audio Unit has to get its audio input from somewhere. So if you imagine a chain of these Audio Units connected together one after the other, you would pull on the very end one and in turn it would go down the chain. Each one would fetch its input from the one before and you pull the audio through the chain. It's possible to connect these Audio Units up in more complicated configurations than just simple linear chains, but the idea is still the same.

Chris Rogers Audio Unit I/O Management Audio Units process audio or deal with audio input. Chris Rogers Audio Unit I/O Management Audio Units process audio or deal with audio input Audio Units specify their number of inputs and outputs through properties. The format of the audio data is canonically n-channel interleaved audio. Or it could be sideband data such as spectral analysis data. This is linear PCM 32-bit floating point.

Okay, I've been talking about connecting these Audio Units together and this is just a picture showing that. I'm going to talk about how to get audio into an Audio Unit and then I'm going to talk about how to get audio out of an Audio Unit. There are two ways to provide an Audio Unit with its audio data. The first way is using a property called kAudioUnitPropertyMakeConnection. With this property, you specify a source Audio Unit and its output number. And then that connection is made and the Audio Unit knows that it's to read its audio data for a particular input from another Audio Unit's output.

A second way of providing audio data to an Audio Unit is by registering a Client Callback. This is the standard sort of callback that we get in the Audio Howler, in the Sound Manager, or in many other audio systems, such as ASIO. Using this method, the Client just specifies a callback function, and then every buffer slice cycle, this callback function is called, and the Client is able to provide the audio or a particular input of an audio unit.

In the case where Audio Units have multiple inputs, each input may be configured separately using either of these two methods. So if an Audio Unit has two inputs, the first may be connected to the output of another Audio Unit and the second input may be

So, I've just gone over two ways that you provide audio input data to the Audio Unit. Now, it's important to understand how do you actually get the rendered audio out of the output.

And you have to use the Audio Unit Render/Slice function call for this. It pulls the audio or reads audio from a particular Audio Unit output. And it must be called once per output. So if there are three outputs, then it would be called three times. Each audio output can be an n-channel interleaved stream like I said before. So it's possible that an Audio Unit might just have one output and provide actually eight discrete channels internally in that one output.

I'm going to go over some of the important arguments to the Audio Unit Render Slice function. It's actually very similar to the Audio HAL callback arguments. The Audio Timestamp is exactly the same Audio Timestamp structure as in Audio HAL and it specifies the start time of the buffer that is to be rendered.

It provides information about the current host time of the machine or the host time that corresponds to the start of the buffer along with the sample time that corresponds to. It also provides information about the current host time of the system and how to synchronize other system events such as MIDI events which use this host time to the audio stream which are synchronizing based on sample time.

The Audio Buffer argument is where you can pass a buffer that you expect the Audio Unit to render into. Or, optionally, you can pass a null for this and it will pass back a returned buffer that it caches. The Audio Buffer is also a structure used in the Audio HAL. We share the same data structures between the high level and the low level parts of the audio stack.

So with Audio Unit Render Slice, it's optionally possible to schedule events before a given I/O cycle. So before Audio Unit Render Slice is called, you may want to do some scheduling. Parameter changes with Audio Unit Set Parameter, or in the case of Software Synthesis, you might want to schedule some notes to start or stop playing during that slice of audio. So there's this two-phase schedule render model. In step one, you would schedule events, and in step two, you call Audio Unit Render Slice.

It's an optional callback that a client can install on an output of an Audio Unit, which gets called before rendering occurs and after rendering occurs. So every time that Audio Unit render slice is called, The Audio Unit will call this notification callback, perform its rendering, and then call the callback again to say that it's done rendering.

This allows the client who installs this notification to perform arbitrary tasks. Some examples would be scheduling, which you might do in exactly the same way as... as I described here before every render slice. Another use that you might make of this is to perform CPU load measurement. You can actually measure the time that the Audio Unit takes to perform its rendering.

How do you write one? Audio Units are components, the same kind of components that have been used in QuickTime for a very long time. Anybody who is familiar with writing components in QuickTime should be able to get up and running pretty quickly with Audio Units. For people who haven't written components before, it can be a little bit difficult unless you have some help. We have an SDK that makes this very simple.

All you really have to do is override one or two basic functions. Um... and in fact, at the base level, you really only have to override one, and that's the function which does your actual DSP processing.

This SDK will be placed on our website soon, in the next few days we hope.

So, I'm finished talking about Audio Units at the lower level. I'm going to talk about AUGraph. It stands for Audio Unit Graph. And this is a set of APIs for connecting these components together. represents a high level representation of a set of audio units and the connections between them. It allows you to construct arbitrary signal paths, fairly arbitrary anyway, and represents a modular routing system. And it can deal with large numbers of them.

Why use AUGraph? One of the reasons is that connections and disconnections between the audio units can be performed while audio is actually being pulled through the chain. If you are talking to the audio units at the lower level, you have to take great care at changing connections, making connections and disconnections while the audio is being pulled through. You can get all kinds of bad things happening. There's a lot of synchronization involved. AUGraph takes care of this for you. Another reason you might want to use AUGraph is that it maintains the representation of the graph even when the audio units themselves are not instantiated.

It also takes care of and embodies the state that the audio units are in, as we'll see. AU Node is the primary object that's represented in an AU Graph. An AU node actually represents a particular Audio Unit in the graph, whether that Audio Unit is instantiated yet or not. It's a placeholder representing that Audio Unit.

To create a node, you can call auGraphNewNode. The next argument to this function is Component Description, which specifies the type, subtype, and ID of the component. This information that you would get from Find Next Component to Component Manager Call. It's also possible to pass what we call Class Data to initialize an Audio Unit with. And we'll talk about that in a minute. AU Graph Remove node just takes it out of the graph.

To create connections between these nodes, once you've added a bunch of nodes, you've got Connect Node Input, Disconnect Node Input, Clear Connections that clears all the connections in the graph, and Update for synchronizing real-time changes. You can also walk through the graph and look at all the nodes with Get Node Count, Get Node, Get Node Info.

These are in the AUGraph.h header file in our Audio Toolbox framework. I think that the APIs are fairly self-explanatory if you're looking at header files, so we're not going to go into detail with all the arguments. Just like the Audio Units themselves, the AUGraph mirrors the state of the Audio Units in them. This slide is just pretty much going over what I talked about. Um... The important thing is that the AUGraph state corresponds with the Audio Unit state.

So the state APIs for AUGraph are Graph Open, Close. So AUGraph Open, when you call that, it actually opens all of the audio units represented by the nodes in the graph. And Close closes all the audio units. And correspondingly, Initialize, Uninitialize, it calls the audio unit functions. And Start and Stop, same thing. In addition, there are inspection or introspection APIs. AUGraph is Open, is Initialized, is Running. So you can determine what state the graph is in.

This class data. When an AU node is added to the graph, it's possible to pass arbitrary data in with a void star pointer and a length, similar to the properties. And if you add this information, this optional class info, when the Audio Unit is opened, a property call will be made on the Audio Unit with this data.

and the Audio Unit is able to make use of this data in any way that it wants. There's no data format that's defined currently for it. But it's a way to maintain persistence because when the graph is closed, data can be gotten from the Audio Unit and stored in the node. So it can be recreated in the same configuration that it was.

So, a little bit earlier I tried to explain the notion of this head Audio Unit in a graph. In a very simple case, you would have a chain of Audio Units. Say you might have three. First one connected to the second, connected to the last one, the third Audio Unit, which is maybe an output device. And this last one represents the head of the graph. And we talked about the pole model.

The Head Audio Unit is very important because it controls the hardware. It controls whether processing is being pulled through the graph or not. There are special Audio Unit APIs, Audio Unit Start and Stop, which I talked about before. The AU Graph has corresponding calls, AU Graph Start and Stop.

Okay, done talking about Audio Units and AUGraph. So finally I'm going to talk about some Sequencing Services for performing scheduling. A few slides back I had a diagram up showing the Schedule Render Model where in step one you schedule events to Audio Units and secondly you would render a slice of audio. The Music Sequencing Services actually would be one mechanism where the scheduling could occur. It's a simple API for creating and editing multi-track MIDI sequences. Individual events in the tracks can address Audio Units and Software Synthesizer Music Devices.

It provides a runtime for the playback of these sequences and has a bunch of APIs for doing live edits, cut, copy, paste, merge, this type of thing. So while the sequence is playing, you can be editing. It's important in a sequence or application to be able to do that.

For those developers who already have a sequencing engine in their application, this set of APIs might not be so interesting. But for somebody developing a new application, because the APIs are there for doing the basic edits and for actually providing the runtime system for playing back the sequences. It's only necessary to write a UI.

The Music Player is the highest level object and it provides transport. So you can seek the sequence to a particular time and start the sequence playing and stop the sequence playing with the Music Player. provides the appropriate synchronization between host-based processing and MIDI. Music Sequence is a next level object. A Music Sequence contains multiple playback tracks, which are called Music Tracks. Each Music Track contains a series of time-ordered, time-stamped events.

Each Music Track with these events addresses a particular AU node in the AU Graph that we spoke about before. So for instance, if you have a graph with a software synth, there would be a node representing the software synth, and one of these tracks might be addressing its events to the software synth. There might be another audio unit in the graph, which is a filter of some kind. There might be another track which is addressing parameter changes to the filter to sweep the filter cutoff.

There are also attributes to the track, Mute and Solo, as I said before. You can set up the loop length, number of repetitions of the loop, and also offset the start of the track relative to the other tracks. There are several different types of events. The most basic one is MIDI Note Message. It's MIDI Note Number with velocity, MIDI Channel, and a duration. The Note Off is not a music event as such. It's implied in this MIDI Note Message using the duration.

There's a MIDI Channel Message for all the other channel messages. There's Raw Data for System Exclusive. There's a Tempo Event for doing tempo changes. And there's this Extended Note API, which is very much like the MIDI Note Message, but it provides for more sophisticated control in esoteric applications. MIDI Note Event only has a Note Number and a velocity and a MIDI Channel associated with it.

This Extended Note Event allows for an arbitrary number of floating point arguments to be associated with the MIDI Note. There's a Tempo Event for doing Tempo Changes and there's this Extended Note API, which is very much like the MIDI Note Message, but it provides for more sophisticated control in esoteric applications. instantiation of a note.

Not everybody may be interested in that, but it's there. The Extended Control represents a 32-bit floating point parameter change. And ultimately, it may end up calling Audio Unit Set Parameter at exactly the right moment in the audio processing. For sweeping filter, for changing volumes, for panning, positioning audio, all different kinds of parameter changes. There's also a User Event where arbitrary data can be placed and a Meta Event for MIDI.

So here's one example. There's an API corresponding to each one of these types of events. And all of them are very similar to this one. So I'm just going to go over the one. This is how you add a MIDI note event. The first argument is the music track you're adding the event to.

The second argument, I think they got the arguments lined up a little bit funny. But the second argument, if you can make sense of that, is the timestamp. It's 32 bit. Actually, I think it's a 64 bit. It's a double precision timestamp representing the beat. And then you have a pointer to a structure, the MIDI note message structure, which essentially just has the note number, velocity, MIDI channel, and the duration of the note in it.

So this is a diagram showing how you can have a Music Sequence with multiple tracks, three in this case, where each track is addressing a separate Audio Unit in an AU Graph. This corresponds with the previous diagram, the two-phase schedule render model, if you remember. There are some editing APIs for copy, cut, paste, this type of thing. And I think if you look in the musicplayer.h header file in the Audio Toolbox framework, it will be very clear how to use these.

One of the lowest level objects in Music Sequencing Services is the Music Event Iterator. It allows you to actually walk through the events in a track, see what they are. It could be useful if you're writing a sequencer application, you're writing the UI sitting on top of this engine for just walking through the events, seeing what they are, so you can display them. Also, if you want to create custom editing operations on track, you could use this Music Event Iterator to walk through all of the events in a particular time range.

So there are a couple different APIs for dealing with these iterators. The Music Event Iterator is a first class object, so you create it with new and dispose. And you can seek the iterator to a particular point in the track. It finds the first event at or after the time that you're seeking to. You can move the iterator one event forward in time with next event, one event previous with previous event. And you can actually ask the iterator if it's at the end or beginning of a track with has previous event and has next event.

Then the Music Event Iterator Get Event Info function can be used to find out what type of events you have at that point. So you can find out what the event type is, what time the event occurs at, and you get a void star pointer which you cast to the appropriate event type to get access to the particulars of that event. For instance, if it's a MIDI note event, you can find out what its note number is, and so on. You can delete an event at the point where the iterator is, or you can change the event's time.

Okay, I'm going to play a demo. It's kind of a little bit vague what I'm trying to illustrate here, but I think what I'm trying to show is how all of these services work together. So, in the example, I've created a graph. I have a software synth, a DLS music device that's loaded up with a 12-string guitar sample bank, and it's playing through kind of an arpeggio really quickly. And then I have some additional audio units in the graph.

I think there's a digital delay in there. There's a look-ahead limiter node to keep the volume pretty steady. And there's a reverb at the end. And I also have a bandpass filter in the graph whose center frequency I'm sweeping using the Music Sequencing Services. So I've set up a music sequence, and I have one of the tracks addressing the node in the graph, which is the bandpass filter, and sweeping that around.

And it basically just ends up sounding like some kind of ethereal wash. But you should hear that there are these kind of filter sweeps going in and out, and there's a reverb in there for sure. Let's see if I can wake this machine up and get this going here.

Okay, got my cursor. Unfortunately, it's not nearly as impressive as Jeff Moore's multi-channel demo the other day, but this will give you an idea. Okay, I'm only launching an application actually. It's nothing too visual, it's just to hear. But I'll show you. Double click, and you should hear something.

But it's showing all the parts of the system that I talked about. And I think, Bill? I'd like Bill Stewart to come up and he's going to show us some more demonstrations and talk about the Java interfaces. I thought I'd make mention that all of that sound you heard was just some notes coming from the 12-string guitar. So it's pretty interesting, I think. Now, if I can have the demo machine back. I've got to get my mouse here somewhere.

This is cool, I can just show you what I want to show you. So this is my Java application. and do we have sound coming out from this? I'll very quickly walk through the code with you after this. Basically, I'm using a sound font that we got, the same sound font that Chris is using from Emu, which is a 12-string guitar.

So I'm going to quit this guy and let me just bring up the project. You have to just

So there's a bunch of stuff here just to sort of set the thing running. I'm loading the sound bank here which I've just installed with Mac OS X.

We also wanted to make sure that there were directories that were available as standard places for you to put standard resources that you may want to share between different applications. So in this case I'm putting the sound bank in the /library/audio/sounds/banks and then this is the name of the sound file here.

And there's also a mirror of this directory in the user's directory as well. So sound banks that were stored here would be available to any user that came onto the machine. So you might imagine like a lab where you could have a setup that you want available to every user.

And then individual users can have their own sound bank. So you might imagine like a lab where you could have a setup that you want available to every user. sandbanks in their own user version of this thing. And there are fine folder constants that you can use for look for these in systems that have been localized.

There are other folders for Audio Units if you want to make them available for different types of sounds. There's a whole bunch of MIDI stuff here that I'll go through later on. What I want to show you is basically how the graph is set up.

constructor here is creating the graph and that just calls through to the C API to create the graph here and then this is me creating to synth two nodes one for the DLS music device and I give it a component description and I already know what I'm looking for here I could use the fine component staff to actually hunt around for components I'm just creating two nodes I'm creating a music device and I'm going to use the DLS synth as the that one for that you know we're very hopeful to see other music devices from you guys like you know some acoustic modeling synthesizer devices and all that kind of stuff and if I get the mouse pad here and then the second one is the default output unit I'm just going to create that because I want the sound to actually come out of the computer and then to connect those I just tell the graph to connect the node and the output unit to the DLS synth and then I'm going to create the DLS synth and then I'm going to create the DLS synth and then I'm going to create the DLS synth and then I'm going to create the DLS synth and then I'm going to create the DLS synth and then I'm going to create the DLS synth and then I'm going to create the output from the synth into the input of the output node.

And just to show you that there's some stuff going on with the graph, after I create the graph, I'm going to get the node count here and I get the index node starting from zero to the number of nodes and I print that out. And I can show you what that guy looks like.

So you can see there that it's telling me I've got two nodes and there's a component which is the music node and the default output node. And I've lost my mouse again, there it is. Okay, so after I do the connection of the nodes, I'm going to open the graph.

And this will open the audio units and that will actually, it's really a constructor. Until you do open on the graph, you don't have an audio unit, you just have the node itself. It's an abstract representation of the graph. So when I open the graph, I can ask the graph to give me the audio unit that's associated with a particular node.

And if we have a look back up here, you'd see that I had kept this guy around. I could actually go and look in the graph and find out the nodes here and I could ask it for information. But I've kept this node here, so I know that what I want to do is get the music device, which is a subclass of audio unit for that particular node.

And what we do in the core audio Java APIs is we actually give you the subclasses that correspond to the extensions of the component selectors. So you could have a basic audio unit, which is your parent class, and you've got an extension of that with your output audio unit. Another extension is the music device.

And that, in component terms, just provides additional component selectors, additional functionality. If I've specified a set of components, I can just add them to the component selectors. And if I've specified a sound bank, which I'd be kind of a bit lame if I didn't, then I'm going to set the property of the sound bank on the music synth. And this is a convenience routine in Java. It does the same thing as what Chris showed in the C file. And it just takes the Java file object.

It does all the necessary conversions to the native structures and instantiates that. And you notice that I'm doing this when I've opened the graph but before I initialize it. So this is one of these things where this is a property that we want the synth to have. We want the synthesizer to have when it's initialized. And until the synth is initialized, it's not going to try to use a particular sound bank until you've initialized it.

And then just to get the synth sort of up and running, I'm going to set some default volume and channel stuff. And there's some commented at code there I'm not going to worry about. And because I want this to be responsive, I quit it, didn't I? Because I want this to be responsive, what I'm actually doing at this point is to get the Audio Device Output Unit, which is the default output unit that I'm sending out to. From that guy, I'm going to get the device that it's attached to, and I'm going to set its buffer size.

That's going to basically take the default callback buffer, which is about 11 milliseconds, 512 sample frames, and I'm going to set that to 64 sample frames. The responsiveness of the system is about 1.5 milliseconds. I create a memory object that I can pass that information in, and then I just set the output property straight on the device.

I get the device from the Output Unit, and it's actually a device output unit, and I set the property to 64 frames, which is 64 times 2, because it's a stereo device, times 4, which is the size of a float. Of course, in Java, you don't do size of float, because it's Java, not C, I suppose.

Then I actually get that property as well, and I print it out, and you can see, if I can find my mouse, and you can see there that it's telling me that the buffer size for the IOPROC is 512, which is what we'd expect, 64 by 2 by 4.

So that's all we do, and then after I've set the buffer size up, and I've initialized the synthesizer, I then just start the graph, and that will start the device. It'll start pulling through the chain, and away we go. And then in the next session, I'll go through how I -- I'll set up the MIDI parsing stuff, so that I can actually send MIDI events to the music device. So can we go back to slides? Okay, so just some notes about the Java. There's also a Java doc available on the website. We're also preparing a PDF of just the Core Audio architecture itself and that will be available at the website early next week.

And as we stressed yesterday, we sort of look at Core Audio as basically an architecture that you should understand how it works. And that's really a language neutral feature. The Java and the C APIs represent the same API to the same functionality that's implemented underneath and in a variety of different ways.

So we also have a mailing list. It's at list.apple.com. It's called Core Audio API. And we welcome you to involve yourself in that mailing list. And it's a good way to send information to us and we'll answer questions on that list. And there's a website. I don't know if it's completely up yet.

If it's not up today, it will be over the next few days. developer.apple.com/audio. There are some related sessions for the DVD viewers of this wonderful thing. The next session, which should be MIDI, is at 5:00 today. Audio Services and Mac OS X was yesterday and there's the Sound Networking Games before this one.

If you have any questions related to FireWire and USB, there's a lot of USB audio devices and MIDI devices and stuff that's coming out, you can contact Craig Keithley, [email protected]. If you're interested in seeding, we're obviously constantly working on these technologies. If you're interested in getting access to that before we release it publicly, you can contact us at [email protected] or talk to your Developer Relations Manager and inquire about seeding.