Welcome to Accessing the iPod Library. I'm Lucas Newman, I'm a member of the iPhone Software Engineering Team, and today I'm going to talk about how you can provide the functionality of the iPod to your application using the new Media Player APIs in iPhone OS 3.0. And some of you out there might be creating the new application that's tailored explicitly for the new APIs in 3.0.

So I'm going to show you how quickly and easily you can add media integration to your App. Others of you out there maybe you already have an App on the App Store and you want to enhance it to provide a better user experience for your users. So I'm going to show you how you can add these features and also what affect it will have on the rest of your application.

So, let's talk about what we're going to learn in this session. There's three main things: The first thing is I'm going to show you how to use the prebuilt UI pieces provided in the media player framework to allow your users to choose media from directly inside the application. Second, I'm going to show you how you can programmatically search for media in the iPod library using the media query interfaces. And third, I'm going to show you how you can actually start audio playback from directly inside your application.

And you can play music, audio podcasts and audio books through the iPod. In addition to what we're going to learn, is a couple of things I'm not going to cover in this session and clearly don't support in the media player framework so I'm just going to get them out of the way.

So, the first thing is you can't actually modify the iPod library from your application right now. It's currently read-only and the second thing is we're not going to provide access to the raw audio data at this time. Now I know a lot of you want these features and as always we encourage you to provide feedback to us.

You can go to bugreport.apple.com and tell us what's important to you in your applications so we can take that into account as we evolve this API in the future. So, with all that out of the way let's look at an overview of the new media player framework APIs.

There's three main parts here: The first is the iPod Library, and the iPod Library is where all the media is stored on the device. To get media out of the iPod Library, we can use the media picker and the media query interfaces. And then once we add media in our application, you can use the music player to actually start media playback.

So first let's talk about the iPod Library. What's actually in the iPod Library? Well, the most basic item in the iPod Library is what we call a media item. This represents a single song, audio podcast or audio book on the device. In addition to media items, there's also what we call the media item collection; and this is simply a group of related media items. There's also a special kind of item collection and that's called a playlist.

This is just like a playlist that you create in the iTunes application on the desktop. So let's talk about MediaItems first. What's the anatomy of a MediaItem? In addition to just the audio data, there's also some metadata associated with the MediaItem. This includes things like the media type-- maybe there's an artist, a title and an album associated with MediaItem. There could even be some art work. There are also many more metadata properties available. I'm not going to cover them all today but you can check out the documentations in the headers for a comprehensive listing.

So, we have all this metadata associated with the MediaItem. How are we going to get it out? So we've got this MPMediaItem class for you and there's only one method you need to know in order to get metadata out of the MediaItem-- that's the valueForProperty method. So all you have to do is ask the MediaItem's valueForProperty, and we provided a set of predefined string constants for you. And you can use any of these string constants to get the specific metadata property that you're interested in. As you'll notice, all of these constants have a common prefix, so they're really simple to code complete in Xcode.

This is similar to many other Cocoa Touch Frameworks that you've been using. So let's look at a quick example of getting metadata from a MediaItem. Here I want to get the title property of a MediaItem. So all I have to do is ask the MediaItem for its valueForProperty and then specify it as Title property string constant, and what I get back is an NSString representing the title.

Note that you'll always get an object back from the valueForProperty method. So, let me talk about artwork for just a minute. Artwork's a little bit special when it comes to MediaItems. When you ask for the value of the artwork property, what you actually get back is an instance of the MPMediaItemArtwork class. And from this instance, you can actually get a UIImage of the specific size that you want to use.

And the reason we've done is, is because it's actually a little bit more efficient for us to give you an image of the actual size that you want to use instead of using the full size image every time. So what you want to do is make sure to always specify a size of the artwork that you're going to use in your application. So that was MediaItems, the most basic item in the iPod Library. Let's talk about the other two things-- the ItemCollection and the Playlist. To visualize itemCollections, I want you to think about the albums listing in the iPod App.

Each of these albums is itself a MediaItemCollection. The items are all related by the unique artists that they have. The items inside are what we call iTunes ordering. This is sort of a natural order that arises when you're using the iTunes application on the desktop or in the iPod application on the device. Once you have a collection, to access the items all you have to do is ask for its items property, and you get back an array of MPMediaItems in this iTunes ordering. There is also some extra metadata associated with the MediaItem Collection.

You can get information like the number of MediaItems and all the contained types in the collection without actually iterating through all the items in the collection for performance. Now, let's say you want to present information about a collection in your user interface. How are you going to do this? What you want to do is use the representative item for the collection.

And this is just an MPMediaItem that comes out of the collection that you can use for common properties in your user interface. So, for instance if you wanted to get the artist of a collection, that all the items in the collection have the same artist, so you want to access that information.

All you have to do is ask the collection for its representative item, and then once you have the representative item simply ask the valueForProperty and specify the artist's property, and you can use that for instance in your tableview of all the artists. Now, you want to make sure when you're asking the representative item for a property, that it actually makes sense to collection. So if you've got a MediaItem collection where all the items have a unique genre but may have different artists, it doesn't necessarily make sense to ask for the artist property.

So, you just want to always make sure that the property you're asking for is actually applicable to the question. All right so MediaItemCollections-- pretty simple. Let's talk about playlists. As I mentioned it's a special type of ItemCollection. It's just simply a subclass of MediaItemCollection. It also has some extra metadata; this includes things like the name of the playlist and any attributes it has such as if it's a genius playlist or a smart playlist.

OK. So we've talked about the MediaItem, the MediaItemCollection and the Media Playlist. Well, there's something that all these classes have in common-- and that's they all confirm to the NSCoding Protocol. So, if in your application you have a MediaItem or an ItemCollection and you want to save this to your application preferences so you can restore it at a later time, all you have to do is use an archiver such as the NSKeyedArchiver that impliments the NSCoding protocol, and in this case I'm just going to ask for the archivedDataWithRootObject and them all I have to do is provide my MediaItem. And I get back an NSData representation which I can then store in my application preferences, and then recreate the MediaItem later using the NSKeyedUnarchiver for instance.

I should also mention that MediaItems and playlists have a unique, persistent identifier associated with them. So you can use media queries to actually locate these items in the iPod library using their PersistentID. I'll cover media queries in just a few moments. So, that was the iPod Library. We know what's inside the iPod Library, it's the MediaItems and the ItemCollections. So how are we going to get this information out of the iPod Library and into our App? Well there's two ways to do this. The first way is the MediaPicker. And the MediaPicker is simply a UIViewController subclass for picking media items.

And it comes in two different forms. The first form is the single item picker and this allows your user to just choose one item and then immediately dismiss. The other is the multiple item picker in case you want your user to choose a playlist for instance. So how are you going to create an MPMediaPickerController? Well it's super simple. It's just like any other UIView Controller. All you have to do is just alloc init a new picker.

We're going to set ourselves as the delegate of the new picker so we can get delegate callbacks when the user takes some action. I'll discuss the delegate callbacks in just a moment here. And then just like any other viewController in the application I'm going to present the picker as a ModalViewController on top of an existing viewController.

So what does this actually look like in an App? So I'm playing a game here-- I've got Texas Hold 'Em and I want to allow my user in my App to choose a song so they can actually listen to something while they're playing. So I bring up the MediaPicker here and once they've hit the cancel button or they've picked some media, we're going to dismiss it and look at one of the delegate callbacks for the MediaPicker. The first, is the MediaPickerdidPickMediaItems.

And what this returns is an ItemCollection that represents all the items that the user picked. Alternately, if the user didn't pick anything or they just immediately hit the cancel button, you'll get the mediaPickerDidCancel delegate callback. Now you should note that you always get an ItemCollection if you're using the single item's title.

So, if you want to get that single item back from the delegate callback, all you have to do is ask that collection for its items array which is only going to have one object in it, and then I'm going to use the NSArray method's last object here to just get that single item out of it. So, with all that, I'm going to have Henry Mason come up. He's going to show you a demonstration of using the MediaPicker in an application.

So I've been working on a little App and I'm calling it Post a Song. And the idea of this application is that it allows you users to pick a song out of their iPod Library, and then send the name, artist and artwork of that song to a friend so they can say hey check out this cool new song I found. So let's take a look at how it works so far. It's got a couple of buttons and when we tap the mail button we use the new iPhone OS 3.0 mail compose API, and we say check out this super cool song, No by No.

Well that's not a very cool song-- I can tell you. So let's take a look at our code and see how we can make this a little cooler. So Post a Song is a pretty simple little application. It's got one root view controller, it's got some instance variables to store an image view and some buttons, and it's got some actions.

I'd like to point out the big feature is this picked item. This is an MPMediaItem that we store as an instance variable in our root view controller, and when the user presses that envelope button, we get the title and artist out of the picked item using valueForProperty, title and then artists.

And then we compose a little string and set that as a message body. And then to create the album art, we get the image out of the album image view and make a JPEG out of it; and the we present a mail composition. But we're never actually setting picked items to be anything so let's change that.

So when the user presses that pick button, let's create a MediaPicker. Now we don't really want to have people picking podcasts or audio books so we use initWithMediaTypeswith media types so we can narrow the scope of items that your users can pick to just music. We're setting ourselves to be the delegate so the root view controller is now the delegate and then we'll present it as a ModalViewController.

So let's see how that works on our device. All right well now we can pick and you'll notice that the interface looks just like iPod so your users won't have to learn a whole new interface. It looks just like the UI they know and love. And you can browse by albums, artists, playlists.

You'll also notice that audio books and audio podcasts aren't listed because we specified a media type when we constructed our Picker Controller. You'll also notice that when we touch things nothing happens; even when we cancel it stays up. And that's because we haven't implemented any of the delegate methods that we need to implement to handle the user picking or canceling to dismiss the controller. So first let's say the user picks something. The MediaPicker will call back to us with MediaPicker didPickMediaItems that will give us a media collection. And we'll set our picked item to be the representative item of that collection.

We could also have used the last object of the items array like Lucas did. And then when the user cancels, we'll just dismiss the ModalViewController. We'll dismiss the ModalViewController in both cases but when we just cancel we don't modify what the picked item is. So let's send that over. All right so now we can pick a song, and when we mail it the artist and title of the song that we've picked is in the mail composition.

But this isn't very nice because we haven't really updated the user interface, there's no feedback that the user actually picked something, although we know internally we know we've set a picked item. So let's change that. So in our setPickedItem method, instead of just updating the iBar we're going to do a little bit of work.

First we're going to set our navigation title of the navigation item of the review controller to be the title of the item that was picked. Secondly, we're going to get the artwork out of the picked item using the artwork property. And as Lucas said, the object that returned is a special object called an MPMediaItemArtwork object. We need to get an image of representation of that using imageWithSize.

And we'll use the size of the bounds of the image view that we set up using Interface Builder. And then we'll set that image to be the image of the album image view. And if we couldn't get an artwork image because it wasn't available at the size or the track just didn't have any album artwork, we'll use that empty album filler art we've been using.

So now we can pick things and you can see now the album artwork is updated and the title of the navigation item is updated, and since we're reading the image out of the image view for the mail composition, you can see now the image that we get and the mail compose view is updated. The problem is when I quit the App and come back it's forgotten what I've picked; and that's because the picked item is reset to nil every time we construct a root view controller. So let's save the item that we picked out to preferences.

So when we send our picked item outside of just updating the UI, let's archive that item using a KeyedArchiver, and then set the archive data as a standard user default key. And then when our view loads, we'll do that backwards. We'll get the data for that key, picked item, if we've got it we will unarchive it using an NSKeyedUnArchiver and then we'll set that to be our picked item. Now we can pick a song and then we can leave, come back, and it's like we never left.

So I'm going to give it back to Lucas to talk about another way to get media items out of the user's iPod Library.

All right. So we've talked about how you can let your users take media using the MediaPicker. What if you want to get media out of the iPod Library programmatically? The way you can do this is you can use a MediaQuery.

An MPMediaQuery is an object that matches a specific subset of items in the iPod Library. And the way it does this is it uses a set of what we call MediaPredicates that match some specific criteria of items to determine which one to choose. So, how are we going to create an MPMediaQuery in our application? Well, we provided a set of common queries for you already.

These roughly correspond to the tabs in the iPod application; so for instance if you just wanted to get all the songs in the device, you can use the factory method songsQuery on the MPMediaQuery class. If you wanted to create your own query and then refine it with media predicates, what you can do is just create one using the standard alloc init method and then by default this query will match all the items in the library.

I'll show you how we can refine this query with MediaPredicates in just a moment. So now that we've got a query, how are we going to give the items that the query matches? Well, all you have to do is ask the query for its items property. And this gives you back an array of MPMedia items as you might expect. Now let's say you didn't just want the items. Instead you want the collections from the query which is just all the items in the query but grouped by a specific property.

So what I can do is I can set the property that I want to group by. So in this case I want to group all the items by the unique artists that I've found with the query; and then when I ask the query for its collections property, what I get back is an NSArray of MPMediaItemCollections, and all these collections will have a unique artist. So, as I mentioned we can refine a query using the MPMediaPredicate. And the base MPMediaPredicate class is actually abstract. What this means is you're not going to allocate it directly; instead you're going to use one of the subclasses of MPMediaPredicate that we provided for you.

This gives us a little bit of flexibility to add more predicates in the future. Currently we provide one subclass for you-- that's the MPMediaPropertyPredicate. And the MPMediaPropertyPredicate allows you to match a specific property of a MediaItem to a specific item. So an example of this, here I'm going to create an MPMediaPropertyPredicate and I use predicateWithValue, and I've chosen The Beets for the property artist.

Once I've created this predicate, I simply add it to my query, using the addFilterPredicate method. And now all the items that this query matches will have the artist The Beets. So when I ask for that items property, I get back an NSArray of MPMediaItems. And if I were to iterate through them and ask for the artist property for each one it would always be The Beets.

So there are a couple of things you should know when you're using MediaItemsProperties to filter. The first is that not all properties can actually be used to construct predicates. For instance, it doesn't really make sense to filter all your items by the artwork property, but it does make sense to filter by title or artist or album. The way you can find this out is you can either check the headers of the documentation as each string definition metadata properties will be annotated with this information, or you can check at runtime using the class method canFilterbyProperty on the media item.

And all you have to do is specify the property that you wish to filter by and it will return you yes or no depending on whether this can be used to create an MPMediaPropertyPredicate. There is also a couple of caveats associated with using MPMediaPropertyPredicate. The first is that all the predicates are going to be combined currently in the query using the AND operator. What this means is that as you add more predicates your query is going to get more and more specific.

Additionally, there can only be one predicate for any given property in a query. So you want to make sure that you don't have a bunch of different predicates in a single query for the artist property; otherwise your behavior is undefined and you rely this in your application because it might break in the future.

All right. So now you know now to use MPMedia queries, I'm going to bring Henry up again and he's going to enhance the application we're building.

All right so now our users can pick items out of their library using interface they know, but what if we just want to have them pick some item? It would be really cool for you to have this random button do something like randomly pick some song out of their library. Let's implement that now.

If we just want to pick something, we should probably just start simple. So, let's create a songs query and then get the items for that query and then just pick the last object. So this will just take the query that matches every item in the library that is a song, and then we get an array of items for that match thatquery, and then we'll set our picked item to be the last item in that array.

So let's try installing that on the device. All right well we have now picked some item, but if we tap it again it's going to pick the same thing and that's because it isn't really random; we're just getting the items as we said earlier in iPod order. So we're just going to get everything in like the songs list in iPod and then we're picking the last one on that list.

And that's listed alphabetically, so that's not a very good random function. Let's change that a little bit. So the other thing in the last item, let's see if we have some items and if we do let's just pick a random index into that items array and then we'll make that our picked item.

So let's see how well that works. Great. So now every time we press the random button, we're creating a query for all the songs in the database, and then getting all the items out of that query and then randomly indexing into that array. So this will give us a pretty good random sampling of songs in the user's library.

OK. Let's say we want to narrow our scope down a little bit. Like I don't want to match every song, I just want to match all rock songs. So, we can do that by adding a MediaPredicate to our query. So, just to be extra safe, we'll ask MediaItem if it's OK to filter by the property genre, and it if is we'll create a PropertyPredicate with a value of rock for the property genre. So now this will only match items that are songs so they have a music media type, and have a genre of rock. So, let's install that and see if that works.

Great. So now we're matching random rock songs in the database. Cool. All right I'll give it back to Lucas to show how we can play some of this stuff.

So now we know how to get media out of the iPod Library using both the MediaPicker and the MediaQuery interfaces.

So what if we actually want to start music playback from within an application, the way you can do that is you can use the MPMusicPlayerController and it comes in two different flavors. The first is the iPod Music Player. And when you're using the iPod Music Player it's exactly like controlling the iPod application on the user's device.

So you set up a queue of items and you start playing, and even if the user presses the Home button and quits your application, music playback will continue. Additionally, if they go into the iPod application, they'll see the music queue that you've set up that will be already playing. Alternately, you use the Application Music Player.

And this is when you want to provide background music. For instance you have an immersive game and you just want to play music in the context of your application. So this doesn't modify any state of the iPod application, and when the user presses the Home button on the device and quits the application it will automatically fade the music out as your application quits.

So, how are we going to create an MPMusicPlayerController? Well first let's look at how we're going to set it up. There's two ways to set it up-- you can use the setQueueWithQuery and provide a media query directly; or you can use the setQueueWithItemCollection and you can provide the ItemCollection such as the one you got back from MediaPicker. Once you set up a queue for playback, all you have to do is call Play on the music player and playback will start.

You can also call Pause to stop music just like you'd expect. So a quick example of this, in this case I'm going to use the iPod Music Player because I actually want to start music on the user's behalf, and I'm going to set up a queue with the songs query which just means play all the songs on the user's device. And then after that, all I have to do is call Play and I've started playing all the songs in order.

So just three lines of code-- pretty simple. Now what if you want a little bit more control over playback? Well, there are a lot of knobs you can twist in the music player. If you want to seek through the track, you can use the currentPlaybackTime property; you can also seek, fast forward and rewind using the beginSeekingForward and beginSeekingBackwards methods.

Additionally, you can actually move through items in the queue using skipToNextItem; skipToPreviousItem just like tapping the fast forward and rewind buttons in the iPod App. Now let's say the user is already playing music in the iPod Application and they launch your app; and your app wants to find out if something's already playing because you want to take some action based on that.

So what you can do is ask the music player for its PlaybackState property. And this could be playing, it could be stopped, it could be paused and maybe even the user has accepted a phone call while they were listening to music. And so they're in the phone call and they launch you app, so in that case the playback could be interrupted and when they end the phone call, the music playback is going to automatically resume.

Now in addition to just asking the music player for PlaybackStateChanges, you might also want to watch for playbackStateNotifications. Because the user can do things like pull out the headphone jack on the device and it will automatically stop playback. So to do this, you're just going to use the standard NotificationCenter mechanism and foundation, and what you want to do is watch for the MPMusicPlayerControllerPlaybackStateDidChangeNotification.

In this case I've set it up so that every time a playback state changes, I'm just going to hit the playbackStateChanged method called on my object. And then once I've set up this notification, I just ask the MusicPlayer to beginGeneratingPlaybackNotifications so I can actually start getting these delivered. Now just like we can ask if the music player is playing, we can also ask it what's playing. So, I can use the nowPlayingItem property and simply get a MediaItem back for the currently playing track.

And just like I can watch for playbackStateChangeNotifications, I can watch for nowPlayingItemChanges. This happens when the track ends and you go into the next one, or maybe the user is skipping through songs using the iPod HUD from double-tapping the Home button. So, now that we know how to use the music player, we're going to have Henry come up, he's going to show you how we can preview music in the application that we're building.

So let's say I've picked the song, I think I want to e-mail it to my friend but I kind of want to remember what it sounds like. What I want to do is switch back to iPod and select the same song to listen to it. It would be really nice if I could just tap this Play button and actually have it play the song. So, let's implement that now. Now the first thing you have to decide is which music player do we want to use.

Now if I'm in the middle of listening to say, a 500 song playlist and I get half way through and then I open up Post a Song and then Post a Song blows away that playback queue and inserts just the one song that I've been trying to preview. The probability of me uninstalling Post a Song and giving it very, very low ratings on the App Store is pretty high. So let's use the application music player so we don't mess with the user's iPod playback queue.

All right so first of all when we set a PickMediaItem and we change our UI, we're also going to set up the application music players playback queue. So, we're going to get our application music player, we'll set our queue with ItemCollection and there's a little bit of code here but basically we're just making the one item ItemCollection. And that one item is that one item that we've picked-- the picked item.

So now we've got a playback queue set up and then when the user presses the Play/Pause button, we'll once again get our application music player, we'll check its playback state, if it's playing we want it to pause; if it's in one of the other states so Stop, Pause, Interrupted, we want to start playback.

So we'll tell it to play. So, let's see what that sounds like. All right so now I can pick song and I can start playback and I can pause playback. And I can play again and I can pause again. So, that works but you'll notice that while we're playing the button still says Play even though touching it pauses. So we probably want to change that to be a Pause icon.

So looking back at the code, it might be tempting in the toggle Play state method, just change the title of the button immediately after we call Play or Pause. That's not what you want to do though. Play and Pause are asynchronous so you're not actually technically playing or pausing immediately after Play and Pause are sent, and there are things outside of your control that can change the playback state of your application music player. If the user unplugs headphones, if your song ends, there's a whole bunch of things that can cause that to change. So the correct way to solve this problem is to observe playbackStateChangeNotifications.

Let's do that. So in our view loads, we'll once again get our application music player, we'll add ourselves as an observer to the MPMusicPlayerControllerPlaybackStateDidChangeNotification for the application MusicPlayer and then we'll tell the player to beginGeneratingPlaybackNotifications. And when the root view controller goes away just to be safe, we'll do the opposite. We'll remove ourselves as the observer and we'll end playback notifications on the MPMusicPlayer.

When that notification gets posted and we receive it, we'll update the title of the button. So this nicely mirrors what happens to the toggle play state button. When the playback state is playing, we'll set the title to Pause because we call Pause and in any other state, we set the title to Play because that's what happens when the user presses it in toggle play state. So now we can play and it changes to a Pause button. Let's do it again, now it's a Pause and stopping it pauses it.

So, that's how to correctly change or update a UI to reflect the playback state of the MusicPlayerController. All right I'll send it back to Lucas to take us home.

So you know how to start playback from within your application, but what effect is this going to have on the rest of your application? Maybe you're already playing audio in your application.

So let's talk about Audio Mixing. What you want to do when you want to mix audio in your application is you've got to set up an audio session. In this case I'm going to use the AVAudioSession API from the AV Foundation Framework, and I'm going to choose the category ambient which means we're going to mix audio played in our application with audio playing in the background.

So I'll set up this category and I'm going to make it active, and then anytime I just play audio as usual in my application, it will automatically mix with the iPod audio. Also, when you're playing audio in your application, you want to think about audio routing a little bit. As we mentioned before, if the user is listening to music and they go along and unplug their headphones, you don't music to continue playing out of the back speaker of the device. This would be annoying to the user.

So you're going to want to be able to handle audio route changes in your application. Well when you're using MPMusicPlayerController, what does the code look like to actually handle these changes? Well there's no code; it actually just works for you. We've done all the heavy lifting. All you have to do is just let the MPMusicPlayer control all the audio route changes for you.

Now in your own application you may have to actually do a little bit of extra work in order to get this behavior. So we put up a sample on the session webpage. See "AddMusic" sample and it gives you examples of audio mixing and routing. Additionally, you want to know that the music is going to play from the music player regardless of the ringer switch setting. This is just like the iPod application. We always play music regardless of the ringer switch setting.

So take that into account when you're starting music playback. Additionally, if you're playing formats like MP3 or AAC in your application, you're going to fall on the software decoding path if the iPod is already playing audio because the iPod is going to be using the audio decoding hardware. So this can actually cause a performance impact in your application and it will lower the battery life of the device. So you want to make sure to take that into account when you're playing audio.

There is more information examples as I mentioned. We've got the "AddMusic" sample on the session webpage. There's a URL for you right there, and there's also a session on it after the conference when they put up the videos. You can check it out and see Audio Development for iPhone OS that one's on Tuesday at 3:30. They go really in depth into Audio Mixing, Audio Routing and setting up the categories. It's a pretty complex topic; we don't have time to cover it all today but you can go check out that session if you want more information.