Hi, everyone. I'm Donna Tom, and I'm a TextKit engineer. And today, my colleagues AppKit engineers Jeff Nadeau and Taylor Kelly will be joining me and we're all really excited to talk to you about advanced Touch Bar concepts. Now, hopefully most of you got a chance to watch or attend the Fundamental session earlier today. If you're following along at home, go watch that now. If you're here, please stay.

It's a really great talk, I do recommend you go and watch it later if you've got the time. In it you've learned about how to create your own NSTouchBars and how to use their input capabilities to enhance the user experience in your app. And you can easily adopt Touch Bar by using just what you learned in that fundamental session.

But you're probably here because you want something more to give your app that extra edge. Maybe you'd like to customize the standard bars offered by Touch Controls like the Mail app's email Autocomplete item. Or maybe you'd like to add a Scrubber to your app so the user can quickly scroll through and select content like the Calendar date selector.

Or maybe, you'd even like to add your own completely custom control like the system color picker. Now, in this session, we are going to show you ways to build all of these things. We'll dive deep into techniques and best practices for customizing the standard components as well as building your own custom controls giving you the power and flexibility to build rich experiences for your users.

So, first up, is to customizing TextBars. We're going to start by going over the standard items, then we'll talk about how to disable the standard items, and how to add your own custom items. And finally we'll learn about the Candidate List item and we'll take a look at how to build an email Autocomplete item similar to what you'd see in the Mail app.

So, let's get started with standard items. Now, we built a lot of great functionality into the standard TextBars. If you're using a TextView or a TextField, guess what? These components support NSTouchBar right out of the box. Now, when a TextView or a TextField becomes the first responder on your app, it'll automatically get a bar populated with items that are driven by the properties and content of the control. So, by default, here's what you get.

QuickType suggestions, which can be expanded and collapsed using that little angle bracket on the left. Top bar shows what it looks like when it's collapsed and the bottom bar shows what it looks like when it's expanded. You'll also get a character picker, for all those great emojis. And if rich text is enabled, you'll get a color picker, formatting controls, and alignment and list controls. And you get all of this for free when you use NSTextView or NSTextField. You don't have to write a single line of code.

But, how does all of that actually work? Well, NSTextView and NSTouchBar are very tightly integrated in order to be provide all of that great functionality. Since the items in the bar may need to change depending on the contents of the entered text, NSTextView actually makes itself the delegate of its own NSTouchBar instance.

And if you're using an NSTextField, remember that an instance of NSTextField serves as the field editor for the currently active TextField. And so when you place items in the TextField's bar, that's not the same instance of NSTouchBar that contains the standard items. The field editor's bar will be the one that contains the standard items.

And while we're on the subject of the field editor, remember that the field editor is shared among all text fields in a window, so if you make changes to that field editor's bar, those changes are going to affect all the text fields in the window. And this is an important concept to keep in mind when working with text fields.

So, now that we understand a little bit more about how these components work together, let's talk about how to disable the standard items. It's actually pretty simple. All you have to do is set the corresponding property on your textView or your textField. And when you set these properties, the textView will automatically get notified that its bar items need updating, so there's nothing else you have to do.

QuickType and the character picker are each controlled by their own property. But the rest of the standard items are text styling controls and these will be present if rich text is enabled on your text field, and if it's not, then you won't get those items. Now, before you go and disable QuickType on all your text controls, please consider your users.

I know some of you might not like to use the QuickType item, but I'd like to emphasize that your users might not share your preferences. In fact, your users can disable QuickType suggestions for themselves either through the System Preferences pane as shown here, or by customizing their Touch Bar. So, please, consider leaving QuickType enabled and allowing your users to make this decision for themselves. So, now that we know how to disable the standard items, let's talk about how to add your own custom items.

So, here are a couple examples of items that have been added alongside the standard items in a TextBar. The bar on the top might look familiar if you've seen the Fundamentals talk. It has the Send Mail button placed all the way on the left. The bottom bar has a Select All button placed just to the right of the character picker. So, how do I go about adding buttons like these to my TextBar? Well, you might be tempted to put something like this in ViewDidLoad.

Grab your textField's Touch bar, append your new button to the defaultItemIdentifiers and that's it, right? No. Please don't do this. You're going to have a bad time guys. And why? It's because the text view needs to be able to reset its bar items based on the current configuration.

And so here's what's going to happen. You're going to just have a bar that looks something like this after you add your new button, which is just as you would have expect. But, if you the user disables rich text while your app is running, that text view will get notified and it's going to need to reset its bar items. And it's going to do this by removing all the existing items and then just recreating the ones it needs based on the current configuration.

But remember, that the TextView is the delegate of its own NSTouchBar instance. So, it's going to recreate these items by calling its own implementation of makeItemForIdentifier. And so it will recreate the QuickType item, it'll recreate the character picker, but it's not going to know how to recreate your custom item. So, it just won't get recreated. Now to avoid this you're going to need an approach that works with the TextView and there are a couple of different ways to do that.

The first approach is to use a higher-level responder to provide your custom item. And this is a good approach to use when it makes sense for your item to appear in more than one context. And our Send Mail bar from earlier is a good example of it. So, here, we have a Send Mail button and ideally we'd like this button to be present in the bar, when the Compose Mail window is open.

It doesn't matter which TextView or which TextField within that window has a first responder. We want this button to be present for any of them. And so, this is a good situation in which to use a higher-level responder like a ViewController to provide your custom item. And here's what that might look like. Here we have our ViewController. We're going to override makeTouchBar.

We'll create our custom Touch Bar item and set its view to the Send button. And then, we'll create our Touch Bar, we'll configure it with our custom item, and then we'll return it. Now, note the use of the otherItemsProxy here. And this is going to nest the bar items from the field editor so that they appear alongside our custom item.

And that's a basic example of how to use a higher-level responder to provide a custom item along the standard items. So, now onto the second approach, subclassing NSTextView. Now this is a good approach to use when you have an item where you only want it to appear in the bar when your TextView is the first responder.

And to see what I mean, let's take a look at the second example from earlier. Here we have a Select All button that selects all of the text in the TextView. Now in this case, we only want our button to appear when our TextView is first responder because it wouldn't really make a whole lot of sense for a Select All button to be there if nobody's editing any text. And so this situation is a good one to subclass NSTextView and put your button in that same NSTouchBar instance as the standard items.

And so there's two methods you're going to want to override in your TextView subclass. One is updateTouchBarItemIdentifiers and you want to override this method because this is called each time the state of the TextView's bar needs changed. And this method is where you'll actually add the text bar to display your custom item, to the defaultItemIdentifiers array. And the second method to override is the delegate method makeItemForIdentifier. And you'll want to override this method because this is what the TextView uses to actually create its bar items. And so this method is where you'll create and you're configure your custom item.

Now, with both of these approaches you're going to want to make sure that you call the superclass implementation before you do any of the work associated with your custom item. And by using this approach you'll ensure that your custom item will be recreated along with the standard items whenever the TextView needs to reset its bar.

And that wraps up the two approaches for adding custom items. Let's shift gears and learn about the Candidate List item. The Candidate List item is a pretty nifty control for giving your users a list of contextual suggestions to choose from. And we'll refer to these suggestions as candidates.

And this item is also tightly integrated with NSTextView. So, every TextView has a reference to the Candidate List in the bar, accessible via the candidateListTouchBarItem property. Now once again, remember the Candidate List you see with a TextField is actually in the bar associated with the window's field editor.

Candidate data generally comes from these three sources. In the QuickType case, the data for the Candidates is supplied by NSSpellChecker. But the control can also be configured to show Candidates from other sources, like the system input method, which allows you to type Chinese, Japanese, Korean, or Indic characters using a Latin Keyboard.

But what most of you are probably interested in is the ability to provide your own completely custom Candidates. And there are two different approaches your can take to do this. The first approach is to use a delegate method. The benefit of using this approach is that it's the simplest one since the TextField does most of the work.

So, to use this approach, you'll implement a delegate method candidatesForSelectedRange in your TextField or your TextView. And you'll get the original text control, the field editor if applicable, and the range of the text, which you can then use to determine your Candidates. So, you'll simply return an array containing your custom Candidates, and they'll be displayed in the standard Candidate List. Now since this approach uses the standard list, there's nothing else you have to do. The TextView will do the rest.

The second approach is to use a higher-level responder. And the benefit of this approach is it allows you to use model objects rather than strings for your Candidates. So, this is similar to the higher-level responder approach that we looked at a little bit earlier for adding custom items to your bar. Here, you'll use a view controller or some other object to create your own instance of NSCandidateListTouchBarItem and provide that instead of using the standard one.

Now, this approach is a little bit more complex than the delegation one, but again it's really useful if you'd like to use model objects for your candidates instead of strings. And we'll go over this approach in greater detail a little bit later. But for now, let's take a look at implementing a simple version of the email Autocomplete item that we've been teasing you with since the beginning of this talk. We're going to use the delegation approach for this.

But we have a slight problem here. Now, since our text field is intended for email addresses, it really wouldn't make a whole lot of sense for it to have emojis in it, or rich text. But if we turn those properties off on the bar, look at what happens when we collapse our Candidate List.

You've kind of got this weird empty bar just sitting there, and that doesn't really look good, so we don't want that to happen. And we're going to prevent that from happening by turning off the collapsing functionality using the allowsCollapsing property. And when we set this property that little angle bracket on the left goes away and you can no longer collapse the list.

But now, we have another problem. We set that allowsCollapsing property on the field editor Candidate List item and the field editor is shared among all the text fields in the window. And so, if we set the allowsCollapsing property on this field editor's Candidate List it's going to affect all of these text fields, which we might not want.

And so we're going to solve this problem by providing our own field editor for our email TextField and then that way, when we modify its properties, it won't affect the shared editor. And so, to create the custom field editor, we'll subclass NSTextFieldCell. In our subclass, we're going to override fieldEditor for controlView.

And then we'll create a fieldEditor and return it. And that's all there is to it. We're not adding any custom behavior. We're just providing our own instance of the stock NSTextView to use as the field editor whenever someone wants to edit the contents of our Text Field. And don't forget to set your TextField to use your new subclass, either through Interface Builder or Programatically. So, now, we can implement our textField delegate method. Here, we'll turn off the collapsing behavior of the field editor and return to our array of custom Candidates. And that's a simple implementation of email Autocomplete with the Candidate List item.

Now earlier we touched on a second more complex approach for returning custom Candidates by providing your own Candidate List item from a higher-level responder. Now if you need finer grain control over the candidate list item, this is the way to go. And if you want to use this approach, here's what you'll have to do. First, you want to disable QuickType for your text field or view. Since you're providing your own Candidate List, there's no need for the standard one.

Second, you want to implement the NSTouchBar delegate method makeItemForIdentifier. And this method is where you will create your custom Candidate List item. And if you're using model objects as your candidates, this is also where you're going to map your model objects into string representations that can be displayed in the Candidate List item. And so to do this, you're going to use this block property attributedStringForCandidate.

Third, if you're using the model object candidates, you need, you're going to need to implement the delegate method endSelectingCandidateAt index and this is where you're going to provide a mapping in the other direction from user's candidate selection to an instance of your model object. And finally, you want to update the candidates that are displayed when someone types into your text field or view. And so you can do that by overriding controlTextDidChange and then calling setCandidates for selected range and string on your custom Candidate List item.

And that wraps up customizing Text Bars. So, to recap, we covered two approaches for adding custom items in text bars and two approaches for providing custom candidates in the candidate bar. I encourage you all to try out customizing your text bars using these approaches that we recommended today. Now, next I'd like to invite Jeff Nadeau up to talk about NSScrubber, a control especially created for NS Touch Bar. Jeff?

All right, thank you. So, as Donna just said, NSScrubber is our very first Cocoa control that we've designed especially for the Touch Bar environment. And, the central theme of this control is taking your app's content and bringing it forward conveniently right at your fingertips. Now just some examples, we've been showing you this nice Calendar timeline as an example of NSScrubber, but in fact we use it in several places across the operating system.

And two more of them are Safari's tab picker, and also Keynote's slide navigator, which appears while you're giving a presentation in Keynote. And just putting these side by side, you can tell that this control is very versatile. It's capable of expressing a wide variety of visual and interactive designs.

But before we dive into any of the API, we should probably answer the question, what is NSScrubber? It's a collection-like control that is all about arranging a list of your app's content and then providing touch gesture-based highlighting and selection of that content using both tap and pan gestures. And if you used NSCollectionView before, this API is going to feel very familiar. However, it has been streamlined and adapted for the unique Touch Bar environment.

Like NSCollectionView, NSScrubber has a compositional interface, which means that it delegates a lot of its responsibilities to other objects. And the best way to see these is to actually pull the control apart and take a look at each piece in turn. First, we have the control view itself, NSScrubber, which manages all of the user interaction with the control. It also provides a couple of nice cosmetic properties like backgrounds and other such things.

Next, we also have the selection decoration. These are the views, which appear to indicate that a specific piece of content is selected. We have a dedicated layout object, which abstractly defines how the contents of the controller are arranged. And then, of course, we've got your content, which is provided using a data source protocol.

Now, we're going to look at each of these pieces in turn; however, we should first look at what we can configure on the control view itself. And the first and most important thing is the mode of interaction, which is controlled appropriately enough with the mode property. NSScrubber has two primary modes of interaction. The first, called "fixed", offers a touch-based selection that directly selects and highlights the item underneath your touch.

The second, "free", is kind of like a free wheel, which you can freely scroll the way that you would any other touch driven scroll view. Once you've chosen between these two modes, there's also the continuous property. This is a familiar name from several other Cocoa controls and we typically use it to mean that a control continuously in response to user interaction is updating its value or sending its action.

Now, when continuous is false, in an NSScrubber you can see that tapping on an item and panning across highlights an item but the selection stays put until we finish our gesture and we commit a new selection. But if we turn it to true, we can see the effect in something like Safari. Where as we pan across tabs, they are immediately selected as your touch passes over each item.

This also goes for free style scrubbers, for example, the Keynote slide navigator we just looked at. We can scroll through here, look at what, you know, navigate our content and then we can tap items to select them. But if we set continuous to true, just the act of scrolling is going to be fluidly and continuously updating the selection within the content.

And so this gives us a very stark distinction between continuous being false, where selection is a very deliberate thing, which might be appropriate if it's kind of a heavyweight action, and continuous being true where navigating and selecting viewer content is very fast and fluid, and may be ideal for lightweight interactions.

Once we've decided what we want or how we want to select our items, we need to decide what we want that selection to look like. And that's controlled with the selectionBackgroundStyle, and OverlayStyle properties. We provide two built-in styles. We have the outlineOverlay style, which is that nice bold white outline that you've seen in several of our examples.

And then we also have a roundedBackground style, which looks great behind text items. Now if you're particularly observant, you may have noticed that the NSScrubber selection style declaration is not enclosed enumeration. It's actually a class. And so you can subclass this and define your own selection styles if you'd like.

One other fun cosmetic option for selections is this floatSelectionViews property. When it's false, changing selection looks pretty much like you'd expect. The old selection disappears and then the new one appears immediately. But if we set this to true, we actually get a nice, smoothly floating and gliding selection, which is kind of a very cool effect.

There's one other trick that you may have noticed in several of our examples. Some of these controls, when you scroll, flick, or pan through them, always glide smoothly so that some item always lands neatly in one place, in this case the center. And that's controlled with the itemAlignment property.

Now by default this is none, which means that we won't do any kind of adjustment on your scroll, but if you set it to leading, trailing, or center, after a scroll event, flicking, panning, momentum, anything like that, will always adjust it so that some item appears neatly aligned with either a leading, trailing, or center of the control.

Finally, NSScrubber as a couple of nice cosmetic options for the area behind your content, including a background color, or if you want to draw anything you like, you can place it in view behind your content. And we actually provide a named color on NSColor called the scrubberTexturedBackground color, which works great in the backgroundColor property to provide this cool textured vertically hashed appearance which you may have noticed in apps like Calendar.

Now that seems like a lot of properties, but if we pull it all together, we can fully configure an NSScrubber in only a couple lines of code. So, we create our control, and then we assign it a layout, a delegate, and a dataSource. We won't dwell on this, but I promise we'll look at each of them in turn in just a moment.

We decide how we want to interact with the control. In this case we want one that scrolls freely, but has a continuous selection as we scroll. And then we also want the itemAlignment to be center so that after we scroll, some item always lands neatly in the middle.

Then we'll choose what we want our selection to look like, in this case we're going to apply both of the built-in effects. And we'll turn on that nice floating selection so that as the selection changes, it floats neatly between the views. And then finally we'll apply that cool textured background color. And with just these couple of lines of code, we've produced the blueprint for exactly Calendar's scrubber that we've been looking at this entire time.

Now that we've configured the control the way that we'd like, we need to layout the items. And that's done with the NSScrubberLayout class. This defines the arrangement of all of the content within the control and it does this abstractly using these NSScrubberLayoutAttributes objects, which are essentially value types that bundle up in itemIndex, and where it should be laid out within the coordinate space of its content.

We provide a couple of built in layouts that you can just pull off the shelf and use if you'd like to, you know, just really quickly get started and that includes a flow layout, which is comparable to NSCollectionView flow layout. But if you want to define your own, it's really not very hard. There's only three key methods that you really need to implement. The first, scrubberContentSize defines the entire size of everything in the control, and this defines how far we can scroll within all of the content.

There is layoutAttributesForItems in rectangle, which provides a set of the attributes for every item that falls within some specific rectangle. And that might be the currently visible rectangle or it might be one that the user is about to scroll to and you want to get ready and lay everything out. And then finally, layoutAttributesForItem at index, so we can determine the layout for some specific item if we need to.

The other half of the layout life cycle is invalidation. InvalidateLayout signals to the control that your layout has changed in some way and it needs to refresh new fresh layout information from the NSScrubber layout. Now, if your layout depends on certain information like the selection highlight or visible rectangle like maybe you want to make the selection be twice as large as every other item, you can optionally request that your automatically invalidated when those things change. And this prevents you from having to track that information yourself and manually call invalidate layout.

Finally, after your layouts been invalidated, before we do any further layout passes, we'll call the prepare method on your layout object. The base implementation doesn't do anything but this is a great opportunity for your subclass to run some calculations, do some measurements, and prepare some caches so that subsequent outcalls are really fast.

And that's all you need to do to put together a layout. Now that we've configured our control and we want, we know how we want to arrange all of our content, we need to get our content into the control in the first place. Content in NSScrubber is represented using simple views, subclasses of the NSScrubberItemView class, and that's comparable to NSTableCellView.

Like NSTableCell, or like NSTable and NSCollectionView, NSScrubber provides a reuse queue so that you can efficiently recycle views as they cycle out of the control, rather than allocating a brand new one every time it's requested. The dataSource protocol for NSScrubber is really super simple. There's only two methods that you have to implement. The numberOfItems in the entire control, and then you just need to be able to prepare a view that represents a specific item.

To aid you in this task, we have two built-in NSScrubberItemView subclasses. We have a TextItem and all you need to do is provide a string that you'd like to present and we'll present it in a way that is consistent with many similar controls across the operating system. And then we also have an image view class, which provides aspect fill presentation of your image and then allows you to specify an alignment beyond that.

Now, if neither of these are exactly what you're looking for, you can alway subclass NSScrubberItemView and just do whatever drawing you'd like. When you do this, you get access to a couple of nice properties. You get isSelected and isHighlighted, which you can use to alter your drawing based on that state. And your drawing will be automatically invalidated when these change so you don't need to observe them yourself. And you can also override the applyLayoutAttributes method, which allows you to inspect and interpret the attributes, which were produced by the layout object.

And that's how you provide content. Finally, we assume that the user is going to use this, this control and so we want to respond to that. And that's done using a delegate protocol. You can be informed when the selection changes, the highlight changes, and if the range of the visible items in the control changes due to either scrolling or layout.

You can also be informed when interaction begins, ends, or is cancelled. And this might be useful for example, to create an undo grouping around a single interaction with the control. So, as the user is rapidly scrubbing through, you can bundle all of those changes into a single undo grouping.

And that's all you need to do to put together your own NSScrubber. I hope you've, you've come up with some great ideas for content in your application you can surface with this control. But, we've -- between the Fundamentals talk and this one, we spent a lot of time explaining how to use AppKit's built-in controls and classes to really build up your Touch Bar experience. But if you want to, to really unleash your creativity and build something truly custom, you're going to want to stay put for the next section where Taylor is going to walk you through doing -- doing completely custom controls for Touch Bar. Thanks.

Here they go. Hello. So, when it comes to building custom controls for the Touch Bar there are four areas that you want to keep in mind. The first is handling the touch events that are coming in. Second is styling your view so it looks and feels at home in the Touch Bar.

Third is making sure it's the right size and position amongst all the other controls. And finally it's tying everything together by applying a little bit of animation. Let's get started with event handling. Now the obvious thing here is that we have direct touches on the Mac. And I really want to qualify the difference between direct and indirect touches.

Those of you who are might be iList developers are already familiar with this distinction where you have direct touches, where you're directly manipulating content on the iPhone or iPad versus the indirect touches on the Siri Remote's touch surface. Where you're remotely interacting content that's actually on the TV.

On the new MacBook Pros, we have both of these in a single device. We've already had indirect touches on the gesture-rich trackpads, where you can get access to individual touches on that trackpad. Now, with the Touch Bar, we support having direct touches where you can directly manipulate content on the screen.

So, the existing NSTouch class has now been extended to add a TouchType to describe whether it is this direct or indirect touch. For direct touches, you can now get the location of that touch provided some view. And providing that view is extremely important because it provides the coordinate space for what that point will be relative to. As an example, when the user taps down on the slider, we want to get the location of that touch, relative to the slider as a whole, and we can take that offset and simply divide it by the overall width to get our new value. It's pretty straightforward.

Now, one very important difference with NSTouch and UITouch is that while UITouch is this long lived object whose location will update over time, NSTouch is essentially a value-type snapshot of that touch on -- at a certain point in time in that gesture. And you can use the identify property to tie a sequence of touches together when forming a gesture. So, as events are coming in through the Touch Bar, each of these events will contain a collection of one or more touches. And each of these touches are unique instances, so again the way you tie them together is by looking at that identity property.

So, even when the user touches down with multiple fingers, we can distinguish which is the one we're tracking by comparing its identity to the one that we're tracking. We can make this a little bit more concrete with some code. So, we want to keep a variable to hold onto that opaque identity object.

When we first get some touches on our view, we go through those, pick up the first one, and save its identity as the one we're tracking. Later, when additional touches come in, we go through the ones that have changed, and compare each of their identity to the one we're tracking to find our new touch. We can now use that new location to update any events in our view.

Now, where do these events and touches come from? Well, one place are these existing NSResponder methods touchesBegan, Moved, Ended, and Cancelled, which will be called back over the lifecycle of our view. We can opt into receiving them by setting the allowedTouchTypes to include direct or indirect touches, but once you link on the Sierra SDK, you'll be automatically opted in to receiving direct touches.

One really important method here is toucheisCancelled. While there's not really a notion of cancelling mouse tracking events, this is extremely important for the Touch Bar because it's so context sensitive. As the user switches active applications, or active windows, the content in the Touch Bar will reflect that and if the user's interacting with any views at that moment, we'll send touchesCancelled so it can clean up any event handling, but separate that from the user actually lifting their finger.

However, for the most part, you actually don't have to worry about this, and you can just use gestureRecognizers. These have gained all these same methods to for subclasses to implement so they can handle touches, as well as the same opt-in method with the only difference here being your explicitly have to opt-in, there's no automatic link check.

And pretty much you can use all the built in GestureRecognizers, which now work in the Touch Bar. For instance, the color picker simply uses a combination of press and pan recognizers to implement all of its event handling. It had to do no extra work. But it's important to remember even for these you still do have to opt them in to receiving direct touches when you do use them in the Touch Bar.

Now, one really interesting context when it comes to event handling, are inside Press-and-Hold popovers. In the Fundamentals section you heard how you can set the Press-and-Hold Touch Bar of your NSTouchBar to have it when the user long presses on that button, for it to immediately present some very simple lists like UI.

You can put your own custom views in here and as the user continues that gesture and pans over your view, you'll get call to touchesBegan, Moved, and Cancelled as they exit the view, and finally once they lift their finger, the Touch Bar will be dismissed. However, because of this very unique event handling model, GestureRecognizers are not supported in this context, so you have to use those NSResponder methods. However, again, outside of Press-and-Hold popovers it's so much simpler just to use GestureRecognizers.

One very interesting aspect of the Touch Bar is around event modality. Since the Touch Bar is this input device akin to a keyboard it should always be responsive. Just because the user's dragging something around on the screen doesn't mean your keyboard stops working. So, even where you previously had a modal event-tracking loop, the Touch Bar will still be responsive and still be able to issue state changes.

So, you might want to go back and actually look at those event-tracking loops to make sure they can handle the type of state changes that could occur. In addition, the Touch Bar itself is capable of multitouch. So, while the user's interacting with one control, they could begin and end interacting with another. And again, you want to make sure you can handle those state changes.

However, once you do that, you're afforded a lot of really interesting interaction opportunities by combining all the input devices that exist on this device. You can create either really advanced work flows for your users or these kind of delightful experiences that the user's might discover by playing around. For instance, in the customization UI, we've added individual pan recognizers to each item, so that users can tab down with multiple fingers and reorder multiple items at once.

Similarly, the color picker has Press-and-Pan recognizers on each individual slider, so you could be editing multiple components at once, or while your'e editing that color, you can save it to your favorites all at the same time. And that's just using Multi-Touch. Once you combine it in the keyboard or trackpad, you can really take things to the next level, such as being able to edit the color of your text as you're entering it, or editing properties of your cursor as you're moving around. So, I really encourage you to think about how you can easily use gestureRecognizers to add this event handling support for your app while also simul-handing these simultaneous state changes.

Next, let's take a look at styling and appearance. And so, obviously the styling inside of the Touch Bar is extremely different from the [inaudible] on the main display. I'm going to cover a number of these areas, but I want to start with something that you might not have even noticed. This is something that you can only see by taking your MacBook Pro into different environments.

So, our standard stock photos look something like this with nice even lighting, if you use it in a bright studio environment, or maybe even take it outside, it might look something closer to this. If you're like me you might primarily use it in the dark, and it's pretty cool how the blacks on that OLED display basically disappear.

And finally, if you get pretty tired as it gets later on in the night, you might turn on the keyboard backlight and get that nice glow. And so, while physically the MacBook Pro looks very different in all these environments, the Touch Bar content is actually adjusting as well. The bezel colors of the different controls is adjusting to that ambient brightness, and even the glyphs within the controls is reacting to that white point change of the keyboard backlight.

And so if you implementing your own custom controls, you want to make sure that you can do the same. And thankfully this is pretty easy by using system colors. These are existing named colors that semantically describe how they should be used, and once you go to draw with them, that's when they'll be dynamically resolved against their current context to determine what color should be drawn with, taking into consideration if it's being used in the Touch Bar versus Aqua, as well as whether the white point or brightness has changed. In addition, new in High Sierra is this expressive palette of colors that you can use to match system UI and have been specifically designed for the out glow appearance and for the Touch Bar.

So, one important thing to keep in mind is how you draw with these colors in order to take advantage of all that context sensitivity. So, if you're using layers to draw your views, you might be tempted to init with frame, ViewDidLoad or immediate response to event handling, immediately update the properties of your layers.

And you don't want to do this because when you get that CG color to set it on the layer, that will immediately resolve those RGB values against whatever the current context happens to be. So, as the whitepoint or brightness changes over time, your layers will stay exactly as they were at this moment.

So, what should we do instead? Well, we can override this method called updateLayer and well, update our layers there. This will get called automatically for you when your view is displayed, such as the first time it's shown on screen or windows white point or brightness changes. And if you have a dynamic color that you actually want to set, it's important that you hold onto that color as an NSColor because again this maintains that semantic nature of the color as well as the context sensitivity. Any time that color changes, you just set needsDisplay to true and we'll recall this method for you. This same exact approach can be used if you're using DrawRectBasedViews, where here the only difference is that you're setting that color against the current context.

So, that's how you can use these custom colors and your own custom drawing. However, a number of our controls support customization of their colors just out of the box. You can now set the bezel color of your NS buttons, the selected segment color or even the fill color of your sliders.

You can use our built in system colors or even your own customer colors if you have some overall app theme. What's really cool here is that the text and image effects within these controls is actually adjusting depending on the lightness of the colors you're setting. You actually don't have to do any work to get that.

One thing you might have seen across the system are these default button blue colors. And it's important to note that this not achieved by setting the systemBlue color as the bezelColor. There's actually a subtly different color that's used for these, and you can get that keyEquivalent to the return key, just like you would have done on the main display.

One thing you might have noticed in these past few slides as well is that the Esc key has been replaced by this Done button. And this is pretty easy to achieve by setting the escapeKeyReplacement ItemIdentifier on your presented NSTouchBar and whatever control that references will be placed in that Esc key region.

However, it's important to consider if an when you should be using this. It should really only be used when the user has entered into some modal context and they can use that control to exit that context. Any actions they take in this context should be undoable, so if I edit my photo here and tap Done, I can still undo that change afterwards.

And finally, you don't want to be adding explicit constraints to this button to try and make it be the standard size of the Esc key. We'll automatically apply metrics to make it be that standard size as well as even adjust the padding around the text to fit the most characters possible. So, for instance, we found that the longest translation of Done is [foreign language] and even that is able to fit in with these adjusted metrics. However, as a last resort we will allow this control to grow in case there are some extenuating circumstances.

Now another really important aspect of the styling in the Touch Bar is when it comes to the font. Now you might have noticed that the font has changed within the Touch Bar on the main display in Aqua, we use a standard font of San Francisco UI size 13. And in Touch Bar we use something actually closer to the Watch, San Francisco Condensed size 15. So, the family and the size have both changed.

And those of you who have followed the font Mac font transition perhaps all the way from Chicago to Charcoal, to Lucida to Helvetica to San Francisco, know that you should not be hardcoding your fonts in your application, and instead you can use systemFont of size 0, which will give you back this dynamic font that when used will be resolved against its current context, just like those system colors, and pick the right family and size.

In addition, there's a weight variance that you can get thinner or bolder fonts, and another really interesting aspect of font you might have noticed are these monospaced digits used in things like the AVKit Player or even the component value of the sliders. And even as the value changes, those numbers aren't jiggling as the metrics adjust.

They're monospaced. And so you can get this by using the monospacedDigitSystemFont of size weight to get this same effect. Now, there are actually a ton more interesting typographic features when it comes to San Francisco, and there are some great talks in the past few years that I'd recommend checking out if you're interested in learning more.

However, the other important type of glyphs in the Touch Bar are, of course, images. And, since the Touch Bar is this P3 Retina display, all you need to provide are 2x assets for the images used there. However, they should be specifically designed for the Touch Bar. You shouldn't just take your toolbar icons and plop them in. In addition, you should be using template rendering to take advantage of those white point and brightness adjustments I mentioned earlier.

To illustrate this, let's say we're making a custom light button. We've specifically designed it nicely for the Touch Bar, and we're using it alongside a bunch of other standard icons. For the most part it looks pretty good, except for later that night, we take it home and we turn on our computer backlight and it looks something like this. All the other icons adjust and our icon basically sticks out like a sore thumb. So, let's not do this.

Instead what we can use, is provide an image that communicates its shape purely via the alpha channel and uses template rendering so that AppKit can take care of all of the different styling that might occur for that image such as the white or even the blue styling when the whitepoint is adjusted.

Now, before you have your designers go off and create a bunch of awesome icon specifically for the Touch Bar to take advantage of template rendering, do be sure to check out all of the standard icons that the system already provides. There are really a lot of them here and the Human Interface Guidelines goes over all of them. And what's really cool is that these are all public constants provided by NSImage so you can very easily drop them into your application with really not a lot of work.

So, those are some aspects of styling an appearance, what's really cool is that for the most part, you can just use system standard techniques and get a lot of this behavior for free. Next, let's talk about layout. In the Fundamentals section, you saw how the standard item layout is to simply stack the items along inside the Touch Bar.

You can use flexible spaces to give your items a little bit of breathing room, or you can use the principal item to center a single or group of items. Now, layout in NSTouchBar works very similarly to NSStackView, where the items themselves provide their sizing information and NSTouchBar positions them based on that size. And when it comes to describing that size, it works exactly like it does outside of the Touch Bar.

For instance, you do override intrinsicContentSize and calculate the size you want your control to be and just return that. In case any state changes, such as the button's title changing, and when you want to note that the intrinsicContentSize has changed, you can call invalidateIntrinsicContentSize and NSTouchBar will relay everything out. Another approach that you might have used in the past was to add an explicit widthConstraint to your control and then update that constant over time. This again works, the same way, and NSTouchBar will again lay everything out.

When you're building custom controls, you also might have flexibly sized controls similar to NSSlider or NSScrubber, and here the only difference is that that simply doesn't have a width constraint, or it has an intrinsic content width of noIntrinsicMetric. NSTouch Bar will take all of the available space within the Touch Bar and divide it up amongst all these flexible controls. In this example, the slider is the only control in there so it takes up the entire application region.

Now, if we want to restrict it to a minimum and maximum size, we can simply add inequality constraints just like we did outside the Touch Bar. NSTouchBar will follow that right along. What's really great about having this flexibility is, as the user customizes in additional items, they're still some flexibility around how big that control can be, and the user can really make the most out of their Touch Bar.

So, that's how you can customize the sizing and layout of individual items. But you also might want to coordinate the layout across groups of items. One new thing in High Sierra is that prefersEqualWidths property on GroupTouchBarItem where you can have all the items within that group prefer to be equal width.

What's cool is that this works with user customization. So, not only can users customize items in and out of this equal sizing group, but the equal sizing only effects the items that are actually present in the Touch Bar, it's not relative to all the possible items that the user could add.

This also works really nicely with localization. In case one of these buttons becomes extremely long when localized, we'll choose to break that equal width rather than overflow the Touch Bar and then cause items to be hidden. However, sometimes that overflow might just happen. And that's where visibility priority comes into play.

Here you see some start window UI that looks pretty nice in English, but when we initially run it in German, it looks something like this. There's no clipping, however the New Collection button has become missing. And the reason is that all the items in the Touch Bar can't fit in the space that we've allotted and so NSTouchBar has overflowed that New Collection button and hidden it.

In this example, we might instead want to hide that Bookmarks button instead of the New Collection button and we can do that by setting its visibilityPriority to be lower than all of the rest. And NSTouchBar will detach the lower ones first. And this time we'll get something like this, which is a bit better but really it's not a great user experience to hide any items. So, in this case, we could probably do a bit better. And using the new Compression behavior in High Sierra, we can.

So, in this example, we can note that the titles of the different buttons actually fully communicate what effect they'll have. And so instead of dropping any items, we can drop those hidden images instead. We can set the prioritize compression options of this group to be hideImages and now when we run in German, it looks like this.

What's really cool is that this happens atomically. We don't end up with half the buttons having images and half of them not, it's an all or nothing thing. And there's really a lot of flexibility here. We can instead think that our icons are descriptive enough and opt to hide the text instead in which case we get this. And you can even go so far as if you have your own custom ways that your custom controls can compress, you can add additional options to these prioritize options to describe that your custom control should compress in those different ways.

So, these are a few ways that you can use different group Touch Bar properties to control the group layout of your items. However, the container views that you're used to using outside the Touch Bar, can also be used within. There was a really great talk earlier today "Choosing the right Cocoa Container View" that discussed some of the differences between these as used outside the Touch Bar, but I'd like to highlight a few of the ways that you can use them within.

You can use NSStackView to have really precise control of the spacing or sizing of your items, such as what Calendar's doing here where they've completely eliminated the spacing between these two buttons. You can also put that StackView into a ScrollView to enable scrollability of these small list of items right inside the Touch Bar.

Jeff talked earlier about how you can use NSScrubber when you have these unbounded list of items where you want to maintain a selection over time as well as benefit from some of the other aspects that Scrubber can provide. Finally, you can go all the way to using NSCollectionView right in the Touch Bar to really have precise control of the layout, or completely custom interactions, such as what the favorites list in the color picker does.

So, that's layout. What's really cool is that for the most part, layout works exactly like it does outside the Touch Bay, so any concepts you're used to are using, you can use within. Now finally, let's talk about how we can use layout to really tie all this together.

Before we do that, I really want to reiterate that the Touch Bar is an input device and not this extra display. It's not the right venue for you to be showing off your different animation skills with animations that might be distracting from what the user's trying to focus on the main display. And any animations that you do add should always be relative to user input. And because that user input can change at any time, your animation should be interruptible and updatable.

So, one obvious thing we might want to animate are the sizes of our items as they're changing. We saw earlier how we can add an explicit width constraint to control that size, and what's really great is that to actually animate that size change, all we need to do is use the animator proxy on that constraint and update its constant instead. And we get all the animation benefits that you'd expect, such as simultaneous animations and even interruptibility just with this one change.

If instead we opted to use intrinsicContentSize to express the size of our items, and then invalidateIntrinsicContentSize to note it changed, what we can do here is we can wrap that in an animation group, set allowsImplicitAnimation to true, and then call layoutIfNeeded. And again we get all the same animation benefits that we saw in the other approach.

Something else we might not want to animate is whether an item is flexible or not. So, we can see here as the item takes up all the available space animatedly, and the same exact technique of using that animation group can be used here with the only difference being the same as before, where when the item's considered flexible, it has a intrinsicContentSize of noIntrinsicMetric for its width. Now we can tie all this together and start to build something similar to the color picker where as you change the active item, it grows to take up all the available space and the other items shrink down.

So, we can start by defining this custom AccordionView that has an active state as well as defines an intrinsicContentSize. When it's inactive, it has this small fixed size. However, once we set it to be active, we want it to have that flexible size that takes up all the available space. And again, we can use noIntrinsicMetric to achieve that. Now we stack a few of these items together in our Touch Bar, and we add pressRecognizers to each individual one.

When the user taps down on one of our accordion views, we can handle that press and then take the oldActiveAccordionView and set it to be inactive, so to have that small fixed size, get the newActiveAccordionView and activate it. And finally using that allowsImplicitAnimations simply layout that change. And we get exactly what you'd expect. It's pretty straightforward.

So, that's how you can animate different item size, but you also might want to animate the content within the items, such as the content within the different scrubber. Now, if you're using constraints or overriding layout and setting your frames or updating your properties there, what's really great is that there's no extra work needed to actually achieve this.

So, as we resize this custom item animatedly, you'll see that the Scrubber's items flexibly resize down, fluidly resize down and even the images within them nicely resize up. And what's really cool is that there is no extra work needed on my part or by NSScrubber to achieve this effect. Because NSScrubber overrides layout to achieve all its layout and view properties, all of this came completely for free.

The other type of content that you often use in your Touch Bar are buttons. Now as your state changes, you might in the past have updated the properties of these buttons, but you might want to achieve something similar to what FaceTime does where it actually animates those property changes. New in High Sierra is the ability to use the animator proxy on these buttons and update your properties there, and again you'll get these animations completely for free.

If you focus in on that Remind Me Later button, you'll note that as the image position changes, so does the overall layout of the Touch Bar and we can combine all these approaches of setting the imagePosition against the animator and then using allowsImplicitAnimations to apply both of these animations all at the same time.

Now, when it comes to building your own custom controls, you might also have completely custom animations that happen alongside item sizes, or maybe even independently, such as the fluid knob resize happens when you activate these sliders. Now there's a few different approaches you could actually use here, such as using the animator proxy on your custom view to animate built-in view properties or your own custom properties.

You can also take explicit CAAnimations and add them to the content of that view, or if you're able to express this as a state change in your layout override, where you update your view properties there, you can again take advantage of that allowsImplicitAnimation call and actually do no extra work as part of animating that change. There was a talk from a few years ago that actually went into a lot of detail about these three different approaches and their pros and cons "Best Practices for Cocoa Animation." I'd recommend checking that out if you want to know more.

So, we talked about a pretty wide variety of content today. And I hope you can take inspiration from some of these areas and actually go back and create some really rich Touch Bar content for your applications to really create a nice user experience, taking things like custom candidates in the Candidate List item, creating your own unique Scrubber layout, or combining gestureRecognizers in really unique ways to really make your UI pretty interesting.

For more information, you can see this website where we have links to documentation and the Human Interface Guidelines. And most of the related sessions we have are in the past, but there's actually one tomorrow at the same time "Building Visually Rich User Experiences" that will actually go into a few tips and tricks of using Core Animation to achieve some of these really interesting effects that equally apply when used in the Touch Bar. That's it for our talk and I hope you guys have a great rest of your WWDC. Thank you.