Video hosted by Apple at devstreaming-cdn.apple.com

Configure player

Close

WWDC Index does not host video files

If you have access to video files, you can configure a URL pattern to be used in a video player.

URL pattern

preview

Use any of these variables in your URL pattern, the pattern is stored in your browsers' local storage.

$id
ID of session: meet-with-apple-292
$eventId
ID of event: meet-with-apple
$eventContentId
ID of session without event part: 292
$eventShortId
Shortened ID of event: meet-with-apple
$year
Year of session: 2026
$extension
Extension of original filename: mp4
$filenameAlmostEvery
Filename from "(Almost) Every..." gist: ...

Meet with Apple • Session 292

Touch on Mac Group Lab

2026-11-18 • macOS • 1:00:40

Connect with Apple engineers and designers online to ask questions, get advice, and follow the discussion about Touch on Mac.

Open in Apple Developer site

Downloads from Apple

HD Video (1 GB)
SD Video (189.6 MB)

Transcript

Hello world. Welcome to today’s Touch on Mac Group Lab. My name is Tomas, and I’m a technology evangelist in Cupertino. Bringing touch to the Mac introduces so many exciting possibilities for how people interact with apps. For the next hour, my colleagues and I will answer your questions about direct manipulation, pointer parity, hit targets, and more.

If you’re watching on the Apple Developer website, you can submit your questions and vote for your favorites in the panel on your right. If you are watching on YouTube and would like to ask questions or upvote the ones you want answered most, please follow the link in the description below the video.

Today, we’ll be focusing on the most popular questions and topics that help the broadest audience. If you have code-specific bugs or issues, please bring those to the Developer Forums or file feedback in Feedback Assistant. Now I want to give a big shout-out to our team behind the scenes helping triage all your inbound questions. Please keep them coming.

And before we dive into questions, let’s meet our panel. Why don’t we go down the line with a quick round of introductions? Please tell us your name, your team, and what you worked on. I’ll start with you. Hi, Tomas. Hello, everyone. I’m Abrar, I’m on the WebKit team. I generally worked on bringing up touch support for web content, primarily on Safari, but generally for all WebKit clients. This includes Mail and a bunch of other internal clients, but also the broad internet.

Hi, I’m Raleigh Ledet. I work on the AppKit team, and I brought up touch and anything event routing in AppKit. That’s my area of expertise. And so we got touch working in AppKit and all the common controls and providing API for you. Hi, I’m Cas. I’m on the Human Interface Design team at Apple, and together with the macOS team, we helped bring touch to the Mac in terms of design.

Hi, I’m Joakim. I’m an engineer on the Photos app team, and I worked on adding touch support to the app. Hi, I’m Henry. I work on the Calendar app for the touch support. Thank you all so much for joining me today. While our behind-the-scenes team is organizing the incoming questions from the audience, let’s set the stage.

Cas, to get started, for developers adapting their Mac apps, what’s the core mental model or design principle they should keep in mind when thinking about touch for Mac? Yeah, it’s a good question. Maybe the first thing to keep in mind is touch is an input we’ve added next to keyboard and mouse. The majority of Macs still use keyboard and mouse. And so it shouldn’t really drastically change your app too much.

It’s really where touch can really feel like it speeds you up, because it just is a bit quicker than trying to do it with the mouse or trackpad. Those are some really good interactions to add, like things like pinch and zoom, for example, is a good example there, where it’s just a bit quicker to just put your finger down on where you want to zoom into and just pinch, versus having to move your mouse to that location and then doing the pinch on the trackpad.

That’s a really good example of just how fast it can feel. Other unique things are the fact that you can just use multiple fingers to interact with multiple UI elements, like two sliders or, you know, two switches, for example. All of that can happen simultaneously. Those are really the unique parts. And that’s really mostly it.

I think another small thing is if you do feel like some controls are too small or some areas are too small, there are some ways to optimize it based on the input you start with. We do that for menus and a bunch of controls. So if there’s any area there that you feel like we can do something touch-optimized there, when you invoke it with touch, that’s another area to improve. But overall, it’s really just about getting over some speed bumps you have with trackpad or mouse and otherwise trying to stick with the design you have as much as possible. Thank you. We are getting some questions. Let’s start with the first one. This one is from Felix.

“Are there events sent through Sidecar from an iPad identical to what the app will see when a user uses the touchscreen on a Mac?” So the other question is kind of, “Will I have to get a touchscreen Mac for full testing, or will an iPad with Sidecar do?” An iPad with Sidecar will work great for now. With the latest SDKs and betas, they are exactly the same that you see. You get all the gesture support, and the touches look the same with Sidecar and with a touchscreen MacBook.

And so yeah, it’s a great way. Like I mentioned that in my video, the best way to start, while you’re waiting for a new hardware to come in, is to just grab the iPad and Sidecar that you already have. That’s also the way we use internally to first do the adoption. At first, Sidecar iPad, when we don’t have the hardware before. Thank you. “My app uses a third-party UI framework. Is there a best way to add touch support?”

For when you’re using a third-party framework, your best bet is to be sure to give them the feedback that you want them to update their framework to add touch support, and then you can adopt their new version of their framework. Until then, depending on what the framework does, you’re sort of at their mercy.

And there is a key that developers need to add to their Info.plist once they’re done adding touch support. For some apps, we detect if you’re using a third-party UI framework, and if you’re using a non-Apple UI framework, we’ll automatically turn on some additional mouse emulation techniques. So as you become touch native in your apps or your frameworks become touch native, then you can add this key to opt out of that, if we’ve automatically opted you in. And so there’s the NSIsTouchNative. It’s a key. And you add that and add true, and see if that changes the behavior in your app. And you can start getting the touch-native events and taking care of it there.

And to clarify, I’ve been working some with some developers. Not every developer needs to add this key. Only those who are opting out of the translation for third party. Right. Okay. By default, we’re native. If you’re using standard AppKit controls, we already are touch native across the board on all of that stuff.

And so it’s if you’re using a third party, just a third-party plug-in that provides you some new kind of control, then you’ll have to get that plug-in updated, and you’re likely going to be, in that case, just a standard Mac app, which won’t have the extra support for emulation for third-party frameworks. But if you’re using a completely different UI framework, then we’ll automatically kick you in, and you’ll need to opt out once you have touch support. Thank you for clarifying. I have a question from Emmanuel. “Since it’s a Mac app, we rely heavily on hover a lot.

Going forward, should we continue using hover in our application, or should we start moving away from them and consider an alternative?” I think this one might be for you. Yeah. I don’t think you should try to avoid hover. It’s a very unique mouse interaction that I think is really helpful to make your app look and feel kind of clean and not overwhelming in terms of how many controls we’re showing.

In terms of trying to make hover work for touch, there are some techniques. So trying to hold on the same area to which the hover applies can bring up a context menu. And in there, you could bring up the same actions that would appear on hover. And then interacting with that area or with the content to which the hover action applies, that could also just show those actions.

So that you have a brief moment where you can use them with touch, or see them with touch and interact with them. We most of the time try to keep them around until you start interacting with a different area with either touch or mouse or a different app, or if you just start using the mouse.

In general, we try to hide those again because they will appear on hover with the mouse. So in general, definitely don’t take them away over time. I think they’re very important to keep. And we have some good ways to make them appear with touch, if you follow the guidelines.

Yeah. I was wondering what you guys did in that world. Yeah. For example, in the Photos app, if you go into the edit UI and we go into the Adjust tab, and there’s this little Levels break, which exposes additional controls when you hover over it. So what we did is we added a touch-only gesture recognizer where you can just touch on the background of the control, and it makes those additional UI elements visible for a short time, and then you can interact with them using touch, and then they fade away after a timeout.

In Calendar app, we added a grabber. When you drag an event to resize, when you drag down the edge of the event to resize the event with mouse, they used to have the cursor. When the cursor hovers on the edge of the event, the cursor changed to let you know that it’s resizable.

It’s draggable, but with the finger, you don’t have a cursor, you don’t have a hover state. So that we added a small visual grabber on both top and bottom edge of the event. So it was much larger hit area to let you know that it’s there for you to grab and to resolve the hover state issue.

Thank you. Here’s an interesting question for Cas. “If my app depends heavily on lists, should I adapt all list items height to be at least 36 points, or is there any other way to adapt based on the user behavior?” If you’re using our frameworks’ lists, you should normally be fine and keep the presentation as is.

I think this is the same principle as before, that your app shouldn’t necessarily just make everything bigger just because the Mac now supports touch. I think we’re trying to keep the same density and the same amount of information on screen as normal. That’s very important for Mac apps, because that’s what they’re known for.

I think on the framework side, we’ve done a lot of work to make these lists work really well with touch. And so hopefully, if you’re just using our native frameworks, try it out and see how it feels. And most likely, it’ll be fine. If you’re using custom frameworks, then I wouldn’t necessarily make them bigger than what you had before. I would just make sure that interactions like dragging and selecting work really well. And I think the frameworks are a good example of trying to see how the functionality works there.

Yeah, we did a lot of work in AppKit optimize all of our controls, including table view and collection view, and make sure that they work great with touch, even when you have small row heights and stuff. Yeah. The fun one was checkboxes along the left edge of a table view, and being able to tap those and just go up and check the ones you want. - Yeah. - The Calendar app, we do have a lot of small controls that we didn’t want to enlarge them just for touch, but we enlarge the hit area.

But one thing to keep in mind is that when you have those small controls next to each other, you want to make sure those enlarged hit areas don’t overlap with each other. One debugging technique I used was to actually visualize those enlarged hit areas with semi-transparent color, so that you can literally see whether they are colliding.

It’s a good strategy. I have a question that sounds like a follow-up for video games. This is from Ronald. “My games use tap and drag, like steering a ship and swapping tiles. Do touch Macs need anything different from iPad, such as bigger tap areas or hover effects?” So that’s a good question.

If you made it work for iPad, it’s probably a good idea to just use the same tap targets when you’re listening for touch events, at least if the spacing and the design of your UI or your game has sufficient space for those. And yeah, absolutely. I think that’s probably a benefit you get from actually having done the work already on iPad.

So I think for sure, yeah, a lot of iPad apps that we have here at Apple that run their iPad version on the Mac, like News or Stocks, for example, or Voice Memos, they do the same thing. They get so much benefit from being touch-optimized already that out of the box, it just works really well.

Yeah. If it’s a Mac Catalyst app, we send the touches over to the UIKit side. And so as it works on the iPad, it should work the same way on the Mac. And that’s pretty great. If you have an iPad app or game, and then you’re using AppKit instead of UIKit on the Mac version of it, then you’ll want to use similar gesture recognizers.

We have comparable gesture recognizers on both platforms. They’re just slightly named differently. We use NS instead of UI, and it’s an NSClickGestureRecognizer, because it started with the mouse-based gesture recognizer, instead of a UITapGestureRecognizer, but they’re equivalent, and a click gesture recognizer can handle both the mouse and a touch on macOS.

This is also a question we’ve been getting from developers I’ve been working with who are migrating their apps or updating their apps. “Can they take their UI gestures and just replace UI with NS, and it will mostly do the work? Or are there any differences to be mindful of?”

The differences are in the mouse interaction stuff. They evolved at slightly different times, and… So we’re trying to keep them as similar as we can and maintain binary compatibility. So when you look at your buttonMask and how they operate, but it’s the same exact delegate API call on how to deal with them, modulo some minor differences, because it’s a shouldReceiveEvent instead of shouldReceiveTouch withEvent. And there’s now a new shouldReceiveTouch delegate API, so you can distinguish between the two.

So there’s some minor differences like that, but it should mostly just carry over. And if you’re looking at the grander concepts of “I need a tap,” I can just use NSClick for that, and I can adjust if you need it to be just touch-specific. You can make the buttonMask 0, and it’ll be touch-specific. And that’s all you really have to do from that point on.

But it’ll be a tap, - just like in your… - Fantastic. Anecdotal experience on the WebKit side, we have some UI event primitives that we build on gesture recognizers, and we’ve already done a bunch of this work for iOS with UIGestureRecognizer. And when it came time to adopt NSGestureRecognizer, we kind of just made that file not just build on iOS, but for Cocoa, and just adopt the new names. And mostly it worked.

You have some interfaces that are named a little differently, but it mostly did just work. And that was a pleasant surprise for us, because it saves some… less typing, it saves some work for us. - Oh, good. - Yeah, it just worked. Speaking of expanded touch controls, “Is there a best way to implement custom expanding touch controls in SwiftUI?”

SwiftUI… We don’t have a SwiftUI expert here. Okay. In AppKit, there are a number of different techniques you can use for that, between putting up your own child window, if you needed it to escape your window bounds, or putting up your own view on top of the control that you’re trying to expand. There’s just… I could spend all day with all the different techniques.

We have time for all of that, but no. So it’s good to know that you can also use a custom window to expand. - Yeah. - Okay. I mean, that’s… From a design point of view, I think what’s important there is that there’s always an element that stays persistent throughout the interaction.

So for example, when we worked on the dial for rotation, the circle itself expands and becomes the glass, and then the content inside that container, which is first… It’s like the little arrow on the small one when you’re using the mouse. But then when it expands, you get this bigger, like, knob you can drag around and a track where it’s in, and that all just fades in and fades out.

I think that the circle and the knob are the two things that transition, but what makes those expanded controls feel so nice is that it really feels like the control itself is expanding and growing, versus just having something appear out of nothing because you invoked it with touch, for example, that will make it feel a little bit less intuitive. And so trying to see if there’s some common view, or, like, some view you can keep in throughout the whole transition that just moves them and changes, I think is a key part to making that feel really nice.

I just want to say from a technical perspective, the initial touch gesture recognizer that you’re using to see that you’ve started to interact with the control that you want to expand, from that moment of initial touch, and while that touch is continuing to be tracked, that gesture recognizer is what’s going to continue to get the touch events and going to continue to send action messages. And so you’ll need to watch that to update your expanded control’s values and moving it around.

And then once they lift off, if you stay in sticky mode for them to continue adjusting, and then at that point, a new touch coming down will come down on the view for the expanded control, and your gesture recognizer for that view will take over. It’s good to know.

I have a question from Eduardo. “I was wondering if you support custom gestures. For example, in our app, if you want to do a multi-object select, you long press with one finger on the screen and then select the elements with another finger. Will this work on Mac with touch?”

I’m a little confused. Are they asking if you can have multiple gesture recognizers working, or one custom complex gesture recognizer? It sounds like a custom complex gesture recognizer. Okay, so you can do custom gesture recognizers, and you can get the… That’s how you would get touch events on macOS, and you can use that however you want. Be sure, though, if you do that, that you implement all of the touches began, changed, ended, and especially canceled, and override them all so that you don’t get your gesture recognizer in a bad state if you miss a canceled event.

But given that you have the events, and you can decide what is a gesture and what’s not, though if you have multiple different objects in a canvas, I would typically have gesture recognizers on those objects, or multiple gesture recognizers, depending on your layout and how it goes. And I would set the flag that says that I can handle multiple gestures at the same time on that view. Then they can just come in and be operating all at the same time independently, because if you’re moving different objects, that can be independent things happening.

And before writing a custom gesture recognizer, I would always start with the built-in AppKit ones and use the delegate methods to make sure that you can use two simultaneously. And if you exhaust all the possibilities of the built-in gesture recognizer, then maybe it’s a good time to think of implementing your own. Thank you for the reminder. That’s a very good recommendation. I have a question from Bartek that says, during his testing, “How can we tell whether the interaction came from a mouse or a touch input?”

That is a great question. Hit testing on macOS is just hitTest, so you don’t have any context, which is interesting, because we do hit testing for all sorts of things, even outside of events. And while I have the opportunity, I’m going to say here that NSApplication.currentEvent is dangerous, especially in this new world where everything is coming through gesture recognizers, and when gesture recognizer action fires, the event is probably long gone.

So if you ask for the current event, it’s not going to be what you expect. So we’ve seen a number of internal app developers get caught by that. So hitTest can be used for accessibility. It can be used for during events. A lot of different places. You don’t know the context.

That said, if you’re using a gesture recognizer, there is the delegate method where you get the event at that point in time. So you’ll know if it’s a touch event, or you’ll know if it’s a mouse event, and you’ll get the one of the two delegates, shouldReceiveEvent, which will be one of the mouse events, and you can look at the location, and that can help you out when you go hit testing. You can provide hints to the rest of your view hierarchy. Or shouldReceiveTouch, and you can get the location of the touch and know what to do from there. Thank you.

We have a WebKit question from Sunil. “So with the new touchscreen on MacBook Pro running macOS 27, how does desktop Safari expose touch input to web content, specifically around pointer events versus touch events?” Yeah. So let me answer literally the first two bits. We have the pointer event API working as we always have, and we don’t have the touch event API working. So what that means is, on web content, what the user sees when you do like some kind of touch interaction is it’s essentially a mouse.

Web pages aren’t supposed to know yet whether it’s a touch interaction or a mouse interaction. We kind of just do that… What you said, mouse emulation. We do, like, a smart mouse emulation, where we understand what’s under the web content and try to give it, like, smart mouse events accordingly. Kind of, I would say, rolling on from that answer, kind of advice for the web is generally just use the pointer event API. The pointer event API is meant to be an abstraction that kind of hides away what the underlying input method is.

And right now, to be honest, if someone does a touch interaction on the web, they will get… They will just see a mouse event. They won’t know whether it’s a touch event or a pointer event. We have discussed internally if we want to expose the touch event API. It hasn’t happened yet on the web. But yeah, that’s where we are. Thank you.

I have a question about assistive software. “Will there be an API that allows generating touch interactions programmatically, like one can do with mouse and keyboard events using APIs like CGEventPost?” We don’t have any API currently, and I can’t comment on what may or may not have in the future. Please file feedback using Feedback Assistant. Let’s see.

Looking for another question here. Here’s a question for Cas. “For video games that show on-screen touch joysticks on an iPhone or iPad, should those make their way to Mac as well, or should they… Should video games rely on keyboard and mouse interaction?” That’s a good question. I think it just really relies on how you would like people to play the game on a Mac.

If you believe that those joysticks are crucial to the experience and a fun way to play it on a MacBook Pro, then by all means, feel free to add them. If you have done a lot of work to make it really enjoyable and fast and cool to play it with keyboard and mouse, then maybe they’re not necessary.

Another option is that you can, if you have it built… For example, let’s say you have built it for iOS already, you could hide it behind a switch, maybe in settings, to say people can choose to bring this up if they know them from iOS and they want to play in the same way, or they can just use keyboard and mouse otherwise. Yeah, I think it really just depends on your design.

And I think this goes back to the first comment you made, where the majority of Macs out there do not have a touchscreen attached to them right now. So you got to make sure you still have a first-class keyboard and trackpad experience, and then add touch as another, you know, dimension to the whole gameplay aspect. Yeah, exactly. I have a question for Joakim. “Is there an example you can give us on how touch make your app better?”

One thing I would say is that it makes it more fluid to navigate through the app. And what I mean by that is, for example, if you’re scrolling through your photos in the grid and you want to change the zoom level, you can, for example, while it’s scrolling, start a pinch gesture, and we’re going to switch automatically to zoom in the grid and not interrupt and force you to, like, do multiple gestures. Another thing, for example, in the edit UI, if you want to crop, straighten, and pan your image, you can do it all in one gesture without ever lifting your hand, which can be a great accelerator.

Yeah, I would add to that. I think that’s a hero example right there of just being able to just do it with your hands. And I think Photos is also a really good example of putting the controls in the right place to make it work really well for touch without actually doing any changes. It was already that way. But the fact that in edit mode, all your sliders are on the right side makes it really easy to, like, grab the screen on the edge and use your thumb to be really precise on those sliders.

And I think for something as creative as editing your photos, sometimes touch just feels a bit better there, because you’re not really looking at the slider in terms of what value you’re picking. You’re looking at the photo and seeing when it looks right. And so we’ve done a lot of work to make sure that these sliders feel really precise, and that if you lift your finger, the value doesn’t change just because you lift it up.

That all work has been in there just so you can have that experience of editing photos and seeing a change with your fingers and making it feel very natural. So I think Photos is a prime example. And another example is if you open up a photo in the viewer, one-up, and then you zoom in, and then you pan, like, to the edge of the photo, you want to be able to pan further if you want to go to the next photo or to the previous ones. Those are, like, features we added specifically for touch.

What I find with it is it’s a lot of little delightful and surprising moments that are really nice. For example, when we added touch support to the signature in Preview… And I did the trackpad when you could do it with the trackpad, and that was great. But when they added that, I hurriedly went in there and added my signature right there on the screen, and now it looks even better.

And those great things. And another nice thing is I use a dual-display setup. So I have my main larger display, and I have my MacBook sitting next to me. So if I’m often working over here, I could just reach over and swipe away Mail messages, or scroll, or do something like that.

And those interactions, without having to move my cursor all the way to the other screen, and being in Photos and going, “Oh, I want to just read out for the eyes” or something, and being able to touch it and get instantly what you want, instead of moving the cursor all the way there. It’s a small interaction, but it’s delightful and it’s nice. And that’s what really impacts the experience and makes it great. Yeah. You don’t have to move your mouse all the way to other display just to do something, which is pretty nice. Yeah, yeah.

Have you started touching the other display? No, because it’s a big screen over there. So it’s out of reach. But I keep my MacBook right there next to me. Cas, this one is for you. “Our app UI is dense. How do we make it touchable without losing that density?”

That’s a good question. That’s something we struggled with a lot too. I think our frameworks have done a really good job at allowing touch to work really well, even though the UI is quite dense. Like, if you look at the places in Apple’s apps that have dense UI, like, let’s say, like, inspectors in Keynote, for example, or some of the controls you see in Logic or in Final Cut, or Xcode, for that matter, they’re all very dense, but we didn’t change any of them.

They’re all exactly the same as we did in previous releases, or at least they weren’t changed just because of touch. And so we did a lot of work to make sure that the touch targets, to your point, they don’t overlap, that we’re really smart about picking which one we think is under your finger and allowing you to change it. So hopefully, again, with our frameworks, you’re off to a good start by using those.

And then I think where dense really benefits is of some of these really small controls, like the stepper and the dial, and maybe sliders that are maybe a bit short. And for all of those, we’ve made expanded controls. So if it feels like maybe when you’re looking at it, it’s maybe too small to use for touch, I think with these expanded controls, you might actually surprise and delight some people by letting them tap in, and it just expands.

Like, they’re not going to do anything wrong, because if they would tap on a slider, thinking that they’re going to change the value, we’re going to expand it for them and not change the value, but just give them the bigger slider to say, “Hold on, we’ll give you something bigger so you can actually be more precise here and make it really easy to use with touch.” And all of that happens without changing your UI, or without making the UI any less dense. So yeah, use frameworks as much as possible and build on the expanded controls we have, I would say.

And another example is if you have, like, section headers. So on the sidebar, AppKit does it for you. In the past, you had to click on the little disclosure triangle to open or close a section. And now you can just click on the whole header. And if you have, like, custom UI, or, for example, in Photos, it’s the edit UI again, where you have those little sections, instead of just clicking, making the triangle clickable, you just make the whole header clickable to make it easier to use with touch, or even with mouse. Yeah. I kind of have a follow-up question from Amanpreet regarding expanding controls and hit target behavior. “How should developers implement the hit target, expanding hit target, for controls?”

So one way you can do that, if you want to make it touch-specific, if you need to expand them, is you can add, you know… Depending on the control, if it’s a custom drawing or something, then you can make your view larger but have it still draw in a smaller area. That’s one technique.

Or you put your view inside of a larger view, and you can put your gesture recognizer on the larger view, and you can see when that gets touched, and you can expand controls from there. Calendar did a lot of this. So they might have some interesting techniques that they used.

We just use AppKit standard controls, and AppKit handles the heavy lifting. You talked about earlier about making sure that, with the debugging, turning on a background color so you can see that the controls are the right… What area of Calendar was that for? And it was for the little control area on the bottom-left corner of the main canvas that shows the little mini month view underneath the calendar list that has many, like, small buttons in it. Also the year view. Like, you can imagine in a month, you can have, like, little days. - 30 days. - Yeah.

Okay. So did you put those… Did you put the AppKit control in a larger view, or did you… Or what were you…? It depends. Like, for larger buttons, there should be hit areas, like, by a large size. We do that. But for the small views, like the days, we don’t do that.

Yeah. My point that I was trying to get to with the line of questioning is it totally depends on your situation and the technique that is going to be best suited for you there. So it’s one of those really technical questions that I would need to see your app and be there with you, and we can discuss options.

So to give you the definitive solution right here is kind of difficult. Yeah. But thank you for bringing up the strategies. And a simple example is if you have borderless buttons, which only show a glyph or something like this, which we have in Photos as well. And they used to be just the size of the glyph, but there was space around it. So it was very easy to just make the button larger, and it looked exactly the same, but it’s way easier to hit.

Maybe just to go back on the previous question about density UI, because it made me think of what you folks in Calendar have done, is I think that the create event sheets or popover that comes up is a good example of an alternative you can do in terms of dense UI. It’s using FormStyle. FormStyle is what we use, for example, in System Settings.

And it’s an API that I think Calendar has used really well to make the creating an event really easy to scan and parse. And it has the benefit that, because they’re quite big rows that are 36 points tall, they work really well with touch. And so that’s another way. If you’re looking at your settings pane, or you’re looking at an inspector that’s not an enormous amount of controls, trying it out, and FormStyle might actually be fine, too, both for cursor and for touch.

And the only thing I would add to that is, I think if you look around on System Settings, the different panes, we try to do a lot of work to not make it just a long list of a label on the left and a control on the right and just line after line after line. We do a lot of work breaking it up into sections or making some parts more visual.

Like, if you look at the Appearance settings pane, there’s colored dots to choose your accent color, or small visualizations of your menu bar to tell you if it’s light or dark mode. So you kind of get a bit nicer design that’s a bit easier to scan than just long lists you have to read one line by line.

So that’s maybe one thing to keep in mind. But otherwise, FormStyle is, I think, a good example, and Calendar’s is a good example of that to follow, I think. Thank you for bringing that up. Here’s a question from Harken. “As touch on Mac introduces new ways to interact with apps, how does Apple recommend evaluating the user experience and identify usability issues across different Mac and iPad configurations?”

That’s a tough question. Can you repeat one more time? How should developers evaluate their app’s UI now that they can be touched on Mac? How should they audit their apps? And a couple of guidelines to keep in mind. I think from a design point of view, the whole point about touch on almost any of Apple’s devices is that it should feel very responsive and alive. Like, that’s the whole feeling you should get, because it’s your hand and your fingers. It should be appreciated by the device and just react to whatever you’re touching.

So anything that is interactive, anything that would do something if you touch it, should react when the finger goes down. And so I think for Mac, given that it’s now a platform that starts to have touch, I think we should look… We should be quite careful about blocking touch entirely. Like, even if you have very small controls, or controls that are just really hard to use with touch, you should still make sure they work with it. You might be surprised at how specific people can be and how well they can use touch even with small targets.

But we definitely don’t want people to, or apps to, just not support touch for certain gestures, because it feels very buggy, very, like, dead on arrival. Like, it just doesn’t react. And therefore, you kind of start guessing as in what does work and what does not work. And now you’re going to stop trying using touch because it might not work. And then you’re kind of deprecating the feeling of that it’s alive and that you can just go up and do it. So that’s maybe one thing.

I think touch targets are another big part. Trying to do tricks like you were doing with overlapping transparent ranges around it, so you can see where it overlaps, and then trying to be good and smart about which one should prioritize. Also testing maybe how it works with system gestures, making sure you’re not overlapping on those, trying it out, like, on the edges of the display, or places where, like, for example, Notification Center could come in, or the Dock, or Stage Manager strip. Those are, I think, are good tests to run.

Yeah, those are the ones I can think of. But I don’t know if anybody else has… Let me answer that question - by telling you a story. - Yeah. So we started working on adding touch to AppKit and to all the standard controls and things, and thought we were doing great. And then we introduced QA to the project.

So they went after it with fresh eyes and looking at it differently. And they took it from the approach of not knowing anything and just that there’s now touch. Go. And so what were their expectations? And that was a big eye-opening. We got all sorts of radars back that they were expecting this to work this way or work that way. And that was the start of a conversation.

And so we pivoted where we need to pivoted and added things where we needed to add them. And it was a great way to look at that. And further down the project, we had to go, and it’s like, “Well, let’s go talk to a different QA group and get fresh eyes to look at it again.”

And what were their expectations coming in at fresh? And we’ve got a new round of bugs came in and things to consider and to discuss about, and decided how we wanted to make it work. And I think at the end, you know, having fresh eyes look at it every once in a while, getting new perspectives really hones in and gets that extra polish and making your app feel like it’s just supposed to work. You know, the best feature is the one that a user uses intuitively and doesn’t realize the amount of work you had to do behind the scenes to get that interaction to happen.

Yeah. I think actually the pinch and zoom was a good example in Photos, because on trackpad, you’re locked into one of them if you start doing it. So like, if you would pinch and then rotate a bit, we would stick to pinching to zoom, and we would ignore the rotation, and vice versa. If you start rotating, we wouldn’t zoom in anymore.

And that felt good on trackpad for a while, because it was kind of difficult to maybe do both, but it felt really wrong on the screen with touch, because it just felt like we were deliberately ignoring something you were doing, and it didn’t feel like it was really, again, following your fingers and feeling alive. So in that case, we made it work for both, just like you can do on an iPad too.

I would say the same thing about the work we did for Logic with the sliders. Like, the fact that you can move one slider is great, but then it really feels strange that if you put your second finger on the slider right next to it, that one wouldn’t follow.

And so that was a lot of work we did on allowing those multiple interactions at the same time, so that you can do up to 4 or 5, however many fingers you have, to just start to move sliders. It’s all of that, making it feel alive, and trying to test your app and just see, like, where does it feel off? Or where does it feel that it’s just not following my fingers as much as I would hope it would? And you might be surprised that some of the things that feel perfectly fine on trackpad might just need a little bit more attention for touch on screen.

Let me take this opportunity to mention gesture exclusivity, the gesture exclusivity API that exists in AppKit. By default, we found we needed to, for binary compatibility, to only allow one gesture at a time in most cases. So like sliders, and it doesn’t seem to want to work at all.

And you need to go onto your view and turn off gesture exclusivity for the parent view, at least. And then you can get multiple interaction. And, you know, I think what we’re looking for here as we move to the future is being able to do multiple interactions just as a common thing across your entire app.

So that’s where we want to get to. And we realize right now, for binary compatibility, we have to start off with only allowing one, but it’s definitely something that you should investigate with your applications. Like, if somebody comes in and is, “I expected to do this with two fingers at the same time,” well, we have API to allow you to get to that and get it working.

Go back to Cas’s point. One important feedback we got for Calendar app, when we do the fresh testing, was that with the mouse before, when you drag an event on canvas, it snaps to the nearest 15-minute time grid or the nearest date column, but there’s no animation in between. If you drag something from one day to another, like, it just… The event just flies there. We borrow the animation, lifting animation from iPad to make the drag feels much smoother, too, actually.

Now you press and hold on event, we render a little shadow to make the event, like, lifting from the canvas, and it follows your finger along the way. And we also added a little timeline at the edge of… We add a little mini label on the edge of the timeline to indicate where the event is going to land. And that corresponding to the Human Interface Guideline, asks for the continuous feedback throughout the whole drag. - Yeah. - Exactly. Let me tie into that. That drag change in behavior works the same way now for mouse too, right?

Yes. And I’ve found that I appreciate that when I’m using the mouse to move calendar events around. That little marker on there is something that I was really pleased about to have for the mouse as well as for touch. So I mean, my point there being that often something that you think you want to do just for touch can also be really useful for the mouse as well. And it’s just a better overall user experience altogether.

Yeah. Driving this point home for us in WebKit land, same thing. You know, if you have some kind of touch reference on iOS, you know, our Mobile Safari experiences, we kind of had to learn from, you know, what looks good or feels good with touch. Initially, when we brought up touch support, and you tapped on the link, it would just navigate to a link.

You wouldn’t really get feedback. So your touch sometimes felt dead, especially if you’re imagining slow internet connections. You tap and then nothing happens, and 5 seconds later you have a web page. And we were kind of evaluating. We asked some fresh eyes to look at it, and they were like, “Oh yeah, on iOS, I see there’s some kind of highlight, like, tap highlight when you click on the link.

So there’s some kind of feedback. We don’t have that on Mac.” So, like, let’s adopt those kinds of primitives that we have already working on iOS that feel good. So yeah, there’s things like that. I feel like all of us have had that experience. That’s a great segue into the next question. “iPhone uses haptics to confirm a tap, or at least this app does. What’s the best way to give users the same confirmation on Mac?”

Well, we don’t have haptics in the touchscreen, and so… It would have to be something visual. I’m going to throw it to Cas. Is the question more about how do we indicate, for example, that you can drag, or that something is attached to your finger? No, I think it’s more of giving a confirmation that something has been tapped, so just like Safari did with the links. Yeah. I mean, I don’t think iOS always gives a haptic.

  • No.
  • Yeah. Like, it is a choice of the developer to say if they want that or not. I like, for example, if you’re on your Home Screen on your phone and you tap an app to open it, it’s not going to give a haptic to say that you did. So it’s really just a design choice.

I think for the Mac, given our limitations in hardware, we can’t… I think we’re trying to be as visual as possible. I think there are some examples, like on Home Screen on the phone. I think you do get haptic when you get into edit mode or you start dragging something. And so we can’t do that on the MacBook Pro, but we did try to give you feedback by enlarging the element that you’re going to drag to give you the idea it is now attached to your finger.

And that’s another big difference, where with the cursor and with trackpad and mouse, we didn’t have to do that, because you can just hold your mouse down on the element and immediately move it away, and at that point, you’re dragging, because scrolling was a different gesture. That was either two-finger scroll on a trackpad, or on a Magic Mouse is moving, you know, sliding your finger over the mouse, not moving the mouse itself.

And so for touch, we had the conflict of, well, dragging could mean you’re scrolling, but it could also mean you’re dragging. So how do we differentiate? And we added that delay to make sure that a normal moving of the finger would just scroll. And then long wait, like a little delay, would allow you to drag it. And so to tell you that now it’s attached to your finger, we enlarge it and attach it to your finger so that you get that feedback to say, “Now you’re in dragging mode. If you’re going to move your finger, you’re not going to scroll anymore.”

That’s the feedback we’re giving. And I think those are the explorations you need to do when you’re trying to find the alternative to a haptic that you might do on your iOS app, because yeah, there’s no way to bring that across identically. Yeah. And the same thing with applied to iPad as well. Yeah, yeah.

I have a plug for the web developers out there. Since we have implemented tap highlights, you can also customize your tap highlight colors for the web. There’s this WebKit-specific -webkit-tap-highlight-color CSS property that you can set on your web pages. It’s something that already works on iOS. Then now we have support for on macOS.

I don’t think we got it in 27.2, but in the 27.3 betas, you should be able to see it. So thank you. Amanpreet has a fun question here. “When a user interacts bimanually, that would be touching the screen while also using an indirect pointer, how does AppKit arbitrate between active gesture recognizers and pointer tracking without stealing the cursor position?”

So how deep do we want to get into the technical? So ideally, we want you to use touch-native gesture recognizers to interact with controls. And all our controls are touch native. So you can interact with a mouse, depending on your gesture exclusivity, with a mouse and a touch, because there are two different inputs, and that’s fine.

On the occasions where we do have to emulate the mouse, once we start emulating the mouse, we will start ignoring a hardware mouse or trackpad events so that they don’t conflict. And, you know, that’s one of the limitations that we had to make. And why, again, we want to move towards having everything be touch native and not rely on mouse emulation, because we want to open up the ability to interact with multiple fingers and the trackpad and the keyboard all at the same time.

“For developers working on remote desktop apps for controlling a Mac via an iPhone, is there a way to send native touch input directly to the Mac instead of translating it into mouse events?” That is similar to the accessibility developer question we had earlier. Yeah. And I will leave that as a… File feedback, please. “When it comes to testing, is there a way to simulate touches at all?”

Let’s recap. No, there’s no… But I think XCUITest has support for touch interactions now. Okay. Yeah. Yeah. Our QA department has done some stuff with that. And I’m not sure what tools we have that we publish that expose that. Let’s see. I’m looking through the questions. “For iPad apps designed for Mac, will they all automatically translate the same experience on Mac without any effort?”

Yes. I mean, I’m going to qualify that a little bit. For most things, you know, the 90-plus percent should work just great. Now remember, if you have an iPad app that you’re bringing to Mac and you’re turning on the Mac idiom, right? We’re going to put in some of the AppKit controls and make it look and work like a Mac in more ways than if you just aren’t using the Mac idiom.

And that should all come across fine. But this is still a… You know, there’s still a trackpad and a keyboard attached, and there’s going to be lots of Macs out there that don’t have a touchscreen. So that’s where you would really need to focus your attention, in making sure that there’s still really great trackpad and keyboard interaction, which should just come over, you know, again, naturally. We’ve been doing this for quite a while with Mac Catalyst. And then test your touches, and they should mostly just work. I’m trying to think of a case where they don’t.

And I can’t really think of one. But there’s an interesting question there: Is that the right thing to do? And most of the time, whatever you have on iPad is the right thing to do on a Mac Catalyst app. Sometimes it’s not, depending on just your situation and the rest of your layout changes that you might have made for Mac. Thank you. “This may require listing the gesture recognizers that AppKit offers, but for an application that has 3D models, is it possible to implement touch and drag animations on the screen to interact with the model itself?”

So you have… If you use the NSPanGestureRecognizer, you can get x and y pan deltas. And you can use that for both a mouse and touch. You can have a touch-only version and a mouse-only version. And you can have them run simultaneously. So you could see them operating at the same time and figure out what your mouse chording is, if you want your mouse buttons or your keyboard modifiers to do different things, so for which axis you should control as you’re panning. So you have all the same capabilities that you would have with a mouse.

You can do that with touch, and however you want to use to get to the third dimension, you can do that too. You know, there’s lots of things we use in macOS, for example, and a lot of places where you can pull up a context menu, you can Control-tap, and that’ll pull up a context menu as well.

So you can get the modifiers from gesture recognizers and/or from the events, if you end up with a completely custom gesture recognizer for some reason, and you can expose that to the action clients of your gesture recognizer. And thank you. I actually don’t know the answer to this question from Andrea. “Does an external touchscreen behave the same way as a native touchscreen?” So I’m guessing a third-party touchscreen. Are we also reading that as a touchscreen, or is it being translated to clicks?

That will depend on a couple of different things. The third-party touchscreens I’ve seen out there, the way they work previous to macOS Golden Gate, some of them would emulate a mouse, and some of them had drivers and would do whatever their drivers do to recognize gestures themselves and translate that to mouse events, or whatever they do.

If you still install those drivers, they’re still going to work that way. If they expose their HID descriptor as a mouse, then the OS is going to treat it as a mouse. If it’s a digitizer, I’m not sure what we’re doing there. We’re focused on this product. So that starts to get out of my area of expertise on what’s supported, and if we can figure out which screen, which display that digitizer is associated with.

So that would be an IOHID question on whether or not we support that out of the box, or if you have to write your own driver. And then once you have to write your own driver, we’re back to the question of the accessibility and how do you post and create your own touch events, and file feedback requests for that. Yeah, please file feedback. We are running out of time. There are a lot of questions around the differences between using Sidecar and the touchscreen itself. Can you think of anything that developers need to be watching out for?

I found that was helpful for me when I was testing with Sidecar and developing on Sidecar at first was to make sure to put it on some kind of case that you get it more vertical, because that’s how you’re going to be interacting with it, in terms of with a MacBook Pro.

So that was one of the things that I used a lot, was make sure I had it vertically, because you could take it here and you can work with it on your lap. And that’s a little bit of a different experience than if you come at it this way.

Yeah. And I think we added a setting to Sidecar to emulate the system events, right? Yeah. This should be happening in the latest seeds. Yeah. So there’s a setting in Sidecar settings where you can say all the multi-finger gestures to switch Spaces or do Mission Control or App Exposé, normally some of them, I think, still go to iPadOS in Sidecar, but there’s a setting to say, “Give them to macOS,” so that if you are using Sidecar to test your apps, you can simulate those events and see how your app hopefully will or will not react to switching Spaces or something. Like, if you are trying to use a four-finger gesture, for example, and you asked for that in your app, then you can test there to make sure that you did it right, and it’s not triggering system Spaces or Mission Control.

We really worked hard on making sure that Sidecar is a pretty good place for you to test your app for touch. So try to use, yeah, most of these tools to just take it there. Maybe one more thing to try out there too is, I think there’s… It’s not the best experience, but you can change resolutions on the Sidecar too.

And so you can try out different densities and making sure that it all works. That’s a thing that I think is maybe a bit more prevalent on a Mac, is just that we have multiple resolutions. On iPad, we do too, but not as many. So making sure that it works in all those resolutions is pretty important too.

Yeah. And I just want to again reiterate that I think would answer a lot of these questions is to install the latest 27.3 betas, and that has added the extra support to Sidecar that it’ll do all the things that the Mac Pro touchscreen does. And you want to have 27.3 on both the iPad, the Mac, and Xcode.

Yeah, I think so. Okay. Thank you. I’ll ask one more question. Hopefully, hopefully it’s not a hard one. But just, you can do Control-clicking, Command-click, and Shift-click. Can you do Control-finger, Command…? Yeah. That’s what I was mentioning earlier. You can do Control-tap. Okay. Usually in my head, whenever I say tap, I always mean a touch. When I say click, I mean the mouse. Okay.

So I know not everybody thinks that way. I gotta clarify, but yeah, a Control-touch and a tap… If that is somewhere that pulls up a context menu with a long press and hold with touch, then that’s an accelerator to be able to pull up a context menu that way.

So we automatically do that for both. If you’re just using the menu API or menu(for:), we’ll automatically handle that for you. And if you’re there and you overrode rightMouseDown and pop up the context menu yourself, then you’re going to have some work to do to add support for touch, for that to be able to get these menus to come up, and add an accelerator gesture recognizer so that you can get Control-tap to pull up the context menu as well, so that you get the full range of user interaction that everybody’s coming to expect. - Good to know. - Yeah. In general, all modifiers you would get from the keyboard using the cursor should also apply to doing the same interaction with touch.

So things like, if you’re dragging files with the cursor and you hold Option or Shift or any of those modifiers, you get different ways of adding it. Like, you could copy it, you could make an alias. That should all work the same way when you’re using touch. So there should be no difference there.

That’s also kind of what we would expect, that it just… It’s a one-to-one translation, and we’re not taking functionality away just because you’re using touch. But you can add touch-only accelerators. For example, in Photos, if you want to select multiple photos, you can use two-finger taps instead of holding the Command key and doing one-finger taps. It’s a good tip. Thank you.

I’m afraid that brings us to the end of our questions and the conclusion of today’s group lab. Thank you all so much for joining me today. Abrar, Raleigh, Cas, Joakim, and Henry, you are awesome. - Thank you. - I want to also give a shout-out to our team backstage. So, Ed, Travis, Adam, Paris, Linda, Jimmy, Adam, Serenity, Doug, Casey, Sergey, and Brandon, you’re all been working hard behind the scenes. Thank you so much. And to everyone watching on the developer site and on YouTube, thank you for all the fantastic questions.

If we missed your question, please bring it over to the Apple Developer Forums so we can keep the conversation going. And remember to file any bugs or requests through Feedback Assistant. Make sure you check out the updated Human Interface Guidelines documentation and touch on Mac videos. And we’ll be hosting another touch on Mac group lab tomorrow evening. We’d love to have you back for that. Thank you again, and have a wonderful rest of your day. Take care.