Good morning. Who here has an app on the store that uses Apple Pencil? All right, a few of you. And who has an app on the store with Apple Pencil features that is not a drawing or a painting app? Oh, one, yes. Well, today, not only are we going to learn a few things about how to make your app great with Pencil, we're also going to see that Pencil can be for so much more than just drawing.

Pencil can do a lot of things and it can be a bit daunting to add Pencil interactions to your app, especially if you've never worked with it before. Today, we're going to look at seven properties of Pencil that, when implemented, will add entire new dimensions to your app. And to do that, we're going to add Pencil interactions to an existing app.

So, here's my app. It's a fairly straightforward sudoku game. The puzzle screen shows the current puzzle we're solving. Tapping a cell brings up the keyboard where the player can enter a number, and there are a number of buttons for showing or hiding the timer, depending on how competitive you are, for getting a hint, and for changing various settings about the current game. So, even though this is not a drawing app, doesn't it seem perfect for Pencil? Let's make it happen.

And the first thing we're going to look at is that Pencil makes marks. I don't know about you, but if I pick up a pen and I try to write with it and it doesn't write, I toss it out. We worked really hard on Apple Pencil and we don't want people tossing out their Apple Pencil. So, don't require tapping a button or entering a special mode to draw with Pencil. Pencil should make a mark the moment it touches the screen. When I put Pencil down, I can go ahead and write my guess immediately.

Pencil is familiar; it should behave consistently with what people using our app have come to expect. This means that tapping controls should behave the same whether you touch them with Pencil or your finger. Here, I can interact with the various setting sliders using Pencil just like I would with my finger. One thing we definitely never want to do is have controls that will have different consequences, or worse, do nothing when tapped with Pencil rather than a finger. This would be very unpredictable and confusing.

Pencil has a number of sensors. It can sense tilt, force, and orientation, and these sensors can add layers of depth and expressivity to our interactions, but there are a few missteps to avoid. A good way to use the sensors is to make marks more expressive. For example, the continuous variation of pressure and tilt can modulate the thickness of a stroke. Or the orientation to affect the width of a marker.

We don't want to use sensors to affect things not directly related to Pencil input. For example, here, depending on how hard we press, we get a new puzzle of varying difficulty. That's just a confusing behavior. Sensors should affect the kind of marks that Pencil makes, not things disconnected from it.

Another way to handle sensor data is discretely. For instance, to modify the input based on the pressure exerted. So, an idea for a game here is to let people pencil in a tentative guess, depending on how hard they write. Writing lightly lets you put in a guess in lighter ink, and writing with more pressure lets you commit that answer in black ink.

We can think about a sensor reading such as pressor as a continuum and treat it as such to modulate a quality of a mark. We can also split that continuum binarily, as we just did for this pencil-in feature. In this case, we'll want to make sure to give clear feedback as to which side of the split we're on.

We definitely don't want to break that continuum into more than two discrete parts though, as it would become very hard to differentiate. People can tell the difference between writing soft or writing hard, but they can't tell the difference between 5 or 8 or 15 different levels of pressure.

Handwriting is tremendously expressive, and as a result, so is Pencil input. As we get our game for people to try, we notice that people will often write slightly outside of the boundaries of a cell and this is a totally normal side effect of handwriting being so personal and freeform, but it means we need to be flexible with how we handle input.

What we shouldn't do is impose unnecessary constraints on input, like rejecting it if it doesn't fit perfectly within the cell. So, what we're going to do here is assign the answer to the cell it covers the most and we also scale and move it to make it fit neatly into the cell, which is a nice little touch.

Now, an interesting question comes up. What should we do if the player starts drawing or writing outside of the main sudoku grid? One thing we can try doing is just nothing, but it doesn't feel very good. It goes back to our earlier point about making a mark. It breaks the impression of fluidity and it might even lead some people to think that their Pencil is impaired, or the battery is discharged. Something we can try is to show transient marks, marks that disappear shortly after they've been made, to communicate that everything is working fine, just the margins don't do anything special.

Another option we can try is to just let people draw and annotate in the margins just for fun. And this could be particularly great if later we add multiplayer to our game. Now, there is no right or wrong answer here. It is up to you and what you want your app to be; but doing something is always better than doing nothing, even if that something is just to communicate that Pencil is working.

If you're not drawing with Pencil, you're writing. And because you're writing on an iPad and not a piece of paper, we have to ask ourselves questions about whether we want to process the input further. For the case of our game, should we convert handwriting to digital or should we keep it analog as it is? One reason to convert it to digital is to be consistent with keyboard input.

And one reason to keep it analog is that it's fun and personal. Again, here there are no right or wrong answers. It will depend on your specific circumstances. You do want to keep it consistent so that people using your app know what to expect. And if we do choose to keep the input analog, it's always a good idea to digitize it under the hood to enable things like searching for text or, in the case of our game, so that we can validate the final answer to the puzzle the player came up with.

So far, we've been talking a lot about Pencil and it can be easy to forget that there is a human holding it. Different people will hold and use Pencil differently. For example, if I watch a left-handed person use our game, we'll notice that their hand and arm gets in the way of the controls on the left side of the screen. The positioning of these controls wasn't much of an issue when our app was used primarily with touch, but it becomes an issue with Pencil. So, what we can do is move the control to the top or bottom of the screen or just make them repositionable.

And, finally, Pencil is really great when it provides shortcuts. That is, letting people do things faster by drawing and writing rather than tapping on the screen. So, how do we erase a cell in our game? It's a two-step process. The first step is we tap the cell we want to erase, and the second step is we press the backspace key. So, can we turn this two-step interaction into a one-step interaction with Pencil? Something that would feel pretty natural is letting people cross out their answer, so let's give that a try.

Seems to work pretty well. But, as I give my game for people to try, I notice something interesting when they try to erase a cell. Some people will strike it through, others will scribble it over, and yet others might draw something like an X mark. Handwriting is really expressive; different people will cross things out differently.

So, in this case, it's probably better not to ask people to perform a specific mark they would have to remember. In addition to being hard to remember, specific marks might have different meanings in different cultures. For example, in the U.S., it's pretty common to use a checkmark to mean correct. In other countries, that might not mean anything or even mean incorrect.

So, what we do is we just let people erase a cell by scribbling over it. Whether they slash it out, draw an X, scribble over it, draw a straight line, it doesn't matter. All of it works and erases the cell. Let's look at another two-step interaction, getting a hint. First, we'd have to tap the hint button and then we tap the cell we want a hint for. And this is another interaction that we can bring down to one step using Pencil.

Perhaps we can let the player draw a question mark on the cell they want an answer to. And this feels pretty good. And in this case, because the question mark is mirrored in the icon of our hint button, I feel pretty good about using that mark as a shortcut for hint.

Now, these seven properties are by no means an exhaustive list of all the ways Pencil can make your app great, but I hope they've shown you a side of Pencil that perhaps you did not expect and I hope they help you start thinking about what adding Pencil interactions would mean for your app. Thank you.