Good morning. Welcome to Bug Reporting Best Practices. Aren't you glad you didn't have to wait an hour and a half in line for me? I'm Paul Schreiber, a software engineer on the Mac OS X team. So before we get started, I'd like to find out who's here in the audience. How many of you have filed a bug against an Apple product in the past year? All right, good, good. Anyone been really busy filing, like, 10 bugs? Wow. And, uh... Anyone filed, say, 50 bugs? All right.

There you go. So thank you for doing this. Why is it that we file bugs? Well, we file bugs to make better software. And you get two benefits from doing this. Number one, when you file bugs against your own products, you make your software better. And number two, when you file bugs against Apple, you help us improve Mac OS X.

So it's one thing to sit around and complain to your friends or on your blog when something doesn't work right. It's another to go ahead and take that step and write up a bug explaining the problem. So since this is a talk about bug reporting, I thought I'd talk a little bit about reporting. So what is it a reporter does? A good reporter writes good stories.

A good story begins with a snappy headline which draws the reader's attention in. A good bug begins with a clear and succinct title which explains the problem and gets the engineer to look at it. A good story has an informative lead which describes key facts and answers key questions. A good bug report has a concise summary which describes the problem.

What happened and why it's a problem. And a good story answers six questions. This talk will answer six questions about reporting bugs. First, why do you file bugs? Second, what is a bug, and what are the different types of bugs? Next, who should file a bug? How do you file a bug? And that's where we're going to spend most of the session today.

Then we'll cover where do you file bugs when you're filing a bug against an Apple product? And finally, when is it appropriate to file a bug? So we've talked about why, now we're going to go and talk about what. What is a bug? Well, a bug is one of these funny six-legged things over here. But the types of bugs we're interested in are software bugs. So a software bug is one of two things. It's either a defect in a piece of software or an enhancement request, which is a suggestion for a new feature or an improvement to an existing feature.

So, let's take a look at some of the most common types of bugs. Now over here behind me is the crash reporter dialog. And when an application on Mac OS X crashes, you see this. So a crash is definitely a bug. And when you file a crashing bug, make sure to mark it as a crash and not just a serious bug. Now sometimes people will say, "Oh, my whole system crashed," or "I saw this really funny dialog in many different languages." And what that is is a kernel panic. And this is the kernel panic dialog on Mac OS X.

Sometimes people will say my application is hanging, or it's spinning. If your application is unresponsive, that's also a bug. Another type of bug is a documentation bug. Documentation bug can be several things. It can be very simple, like a typo. It could be a case where the documentation is unclear or ambiguous, and you don't know what the author meant. It could be the sample code doesn't compile. It could be a case of the documentation not matching the implementation.

So let's take a look at some bugs in the software itself. First type of bug is a functional bug. Now notice here is the familiar connect to server dialogue in the finder. And I typed ftp.apple.com and clicked connect. But the finder guessed wrong, and it prepended afp:// to the host name. Another type of bug is a cosmetic bug. This is when something works, but it doesn't look right. If you notice in the info window behind me, you'll see the icon for the Mac is jaggy and blurry.

Another type of bug is a usability bug. A usability bug is when something works, but it could work better. So here we see the Energy Saver preference pane, and notice in the bottom right corner, The lock is locked. But look here. The sliders are disabled, as you would expect. However, the schedule button is still enabled.

If you were to click the schedule button, a sheet would appear, and all the controls on the sheet would be disabled. So it would be much better to disable the schedule button whenever the lock is locked as well. Now, another type of bug is a performance bug. A performance bug is when your application is slow, or when an application is leaking something like memory or Mach ports.

Next, a security bug. A security bug describes a security exposure in the operating system. That means when someone gains access to files they shouldn't or services they shouldn't. What a security bug is not is any old bug in a security component. For example, if you saw a typo in keychain access, that's not necessarily a security bug. It is, however, still a bug and still should be filed.

So, now let's talk about who. Who is it that should file a bug? Do I have to ask someone if it's okay to file a bug? Should I check? No. You all should file a bug. If there's something that you see that's wrong or confusing or you don't understand, or you have an idea for how something could work better, file a bug. So now, let's talk about how to file bugs.

There are six components to a good bug report. We have one, the bug's title. Two, the bug's description. Three, steps to reproduce the problem. Four, regression, when the bug does and doesn't occur on different systems or configurations. Five is the configuration of the machine on which the problem occurred. And number six is reproducibility. Is this the bug that recurs? So let's go over the bug's title.

The title for a bug needs to be clear and succinct. You don't have a lot of space in which to write the title, and you want to make sure you have a lot of useful information Think of the title as an advertisement for your bug. Now when you're reporting, you have to make sure there's a clear line between advertising and editorial, but in the case of a bug report, you need to get people's attention. You want them to read your bug, and you want them to fix your bug.

So you need to make the first words of your title count. And a great way to do that is to include the name of the application where you saw the problem, at the beginning of the bug's title. Now because space is so limited, you have to make every word count.

So how do you make every word count? Well the first way you can do that is to avoid any sort of chatter. Sometimes people say things such as, "It would be nice if my software did this," or "I would really like it if the software could do that." This type of exposition is okay in the bug's description, but it's not appropriate for the bug's title. Another thing to avoid is the use of filler words. Any extra prepositions or pronouns or words like "etc.", "articles," those sort of things can usually be eliminated from the title.

Now, sometimes if you're a good reporter and you write a good story, your editor will follow that up with a headline containing a clever pun. In the case of a bug title, a pun is not appropriate. Remember that your goal is to be objective and straightforward. And also, the person reading your bug might not be a native English speaker and might not get the joke.

Now, sometimes when people are selling things on eBay, they use special characters or all capitals to draw attention to their item. In the case of a bug title, this is not appropriate. Not only does using all capital letters impair readability and reduce the amount of information you can fit in that small space, but it's much harder to search for titles with alternate spellings. Now, I haven't seen any bugs that look like this, but before you get any ideas, Leetspeak would not be a good thing to use in a bug's title.

Remember that your bug's title has to stand on its own. It's going to be viewed in a list of tens or hundreds of other bugs out of context. Think about how people will search for your bug. Think about the types of synonyms they'll use. Maybe your bug is about a particular error. If that's the case, it's great to include key words from the error message and the error number in the bug's title.

Now, sometimes bug titles have prefixes. Let's go over a few of the common prefixes we see in bug titles. Number one is the prefix ER. ER means enhancement request. That means this describes a suggestion for an improvement and not a defect. Number two is the build number. And you'll see this a lot when you're dealing with pre-release software or seeds.

This way, when you write the bug, the engineer reading it knows exactly what build of the software you saw the problem on and can compare to the build that they're using. And finally, you'll sometimes see the prefix regression. It's especially common on Safari or WebKit bugs. A regression is when something was working correctly in a previous version and is no longer working in the current shipping build.

When you have a crashing or hanging bug, it can be helpful to include the function at the top of the stack trace at the end of your bug's title. However, if you're doing this, you want to omit function names from low-level libraries, doing things like exception handling, string comparison, objective C messaging, or low-level memory management.

Let's go over a few examples of how we can improve a bug's title. Now, say we're using our computer, and one of our applications crashed. And we could write a bug that says, "crash." But this isn't a very good title, because it doesn't tell us a lot of information. It doesn't tell us what crashed, it doesn't tell us what was going on when the crash occurred.

So, maybe we could say, "It was the finder that crashed." Okay, this is a little better, but there's still more information we need to include in the title. So, let's say we went to the Go menu, and we chose Connect to iDisk, and that's when the crash occurred. So, we'll describe this as "Finder crashed, connecting to iDisk."

Now since this is a crashing bug, we can look at the stack trace, and we can figure out where the crash occurred. And we'll put that at the end of the title in brackets. Now this is a great bug title, and many people would stop here. But we can make this even better.

So remember what I said about including the build number? If we prepend the build number, then here we have a fantastic bug title, which contains lots of information. The engineer knows right away when the crash occurred, what you're doing, and where the crash was. So, good reporters have good editors. But when you're writing a bug, you have to be your own editor. Let's take a couple of examples of editing a bug's title.

Now, say you were in Microsoft Word and you tried to print and you went to your printer and there was no paper there, and you might write a bug that says "I tried to print, but nothing happens." This isn't a very good title. Why? Well, first of all, it doesn't tell us what application we tried to print from, it doesn't tell us what steps we took, it doesn't tell us what we were expecting to happen. So a better way of expressing this problem is a "word printing attempt fails silently." "Fails silently" is a good phrase which conveys a lot of information succinctly.

Now say the opposite occurred. Say we did something, and it worked fine, but we still got an error. So someone might write up a bug that says, iTunes shows me an error. OK. But why is that a problem that it shows you an error? What type of error did it show you? So say you were using the Music Store, and then you downloaded a song, and it worked fine, but you still got an error dialog. So we could say that the iTunes song download gives me an error dialog, but it succeeds anyway.

Now, sometimes people like to use colloquial language or aren't clear in their bug titles. For example, you might be using Safari, and you would write up a bug that says there's smudge marks everywhere in Safari. Okay, well what's a smudge mark? And how did they appear? So a better way to write this title will be something such as, "Safari redraws QuickTime VR movies incorrectly during window resize." So that tells us what we're doing when the smudge marks appear and how to reproduce them.

Now, sometimes when you write a bug because you're really angry, say you were doing an important demo or a presentation, and something didn't work right. And you would be upset, and I too would be upset if this happened. So say you were using a new application called iSmell, and you could say iSmell is a complete failure.

And I would agree that in that case, I too would feel that iSmell was a complete failure. But when writing the bug, think about how it failed. So say you had a new video camera, and iSmell didn't work with your Sony CyberCube 3000MX. Write that bug instead. Sometimes I see a bug title that's so confusing, even I don't know how to fix it.

So, let's summarize the best practices for writing a bug's title. Start off with the build of the software or operating system on which the problem occurred. Then, include an application where the problem occurred. Third, use a verb describing the action. Fourth, explain the situation which was going on when you saw the problem. And finally, in the case of a crash or a hang, include the symbol name. It's important to be clear, concise, and succinct in your bug titles. Convey the important information there. So let's take a look at what is included in a bug's description.

Now some of the best descriptive writing you'll find in the newspaper is in the sports section. And a good sports writer would never say, "Oh, Hernandez threw a curveball." They would say, "It was the bottom of the ninth. There were two out. There were three men on base. It was raining.

They were down three to one." And then, "Hernandez threw a curveball." They're gonna set you up with context. Remember that when you write a bug, the engineer who reads the bug wasn't there when you saw the problem occur. They don't know what happened. They do not have ESP. They might not even have ESPN.

So the first thing you need to do in your bug's description is repeat the bug's title. Why is it you repeat the title? Well, as the bug evolves and is investigated, the title may change to reflect the underlying cause of the problem. When you repeat the title in the description, this makes it clear to anyone reading the bug for the first time what problem you saw. Next thing you need to do is summarize the problem. Describe what happened. Describe why you think it's a problem. Include some details. Make sure to be very specific in your description.

Sometimes people will use vague language to describe the problem, or they will use slang. They'll say something looks bad, or they'll say it's odd, or my computer is acting up. Don't use any of these terms. It's not clear to the person reading the bug what this means. Instead, it's important to describe specifically what happened.

If a bug involves a particular error, include the exact text of the error message. If you were unable to copy down the full text of the error message, for example, it appeared and disappeared very quickly, or you had to restart the machine, give the best approximation you can. And note also that it is an approximation. Many error messages are similar. You don't want someone to spend a lot of time chasing down the wrong error.

When you're writing, it's important to include specific references to items on the screen. For example, instead of saying "the inspector" or "the pallet" or "the window", say something like "the finder's inspector window". Instead of saying "oh, the funny grey round thing over there", say "the spinning indeterminate progress indicator". Now what if I don't know the name of all these things? Okay, that's fine. You can either look it up in the documentation, in the case of the spinning indeterminate progress indicator, you'd see something like that in the HI guidelines.

And in the case of API level things, you could look it up there. But if you don't know, that's OK. Include a screenshot and point it out to us what it is you saw. Another thing to avoid is relative pronouns. When you're writing three paragraphs, it's not clear when you say it crashed or that crashed, which of the things in the preceding paragraph you were referring to. So make sure to be explicit.

Other times, people will use one phrase to mean many things. For example, one might say the system crashed. What does this mean? Does this mean that you saw an application that crashed? Does it mean the entire system crashed? In other words, you saw a kernel panic. Perhaps you meant an application that hung.

It was spinning. It was unresponsive. And that felt like a crash to you. Or maybe the entire GUI was frozen and wouldn't accept any user input. But perhaps the underlying subsystem was still active and you could SSH in. Make sure to be very specific when describing these types of problems.

So other times we'll find vague language in the description as well. People often will use the word nothing. They will say, oh, nothing happens, or this does nothing, or it doesn't work. Instead, describe what you did see and what you did do. For example, in the printing bug we described earlier, What does nothing happens mean?

Was the print menu item enabled? When you chose the print menu item, did the print dialog box come up? When the print dialog box came up, was your printer there? Was the print button enabled? When you clicked the print button, did the progress dialog appear, and so on?

Now, sometimes we're upset when we're writing a bug. And when we're describing it, we might be tempted to be sarcastic. But it's important to be clear and fair and objective. Think about if you were reading that bug, and you saw sarcasm. How would you feel? When you're describing the problem, it's important to be descriptive and not prescriptive. That is, you should describe the problem you saw, and not necessarily what you think the solution should be. Let's give a couple examples of this.

So say you wrote a bug, and you would say, oh, the text on this screen should be green. And what your real problem is is that you can't read the text. So instead, write the bug as, the text is unreadable. Because the engineer may choose to make the text larger, and they may choose to change the color of the background. By describing why this is a problem for you, you can help them choose the appropriate solution. This happens at the code level as well.

So say you would write a bug that says that the NSFritz should use an NSWidget matrix. Well, this is really a problem for you, that it's using a hand-rolled NSWidget list, and this isn't working very well with resolution independence. So instead, describe that. And maybe the engineer will choose to use an NSWidgetSet or an NSWidgetArray instead.

When you're describing performance bugs, it's not enough to say something is slow. For example, Safari is slow. Instead, you need to say, what was going on? What is Safari slow at doing? So maybe Safari is slow at allocating JavaScript arrays. Describe that. Include a sample or a shark profile of Safari.

And perhaps you can include some sample JavaScript which illustrates the problem. When you're writing your description, it's important to describe the facts-- that is, what you saw, did, and heard-- and differentiate that from speculation, which is what you think happened. When you write the bug, don't guess at the cause of the bug.

It's also important to describe what you expected to see and how that differs from what you actually saw. Say you wrote a bug and you said, I saw a yellow triangle. And the engineer comes and reads the bug and says, oh, exactly. They were supposed to see a yellow triangle. Maybe you were expecting to see a red triangle or a yellow square. So make sure that's very clear in the bug. Now when you're writing the bug, it's important to include only the relevant pieces of information to the bug.

If you have a workaround for the problem, include that information. It may help the engineer find the cause of the problem, and you can let people know if it's more important for them to solve another problem for which you don't have a workaround than this one for which you do. Now, sometimes people like to tell us stories when they write bugs. So you could say, oh, you know, I was over at my mom's house for Thanksgiving dinner, and I was eating this really good turkey, or it's California, maybe you're eating a tofurkey.

So, and it was really tasty. And I was talking about how good Apple is, and how great my Mac is, and all the neat things I can do with it. And we were going to show off iChat AV, and isn't iChat AV really cool? And then we went, and we started to have a chat with our Aunt Gilda in Guadalupe. But we couldn't connect, and we were very sad.

So what we have now is a good story, but how do we turn this good story into a good bug? Let's think about what the facts are here. So we are trying an iChat AV connection, but we are somewhere other than usual. What's different about where we were? Well, perhaps they have a different router.

Okay, so let's write the bug's title. iChat video chat failed with the Q-Link XJ47 router. And notice how we've included the build number at the beginning of the bug's title. We can describe what machines we are attempting the video chat with. We're attempting with an iBook G4, which had an iSight connected, and the MacBook, which has a built-in iSight. We've included the build numbers of the operating system on both machines. We've mentioned the name of the router we're using.

We included the firmware version of the router. And since we are trying really hard to impress our family, we're going to know all the steps we attempted to regress the bug. We know at home that it works with our Linksys router. We know that we tried turning the DMZ on and off. And we tried turning port forwarding on and off, and that didn't help. All of this information is really helpful.

Other things you can include in the bug: log files from iChat. On Mac OS X, 10.4.7 and later, whenever an AV connection attempt fails, iChat displays a detailed log and gives you the option of sending that to Apple. You can include that in the bug. Talk about the account types you are using. We're using Jabra accounts, we're using AIM accounts, we're using .Mac accounts.

Another thing that's very helpful to include in bugs is log files. The most obvious case of this is a crash. Whenever you have a crashing bug, it is essential to include the crash log. Now note that the location of crash logs has changed for Leopard. The crash logs are in two places. In library logs, either your home folder or on the root level of your hard drive. But in Leopard, we've moved them into subfolders by host and by application name. In Tiger and earlier, all crash logs are found in the crash reporter folder.

Now if you have a Java application that crashes, in addition to the regular app crash log, you'll also get a Java crash log in library logs Java in either of those two places. The name of the file will be something like javanativecrash_pid, and the PID of the process that crashed.

When you're filing a Java crash, make sure to include both crash logs, the Java one and the regular Mac OS X crash log. Now, in addition to crash logs, there are some other types of logs that are helpful when filing bugs. The first is a panic log. If you see a kernel panic, make sure to include the panic log in library logs.

Secondly, you'll see a lot of useful information about error messages or descriptions of activity in the system log in var log. And finally, the console log in library logs console and then your UID can contain additional information. If you ever see an exception occur, make sure to check the console log for more details. So, another thing that's very helpful to include is information about the application state. And a great way to get information about the application state is to use Shark.

Attach a shark profile of what was going on to the bug. If you want to get information about file system activity, you can get that with the fsusage command. If you want a list of what files are currently open on the system, for example, if you can't eject a volume, use the lsof command.

Shark profiles are great. If you don't have Shark, you can always use Sample. Now, Shark also has an additional configuration called Time Profile All Thread States. And if you use that, it's equivalent to a sample. It tells you what's going on even when the application is busy or appears not to be doing anything. You may have an older application on your system called Sampler in your Developer Tools folder. Don't use this. Instead, either use Shark or the Sample Command Line tool. It contains more useful information in a better format.

Now, many reporters are also photojournalists, and a photojournalist has a great camera. But the camera of choice when filing a bug report is the screenshot. So what do we do with screenshots? Well, screenshot's really helpful whenever you see a dialogue. I need to write a bug about that. Including a screenshot not only will tell people the exact wording of the dialogue, but it will explain what application or process generated the dialogue. It's not always clear.

Whenever you see any sort of a rendering or redraw problem, those are very hard to describe. It's great to include a screenshot there. The picture really is worth a thousand words. If you see something on the screen that you don't understand or don't recognize, as I mentioned earlier, anything with unfamiliar terminology, include a screenshot of that.

Now sometimes a picture's not enough. Say you need to show something in action. A movie is great. So you can use a third party utility like Snaps Pro, or you can grab your camera and film it, and import that into iMovie and attach that movie to the bug. Sometimes you can't use the screenshot facility. Perhaps it's a DVD player which prohibits it, or the machine is hung, or some other problem which looks fine in the screenshot. In this case, get out your digital camera, turn off the flash, and take a picture of the screen.

So what about URLs and error messages? If you have a screenshot of one of these, make sure to type the text from the error message or the URL into the bug. This way, people can easily search for the bug, and they don't have to retype it, and they're less likely to make a transcription error.

Now, this is a lot of information to include in your bug, and you don't have to include all of it. In fact, you probably shouldn't include all of it in most of your bugs. You need to pick just the most relevant information and put that in the bug. But there are a few things you should never include in your bugs. The first is any confidential information. If you have any trade secrets, or you have a file of social security numbers, don't attach that to a bug.

Second thing you don't want to include in a bug is someone else's application. So if you see a problem that you want to report to Apple in a third party application that's not yours, don't attach that to the bug. And definitely don't include your serial number for that application.

So when you're done writing your bug, first thing you should do is read over the bug. Make sure that it makes sense. Make sure that all the information that needs to be there is there. If there's someone else with you, have them read over the bug and see if they understand the problem and why it needs to be fixed. Now let's move on to the third component of writing a bug, steps to reproduce.

When you're writing your steps to reproduce, you need to put yourself in the engineer's shoes. Think about the types of information they're going to want to know. What will they ask for? Include lots of information. It's better to include too much information than not enough. Anticipate the types of questions they're going to ask you. You want to minimize the back and forth.

When someone has to stop to ask for more information on a bug, It means they have to temporarily put your bug aside and work on something else before they can come back to it. This is essential, especially to get back quickly so you catch engineers early in development cycles.

When you're writing the steps to reproduce, think about when this problem occurs. This is a problem that will occur after you log out. This is a problem that will occur after you restart the computer. This is a problem that can occur after you shut the computer off and turn it back on.

Make no assumptions about what people will know about your product. It's important to be very specific in these steps to reproduce. When you're describing the steps, you want to include details of what you did, not high-level actions. So instead of saying, "Oh, I opened a file," well that could mean many things. It could mean you double-clicked on a file.

It could mean you selected the file and pressed Command-O. It could mean you selected the file and pressed Command-Down Arrow. You could have used the Open command in the terminal. Instead, make sure to say exactly what you did. "I double-clicked on the file." While these things, different details, could result in the same apparent action, they could go through different code paths. So it's important to be very specific.

Think about the account type of the user in which you experience the problem. Where is the home directory? Is this a regular local home? Is this a portable home on a FireWire disk or an iPod? Is this a network home? What type of user is it? Is it a standard user, an admin user, or is it a managed client user? Think about all the steps it would take to reproduce this problem on a brand new machine that you just took out of the box.

So let me give you a couple of examples that demonstrate the importance of detailed steps to reproduce. So one time I had a problem with the installer. When I got three quarters of the way through the installation process, the installer would hang. And I wrote up a bug, and I sampled the installer, and I included the sample in the bug.

And I included the name of the packages where I saw the problem, and I attached them. And I wrote detailed steps of what I went through. But the installer engineer couldn't reproduce the problem. So finally, he came over to my office, and he watched me reproduce the problem. And I watched him not reproduce the problem. And we did this for about 15 minutes. And finally, we figured out what was going on.

It turned out that I was using the keyboard to navigate. That is, I pressed Tab and the Spacebar, and he was using the mouse. So if you're using any sort of accessibility features, keyboard navigation or voiceover, make sure to include that sort of information in your steps to reproduce.

Now here's another example. Your dad wrote me, and he said, Paul, I have this web page in Safari, and it doesn't look right. And he sent me the URL of the web page, and he sent me a screenshot. And indeed, there were rendering problems in the web page.

When I went to the page, it looked fine. Finally, after some discussion, we discovered that he had turned on the minimum font size preference in Safari. And when I checked that box, I too saw the problem. So make sure to include information about preferences that you've changed from the defaults.

When people can reproduce your problems, it's much easier for them to investigate them, understand them, and fix them. So now let's take a look at the fourth component of a good bug report, regression. What is regression? Regression is when you describe where the problem occurs and where it doesn't occur. What builds of the operating system, what architecture, what types of hardware?

Say you have a problem and you will see it not on an iBook G4, but only on a MacBook. Include that information. If you have any FireWire or USB devices attached to the machine, try removing them. Does the problem still occur? If it does, great. If the problem doesn't occur, remove devices one at a time until you can narrow it down to the device that needs to be present for the problem to occur.

So now, next, configuration. It's important to include information about the machine configuration on which the problem occurred. And the best way to do that is with a system profile. Go into System Profiler and save an SPX file and not an RTF file. The SPX file contains information in a format that can be easily read and parsed. And when you're saving the report, remember two things. First, include a full report with all of the information. And second, make sure to take the profile on the machine which demonstrates the problem, and not just the machine on which you are reporting the bug.

If you have multi-monitor setup, include that information. Talk about how many monitors you have. Talk about whether they're in mirroring or extended desktop mode. Is this a PowerBook, iBook, or MacBook running in lid-closed mode? What's the physical layout of all these displays? Which display has the menu bar? What about application information? When you see a problem pertaining to a particular application, you need to include detailed information about that application.

It's not enough to say, I was using the latest version of this application, or the current version, or the version that shipped with my computer. You need to be very specific. For example, if you're writing a bug app involving Photoshop, don't just say, I was using Photoshop. I was using Photoshop CS. Say, I was using Adobe Photoshop CS2 version 9.0.1.

Now when you have an Apple application, you'll see the build number in the About box. For example, in Preview, you can see Preview 3.0.6, Build 409. Include this information in the bug as well. Remember that versions will change in software updates or security updates, and the version of a framework could differ from the version of the application. What about third party software? Well, say you have some third party software installed on your machine, like kernel extensions, or plugins, haxies, or input managers.

It's very useful to regress the bug by removing these additional pieces of software and seeing if the problem occurs. If the problem goes away, it's very likely the problem lies in the other piece of software. And if the problem remains, that's one less step engineers investigating the problem have to take.

And what about reproducibility? Well, this one's easy. Is this a problem that you see all the time? Is this a problem that is intermittent? Is it a problem you see sometimes or rarely? Or is this a problem you saw only once and are unable to reproduce? So let's summarize the best practices when writing a bug description. Include a concise summary of the problem.

Describe the expected results and how they differ from the actual results. Include detailed steps to reproduce. Include regression information, what builds and machines the problem does and doesn't occur on. Include the configuration information of the machine where you saw the problem. And describe how reproducible the problem is.

So, since you've been such a good audience, I have two bonus sections for you. So we have some four pieces of application-specific information for filing bugs on these applications, and we have some tips for how to set up your Mac for debugging. So let's go over the application-specific information.

So first, Xcode. Since you're all developers, you probably spend a lot of time in Xcode. How do you file a really good bug against Xcode? Well, a great thing to include is your project file. If you have a problem that's occurring with this particular project, that is really helpful.

But sometimes you can't include your project file or all the attached files with it. In that case, there's some other pieces of data that are really helpful. You can include a test application, which demonstrates the problem. Perhaps you can include a test application that demonstrates the problem. Perhaps you're seeing a problem in a particular API.

What about a failing unit test? If you're using OC unit and you have a unit test, include that with the bug. If you have any sample code at all, which demonstrates the problem, even if it's just three lines, it's all that's needed, great. Include that in the bug. What about test data? Say you have a problem that occurs only with particular input. In that case, include this test data with your bug.

When you're writing up bugs on Xcode, it's important to include as much evidence as possible. Take precise samples. This is really helpful when you see an intermittent or hard-to-reproduce problem. Even circumstantial evidence is helpful. Any screenshot or any log or any... those things are really helpful. Even if the engineer cannot reproduce the problem, they might be able to prevent it from recurring in the future.

Make sure to include detailed information about the version of Xcode you're using. In the About menu, in addition to the regular version of Build, you'll see three pieces of information. Include all of them in your bug. Now what about if you've installed some additional developer tools, say from Fink or Darwin ports? In that case, it's important to include detailed version information about things like GCC and GDB and LD. If you have any sort of build or linking failure, include the full build log in your bug.

Sometimes removing temporary files can make the problem go away. So move these files aside and see if that helps. You'll see temporary files in library caches Xcode, and then of course there's the build directory inside your projects directory. If moving these files aside fixes the problem, then it's helpful to attach these to the bug.

If you have a problem and ZeroLink is enabled, try disabling ZeroLink and see if that makes the problem go away. Now here's application specific information tip number two: filing bugs against Java. If you have a Java application that's hanging, you can get, in addition to the regular sample, you can get a sample of the Java native threads by typing kill-quit and the process ID of the Java process. This could go in many places.

If it's a command line application, you'll see it in the terminal. You could see it in the console log. You could see it in Safari's Java console, if you have enabled that, which we'll talk about later. You could see it in Xcode's run log, if you're running this in Xcode. Now, it's important to include both the native threads from sample or shark, and the Java threads, which you get from kill-quit.

If you're using a developer preview of Java, you can prefix the bug title with the version of Java. For example, Java 6 release 1 DP4. And the Java team finds it very helpful if you regress the bug against the shipping version of Java. So this is a problem that occurs only in Java 6, and not in Java 1.5. And speaking of Java 1.5, it's important to include detailed version information about the version of Java you have on the system.

The version number of Java looks something like this. It's 1.5.0 under bar 06. If you use Perl, you'll see something like that as well. But this is not all of the version information for Java. There's even more. The best thing to do is type java-version, and you'll get some output that looks like this. And if you include all of this, this way the Java team knows exactly what version of Java you have, and where you saw the problem.

I think that's really helpful is a JUnit unit test. If you're a Java developer and you can create a unit test that demonstrates the problem, there are two benefits to this. Number one, it makes sure the Java team is seeing the same problem that you're seeing. And number two, they have the opportunity to include that test in their test suite, which would make sure the problem doesn't recur in the future.

All right, third application-specific tip. What about when you're writing bugs against your own applications? Well, you have to assume that the engineers reading the bug have never seen your application. They don't know what it does. They don't know your terminology specific to your application's domain. So you don't want to write about something like receipts, or debits, or credits, or reconciling. Instead, you need to be very specific in your steps and tell people what menu items to click, and what buttons to press, and so on.

Next, you want to make sure the engineers have access to your software to reproduce the problem. Even if it's a problem that you see in the system that only occurs with a particular piece of software, they're going to need that piece of software to reproduce the problem. And when you give them the software, make sure they can use it.

So if your software requires a dongle, if it requires any serial number, any sort of activation, any username or password, make sure they have access to that. And make sure also, this is not a trial version of the software that will expire by the time the engineer gets around to investigating the problem.

All right, Safari. Here's some tips from web developers who are writing bugs against Safari. So the first thing you can do is take a look at this web page that demonstrates the problem in other web browsers. Does this problem still occur? OK. Well, if the problem goes away, perhaps the site is doing browser detection.

And you can have Safari spoof its user agent and pretend to be something else. So how do we do this? First, we need to issue a defaults write command. So go into terminal and type defaults write com.apple.safari include debug menu 1. Then launch Safari and you'll see the Debug menu.

At the bottom of the debug menu is the User Agent submenu. You can pretend to be Firefox or Internet Explorer for Windows, or an old version of Netscape, for example. Now go to this unfriendly site, and see if the problem goes away. If it goes away and they are using browser detection, write up a bug.

Say that this site is using browser detection. What if the problem still occurs? Maybe they let you in this time, but the site doesn't work. They could be using browser-specific HTML or JavaScript, something like document.all, an old method in DHTML. In this case, still write a bug. The Safari team will work with the website developer to make sure that the site uses web standards and is compatible with all web browsers.

What about plugins? Say you have a particular plugin. It's PowerPC only. Remember, if you have a new Intel Mac, that plugins aren't loaded under Rosetta. So if you need to use this plugin, you'll need to open Safari in Rosetta. If you have a problem related to Flash, make sure to include detailed version information about Flash. Which dot release of Flash do you have? Do you have Flash 8, or do you have Flash 9 beta?

If you see a problem on a site that's in a foreign language, you need to include step-by-step instructions for the Safari team. You can't assume they speak any language other than English, so take pictures of the site and circle the items they need to click, or what they need to type to reproduce the problem.

A great thing to include is a copy of the web page. If you go to File and Save As and save a web archive, attach that to the bug. This is really helpful for dynamic sites that change all the time. This way you can ensure the Safari engineer sees the same version of the site that you saw.

Now, if you're a web developer, you might want to download the latest nightly build of WebKit. You can check the problem there. This is a problem that's been fixed in WebKit. If so, it might be fixed in the next version of Safari. This is a problem that only occurs in WebKit and not in the currently shipping version.

Well, you definitely want to write that bug up, because the Safari team takes compatibility very seriously. If a site worked in Safari, it needs to continue to work in Safari. So if you're a web developer, it's great to use the WebKit nightly. It's certainly not required to send a bug to Apple, though.

When you're writing a bug about a particular website, it's important to include the full URL to the page or resource that demonstrates the problem, not just the name of the site. If the site requires a username or password, it's important to note that in the bug. If this is a site you own, you can create an account for the Safari team. It's really helpful. But if this is a banking or a brokerage site, don't include your own username and password and send that to the Safari team. Instead, provide them with contact information for the site owner so they can obtain their own account.

Now what about firewalls? Well, say you have a problem on a page that's behind a firewall, and you want the Safari team to look at it. They probably can't. You need to either find some way to grant them access, or you can save a copy of the page using the Web Archive method we discussed earlier.

What about proxy servers? If you're behind a proxy server, or a problem occurs with a particular proxy server, include that information in the bug. Talk about the version of the proxy server, the name of the proxy software, what platforms it's running on. All of that information is really helpful. The gold standard for a Safari bug is a reduction. What is a reduction? A reduction is the minimum piece of the web page that demonstrates the problem.

Creating a reduction is an iterative process where you remove pieces of the page until you have the smallest possible piece that demonstrates the problem. Sometimes it's trial and error, and you'll cut pieces out and put them back in until you find something that works. A reduction is certainly not required. It's really helpful, and the Safari team will love you if you do this. So, last piece of information on how to file a bug is how do you set up your Mac for debugging?

Well, the first thing you're going to want to do is enable SSH. When you enable SSH, you can log into your machine remotely if you need to do some debugging when you're running a full screen application, or playing a game, or a video, or perhaps the GUI is hung or unresponsive. You can do this by going to System Preferences, and then Sharing, and checking the box beside Remote Login.

Next thing you want to do is install a couple of pieces of software. When you install the developer tools, you get things like GDB, which are really helpful for debugging. If you install the Chud tools, that's where you get Shark, which is super handy whenever you see any sort of a performance problem or a hang.

Next thing you want to do is take notes. Reporters have their reporter's notepad. And in this case, you really do want to write things down on a piece of paper. When your Mac is in the state where it's experiencing the problem, you might not be able to get at the information you need.

So what do you want to write down? First thing you want to write down is the IP address of your machine. So that way you can log in remotely. Now, maybe the IP address is changing or it's gone away. Perhaps your DHCP server went down. In this case, writing down the Bonjour name of the machine is really helpful. And that way you can still log in. In other cases, the host name will be helpful if... If that stays the same.

And finally, you can get the hardware address of both the Ethernet and airport interfaces. This information is really helpful for diagnosing any sort of a network problem. Now, you never know when a problem is going to occur. And if you're ready to debug it when it does happen, you don't have to wait around and reproduce it. And this is especially important for any sort of intermittent or hard to reproduce problem.

So now, let's move on to the fifth question. Where do you file a bug against an Apple product? Well, there are many places in Apple products where you can provide feedback to Apple. When you see a crash, there's the crash reporter dialog. And if you click Report and Send to Apple, that goes right to us. If you're a WebKit developer, you could be using Bugzilla to write out bugs. If you're a Safari user, you might click the Report Bug to Apple button in Safari.

In applications like Keynote and iWeb, you could use the Provide Feedback menu item. You might have an email address you use to communicate with Apple. But in all of these cases, you should use Apple Bug Reporter. When you write a bug using Apple Bug Reporter, you get a bug number, and you can use that to track the problem. If people need more information about it, they can contact you. If you find additional information about the problem, you can add that to the bug. So where do you get Apple Bug Reporter? It's bugreport.apple.com.

So, finally, when is it appropriate to file a bug? People ask, should I file a bug? The answer to that question is yes. Now, there are two objections to this. Number one, people will say, this bug is trivial. It only affects me. Maybe it only affects two people. Maybe it's just a typo. It doesn't matter. File a bug.

The second objection that people have is that this bug is obvious. They say, Paul, surely Apple has 12 copies of this bug already. Should I file this bug? Yes. File the bug. So, why would you do that? Well, it's much better to have three reports of a problem than to have no reports of the problem. And secondly, having multiple reports helps engineers prioritize. They know how widespread the problem is.

So, to sum up... First thing a reporter does is a reporter gathers tools. So they'll have their notebook. In your case, you're going to have your Mac, and you're going to have the right software installed. You're going to have remote login enabled. A great reporter will chase the story. They're going to interview witnesses, and they'll gather documents.

And they'll run around town. In your case, you're going to get crash logs, and you're going to include screenshots, and you're going to have All sorts of log files which are relevant to the problem. And a great reporter writes clearly. You need to write your bug report very clearly so people reading it understand the problem.

So, why again do we write bugs? Because writing bugs can be very rewarding. You get great software, and when you have great software, you have really happy customers. So, when you go home today, I want you to write a bug. I want you to employ all the practices we talked about today. Make sure your title is clear and concise. Make sure you summarize the problem well. Make sure you include detailed steps to reproduce.

Make sure you include all the relevant configuration and regression information. So think about whatever is bothering you. Think about some suggestion you have for an improvement, and write that up. So go ahead and take that step, the first step, towards solving the problem. Now while great bug reporting won't win you the Pulitzer, the great software that results could get you one of these. Thank you very much.