Adobe Robohelp

Get new posts delivered straight to your inbox.

Subscriber count: 3,505

Stitcher radio

follow us in feedly

Want more tech comm blogs to follow? See my Tech Comm Collection of Blogs on Feedly.

Adobe Robohelp

Podcast: How do design, length, and relevance affect how people use API reference docs -- interview with Bob Watson

Jul 30, 2015 • api-doc, podcasts, stitcher

Bob Watson recently finished a PhD with research that examined how the design and content of API reference docs affects the user's performance. In this podcast, I talk with Bob about his findings and his other research interests, primarily around goal testing to measure documentation's effectiveness.

Listen to the podcast

You can download the MP3 file here.


Bob was a software engineer for 17+ years. He decided to get a PhD from the University of Washington’s Human Centered Design and Engineering because he felt that if the documentation for his software were better, the adoption of the software would increase. This led him to pursue research in technical communications.

He soon found that many other factors besides documentation contribute to product adoption. Still, the tech comm domain held his interest, as it contained many interesting problems to solve. For his PhD, Bob examined several questions related to API documentation:

  • How does the design of API reference documentation affect the user experience?
  • How does the relevance of the content on API reference documentation affect the user experience?
  • How does the length of the content on API reference documentation pages affect the user experience?

He found that his changes to the topics’ visual design didn’t affect performance — whether your API documentation has lots of design elements or not didn’t affect how quickly people found the information to achieve a goal. A strong visual design did, however, affect people’s perception of the documentation’s credibility and professionalism.

He also found that whether reference topics contained relevant information or not, people spent the same amount of time evaluating the topic for relevance (about 44 seconds).

Finally, he found that topic length influenced performance. Shorter topics were quicker to assess, but longer topics increased the user’s perception of the topic’s credibility and professionalism.

Goal testing documentation

We also talked about testing. Bob said his background as a software engineer, a context in which code is heavily tested, led him to wonder why documentation wasn’t also tested with the same degree of thoroughness. He found that testing documentation content was much more difficult. It’s harder to determine whether documentation is achieving its goal, especially with API reference documentation, since a user may not implement the knowledge gained from the content until later, such as when the user writes code for an application he or she is building.

Bob noted several practical approaches to testing. One method is to get a new engineer in your company to test your documentation. He recommends designing 10- to 30-minute tests focused on a specific task. You can encourage people to participate by giving them Starbucks cards or other perks. The outcome of nearly every test, Bob noted, is that you realize that some assumption you had was wrong.

While you can enlist other non-engineering colleagues at your work to test your API documentation, you have to recognize the limits of your subjects. If you’re asking a user (such as a product manager) to do something he or she wouldn’t normally do, the results may not provide you with the information you need to optimize your information.

Product-market-audience framework for analysis

One of the frameworks that Bob mentioned in making decisions about documentation is a product-market-audience framework. Whether you decide to include a strong visual design in your docs, or add a lot of detail to handhold users, etc., really depends on how these three factors intersect: product, market, and audience.

If your product is new in the market, and users are unfamiliar with it, then you probably want to add more detail in your product’s documentation. If your documentation’s purpose is to help sell the product and increase credibility for your product in the community, then you might want to implement a strong visual design. If your product is a familiar technology in the market, you may not need as much doc content. By analyzing the product, market, and audience, you can make decisions about what to emphasize in your documentation.

Resources to learn more

Here are some links for some of the resource mentioned in the podcast:

follow us in feedly

Get new posts delivered straight to your inbox.

Subscriber count: 3,505

About Tom Johnson

Tom Johnson

I'm a technical writer based in the California San Francisco Bay area. Topics I write about on this blog include technical writing, authoring and publishing tools, API documentation, tech comm trends, visual communication, technical writing career advice, information architecture and findability, developer documentation, and more. If you're a professional or aspiring technical writer, be sure to subscribe to email updates using the form above. You can learn more about me here.