Should you add your testing scenarios into your documentation?

by Tom Johnson on Jul 27, 2015
categories: technical-writing

This is another post in my series on testing documentation.

What I mean by testing

First, I want to add a quick clarifying note about terms. By “testing” docs, I’m not referring to the same rigor or perspective that QA teams apply to code. QA teams look at more than whether the product meets basic expectations. QA teams try to break the code by scaling the load, or they set up automated tests that run through hundreds of scenarios in scripts, or they test the code on dozens of different simulated devices.

The testing I’m talking about is more like a simple user test, a test that approximates how the actual users might use the product. In these simple tests, you set up a small scenario to test something out in a practical situation.

Sample test case

An example might help clarify things. A typical documentation task might look like this:

In contrast, a typical test case might look like this:

What’s the difference? The test case uses actual values in a more realistic scenario to produce a specific and measurable end. The test case might also incorporate specific business logic that is usually left out of the generic task in the documentation.

Tutorials help users learn to use the product

There’s a funny picture going around Twitter:

Every coding tutorial ever written. pic.twitter.com/t8AKl7EAge

— Rich Rogers (@RichRogersHDS) July 25, 2015

It’s actually talking about tutorials, not tasks. But tasks are probably even a better use case to illustrate the point. A task in the documentation might explain how to achieve a certain end. But without defining any of the values or specifics needed to achieve that end, the tasks can leave users scratching their head, looking at the two primitive circles instead of the finished owl.

Tutorials include all those other steps

Documentation often breaks up all tasks into separate topics (an effect of topic-based authoring). In contrast, the tutorial brings them together, or at least links out to them in appropriate places, so that new users aren’t left wondering what else they need to do before or after in order to get the result they want. Further, tutorials help guide users to understand the results available to them.

In short, test-based tutorials tell the user what values and options to select each step of the way, whereas doc tasks usually omit this level of detail. These omitted details can be useful to new users who are less familiar with the product.

Tutorials are especially useful for API documentation

With API documentation, most of documentation consists of reference topics that list the endpoints, parameters, and other details associated with the API in a general way. You don’t have the same number of tasks that GUI documentation has. As a result, it’s less clear how people are supposed to actually use the API.

Here’s an example. A typical reference topic in API doc might look like this:

In contrast, with a tutorial, you might write it like this:

http://myapi.acme.com/apikey/12345/acmegluetrap?power=10&duration=300?amount=50

curl 'http://myapi.acme.com/apikey/12345/acmegluetrap?power=10&duration=300?amount=50' -H 'Authorization: 43uo3jkfp' -H 'Accept: application/json'

{
"adhesionResult":{  
       "roadrunnersTrapped":"3",
       "glueAmountApplied":"30",
       "effiencyRate":"8",
       "powerLevel":"titan"
    }
 }

Why do API docs lack task-based topics?

The task-based topics that you usually find in GUI documentation are often missing in API documentation. Why aren’t they present? Well, you provide the endpoint, and you explain the parameters. The idea is that users can simply choose how they want to use the endpoints. There’s no need for actual beginning-to-end scenarios. Right?

Maybe for advanced users, sure. But beginning users can benefit from seeing the larger picture. It might be helpful to provide a series of tutorials that walk through different use cases or business solutions. (Similar to Getting Started type tutorials, but getting started with different use cases and scenarios.)

If you’re already testing the docs with certain values, it wouldn’t be hard to publish those test cases in the documentation as a kind of tutorial that helps users through actual steps and tasks with the product. They won’t be staring at a list of endpoints with no real idea of how to use them in actual business scenarios.

However you decide to include them, I definitely think that tutorial topics build from test cases have a valid place in documentation.

About Tom Johnson

I'm an API technical writer based in the Seattle area. On this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, AI, information architecture, content strategy, writing processes, plain language, tech comm careers, and more. Check out my API documentation course if you're looking for more info about documenting APIs. Or see my posts on AI and AI course section for more on the latest in AI and tech comm.

If you're a technical writer and want to keep on top of the latest trends in the tech comm, be sure to subscribe to email updates below. You can also learn more about me or contact me. Finally, note that the opinions I express on my blog are my own points of view, not that of my employer.