Search results

Should you add your testing scenarios into your documentation?

Series: Testing your documentation

by Tom Johnson on Jul 27, 2015
categories: technical-writing

When you test a product, you usually put together simple test scenarios that include actual values and other specific details. You can repackage these test scenarios into documentation-based tutorials that help users understand how to use the product. By including specific values and instruction with a defined end, new users can better understand how to use the product for different use cases and scenarios.

This is another post in my series on testing documentation.

What I mean by testing

First, I want to add a quick clarifying note about terms. By “testing” docs, I’m not referring to the same rigor or perspective that QA teams apply to code. QA teams look at more than whether the product meets basic expectations. QA teams try to break the code by scaling the load, or they set up automated tests that run through hundreds of scenarios in scripts, or they test the code on dozens of different simulated devices.

The testing I’m talking about is more like a simple user test, a test that approximates how the actual users might use the product. In these simple tests, you set up a small scenario to test something out in a practical situation.

Sample test case

An example might help clarify things. A typical documentation task might look like this:

Generating a Widget

  1. Click the Widget button.
  2. Configure the system with the settings you want:
    • Height: Does this...
    • Width: Does that...
    • Type: Optionally does this and that.
  3. Copy the code to your directory and publish.

In contrast, a typical test case might look like this:

Generating a 300x200px ABC Widget

  1. Click the Widget button.
  2. Input value 123 into the system.
  3. Configure the system with the following settings:
    • Height: 200px
    • Width: 300px
    • Type: Visual
  4. Copy the widget into directory 3 and restart the service.

You should see the ABC-123 widget appear filling a space 300x200px wide.

What’s the difference? The test case uses actual values in a more realistic scenario to produce a specific and measurable end. The test case might also incorporate specific business logic that is usually left out of the generic task in the documentation.

Tutorials help users learn to use the product

There’s a funny picture going around Twitter:

It’s actually talking about tutorials, not tasks. But tasks are probably even a better use case to illustrate the point. A task in the documentation might explain how to achieve a certain end. But without defining any of the values or specifics needed to achieve that end, the tasks can leave users scratching their head, looking at the two primitive circles instead of the finished owl.

Tutorials include all those other steps

Documentation often breaks up all tasks into separate topics (an effect of topic-based authoring). In contrast, the tutorial brings them together, or at least links out to them in appropriate places, so that new users aren’t left wondering what else they need to do before or after in order to get the result they want. Further, tutorials help guide users to understand the results available to them.

In short, test-based tutorials tell the user what values and options to select each step of the way, whereas doc tasks usually omit this level of detail. These omitted details can be useful to new users who are less familiar with the product.

Tutorials are especially useful for API documentation

With API documentation, most of documentation consists of reference topics that list the endpoints, parameters, and other details associated with the API in a general way. You don’t have the same number of tasks that GUI documentation has. As a result, it’s less clear how people are supposed to actually use the API.

Here’s an example. A typical reference topic in API doc might look like this:

ACME Endpoint: acmegluetrap

Description: Catches roadrunners with a sticky glue.

Parameters:

  • power: How powerful the glue should be (integer)
  • duration: How long the glue should remain sticky (integer)
  • amount: The amount of glue to apply (integer)

In contrast, with a tutorial, you might write it like this:

Catch a roadrunner with glue

In this tutorial, you will use the acmegluetrap endpoint to catch a roadrunner with a sticky glue.

  1. Set up authorization keys. (link...)
  2. Copy the ACME endpoint.
  3. For each parameter, enter the following:
    • power: 10
    • duration: 300
    • amount: 50

Power is how powerful the glue should be. Duration defines the length of time the glue remains sticky. And amount refers to how much glue you're using. All values are integers.

The endpoint looks like this:

http://myapi.acme.com/apikey/12345/acmegluetrap?power=10&duration=300?amount=50

With the Authorization headers, the cURL request looks like this:

curl 'http://myapi.acme.com/apikey/12345/acmegluetrap?power=10&duration=300?amount=50' -H 'Authorization: 43uo3jkfp' -H 'Accept: application/json'

The response should be as follows:

{
"adhesionResult":{  
       "roadrunnersTrapped":"3",
       "glueAmountApplied":"30",
       "effiencyRate":"8",
       "powerLevel":"titan"
    }
 }

Why do API docs lack task-based topics?

The task-based topics that you usually find in GUI documentation are often missing in API documentation. Why aren’t they present? Well, you provide the endpoint, and you explain the parameters. The idea is that users can simply choose how they want to use the endpoints. There’s no need for actual beginning-to-end scenarios. Right?

Maybe for advanced users, sure. But beginning users can benefit from seeing the larger picture. It might be helpful to provide a series of tutorials that walk through different use cases or business solutions. (Similar to Getting Started type tutorials, but getting started with different use cases and scenarios.)

If you’re already testing the docs with certain values, it wouldn’t be hard to publish those test cases in the documentation as a kind of tutorial that helps users through actual steps and tasks with the product. They won’t be staring at a list of endpoints with no real idea of how to use them in actual business scenarios.

However you decide to include them, I definitely think that tutorial topics build from test cases have a valid place in documentation.

About Tom Johnson

Tom Johnson

I'm an API technical writer based in the Seattle area. On this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, AI, information architecture, content strategy, writing processes, plain language, tech comm careers, and more. Check out my API documentation course if you're looking for more info about documenting APIs. Or see my posts on AI and AI course section for more on the latest in AI and tech comm.

If you're a technical writer and want to keep on top of the latest trends in the tech comm, be sure to subscribe to email updates below. You can also learn more about me or contact me. Finally, note that the opinions I express on my blog are my own points of view, not that of my employer.