Should you add your testing scenarios into your documentation?
<div class="btn-group"> <button type="button" data-toggle="dropdown" class="btn btn-warning dropdown-toggle">testing_documentation<span class="caret"></span></button> <ol class="dropdown-menu"> <li> <a href="/2015/07/09/should-quality-assurance-test-documentation/">1.1 Should QA test documentation?</a> </li> <li> <a href="/2015/07/10/communicating-feedback-from-testing/">1.2 Communicating feedback from testing documentation</a> </li> <li> <a href="/2015/07/14/should-technical-writers-care-about-more-than-documentation/">1.3 Should technical writers care about more than documentation?</a> </li> <li class="active">1.4 → Should you add your testing scenarios into your documentation?</li> <li> <a href="/2015/07/28/how-to-test-content-beyond-your-skill-level/">1.5 How do you test content that's beyond your skill level?</a> </li> </ol> </div>
This is another post in my series on testing documentation.
What I mean by testing
First, I want to add a quick clarifying note about terms. By “testing” docs, I’m not referring to the same rigor or perspective that QA teams apply to code. QA teams look at more than whether the product meets basic expectations. QA teams try to break the code by scaling the load, or they set up automated tests that run through hundreds of scenarios in scripts, or they test the code on dozens of different simulated devices.
The testing I’m talking about is more like a simple user test, a test that approximates how the actual users might use the product. In these simple tests, you set up a small scenario to test something out in a practical situation.
Sample test case
An example might help clarify things. A typical documentation task might look like this:
In contrast, a typical test case might look like this:
What’s the difference? The test case uses actual values in a more realistic scenario to produce a specific and measurable end. The test case might also incorporate specific business logic that is usually left out of the generic task in the documentation.
Tutorials help users learn to use the product
There’s a funny picture going around Twitter:
Every coding tutorial ever written. pic.twitter.com/t8AKl7EAge— Rich Rogers (@RichRogersHDS) July 25, 2015
It’s actually talking about tutorials, not tasks. But tasks are probably even a better use case to illustrate the point. A task in the documentation might explain how to achieve a certain end. But without defining any of the values or specifics needed to achieve that end, the tasks can leave users scratching their head, looking at the two primitive circles instead of the finished owl.
Tutorials include all those other steps
Documentation often breaks up all tasks into separate topics (an effect of topic-based authoring). In contrast, the tutorial brings them together, or at least links out to them in appropriate places, so that new users aren’t left wondering what else they need to do before or after in order to get the result they want. Further, tutorials help guide users to understand the results available to them.
In short, test-based tutorials tell the user what values and options to select each step of the way, whereas doc tasks usually omit this level of detail. These omitted details can be useful to new users who are less familiar with the product.
Tutorials are especially useful for API documentation
With API documentation, most of documentation consists of reference topics that list the endpoints, parameters, and other details associated with the API in a general way. You don’t have the same number of tasks that GUI documentation has. As a result, it’s less clear how people are supposed to actually use the API.
Here’s an example. A typical reference topic in API doc might look like this:
In contrast, with a tutorial, you might write it like this:
Why do API docs lack task-based topics?
The task-based topics that you usually find in GUI documentation are often missing in API documentation. Why aren’t they present? Well, you provide the endpoint, and you explain the parameters. The idea is that users can simply choose how they want to use the endpoints. There’s no need for actual beginning-to-end scenarios. Right?
Maybe for advanced users, sure. But beginning users can benefit from seeing the larger picture. It might be helpful to provide a series of tutorials that walk through different use cases or business solutions. (Similar to Getting Started type tutorials, but getting started with different use cases and scenarios.)
If you’re already testing the docs with certain values, it wouldn’t be hard to publish those test cases in the documentation as a kind of tutorial that helps users through actual steps and tasks with the product. They won’t be staring at a list of endpoints with no real idea of how to use them in actual business scenarios.
However you decide to include them, I definitely think that tutorial topics build from test cases have a valid place in documentation.