Updated Metrics and Measurement section in API course to remove scoring aspect
- Why remove the scoring aspect of criteria?
- Consolidating the checklists
- A lightweight checklist
- Best outcome of the quality checklists
Why remove the scoring aspect of criteria?
I initially thought that I could give each criteria a score, weight each score, and then give an overall grade or score for developer docs. But as I did that recently at work, revisiting each criteria and its score that I’d assigned one year earlier with a new annual assessment and score, the scores felt silly. I don’t think they worked well and seemed meaningless. So in the spirit of getting rid of things that don’t work, I removed that scoring section from my API doc course. It seems that achieving that elusive metric about how to quantify tech comm work is still something I haven’t figured out.
Consolidating the checklists
Also, I originally had a first-level checklist and a second-level checklist. The idea was that you could only assess some aspects of a developer portal after becoming more familiar with the content. However, I consolidated both checklists into a single checklist for the sake of convenience. I think that most people can naturally exclude criteria they don’t have enough information about. I’d rather have all the criteria in one place.
A lightweight checklist
Finally, I created a lightweight checklist that selects only two criteria from each category. I tried to select what I felt would be the most important criteria, but really it came down to a purely arbitrary decision on my part.
Best outcome of the quality checklists
I made these updates after a recent podcast where we discussed the checklists. The podcast isn’t yet published, but stay tuned for more on the topic of API documentation quality checklists. I’ve found that the best outcome of the checklists is my “Observation 3” that I published in 10 observations after using my API documentation checklist in a real scenario, about a year ago:
Observation 3: The checklist does a wonderful job at opening up a critical look at API docs
Although one of my motivations was to create metrics around quality, as I did an initial assessment of the docs I’m working on, I found that the checklist does a wonderful job in opening up a critical view and analysis of the documentation. It opens up a more critical perspective that lets you slice up different dimensions and aspects of the docs. Regardless of whether the scoring and quantitative metrics goes anywhere, the set of characteristics provides an incisive way to investigate doc quality.
If you’re looking to audit a site and get a sense of what should be improved, try going through the checklist of questions. You’ll have a lot to think about this way, and it provides a more structured approach than random observations.
About Tom Johnson
I'm an API technical writer based in the Seattle area. On this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, AI, information architecture, content strategy, writing processes, plain language, tech comm careers, and more. Check out my API documentation course if you're looking for more info about documenting APIs. Or see my posts on AI and AI course section for more on the latest in AI and tech comm.
If you're a technical writer and want to keep on top of the latest trends in the tech comm, be sure to subscribe to email updates below. You can also learn more about me or contact me. Finally, note that the opinions I express on my blog are my own points of view, not that of my employer.