First-level checklist for API documentation
This section continues from the previous page, Measuring progress against documentation quality goals.
As indicated earlier, my goal is to create a practical guide for measuring quality. Instead of looking at docs against a list of general, abstract criteria, I encourage creating a list of characteristics that, if fulfilled, should lead to all the more high-level categories of the rubric (clarity, accuracy, findability, etc.) automatically. Each of the characteristics must be specific, actionable, and unambiguous in how it would be implemented in your docs. This section gets into the nitty-gritty details of the characteristics.
- Doc scope
- Two levels of assessment
- First-level documentation checklist
Before jumping into the criteria, consider the scope. If you’re working on a developer portal, chances are you don’t own the entire portal. You might just own one little section of the portal. That’s okay. You can limit your review to just the scope that you own. Granted, the user journeys might extend beyond this scope, but start with your stewardship first. The last thing you want to do is start a war with other authors by identifying all kinds of issues with their content (at least not before you address your own issues first).
Two levels of assessment
Another consideration is just how much you can assess without more familiarity with docs. You can’t know if the steps are accurate unless you go through the steps. You can’t know if the docs are consistent unless you’ve read all the documentation. You can’t know if the code works unless you can run it in a test environment. It might take more than a year working with the docs to be able to make these kinds of assessments. You shouldn’t have to spend a year working with the docs before you can provide an initial assessment of some kind.
All too often, you might find yourself situated in a new group, with many pages of documentation before you. What if you’re trying to measure your impact on the docs from day one? If you can’t complete the assessment until a year into the work, after you’ll have already contributed so much to the docs, you won’t have a clear benchmark against which to assess your contributions and impact.
Because some criteria can be assessed quickly, while other criteria requires you to be much more familiar with the documentation, I provide two levels of assessment — a first-level checklist and a second-level checklist. The two lists are distinguished between quick assessments you can make without being familiar with the docs versus more in-depth assessments that require more familiarity.
In this section, I’ll present a first-level checklist, and in the next topic, I’ll present a second-level checklist.
First-level documentation checklist
The following checklist is a first-level checklist that doesn’t require much familiarity with the docs. The checklist’s criteria are in no particular order except those that seem more important (hence have a greater weight) appear first. Also, the list shouldn’t be seen as definitive or as a foolproof recipe for perfect documentation. And some points might apply more than others, depending on your product, domain, and audience. Overall, these are criteria/characteristics that will likely lead to a better experience with developer docs.
- Findable in search. The content is indexed in a general search engine and findable when you create searches with the product name and some key tasks. weight: 1
- Site-specific search available. The doc site has a site-specific search that lets users search within the documentation site itself. weight: 3
- Developer portal hub context. If you have lots of documentation relevant to users, the documentation is organized in a developer portal of some kind. weight: 3
- UI pages link to docs. If there’s a UI (such as with an administrative console), links within the UI connect users to the relevant documentation. weight: 1
- Main organization isn’t an FAQ. The content doesn’t have an endless FAQ with information that should have been integrated into more logical places in the document. weight: 1
- Version selection available. If content has multiple versions, the versions are called out visibly in the topic and might have a selector or link allows users to navigate to the other versions. weight: 2
- Release notes present. Release notes track changes to the product and documentation. weight: 3
- Easy path to top 5 pages. There’s an easy path for users to find the top 5 most-visited pages on the site. This requires you to look at metrics to determine these pages, and then assess the flow to those pages. weight: 1
- Content reviewed within past year. Content has been reviewed by a subject matter expert within the past year. Ideally, each topic should include metadata such as the last-reviewed timestamp, last author, and the group that owns the content. weight: 1
- Timestamps visible. The documentation provides a visible timestamp of the last time it was edited so that users can gauge how current the documentation is. weight: 1
- No broken links. Links point to correct pages or are appropriately handled by redirects to equivalent pages. weight: 1
- Key use cases are documented. The documentation doesn’t just provide reference information (e.g., auto-generated API documentation) but also explains how to use the API with tutorials guiding users through common use cases and journeys. The content should address the most common use cases intended for the product. weight: 3
- Code samples exist. Code samples showing sample ways to use the API (or similar tools) are provided. Ideally, the code samples are available in the user’s target language. This might mean providing multiple code samples. weight: 3
- Support options noted. Options for contact or support are provided, even if the support merely involves posting to a peer-monitored forum. weight: 1
- Reference material is interactive. The docs allow users to interact with the functions, API calls, or other commands often using their custom data. Typically, with REST APIs, an API explorer allows users to try out requests using their own data. Other sites might use Jupyter Notebooks to run code that can be customized. Seeing responses with your own data can make the API seem more relevant. weight: 1
- Feedback options available. A feedback option is included in the docs so readers can relay feedback about their experience. When users provide feedback, they get the sense that someone is listening. This helps users feel more relevant. weight: 1
- Product overview page answers “wh” questions. The overview explains the big picture and describes the problem that the tool or service addresses. Common who/what/where/why questions are answered here. weight: 3
- Overview addresses use cases. The overview provides a high-level description of the main use cases or business objectives of the product. This allows users to get a sense of what the API is all about. weight: 2
- Overview has architectural diagram and explanation. The overview has a diagram of the main components and how they interact. This provides users with a glimpse of the whole. weight: 2
- Overview has index of assets that product offers. If there’s an SDK or developer kit that users can download, the contents of this download are described. This is similar to product instructions that start by identifying all parts that should have arrived in a package. weight: 1
- Subsystems have their own overview pages. For larger systems that might have multiple subsystems (e.g., groups of APIs for different scenarios), these subsystems have their own landing pages that resemble the higher-level overview (with use cases, diagrams, getting started links) but scoped to that specific subsystem. weight: 1
- Access and authorization explained. Details about how to get access, permissions, and authorization to use the API are provided. For example, this topic might cover how to authorize an API call with API keys. weight: 2
- Getting started tutorial exists. A getting started tutorial is provided for users to get started in an end-to-end way with the product, producing a sample output that builds their confidence. This topic might provide info on how to sign up, register, get API keys or permissions, and start using the API. (This topic might link to the authorization topic but is more comprehensive in scope. The purpose of this topic is frictionless onboarding.) weight: 3
- Sandbox environment available. A sandbox environment is available for users to try out the tool/system/API on their own and get a better understanding of how it works. weight: 1
- README file exists in code repo. If the product has a code repository, the repository contains a README file that provides information similar to the overview (but abbreviated), including information on how to contribute if applicable. (The README might simply link to the documentation overview but also include unique sections such as how to contribute in the repo.) weight: 1
- Tasks have prerequisites sections. Each task has a “Prerequisites” section that explains knowledge requirements, tool requirements, essential concepts, etc., necessary for completing the task. weight: 2
- Tasks have numbered steps or sections. Tasks are articulated through a series of specific, usually numbered steps or sections that reduce the complexity of the task. More complex tasks might require more granular and numerous steps or sections so as to reduce the complexity. (This is the primary way that technical writers simplify complexity.) weight: 2
- Sample app available. In more robust docs, a sample app might serve as a reference implementation for users. weight: 2
- Code sample has inline comments. Code samples are appropriately commented, either with inline comments, comments that preface or follow the sample, or both. Long code samples with no explanation aren’t just dropped in. weight: 2
- Reference docs follow industry standards. For native library APIs (or other API types), reference docs (auto-generated from source code comments) are available. This might mean nativelibraryapis_exploring_javadoc_output.html, Doxygen, OpenAPI outputs like Swagger or other reference docs specific to the library. The reference docs should be populated and generally follow tagging standards. weight: 3
- Reference content has consistent structure. Reference material such as APIs follow a common structure within each topic, mostly following a request-response type structure. Typical sections include descriptions, parameters, sample requests or usage, and sample responses. weight: 3
- Error messages documented. Error messages that users can encounter are documented and discoverable through search. weight: 2
- Parameter docs have complete info. Parameter documentation typically includes a description, data type, min/max values, sample values, and optional/required usage. weight: 3
- Response includes both sample and schema (REST APIs). The response documentation for REST APIs provides both a sample response and schema. The response provides an example of what might be returned, while the schema defines all possible elements that might be returned and describes attributes such as data types and whether the elements are required or optional in the response. weight: 2
- Troubleshooting section exists. The documentation has a troubleshooting section (either standalone or included within the section/topic it relates to) that provides information on how to solve common problems. The troubleshooting information indicates where things might go wrong and how to fix them. In other words, the documentation doesn’t just describe the happy path. weight: 2
- Limitations/thresholds documented. If there are limitations and thresholds (for example, max number of API calls, max file sizes, max string lengths, max/min parameter values, max server load), these limitations are called out in the docs. weight: 2
- Locale limitations noted. If a feature is available only in certain contexts (locales, languages, platforms, roles, versions), that information is noted clearly in the feature. For example, an API that is only available for enterprise versions might have a label that says “Enterprise Version Only,” or if only available for a particular platform, might say “Linux Only” or the equivalent. weight: 2
- Sidebar nav has consumable organization at a glance. The sidebar navigation lets users take in a sense of the whole while also allowing users to expand more details as desired. The sidebar isn’t a massive list of seemingly endless scrolling and expansion + expansion + expansion but divides up doc sets into logical groups, like chapters in a book. For systems with large numbers of topics, progressive disclose techniques might be implemented across primary, secondary, and tertiary levels of information. weight: 1
- Sidebar nav highlights current topic. As the user navigates each topic, the sidebar navigation makes it clear where the user is in the navigation (for example, the topic highlights clearly and the navigation sticks open at that level). Breadcrumbs might also help establish site context. weight: 2
- Context remains consistent when navigating. When a user clicks topics in the navigation, the UI doesn’t shift context in jarring ways, such as unexpectedly taking the user to another doc set or changing stable navigation areas like the sidebar and header (which are usually consistent for every page). This jarring navigation often happens when sidebar entries point to topics in other doc sites. If this is the case, the external links have an icon indicating the link takes them to another site. weight: 1
- Doc types have consistent names across product docs. Common topics have similar names across doc sets in the developer portal. For example, the Overview, Getting Started, Troubleshooting, Glossary, Release Notes, and Reference are named consistently to help users understand how to navigate the site. One doc set shouldn’t call a topic “Latest updates” and “First steps” while another uses “What’s new” and “Quickstart.” weight: 1
- Subheadings are descriptive. The subheadings are descriptive enough to allow users to get a gist of the topic by merely reading the subheadings. The subheadings also follow a parallel structure so they can be more easily scanned and read. weight: 1
- Topics start with summaries. Each topic usually has a short summary below the title that encapsulates the main purpose of the topic. This helps users get a sense of the topic at a glance. weight: 1
- Glossary exists. Unfamiliar words and jargon are defined in a glossary. At times, the glossary terms are linked to their glossary definitions. weight: 1
- Code samples have proper formatting and highlighting. The formatting in code samples follows standard white spacing, line breaks, and other syntax for the language. Code syntax highlighting appropriate to the language has been applied to increase the code’s readability. weight: 1
- Responsive view presents content in readable way. The content can be read on a mobile device (e.g., iPhone) in a usable way. For example, the responsive view allows users to navigate the sidebar links and view code samples. weight: 1
Continue on to the next section, Second-level checklist for API documentation.
137/153 pages complete. Only 16 more pages to go.