Search results

My Knowledgebase Ninjas podcast episode -- "Metrics don't work"

by Tom Johnson on Apr 15, 2020
categories: technical-writing • podcasts

I recently appeared on the Knowledgebase Ninjas podcast in an episode titled Metrics Don't Work. In the podcast, I chat with Gowri Ramkumar about documentation processes, why metrics are problematic, advantages and disadvantages of docs-as-code models, why measuring doc traffic falls short, the value of internal documentation, people I've learned from in my career, advice for my younger self, and more.

Episode details

You can view the episode details on Doc360 podcast site:

You can also listen through the embedded player here:

Topics discussed

The podcast covers the following topics:

  • How I got into tech comm — my “reality check moment” working two jobs
  • About the group I work in at Amazon (which includes just two technical writers)
  • What docs-as-code entails — Markdown, Git, CI/CD, static site generators (SSGs), etc.
  • Trends about authoring/publishing tools, including SSG usage, JAMstack, Markdown as a common format across tools
  • Disadvantages of the docs-as-code approach: UX skillset requirements, lack of built-in search, internal development time, localization, content re-use, PDF generation, custom scripting and tool lock-in
  • Advantages of the docs-as-code approach: simplicity, scalability, flexibility to customize, integration of web frameworks (e.g., Bootstrap), modern-looking websites
  • Why metrics often fall short in the value-from-ticket-deflection argument — budgets come from different, non-related areas of business, doc knowledge is internalized and used to close tickets without acknowledging docs, tickets are often never created because users find doc first and get their answers, deals might be made or rejected based on docs without any kind of acknowledgement, onboarding times for internal engineers aren’t tracked, etc.
  • Why measuring documentation usage is problematic — high-impact customers versus low-impact customers, the challenge of interpreting metrics, why time on page, bounce rate, and click rates are problematic, paying attention to trending pages
  • Whether technical writers create value for internal groups — knowledge creation as an intangible benefit, creating information that becomes the lifeblood of the company
  • Whether tech comm teams should be involved high-level product design or just write docs, different specializations tech writers sometimes play, CX knowledge requirements to influence design
  • Who I learned from the most in my career - Mark Baker, Every Page Is Page One
  • Docs I’ve been using lately — Descript, Cave of Programming
  • Advice for my 20-year-old self — turning around the question and letting my 40-year-old self learn from my 20-year-old self to be more open to risk and change

Here’s one takeaway from the metrics discussion:

I think I took more of a devil’s advocate approach regarding metrics during this podcast because other podcast guests seemed to be more enthusiastic about embracing metrics (for example, listen to the episode with Gavin Austin, which I enjoyed). I should have probably acknowledged that metrics are needed to influence decision makers. Also, I should have more advice about which metrics to use for docs. Unfortunately, I don’t have good advice for metrics. Everything I have ever tried hasn’t really worked.

Resources mentioned during the podcast

Subscribe to Knowledgebase Ninjas Podcast

You can subscribe to the Knowledgebase Ninjas podcast by clicking the Subscribe button on the embedded player above. Alternatively, search for the podcast in your favorite player or subscribe to their RSS feed.

About Tom Johnson

Tom Johnson

I'm a technical writer / API doc specialist based in the Seattle area. In this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, visual communication, information architecture, writing techniques, plain language, tech comm careers, and more. Check out my API documentation if you're looking for more info about that. If you're a technical writer and want to keep on top of the latest trends in the field, be sure to subscribe to email updates below. You can also learn more about me or contact me. Finally, note that the opinions I express on my blog are my own points of view, not that of my employer.

© 2023 Tom Johnson