Building out my agent skills course and rethinking product skills

by Tom Johnson on Oct 8, 2026
categories: ai

It’s been a while since I’ve posted on my blog. Here’s what I’ve been up to:

• I’ve been adding to my course on agent skills. I added a whole section on product skills with the following pages:
  • From developer experience to agent experience
  • Anatomy and distribution of a product skill
  • What the research says about product skills
  • Problems with product skills
  • The docs-first approach
  • Making docs accessible to agents
  • Roles for tech writers with product skills
  • Mining users’ AI chat sessions: gaps and forensics
  • Reimagining the documentation experience

This is a section I completely reworked after changing my mind about the value of product skills. A couple of months ago, it seemed like everyone was championing product skills as a key strategy for influencing AI agents. Since we started receiving access to log data at my work, I’ve rethought some of the rah-rah-rah approach to product skills. Now I’m more of a docs-first person, as I can’t think of many things that should be in a product skill that shouldn’t already be in the docs. It just seems like a better investment of time to add whatever info might exist in a product skill into the documentation instead.

I’m not against product skills, as I’m still waiting on more data to fully come to a conclusion. But product skills that do little more than summarize what’s already in the docs and what the models are already trained on seem to do little for agents. A product skill could help an agent learn the right terminology and understand how to search the docs, but all of this seems like marginal work for product skill content. A tech writer’s time is better invested in improving the docs with this info.

Additionally, it’s surprising how infrequently product skills actually fire. And when they do fire, it’s surprising how infrequently they actually help the user.

Some conversations in the #ai channel in WTD Slack were helpful in shaping some of my thinking.

As I’ve been writing out the agent skills course, I’ve been playing more of a director/editor role and telling AI what to write. For my first chapter on agent skills, I wrote most pages as a draft first before bringing in AI, but as I’ve worked on the other chapters, it’s been more of an AI writing experience. That’s okay for at least two reasons. First, this is instructional/informational content, not blog posts. As such, this is more like documentation work.

Second, because so much of the material is in flux, it doesn’t make sense to spend so much time hand-crafting it. Take the section on product skills — the winds could change, and then I’d need to rework large chunks of material. I know how hard it is to keep a large corpus of content up to date. I’ve spent years keeping my API documentation course up to date. Every example, link, instructional detail, etc., has about a six-month expiration date before it becomes stale, outdated, or gone. Try keeping hundreds of pages of material updated regularly — and doing it on the side, in evenings, weekends, etc. It’s a losing battle. If AI can help with this, I’m all for it.

Right now I’m working on the next chapter in the agent skills course: From logs to doc improvements. This chapter is more forward-thinking because I’m just starting to evaluate/iterate on the process I’m describing. The chapter is more of a blueprint, constantly fluctuating and evolving, based on my current experiments and experiences. I have a big yellow note at the top of every page that says as much, warning the reader.

This approach isn’t lazy or hypocritical. I’m using AI to learn, to help accelerate my thinking. The pages are a blueprint for a skill I’m building, and as I get more logs from the chat sessions and can push the skill through many iterations, I’ll be able to see it through. I’ve always treated writing as a means of learning, which is why information-dense writing (similar to documentation) has been less than engaging for me. I like that spark of discovery in writing a post, landing on some detail that I’d never thought of. In short, learning from writing.

When I’m writing informational material, it’s usually a knowledge dump of what I already know. Hence it’s much less engaging to write. That’s not so much the case when I have AI. AI can push me in directions I hadn’t considered and help me think through new or different ideas. Many times it’s wrong and can easily be pushed in whatever direction I nudge it. As such, it can probably lead me astray. But I’ll course correct the content as I go, adjusting and coming round and round to find the content that works, content that can be supported by my own direct experiments and observations. If I can use AI to learn while writing instructional material, that seems like a highly valuable use of time.