Treat code like code and prose like prose

by Tom Johnson on Jun 16, 2020
categories: writing

Some recent experiences have prompted me to rethink the value of treating docs like code in some respects. I want to return to focusing more on content rather than technical workflows, especially when creating new content. Some of the docs-like-code processes exclude too many people in non-engineer roles who would otherwise contribute to the review and development of the content.

“Prose like prose”

I recently participated in a podcast sponsored by Readme, where Andrew Baker of Twilio and I were interviewed about best practices with API documentation — see “Building Great Documentation” – WAPI FM radio hour. When the Readme host asked us doc trends, I mentioned that more tech writers are treating documentation like code. When Andrew, who is the Director of Developer Education at Twilio, gave his response, he pushed back a bit on the idea of treating docs like code. He said their philosophy at Twilio is to “treat code like code and prose like prose.” (This topic surfaces ~15m into the podcast if you want to listen to it.)

Andrew said it didn’t make sense to run through their entire deployment process just to fix a typo, but he didn’t elaborate much beyond this to explain what it means to treat prose like prose. This was the first time I’d ever heard the phrase “treat code like code and prose like prose.” I didn’t entirely understand what Andrew meant. Did they not treat documentation like code at Twilio? Why was he drawing a distinction between code and prose when so many people only seem to draw parallels between the two?

I started digging around a bit to understand Twilio’s documentation processes better and found a presentation titled How Twilio Writes Documentation. In the presentation, Andrew talks about how they wanted their code samples to be fully accurate and functional with each release (listen to about the ~18m to 24m mark). The problem was, when code was intertwined with documentation, there wasn’t an easy way to test the code in an automated way to ensure its accuracy. Code couldn’t be treated with the same stringent deployment processes that code merited. And in Twilio’s research, code trumped nearly everything else in importance to users.

Andrew says they embraced a CMS called Wagtail because it provided a way to embed code from another source such as GitHub (you can see Twilio’s code snippets here). Using Wagtail’s Streamfields feature, they could drop code into a documentation page without the code literally being stored in the docs. This allowed them to treat code like a first-class citizen, running tests on the code to ensure it was functional and accurate (that it compiles or runs) with each release. Storing the code on GitHub also allowed contributor to make pull requests to fix errors.

Hence the phrase “treat code like code and prose like prose” means to not treat code like docs but rather to store and manage your code snippets so that the code snippets can be properly treated like actual code, subject to the same deployment criteria as other code released. As for prose, decoupling prose from code deployment processes allowed them to make dynamic updates and to potentially personalize the display based on user preferences or other logic.

I wasn’t expecting this direction. I now realize that when engineers hear the phrase “docs like code,” they take it much more literally than I do. When I say “docs like code,” I don’t mean that docs should be treated exactly like code with software testing on each release. It can take software teams about a day to push a new release following an official deployment process. Of course fixing a typo shouldn’t require this kind of process. When I say docs like code, I just mean working in a text editor and using version control to manage content and trigger builds to a server.

At any rate, this phrase — “treat code like code and prose like prose” — really prompted to think. I wondered, is there a good reason for treating prose differently from code? If even Twilio pushes back against treating docs like code, and instead encourages prose to be treated differently, should I reconsider treating docs like code, even in the general way that I mean it? Am I trying to force prose into an engineering workflow that isn’t optimal for the inherent and unique characteristics of prose? What are those qualities that make prose unique from code?

Given my background in creative writing, this pushback awoke part of my past. I remembered back to my time in New York, when I was getting an MFA in literary nonfiction. In any creative writing workshop, the default approach is to write something and share it with others. The others mark up your content and provide feedback about your writing. The feedback can be discouraging or exhilarating, depending on the response. But one thing was certain — these workshops, where we were forging prose from blank pages, didn’t treat prose like code.

This dilemma about whether to treat docs like code or prose like prose was especially challenging given that I’d just recently concluded a survey with more than 400 dev doc writers responding about trends. In my survey, I had asked, “How do you review your docs prior to release?” The majority (25%) said they use “the same code review tools that engineers use to review software code.”

I wondered if this trend was misguided. Should I stop trying to force prose into a workflow that isn’t the best fit for what it is? I let this conundrum settle into my mind for a few weeks.

Code reviews exclude non-engineers

A few weeks later, I had another experience that prompted to push back on the idea of treating docs like code. A software development manager at my work recently asked me to start pushing doc changes into code reviews, or CRs. Software developers use code review tools (like Review Board) when reviewing code with other developers. The code review contains a “file diff” snapshot that shows exactly what has changed in the commit, highlighting the differences line by line in red (for what’s removed) and green (for what’s added).

For docs, this meant being more careful about the exact changes I was making to a branch so that the file diff for a specific ticket could be captured and pushed into the code review tool. I could no longer make a variety of changes in a general branch and push it but instead had to scope my changes to a specific ticket, defined by a specific branch, so that the file diff would correlate with the code review.

It took me a while to figure out how to push a file diff to the code review tool. It wasn’t as straightforward as it could have been. The code review tool would often try to guess the changes to include, and sometimes it included changes from some other ticket even when I properly switched branches when working on other tickets.

I spent a few days trying different workflows. I resorted to rebasing and squashing my changes into a single commit and passing a parameter to the CR to look only at the most recent squashed commit. This seemed to work but required more effort on my part. I also had to describe the changes in the CR too as well in a ticketing system (e.g., JIRA), so there was redundancy.

After all this effort, what was the result of committing CRs? Some engineers did provide comments in the CR tool, but project managers, field engineers, and UX did not. They didn’t even try. It was like noise from an automated log message or something. For these other groups, I generated an output and uploaded it to an internal website and sent them the link.

Nevertheless, some writers on another doc team kept raving about their success with CRs, so I decided to stick with it. After all, I had only been trying out the process a couple of weeks.

About this same time, we onboarded a new writer, and I had visions in my had about everyone on our team contributing code reviews for doc changes so that we could all be acutely aware of what changes each of us was making and pushing.

I had trouble getting this idea off the ground, though, and it didn’t make sense for many projects because the project managers whom these others writers were working with never requested docs to appear in code reviews. One of the other writers asked, what’s the point? It seemed like a waste of time.

I still continued to push for doing CRs. Again, even on the project where the software development manager requested that I do CRs, he himself had yet to comment on any CR, and the only other engineers commenting were those who had created the code or feature that I was documenting. No one else really seemed to look at them at the CRs, even though I included them in the list of reviewers.

I continued trying to use CRs for about a month, but eventually I realized this review process seemed to be excluding a large group of people whom I wanted to review content. Not just program managers and field engineers, but the support team, other stakeholders, legal, and more.

Just like in the creative writing workshop, you don’t write prose and have one person review it. You gather reviews from a large group of people, at least five or more, so that you can gather trends from the feedback. One person might like the story, while another can’t stand it. One person might say a theme was apparent while another misses it entirely. Reviewing my docs with just one other person (i.e., the engineer who created the feature requiring documentation) was like showing up to a workshop where the reviewer already knows the story he or she wants to read. This isn’t how prose is created. Prose is workshopped and rewritten as you gather input from many different people and grow the content more complete with each iteration.

The echoes of Andrew’s phrase, “treat code like code and prose like prose” resounded in my head. I started to question whether prose, or rather, documentation, should actually be treated like code. If you’ve ever tried to read changes in a file diff display, it’s not easy. The file diff usually shows you a side-by-side comparison of what’s changed in a before and after way. If it’s a small change, it’s easy to identify a line or paragraph that is new, but in more substantive changes, the changes might span multiple files and include a confusing display of red and green highlighted lines from different parts of the file. Sometimes Git will highlight a changed line when all you did was move it down or change some formatting.

Additionally, the file diff highlights only what has changed. Sometimes when you add a new paragraph or make other edits, you want to read them in the context of the whole, not just in isolation. Imagine a workshop where writers present their revisions in file diffs, where other workshop participants just read what has changed, not the new version in its entirety. There would be no real way to judge whether the revision fixed the problems with the initial version.

The final blow

Despite my reservations with CRs, I continued creating and promoting them until the final turning point in this story: one of my colleagues left for another Amazon team. We have a small team of just three writers, and now as he transitions elsewhere, we will shrink down to just two — for the entire Amazon Appstore and every app development technology related to Fire TV and tablets. My manager asked me to compile a list of all the work for the year so that we could make better decisions about which projects to prioritize, so that we could make a business case for more writers, and so on.

I started copying roadmaps and lists of work from various teams. We serve many different teams in different organizations from Fire TV to Alexa to others. And we engage with them in varying capacities, sometimes acting as authors, other times as editors, other times only as publishers, in different combinations and strengths.

After looking at all the projects scoped for 2020, I realized there would be no way to play the author role for all the work. We would need to rely heavily on product teams to contribute to the documentation. In some cases, we might provide empty wiki pages and ask them to fill in the blanks. Teams might have different contributors for these doc projects — maybe the doc contributor one one team is a product manager, on another it’s a QA person, on another it’s an engineer, and so on. The many different roles and teams we interact with at Amazon are endless. Each project includes email threads with many people I’ve never met and whose roles are opaque to me.

In such a scenario where healthy input from project teams would be essential, how could I possibly tell them all to make a pull request for any changes they wanted to make, or to raise a CR? Engineers would understand, but few others. I saw myself handholding PMs through git workflows and teaching people again and again how to make comments in the code review tool. There was no way this was going to work.

At this point, I decided that the CR process wasn’t feasible for my situation, and we should instead drive new content development through Quip, which is similar to Google Docs. Quip is already the wiki-like collaboration platform many use at Amazon, and so I decided to root ourselves in Quip as we work with teams to create the content they need.

The primary reason for developing content in Quip (or similar) is that it’s more inclusive for different types of contributors and reviewers. If you try to conduct a review in a Code Review tool, you get the engineers commenting but few others. In a more general tool like Quip or Google Docs, everyone can easily jump in: project managers, field engineers, quality assurance, and even engineers. There’s no barrier to entry. This more inclusive perspective and input is essential when writing and reviewing new content. This is how you make prose. It’s not necessarily how you make code. In the writing process, you want to include as many readers and reviewers as possible.

Note that the scenarios I’m referring to here are scenarios where you have to write significant amounts of new content that require input and review from other groups. When I’m writing documentation that requires this kind of input, it’s much easier and more fluid and flexible to use Quip or Google Docs. This is how I operated at a previous company I worked at — Badgeville, where we all had Google Docs. I would write content in a new Google doc, everyone would review it, and when finalized I would copy and paste the content into Drupal. It worked beautifully even if the copy/paste between systems was manual.

I’m not a fan of copying and pasting content from one system to another, of course, and ideally it would be great to operate entirely in Markdown and have people commenting in the same system. But if doing so means excluding 75% of the reviewers and contributors, I’ll suffer with the manual copy and paste as the price to pay. There are always tradeoffs with authoring tools and approaches.

Since making this decision, I’ve felt a lot of relief. I think sometimes we try to push writing processes into more technical software engineering processes even when it doesn’t make sense. Why do we do this? Perhaps doing so makes us feel aligned with engineering, like one of the gang, and it earns us street cred and respect (which is essential to compensate for the low-value estimations we often sense from others about docs). But at the end of the day, documentation really isn’t like code. You can make many typos and mistakes in your writing, and it will still “compile” and publish. Writing doesn’t need to be unit tested or security-hardened. Writing doesn’t include for loops or parameters that you pass in from variables you define elsewhere. One typo won’t usually crash an entire system. You can’t automate testing of documentation like you can with code.

I’m sure people could challenge me on every point here and draw elaborate comparisons between writing and coding, but I’m growing tired of that comparison. I’d like to start treating prose like prose. Prose works best when developed on an inclusive collaborative authoring platform, when it’s readable in its entirety, in a display similar to the output.

If you try to pick out pieces of what’s changed in a file diff display, you can’t see it as a whole as it’s intended to be consumed. You can’t see images and other visuals displayed, and how those visuals give balance to text. It’s not easy to visually browse the structure, or to get a sense of how content is broken up across pages and how you flow from page to page. When you’re developing new content, your focus should be on the content, not on the technical details of the platform and collaboration.

Conclusion

As I move forward into more of a prose-like-prose model, I plan to develop more content in Quip and have a lot of back-and-forth with engineering/product teams. I still plan to keep the other docs-like-code aspects — this includes maintaining and updating content in Jekyll, managing content in Git, and triggering builds through CI/CD on various branches. I just think the initial doc development and review might be better off outside this workflow. I hope that this new process will reduce my workload without putting undue stress on others to develop content.

Your feedback and input

I’m curious to hear your feedback and learn whether similar processes work for you. I like knowing whether the ideas I’m leaning towards in posts have merit, or if I’m just talking myself into a direction that might be counterproductive in the long run. I want to leverage your wisdom and experience as much as possible, so if you’d like to provide input in the following short survey, please do so. You can view the ongoing results here.

About Tom Johnson

Tom Johnson

I'm an API technical writer based in the Seattle area. On this blog, I write about topics related to technical writing and communication — such as software documentation, API documentation, AI, information architecture, content strategy, writing processes, plain language, tech comm careers, and more. Check out my API documentation course if you're looking for more info about documenting APIs. Or see my posts on AI and AI course section for more on the latest in AI and tech comm.

If you're a technical writer and want to keep on top of the latest trends in the tech comm, be sure to subscribe to email updates below. You can also learn more about me or contact me. Finally, note that the opinions I express on my blog are my own points of view, not that of my employer.

M ↓   Markdown
J
Joe Pairman
0 points
5 years ago

This was a very good post. As organizations rediscover time and time again, content is quite different from code. It is both messier (despite the valiant efforts of Mark Baker and others), but richer.

The scope of a piece of content is often different from that of a code module; indeed it can be far more useful if it illustrates how to use a system in a connected way, not just describing one component of that system.

The timing is different as well: some aspects of content can be ready well before the product they document; for other aspects the reverse is true.

There are still cases where it makes sense to manage software docs as code – where the docs are just one step away from comments in the code itself. But for many cases, content is better created, reviewed, and managed in a separated but linked system.

(I have some skin in the game here. Not sure whether this is a disclaimer or a brag but a substantial part of my time over the last two years has been helping to create a system for creating and reviewing structured content in a more prose-like way that keeps the value of constrained structured templates and makes editorial decisions auditable.)

?
Anonymous
0 points
5 years ago

Fascinating article. As a beginner in this area, I'm curious as to how you alert potential reviewers about what's new/changed in the docs using the Quip approach?

Y
Yuliya Bagriy
0 points
5 years ago

I feel like most of the problems with treating docs more like a code are due to problems with tools, processes, or biases around them.

  • submitting to peer review is hard: that's a tools problem, bad UX. There were (and are) times when code reviews were done via mail lists ;)
  • changing typo in docs triggers the same process as the feature release: that's a process problem, because you try to apply the same solution for different areas. It's like running the full product regression tests for every bugfix instead of testing around the bug itself.
  • people don't participate even if tools&processes are ok: bias. I've noticed that sometimes you just need rename a thing in order to appeal to uncommon audience. Because if it's called "code review," non-coders will have a mindset "I'm not a coder, so I won't do it."

I'm not as well versed in tech writing space, but from what I can tell with experience in testing and coding, there are no real prose vs code. Yes, there are certain differences, but when I read your description of creative workshop review, to me it screams "code review." Or "test review." Or "API definitions review." The idea is the same, only implementation differs. Let's take a step a side and look on a seemingly unrelated field: translation strings.

  • usually they are stored in some kind of version control system
  • usually in PO format, for easy consumption by applications and translation tools
  • translation tools abstract all the complexity of versioning, committing, reviewing in extremely user friendly way so that even normal users can participate (in addition to other features like glossary or machine learning analysis)

The last bit is the most important one. Tools are evolved enough to be used by anyone. I suppose that's what is lacking in documentation writing space. Either you have to go all geeky or you abandon some of the automation advantages typically associated with the "code."

J
John Baker
0 points
5 years ago

Hey Tom, Interesting piece. Thanks for providing a little bit of insight into your thought process.

Also, The first link for WAPI radio hour doesn't seem to be working.

T
Tom Johnson
0 points
5 years ago

thanks for the note. i fixed the link.

S
Saravana Kumar
0 points
5 years ago

Tom, that's exactly the point "Andrew said it didn’t make sense to run through their entire deployment process just to fix a typo," I tried to highlight on my length blog on this topic https://document360.io/blog/docs-like-code-is-it-worth-the-hassle/

The product release cycle making sure it passes all the QA test cases and CI/CD is complex enough, you really do not want to add another layer of complexity of Docs compilation and deployment in that process.

I quite like the idea of having direct connection to GitHub and embed the code in the docs, so it's decoupled. We will see if we can bring this feature into Document360 in the future.

M
Mark Baker
2 points
5 years ago

Tom, the documentation world has been flapping back and forth between treating docs like code and treating prose like prose for as long as I can remember. Each approach as significant issues. You are spot on with how you describe the problem with docs like code. But I am sure you would be able to go back in your archives and find posts where you describe the problems that come with treating docs like creative writing -- probably the posts were you described your move into docs like code in the first place.

The case I have been trying to make for the last 20 years is that both approaches are wrong. You should not be treating documentation like creative writing. Nor should you be treating it like code. To sum it up in the same terms, you should be treating docs like data.

Structured writing should not be about making prose into code, nor managing it like code. It should be about making it into data that can be processed by code, and managing it like data. That is a very different pattern. The problem is that most attempts at structured writing -- from DITA to Jekyll -- have succumbed to the temptation to embed logic in their source files. This makes the docs a mix of data and code and leads you right down the path of treating the whole thing like code rather then data. And this leads to all the problems you have delineated in this post.

My latest book, Structured Writing: Rhetoric and Process shows how it is possible to treat content strictly as data, meaning no specialized knowledge or tools are required to write or review the content, and anyone can collaborate without any knowledge of the publishing system. You can do all this while still retaining all of the process advantages the come with a docs like code approach. This is the middle ground we have to find if we want an end to flipping back and forth between the problems caused by treating docs like novels and those cause by treating docs like code. We need to learn to treat docs like data.

If you would like a review copy of the book, please let me know.

T
Tom Johnson
0 points
5 years ago

I'm a big fan of structured writing and have actually implemented in a lot of my docs where it makes sense. For example, specification information, CLI commands, schema descriptions, and other reference material. I often create a custom structure in yaml and then parse through it with liquid in jekyll using some template. For example, see this post: Some specifications docs I've been working on.

However, a lot of content doesn't conform to a specific template or structure, so this approach can't always be used. Most how-to topics in tech docs can't be treated like this. There are general templates and patterns, for sure, but basically most tech docs isn't like a recipe, just as most blog posts aren't like a recipe. I felt like your latest book slanted too much toward treating all docs like a recipe. (I admit I didn't get through most of the book, though. I keep meaning to at some point.)

Also, I see the structure issue as more of a publishing problem. The development of the information, whether it's treated as prose or code or data, is something that requires collaboration and input before you decide on the way to publish it. Sure, if the information fits a common pattern or template, that can drive how the information is gathered and structured from the start. But most of the content I work on resists that cookie-cutter model. At any rate, after the information is gathered, then perhaps it can be structured or single sourced or otherwise handled in a more efficient approach for publishing.