The need for robust tech comm authoring tools

Neal Kaplan recently posted about the search for a perfectly adequate authoring tool, which got me thinking about the tools I use.

At the time I joined my current company, I had to change my help authoring tools quite a bit. At my previous company, I used Mediawiki and Madcap Flare. Both tools worked pretty well. At my present company, we publish on Drupal, which is a web-based CMS similar to WordPress but more robust. All the content lives primarily on Drupal.

At first we authored directly in Drupal. However, authoring and managing tech comm content in Drupal isn’t ideal at all. After a few weeks of working in Drupal, I adopted a combination of Markdown and Google Docs, which I wrote about here: A Simple process for authoring and publishing technical content.

However, while Markdown is a convenient way to produce HTML, it’s not my favorite syntax. Some of the syntax relies on spaces (such as breaking a numbered list with 4 spaces to insert a note or codeblock). And there aren’t any good tools for managing Markdown documents, nor are there opportunities for re-use, assembly into complex help systems, use of variables or conditional text, output to PDF books, and more.

I really miss the power and convenience of a robust tech comm authoring tool. The Markdown method was fine as an interim solution, but not long term. Markdown mostly addresses the needs of programmers who have simple documentation to write (like 5 pages, nothing more). At most, you might put your Markdown documents into a collection of articles, like you can with Nanoc.

One of the main problems with managing content in Drupal is that the content is hard to wrangle. I feel like there are endless pages (“nodes”) that just exist in the database. It’s hard to read through the content and manage, shape, massage, and work on it as a whole. I know readers don’t read the content as a whole, but it’s important to make the individual pieces consistent with each other, especially when terms shift, features are deprecated, links once relevant no longer are, new functionality is added, strategies change, and so on. I need to regularly go through the documentation and make sure it’s internally consistent, logical, clear, and accurate.

The problem with the online database model is that there are so many topics, it’s hard to know where to begin or end. Sure, this may be the paradigm of the web – an endless collection of discrete topics, some relevant, some not, some updated, others not – all filtered by the user’s search.

But that model paints a scenario that is more complex than it needs to be. My products aren’t that complicated. They should be intelligible in a single read, more or less. I don’t want to send the user in 15 different directions.

I would like to return to a time in which I could manage and shape the content in a robust help authoring tool. I’ve decided to try a new approach. I’d like to try the DITA option using Oxygen. (About 90% of the help authoring tools usually available to me aren’t available because I’m on a Mac. Sure I could use boot camp to install Windows instead of Mac OS X, but doing that is too painful.)

One of my primary goals will be to single source both documentation and training from the same source, producing attractively designed training workbooks that rival the current styling and formatting (they’re in InDesign). Figuring out how to customize and XSLT stylesheet in order to product output directly from Oxygen will be one of my first tasks. Getting the content into Drupal from the XHTML output will also be a challenge.

Madcap FlareAdobe Robohelp

This entry was posted in general on by .

By Tom Johnson

I'm a technical writer working for The 41st Parameter in San Jose, California. I'm primarily interested in topics related to technical writing, such as visual communication (video tutorials, illustrations), findability (organization, information architecture), API documentation (code examples, programming), and web publishing (web platforms, interactivity) -- pretty much everything related to technical writing. If you're trying to keep up to date about the field of technical communication, subscribe to my blog either by RSS or by email. To learn more about me, see my About page. You can also contact me if you have questions.

23 thoughts on “The need for robust tech comm authoring tools

  1. John Tait

    Hi Tom

    Emacs org-mode is really similar to Markdown-Pandoc except that it can all be done within Emacs, so you’ve got a single tool. There are a lot more features, including conditional text (by filtering tags) and reusable snippets of text (called macros). LaTeX output needs a bit of tweaking, I’ve found.

    I also wonder why no-one uses Texinfo for producing HTML and PDFs – the PDFs look amazing, it’s real single sourcing, its very mature and fully-featured. It’s a bit like DocBook, but with @ signs instead of angle brackets.

    Maybe the prevalent database model is too convincing.

    1. Tom Johnson Post author

      John, thanks for the tip. It’s good to know these tools are out there. However, I want to try DITA because it has more relevance and infrastructure in the tech comm world. Are you using the Texinfo method you described? I know you were using DITA at one point.

      1. John Tait

        I haven’t used Texinfo for work, though I’ve tested it many times. As well as liking the friendly @ signs, I’ve found its PDF output to be way better than the XLST route, and the HTML is at least as good as DocBook. No lines in PDF tables, though.

        I used DITA for a project. I admire DITA’s technical achievements (specialization, etc.) What I don’t like is all that task/concept/reference stuff, which is of no use to my industry whatsoever. (Requirements and guidance in safety critical industries.) I know I could theoretically specialize.

        But I found DocBook with XInclude is a slightl better fit., especially since Oxygen added something called “Master files”, which let you assemble and cross-reference pieces like DITA topics.

    1. Tom Johnson Post author

      Thanks for the tip. Yes, I already have a copy of the book. It seems useful. One criticism — the author assumes you’re using the DITA OT directly. However, I can’t imagine how it would be more efficient to use the OT directly rather than an XML editor like Oxygen, which is pretty inexpensive.

  2. Karl Hakkarainen

    Thanks for this recap of the state of the tools.
    Funny thing, we had a very good solution 20 years ago. Interleaf and its WorldView info delivery tool provided much of what you’re seeking. We could work and deliver chunks of information as needed or assemble them visually into an information set to work on broad editing reviews. Its desktop view of information chunks was unique and hasn’t been replicated. (Think Visual Studio for information.)
    Interleaf was ripped apart by two things. The pricing model for its workstation products was an order of magnitude higher than comparable PC-based tools (Frame, Word, Quark, etc.). Also, the advent of the web meant that people no longer looked just to local installations, but expected to have integration with information everywhere.
    It remains a puzzle why we haven’t seen a good visual tool for modeling information sets. Even the best mind-mapping tools don’t seem to know what they’ve got.

    1. Tom Johnson Post author

      I don’t think I’ve ever heard of Interleaf and Worldview. Thanks for commenting and letting me know. I guess it’s good to know the tools did exist at one time…

    2. Thomas Tregner

      I’ve heard that sentiment about Interleaf from an established technical writer I worked with several years ago. We were using unstructured FrameMaker. About a year ago I got curious and Googled to find out what happened. Interleaf continued through an acquisition and became QuickSilver. But I don’t know whether the user interface retains the desktop view of information chunks you describe.

      http://www.broadvision.com/en/quicksilver.php

  3. John Tait

    “It remains a puzzle why we haven’t seen a good visual tool for modeling information sets. Even the best mind-mapping tools don’t seem to know what they’ve got.”

    Karl – I’ve been visually modelling content using Graphviz for years. I predict the rise of graph databases like Neo4j to model content. You heard it here first!

    Also check this out from the 80s: http://www.youtube.com/watch?v=7DxYj32cvoE

    1. Thomas Tregner

      Great video! Thanks for sharing that. That is an interesting prediction. I also share your opinion about graph databases. Graphviz is used in some plug-ins for an ontology editor, protégé. I have found those visualizations to be very helpful. In my opinion, one visualization challenge is effectively representing the distinction between is-a and has-a relationships.

        1. Thomas Tregner

          :) I don’t mean for someone using a graph visualization tool. I mean for presenting those relationships to readers. I wouldn’t expect a reader to learn a color scheme or read a key for a graph visualization. I’m talking about how is-a relationships lend themselves to a hierarchical representation of concepts such as with a TOC in topic-based authoring. And to carry the analogy further, has-a relationships reflect a rendering based on something such as a relationship table.

  4. Pingback: The need for robust tech comm authoring tools | I’d Rather Be Writing | TechCommGeekMom

  5. Sue Andrews

    Hi Tom,
    Having tried DITA in a few contexts, I have found that the technicalities involved become so distracting and take up so much time, that the ROI is eaten away. Focusing on how to accomplish basic tasks using XSLT just became frustrating in the extreme. I returned to Flare with huge relief and now enjoy focusing on what I am where I truly add value – knowledge transfer, writing, and design.
    It is very interesting to witness your struggles. They confirm my thinking :)

    1. Tom Johnson Post author

      Thanks Sue. I do really like Flare. However, you can spend quite a bit of time tweaking the look and feel of both the online and print outputs. The problem with Flare is that you can’t easily grab the content and plug it into another system. It’s designed to be its own system. I always wanted to bring its content into something else.

      Re the focusing on the technical versus the content, this is good to be aware of. I think some of the DITA tools and processes have matured a bit. A few years ago I played around with DITA using the OT directly and had a “fun” time trying to validate files filled with errors. Oxygen has been a huge leap forward as it has real-time validation, auto-complete, built-in transforms, an author view, and more.

      I’ll keep you updated on my experiences through blog posts.

      1. John Tait

        Did you know that in latest version of Oxygen (15.2), its version of the DITA–OT can finally generate Indexes for print?

  6. Debbie M

    Sounds like a great project. I hope you decide to publish your experiences here.
    How is it that you’re getting DITA into Drupal?

    1. Tom Johnson Post author

      That is a great question. We can build a connector, but it will cost too much. Currently we have an API import module that we built to bring in JSON files for our API documentation. It works well and we plan to modify it to work with other files.

      The custom-built module has some serious limitations, though. You have to import files into one book at a time, for example. I’m not sure how it will handle links that reference other links not yet published, and so on.

      Building a proper import module is always a possibility. At the very least, if the import doesn’t work at all, I could copy and paste the HTML output into Drupal articles. I know that sounds really primitive and inefficient, but if you consider that we may be publishing only 5-10 new pages a week, the time spent copying and pasting the content into the individual articles may not be too much. Also keep in mind that the import module most likely won’t be able to map to all the Drupal taxonomies, weight selections in book modules, and so forth, so the copy and paste method isn’t so atrocious.

      That said, I think it might be cool to do a kickstarter project at some point to build an import module that would work with Drupal 8 (when released) to be available for the community as a whole. If there’s one complaint I regularly hear, it’s that DITA’s online help needs a nicer interface. Drupal can provide that.

      However, despite Drupal’s web-like feel, I am not a big fan of the software as a model for help. Pages load too slowly, the TOC doesn’t collapse in an AJAX-like way, and Drupal tends to amass content from lots of other places (forums, blogs, announcements, etc.), becoming one massive mountain of info.

      I’d rather publish content into Antidot’s Fluid Topics interface, actually. Or some other online site. So maybe my move to DITA is actually a step away from Drupal. I really don’t like the idea that so much of our content is locked away in Drupal right now. Try getting it out and all you end up with is a massive SQL file that doesn’t easily import elsewhere.

  7. Mark Baker

    Tom,

    The holy grail seems to be simplicity and light weight on the front end, sophistication on the back end. Though the desirability of such an approach is obvious enough — many have asked for it before — I think there is a basic economic reason why it has not yet appeared, which I detailed in a post last year: http://everypageispageone.com/2013/01/03/we-need-a-new-economic-model-for-tech-writing-tools/.

    The SPFE project is my attempt to address this hole in the tech comm tool set. It’s specific design goal is to avoid projecting content management or publishing semantics back into the authoring layer while providing sufficient subject semantics to drive sophisticated management and publishing functions.

  8. lukasz gornicki

    The only solution thah has future in software documentation is nanoc and others. You can integrate them with devs tools and projects. A the beginning you have to spend some time on creating a project but then adding pages with markdown, but not only is easy. Markdown files accepts html if you want. Just take a look at developer.github.com It is entirely built with nanoc. How cool is that! I can`t wait when I`ll publish my findings and research on creating software documentation with docpad!

  9. Dimitri Tetsch

    This is a vendor post
    ****************
    Did you already hear about HelpServer?

    HelpServer is a web-based team-authoring solution geared towards documentation and help.
    The nice thing about HelpServer is that authors can work with a client authoring tool (=workbench), which retrieves/save content from/to a database on a web server. This can be either your web server or one of our web servers since HelpServer is also available as SaaS.

    HelpServer comes with an easy to use WYSIWYG editor and all content that is contributed is stored in a database on a web server. Images or any other files can be uploaded by means of drag and drop, and can be reused by your authoring team.

    Your authors can reuse content objects like for example topics (in HS terminology: only one object will exist in the database and can be reused/referenced in/from different places within your content hierarchy). Another scenario is that your authors can create new separate instances based on existing content objects (=like Xeroxing a paper and then start writing on it).
    The workbench (=the authoring tool) is nice to work in because your authors will feel like they are working with client software (although content is retrieved from / or stored in the database on the server when needed, you actually do the authoring in a client workbench which is installed on the client computers of the authors). The client workbench can be installed on Windows and Macintosh computers.

    When content is ready-to-go, authors can publish content. With HelpServer publishing is NOT about creating static files, as most other help authoring tools do, no instead it is turning a switch in the database to make your content in real time accessible by your target audience. In fact this is an important difference. Because if your team tomorrow needs to change a few topics, you afterwards do not need to publish the entire shebang again, no you only need to publish the topics that have changed. The result is that your target audience will always access the most recent published version of your content via a front-end web-portal (which comes of the shelf with HelpServer) and from this web portal can browse, search, and print content to PDF.

    HelpServer is web- and server-based. This means HelpServer will listen to incoming requests (URLs that are launched and hit the HelpServer server), on the server-side the needed content is retrieved from the database and the content is rendered to the browser in real time.
    Next to delivering dynamic content in true real time via the web, your authoring team can also export content from the database to popular file formats for offline delivery (HTML + PDF + Translation XML + XML).

    If interested visit http://www.helpserver.eu and get started with a free trial or contact me, I would be happy to introduce you to HelpServer during a WebEx meeting.

Comments are closed.