Using Treejack as a Method for Evaluating Your Help’s Navigation

Recently, at my request, one of my user research colleagues did some usability testing on a webhelp file. He did what’s called a “treejack,” which is a method that asks users to identify the place in a navigation system they would go to find a topic.

For example, if you were trying to figure out how to schedule a projector on a calendar (to use a scenario from my treejack), where would think you would find it in the navigation tree? The user then navigates a series of labels to drill down into what he or she feels would be the location.

The method is called a “treejack” because Treejack is the name of the online tool from Optimal Workshop. (I’m sure there’s a more general term for the activity.)

While you could do this same usability test using a series of document folders (nested inside each other), the treejack shows you the unsuccessful paths that participants follow. What I’ve learned is that users have a lot of success when the labels are clear, but when you have vague navigation labels, such as “Managing Calendars,” where a lot of different tasks and topics could be grouped, success rates decline. Words that have subtle distinctions in meaning, such as scheduling versus reserving, also mislead users.

Here’s what a sample treejack output looks like. This is a graph from the Optimal Sort site:

Treejack

This is what the results look like from a treejack. It shows you all the paths the users took. The green paths are the successful paths. Red paths are wrong paths. (Image from Optimal Sort.)

As I looked down the treejack paths to see where users clicked, I could see the logic that would lead a user to think the topic would be organized in that folder. For example, in my treejack, setting up and migrating calendars could have been grouped under Getting Started (it wasn’t), just as “turning off the old calendar,” a somewhat random miscellaneous topic, could have been grouped under FAQs (it wasn’t).

The problem with organizing help is that words have so many nuances and subtleties, they can be used in a lot of different ways. There are shades of meaning, words that accommodate multiple definitions. Using general terms such as “managing,” “viewing,” or “working with…” leads to a variety of topics that the folders could contain. Only when words have unmistakably singular meanings do users navigate to the right topic in the navigation tree almost every time.

Naming Conventions for Concepts and Topics

Another challenge users faced is how to distinguish between concepts and topics. In navigating the webhelp tree, it wasn’t clear to users if a topic was conceptual or task-based. In my content model, I start the first topic of each help folder with a conceptual topic, phrased as a gerund (“Managing the calendar”). I include the bulk of that folder’s conceptual information on that folder’s conceptual topic, and then let all sub-topics in that folder be tasks. I do this because I found that separating each concept out into its own help topic bloats the help and makes it seem unnecessarily complex.

For tasks, I use the imperative form (“Create a calendar,” “Approve a calendar”). I do this partly because I’ve found that the imperative form is much more search friendly. When searching for words, most users type active keywords that express what they want to do – for example, “create a new calendar.” Users rarely type “creating a new calendar” if they’re looking for information on how to create a calendar. However, the help engine in many webhelp files (such as Author-it) doesn’t interpret “create” and “creating” as the same form of the word. Only words that have matching stems (“edit” and “editing”) are treated as the same keyword.

Regardless, this distinction between gerunds for concepts and imperatives for tasks wasn’t clear to users. I could preface concept topics with “About,” perhaps. But as the landing page for the webhelp folder, my page usually matches the folder name. So unless I want “About” included in every help book title, I tend to omit it.

I’m curious to know if others have adopted a standard convention for distinguishing between task and concept topics. If so, I would be interested to hear what it is.

As always, the usability study opened up my eyes to a lot of the difficulties inherent in organizing help. There’s a reason why users often can’t find what they’re looking for, and it’s not just a matter of organization. It’s due to the slippery nature of language. Perhaps I need to think about ways to make language more exact, to shy away from words with vague meaning. I also need to provide more abundant cross-references in various topics to redirect users toward the “right” path.

Madcap Flare Adobe Robohelp

This entry was posted in findability on by .

By Tom Johnson

I'm a technical writer working for the 41st Parameter in San Jose, California. I'm primarily interested in topics related to technical writing, such as visual communication (video tutorials, illustrations), findability (organization, information architecture), API documentation (code examples, programming), and web publishing (web platforms, interactivity) -- pretty much everything related to technical writing. If you're trying to keep up to date about the field of technical communication, subscribe to my blog either by RSS, email, or another method. To learn more about me, see my About page. You can also contact me if you have questions.

25 thoughts on “Using Treejack as a Method for Evaluating Your Help’s Navigation

  1. Vilma Zamboli

    Well, happy to read that some best practices experimented in the past are still working! Even in the far Italy!
    Yes, we have a modular help structure that we apply to all our help. We respect the specific feature of each documentad software and target user and we remove or we add the module needed.
    We push all conceptual text, things “to know”, not to do.. we ad schemas and diagram. In a single sourcing view, this can be interesting for help owners in order to extract conceptual parts maybe to be delivered to some important potential customer to show how powerful are the sofgtware features!

  2. Patty Blount

    A song lyric popped into my head, but WHAT SONG?? ‘…sometimes words have two meanings.”

    While I go crazy trying to recall what song that’s from, let me say this is an excellent post. It never occurred to me that language itself could be the reason for findability complaints.

    I will be sure to pay attention to this issue as I design the information set for a brand new product.

    As for your concepts vs. topics issue, I’m leaning toward omitting concepts from Help entirely, or perhaps linking to them from the ‘imperative’ topics. I think users no longer care how technology works. They simply assume that it does. This touches on one of your earlier posts about how the internet has reduced our attention spans. I see this in my own ability to focus on tasks – there are so many things screaming for my attention. As I type this, a new email arrived and I MUST go click it. There are two instant message conversations blinking in my status bar. The desk phone plus the cell phone sit nearby. And, of course, there is this blog itself, with its links tempting me to click out before I finish this comment.

    I believe most users are like me and want as little information as possible to read in order to use the technology. I’m leaning toward designing Help systems for this audience and adding links to conceptual information for the few users who want to read more deeply.

    1. Marcia Johnston

      Patty, I suspect that you’re thinking of the Led Zeppelin classic Stairway to Heaven. “There’s a sign on the wall, but she wants to be sure, ’cause you know sometimes words have two meanings….”

    2. Tom Johnson

      Patty, thanks for commenting on this post. ” It never occurred to me that language itself could be the reason for findability complaints.” Thank you for this comment. It speaks to the main point of that post, which I wasn’t sure came across so well.

      As for minimalizing conceptual information in products, this is probably the trend in tech comm for sure. By conceptual information, I didn’t mean to refer to explanations about how things work, but rather info about workflows and limitations, reasons and other frequently asked questions whose answers don’t fit into a task users can do.

  3. Jonovitch

    I would argue that most people who go looking for help (on any website) don’t want to *know* something, they want to *do* something. (Yes, I recognize “doing” requires some “knowing, but that’s secondary at best.) Whether it’s in the Getting Started section or the FAQs, when I’m looking for an answer, it’s because I don’t know how to *do* something.

    If this confirms what you found in your tree jack, I’d recommend either moving the “concept” help files somewhere harmless and out of the way, or converting them into “task” help topics, where they can do some real good.

  4. Eddie VanArsdall

    Tom, I mainly stick with the basic “big three” used in DITA: concept, task, and reference. I follow these guidelines:

    * Preface conceptual information with “About…”
    * Use gerunds for tasks: “Creating a Widget”
    * Use the word “Reference” in a reference topic: “WidgetMaker Command Reference.”

    I disagree that any of these topic types should be excluded from help. Some users do want to understand the “why” behind tasks. Others like reference lists and tables that they can print or bookmark. As a user, I fall into both categories.

    1. Tom Johnson

      Thanks for sharing your approach to naming these topics. “Some users do want to understand the “why” behind tasks.” I agree that the “why” can be important to include. I didn’t explain it well in this post, but concepts don’t always answer the “why.” Sometimes concept topics explain limitations, workflows, and other bigger picture ideas.

  5. Marcia Johnston

    Tom,

    Love the treejack image and the fact that this kind of software exists.

    Good insight — well organized information isn’t necessarily FINDABLE information. It’s a challenging question: How do we structure and label topics so that people can find what they need?

    One of my workgroups wrestled with heading syntax. Like you, we went with imperative for task headings (“Create a calendar”) after much debate and research. FWIW, I side with those who call this the “root form” of the verb in this context since there’s no sense of imperative here.

    As for the syntax of concept topic headings, we never reached consensus. I like Kurt Ament’s suggestion (on page 86 of his 2003 “Single Sourcing: Building Modular Documentation”) of using the first word of a heading to indicate module type, for example, “About…” for general descriptions, “Types of…” for lists, “If…” for troubleshooting. But, as in your case, Tom, these extra info-type-indicator words can be problematic. For example, on mobile devices the word “About” might take up half the width of the display. So, no easy answer there.

    Your conundrum reminds me of an article I’ve found intriguing, which advocates creating Help in “task-support clusters.” Each cluster includes the following modules:

    - an introductory “keystone” concept topic (“the target of a context-sensitive link in the UI providing a functional overview including tips or examples that give insight into how best accomplish user goals”)

    - task topics

    - reference topics, if needed (info that users might require in support of those tasks… “might be unnecessary clutter to the average user, but less-experienced or more-experienced users could find them valuable.”)

    - “deep concepts,” if needed (info that’s more detailed than the keystone concept… again “might be unnecessary clutter to the average user, but less-experienced or more-experienced users could find them valuable.”)

    - inter-topic links

    Lots more on this here:
    http://www.uxmatters.com/mt/archives/2007/05/the-anatomy-of-a-help-file-an-iterative-approach.php

    Thanks for the stimulating post, as always.

    1. Tom Johnson

      “FWIW, I side with those who call this the “root form” of the verb.” You’re right. I have been struggling to properly identify this verb form. Imperative is a command form, and these aren’t really commands, even if the spelling is the same. I’ll have to refer to this as the root form going forward.

      Thanks for your other comments about prefacing conceptual topics with “About,” positioning that topic as the introduction, and so on.

      “One of my workgroups wrestled with heading syntax. Like you, we went with imperative for task headings (“Create a calendar”) after much debate and research.” The verb form for task topics is a current debate among our team right now. What research pointed you to adopt the root form? I would love to leverage the same. For me, the argument is simply that users use the root form in searches, so they are more apt to find the topic.

      1. Marcia Johnston

        Tom, You ask, “What research pointed you to adopt the root form? I would love to leverage the same.”

        Ooooo, if only I had captured that research in a way that I could retrieve it now. I no longer have access to the SharePoint site where I had posted a bunch of my findings. Also, as I note in a separate comment, my favorite STC “Hyperviews Online” web page — where I found the discussion on all the angles and nuances of all the possibilities on this question — is no longer available.

        Here’s what remains in my memory from that research back in 2006.

        My work group considered various syntax options for task-topic titles:

        - gerund (“Setting up the XYZ”)
        - infinitive (“To set up the XYZ”)
        - “How to” (“How to set up the XYZ”)
        - question (“How do I set up the XYZ?”)
        - root verb (“Set up the XYZ”)

        It took me a while to give up my defense of the old-school use of gerunds in task-topic titles coupled with infinitive headings (aka “stem sentences”) within the task topic to flag the beginning of the steps. For an excellent discussion of “stem sentences” and why it just might be okay to let go of them, see this link: http://dita.xml.org/wiki/stem-sentences. (This link does still work, yesssss!)

        In the end, I and my department adopted the root verb. The reasons that come to mind are these:

        - People search on root verbs (as you say, Tom).

        - On small displays, every character counts. The root verb (a) makes titles compact, so more of each title fits on the display and (b) puts the most important word first. It’s less than ideal to have a small screen display a bunch of topics that start with “How to…” The root verb form conveys — much more efficiently — the fact that this is a “how to” topic.

        - Gerunds can be difficult to translate. (This may be true generally, but it’s a weak argument when it comes to titles. Several translators commented that gerunds in titles would not present a problem.)

        - Gerunds take longer to process cognitively.

        - Gerunds take up more space.

        - Root verbs were then becoming, and have now become, common in task topic titles. No one would mistake root verbs for commands. (This was my original objection — that users would take root verbs as imperatives. I changed my mind when I noticed how easy it was to use documentation that used root verbs in task titles.)

        Note: My work group deemed gerunds as acceptable to introduce concept topic titles where appropriate. Some of us preceded the gerund with “About” to avoid looking a task topic. Consensus was elusive when it came to the syntax for concept topics.

        Please keep us posted on your group’s findings and decisions, Tom.

        1. Tom Johnson

          Thanks for elaborating more on this. It will give me some good points for discussion when I bring this up with my team. I hadn’t thought of all of these other reasons that you outline, such as the additional space, the cognitive processing, and the translation.

    1. Marcia Johnston

      Just found out that this old link I was looking for is now beyond anyone’s reach. It was part of a publication of the STC Online SIG, whose one-time manager, Ann Wiley, has just told me,

      “The Hyperviews Online posts that were in WordPress weren’t archived, so this does not exist anymore. Sorry I can’t retrieve it for you.”

      Thanks anyhow, Ann. It was an excellent publication while it lasted.

    2. Tom Johnson

      Just curious, Marcia, but how do you keep track of links like this (especially if the links are no longer present)? Do you use Delicious to store and track helpful sites?

      1. Marcia Johnston

        I started out using Delicious to tag my bookmarks. When I heard that Delicious was being “sunsetted,” I converted my bookmarks to Google Bookmarks. It’s been a much better experience, so I’m glad I made the change. I like being able to create my own tags for filtering and quickly finding web pages. I also like capturing my own notes as metadata on those web pages that I bookmark.

        In the case you’re asking about, Tom, the URL for the missing link was preserved in a presentation I put together a few years ago on topic-based writing. I wish now that I had captured the full discussion on that web page instead of merely the URL!

        1. Tom Johnson

          I didn’t even know Delicious was being sunset. I don’t use the tool that often, but I can see how useful it would be. It’s really hard to keep track of all the worthwhile content on the web.

  6. Jimmy Breck-McKye

    Glad to see usability research techniques applied to technical writing.

    Another IA method to borrow is card sorting. It’s a very simple activity – essentially, you write the names of your topics on post-it notes, then ask your users to stick them on a board, grouped into the categories they feel most appropriate. Low-tech, but it’s very easy to do.

    A word of caution about these IA methods, though – they assume a usecase where a user navigates from a homepage or at least another point in your website. In online help, that’s often not the case – your users alight on results via search engine. In that case, you might find that SEO-optimized titles and copy trumps the keyword insights of your IA activity.

    As for title keywords being specific – yes, yes, and yes. This is vital, but I see it done wrong surprisingly often. A topic named ‘Understanding the fizzwidget’ is not going to appear immediately useful to the user, who’s notorious for refusing to make upfront investments in learning time for long term-benefit. Only topics that appear directly linked to the user’s immediate task are ever going to be read – the rest will be shelfware.

    1. Tom Johnson

      Jimmy, thanks for bringing up card sorting. I need to do more of that. In fact, if you’re interested in writing a guest post on this topic, I would gladly welcome it. I’m interested to hear stories of real-world experience and interactions with users as you do card sorts.

      My hesitation with card sorting in the past has been my fear that users will look at a stack of 75 index cards and feel overwhelmed by what I’m asking them to do.

      1. Jimmy Breck-McKye

        You raise a good question. A large number of index cards wouldn’t just be intimidating, but sorting them would be time-consuming and a logistical nightmare to boot.

        You might try stripping your index cards so that you only provide a representative sample of items, and don’t provide an excessive number of items that you suspect will be categorized together in most imaginable organizational schemes.

        For instance, if you were writing a manual for a word processor, you might guess that most task items about setting specific font styles (bold, italic, strikethroughs, small-caps etc.) would be put together in most information architectures – that people will typically place them in the same category, whatever that category *is*. On that basis, you might remove a few items to make the activity more manageable, without being likely to lose many insights.

        Of course, once you do this, you start making assumptions about the ways your users will organize the data – and that obviously risks invalidating your results. In the example above, you might guess that you don’t need index cards for the topics on how to set strikethroughs and small-caps, only to later discover that users would naturally categorize them apart from more common paragraph stylings, putting them instead into their own ‘rare activities’ group instead.

        It’s all something of a dark art, and there’ll always be a level of interviewer bias – even giving the index card a topic title will promote certain organizational strategies over another.

        Of course, there’s always the slightly glib answer that if your users are overwhelmed by the index cards, your document or deliverable has too many topics to begin with. That’s a nice position to take, but probably not a helpful one in a real-world scenario.

        1. Tom Johnson

          Jimmy, thanks for the tips here. Breaking down the number of cards to a representative sample seems like a good idea. Also, I suppose I wouldn’t need users to drill down deep into a navigation hierarchy.

  7. Andrew Mayfield

    Fantastic use of Treejack. Thanks for a great article Tom. You mentioned that there must be a general term for a “Treejack” study, well there is: a Tree Test.

    In fact, before Treejack existed this activity was performed offline using paper based trees and referred to as “reverse card sorting”. This makes sense because a tree test is essentially a method for validating your labeling concepts, which have often been gleaned from an Open Card Sort in the first place.

  8. Pingback: Looking at Search Analytics to Improve Help Content | I'd Rather Be Writing

Comments are closed.