Findability and The Information Paradox

Last year I started a series on organizing content that spanned nearly 30 posts. I want to return to this thread with a summary of why findability becomes an issue for technical writers, and what the information paradox is that we encounter. Then, in an usual ethical twist, I’ll explain why findability might not actually be an issue.

The Documentation Scenario

The help scenario starts out innocently enough. As a technical writer, I document the main tasks users can do in an application. I describe core features and how-to information.

But as times goes by, and I get deeper and deeper into the application, I learn more and more about it — the not-so-intuitive processes, the workarounds, the edge cases, the unresolvable bugs, the usability problems. I transfer this information into the help file because information is my game.

When the app is released, I start to hear feedback from users through various means — calls to the support center, questions during training, emails and conversations. I add much of this information to the help. In the back of my mind, I envision a day when all users can search the help for even the most arcane questions, the most unique situations, and they will find — to their surprise — the answer.

In fact, during one project, I even played a support role for the application. I made it my goal to never answer users’ questions without pointing them to the answer in the help file. This would accomplish two things: (1) it would require me to supplement the help file with possibly missing information; (2) it would increase the user’s confidence in the help. So phone call by phone call, email by email, I added more and more information to the help file until something strange started to happen.

The findability of help topics started breaking down. I reached the point where the help information transitioned from somewhat straightforward to definitely complicated. Each folder had a lot of topics, and some folders had subfolders and subfolders with a lot of topics. I began to forget where some topics were, since they overlapped topically with other folders. I knew that if I forgot where content was, users would have even less of an idea where to look. At this point, I clearly had a problem that I didn’t have when everything was simple.

The Information Paradox

There’s a point at which help files reach a threshold of findability. It doesn’t happen with simple applications — it’s more a problem with semi-large help systems. And it’s also more common when you have a post-release documentation methodology (in which you continue to add the help content after release).

Passing this threshold leads to a paradox that I call the information paradox: The more information you add to a help file, the more informative it becomes because it contains more information. However, as you add more information to your help file, it also becomes less informative because it’s harder for users to find that information.

Here’s an example. Let’s say you just bought a new smart phone. In a weird hypothetical scenario, the store clerk gives you two options for help: you can either have a 5 page quick reference guide, or you can have a 500 page reference manual. But you cannot have both. Which do you choose?

If you choose the quick reference guide, you’ll be able to easily read the information and understand the core tasks, but your advanced questions and tough scenarios might not be addressed. If you take the 500 page manual, it will probably address all your questions, but the process of finding the answers may be too burdensome and tedious to be worth the effort.

The information paradox comes into play here. The more information you add to a help file (in this case, the reference manual), the more informative it becomes because it contains more information. However, the reverse is simultaneously true: the more information you add to a help file, the less informative it becomes, because users can’t find the information. It gets buried among the hundreds of pages.

Here’s an analogy that expresses the same paradox. Let’s say you have a friend who talks your ear off. The more the person talks, the more information is available; with more information, the more potentially informative the conversation is. However, the more a person talks (and yaks on and on and on, talking your ear off), he actually communicates less information, because you start tuning him out. Each new word (or paragraph and topic in a help file) gets diluted in a sea of information until you arrive at the conclusion that he “talked about everything and nothing.”

Document Everything?

Let’s come back to the help situation. My assumption is that there’s a point at which help topics start to break down under a mass of information. When help reaches this threshold, going beyond it actually becomes detrimental to the user because the information starts to lack continuity and focus. It gets sidelined by a myriad of notes, tips, gotchas, quirks, exceptions, side notes, references, extended explanations, details, notices, and other information. A simple process sinks under the albatross of TMI (too much information).

The activity that often pushes help over the edge is the effort to document everything. When I played a support role and tried to answer every user question by pointing users to a link in the help that answered their questions (even if I had to write it before responding), I had an idea of documenting everything. I envisioned a day when I would have a sort of omniscient manual.

But documenting everything is a controversial strategy in technical communication. Some point it out as a myth. Alistair Christie and Graham Campbell had an extended exchange on this topic in one of their podcasts. Let’s suppose they’re right. If you abandon the desire to document everything, and instead only focus on main questions from users, looking at the trends of questions only, do you still run into the information paradox?

If you stay behind that threshold where help still remains relatively findable, are you better off in the long run, and does the whole problem of findability go away? This is a key assumption that begins my discussion on organizing content, because if you don’t document everything, if you only document selectively, chances are your users won’t bang their heads in frustration looking for buried answers. You won’t need to implement advanced techniques for organizing content, and your topic-based folders may not pose any issues with findability.

Expansion and Influence

This isn’t merely a theoretical speculation. I am seriously considering that my strategy for comprehensive documentation is flawed. A couple of years ago, I used to carpool to work with a quality engineer named Kevin. During this time, I operated under the idea that my online help would be a comprehensive source of information for the application I was documenting. It would be the omniscient manual that I was striving for, answering every user’s question. But in our organization, technical writers are scarce resources. Four technical writers try to serve the needs of an 800-person IT department.

In my conversations with Kevin, many times I expressed frustrations with the lack of user assistance on projects in our organization. On many projects, project managers either omitted help or wrote it themselves. Kevin pointed out that if I really felt this way, I should begin a crusade of quick reference guides for as many applications as I could lay my hands on. The information I produce would be short, just a couple of pages, but my reach would be widespread. Not only could I provide the missing user assistance needed for so many applications, he said, with this approach I would also provide users with a form of documentation they would actually read. Like many people, Kevin believed long manuals were unread dinosaur relics that no one used or wanted. My time would be better spent keeping documentation short and sweet. Lots of quick reference guides, lots of projects, lots of influence.

I never followed Kevin’s advice because I couldn’t bring myself to work on a project where my deliverable was only a superficial two-pager with brief information. I couldn’t forego addressing what I felt were the true needs in a help file — those difficult questions that surface from power users and edge-case scenarios. After all, this was my own frustration as a software user: the instructions never seemed to have the answers I needed. But looking back, maybe I should have listened to Kevin.

The 80-20 Rule

The 80-20 Rule: 80 percent of the results come from 20 percent of the efforts.

Kevin’s advice follows a rule know as the 80-20 rule, or the Pareto principle. The idea is that “80 percent of the effects come from 20 percent of the causes.” Let’s say that it takes me 20 percent of my time to create a quick reference guide for an application. Despite only expending 1/5 of my time, the quick reference guide’s effect might influence 80 percent of the users.

This sounds improportional at first, but if you think about it, the idea has merit. Imagine a scenario where you hand out both a quick reference guide and a reference manual to 100 people in a crowd. Assume everyone reads something. About 80 of them would read the quick reference guide , and only 20 would read the reference manual. Right?

Are we technical writers so ethically obtuse that we can’t triage a documentation situation? We end up so bent on writing the all-encompassing reference manual, expending a lot of effort (80 percent of our effort) chasing down answers to arcane questions and edge cases that almost no one needs. Our time would be better spent focusing on the core ideas/tasks and then moving on, leaving the straggling few oddball users to make do without (you know, those users who ask for the full printed manual or who ask about running the app remotely on Ubuntu through a proxy.) Sacrifice a few for the good of the whole. This isn’t murder we’re talking about — just the unfortunate circumstances of time and information. Our influence could be enormous, if we followed logic.

The result of not documenting everything invariably leads to less information. With less information, the help file is smaller and topics are more findable. A quick reference guide that is between 2 and 20 pages in length does not have a findability problem, and the whole information paradox becomes somewhat of a moot point.

In a world of 4 technical writers for 800 IT professionals, the 80-20 rule makes sense. But I also admit that it isn’t ideal. In an ideal world, information is complete; all users’ needs are addressed. Ultimately we continue on with the old model believing that some day, when project managers see our omniscient help systems, which reduce customer support, improve the user experience, and help user productivity soar, they’ll recognize our value and hire more technical writers to document their application. That scenario, however, seems like a day on the horizon that never fully dawns. Maybe we should change our perspective.

Madcap Flare Adobe Robohelp

This entry was posted in findability, general on by .

By Tom Johnson

I'm a technical writer working for the 41st Parameter in San Jose, California. I'm primarily interested in topics related to technical writing, such as visual communication (video tutorials, illustrations), findability (organization, information architecture), API documentation (code examples, programming), and web publishing (web platforms, interactivity) -- pretty much everything related to technical writing. If you're trying to keep up to date about the field of technical communication, subscribe to my blog either by RSS, email, or another method. To learn more about me, see my About page. You can also contact me if you have questions.

29 thoughts on “Findability and The Information Paradox

  1. Ben Minson

    The 80-20 rule you mentioned reminds me of the law of diminishing returns, which an interaction designer once discussed with me while talking about usability testing. Basically, after you test with about five people, you’ve gotten 80%–90% of the findings you’re going to get, even if you tested with dozens more. If you keep going, you’ll mostly get the same results.

    I think this applies here. 80% of users will have the same or similar questions. If you try to go beyond that, your returns diminish—you’re putting forth a lot of effort for not much benefit. So it’s good to focus on addressing the needs of the majority and then move on if you have other big needs to address among the 800 IT professionals that we four serve.

    Maybe some tech writers feel compelled to create all-encompassing documentation because they can’t stand the thought of someone sitting out there helpless and stuck.

    1. Tom Johnson

      Ben, I like how you’ve contextualized this 80-20 principle into usability testing. I completely agree. It makes sense that one would generally move on after reaching the 80% mark, because by moving on, one can knock out an 80% achievement in another dimension of testing or otherwise.

  2. Patty Blount

    I thought the analogy you used, comparing the information overload to a talkative friend, was simply brilliant. I admit, I’ve always struggled with the whole “less is more” theory because I’m a writer and I feel as if I’m shirking my duties by not writing. But I think the key point here is that we’re more than writers. We’re reminded to consider big-picture issues like how do our readers actually use the information we provide?

    Thanks for the reminder.

    1. Tom Johnson

      Thanks for your comment about the talking friend analogy. That rings true to my life, esp. for one of my wife’s friends who talks about 1,000 words a minute.

      The whole “less is more” discussion has been a hot topic lately, esp. given the mass of information that so many people are drowning in.

  3. Julio Vazquez

    Another great post, Tom. A long time ago, the adage was, “80% of our users read 20% of our documentation”. It always seems like we bow to pressure to document everything, even though we can never get a business justification from anyone for those edge cases or “expert” requests. I believe that a lot of the document everything attitude comes from a sense of pride, either from the technical communicator or the developer of an application. In either case, it may be misguided.

    I have a basic mantra when looking at a product when planning my writing. That mantra is “who cares?” If the answer is not the person using the product, I mark that content as a low priority and may even remove it totally from the development effort. If I can’t come up with a user profile that would be a person who cares about the information I’m considering, it also goes on that pile. Is it perfect? Probably not, but it’s been right more often than not.

    1. Tom Johnson

      I know I would be much better off approaching documentation with the mindset you describe. This post may be a perspective-changer for me when it comes to comprehensive documentation. Your persona method seems sound.

  4. Larry Kunz

    Excellent thoughts, Tom. To a writer who wants to get every bit of information into the doc, I’d ask: If no one can find the information, is it really information at all? I’d suggest that it’s not.

    Having said that, my mind loops back to our discussion of the semantic web in your “Organizing Content” series, I really liked your response to, and the discussion that followed, my comment on your post about faceted classification and search. Your subsequent interview with Patrick Warren was also illuminating.

    So, can the semantic web help to overcome the paradox and make information easier to find? I think it can. While it’ll never be a perfect solution, it might turn the 20:80 ratio into something like 10:90.

    1. Tom Johnson

      I still feel very novice about the semantic web, but my impression is that the semantic web may assist with finding general categories of information (for example, finding Darwin Information Typing Architecture rather than Dita Von Tesse when searching for DITA), but once you arrive at that category of information, you have to rely on other findability methods to drill down into the right info. Also, the semantic web is still years out from mainstream adoption. I wasn’t so much exploring solutions in this post as questioning assumptions.

  5. rick

    If help file is context sensitive (and if it isn’t, is it really helpful?) is the overall organization really that important? Do I really need/want to see the forest when I’m only interested in a specific tree… or branch?

    With context-sensitivity, don’t we have *automatic* findability?

    1. Tom Johnson

      Context-sensitive help is a good technique for improving findability, but the topic that a help author connects to a specific screen may not be the one the user is looking for. The topic may also show users the tree instead of the forest. If context-sensitive help solved all findability problems, you’d see a lot more context-sensitive help out there. But many times users don’t even know how to get the screen that would prompt the help topic they’re looking for.

  6. Karen Mulholland

    All-encompassing documentation isn’t a possibility for me, or for you – but even if we had the time and resources to do it, we shouldn’t.
    The information paradox is not the only problem. Its evil(er) twin is the problem of DUHcumentation: our urge to document the forehead-smackingly obvious. If we allow ourselves to write help for features that the UI explains clearly, we are wasting time – nobody’s going to look for help on things they can work out on their own. First-pass documentation triage needs to weed out help topics that people won’t need.

    1. Tom Johnson

      “DUHcumentation” — I love it. I hadn’t heard that term before. You’re right to pass on the obvious information. But for the less tech-savvy users who may need that basic info, are they left in the dust as we follow our strategy to hit the needs of the majority of users?

  7. Anne Sandstrom

    When non-professionals do doc (often in wiki format these days), the organic growth makes information difficult or impossible to find. As professionals, we reach the ‘impossible to find’ threshold in our doc much later. But we do reach it, as you say. It’s like architecting a building. A weekend carpenter could maybe build a two-story building. Professionals can build something 100 stories high (or even ~ 150 in Dubai). But there is always a limit. Oddly enough, elevators are the biggest factor in the limit of building height. It’s (sort of) a navigation problem, where the means to get anywhere within the building actually takes over more and more real estate in the building!

    1. Tom Johnson

      Anne, thanks for commenting on the “impossible to find” threshold. I’m glad to see some agreement with this idea, and you’re right that professionals probably go a lot further before they face this issue.

      Re elevators being the limitation on skyscraper limits, I hadn’t heard that before. Interesting.

  8. Tammy

    Very interesting thoughts. My concern is this, though: doesn’t a quick reference guide typically cover the common tasks that, if the software is designed well, users can probably figure out themselves? Many users only turn to documentation when they hit something they *can’t* figure out – and that’s where the more comprehensive documentation is usually needed (for those edge cases, for example). If we only document the basics, are we less likely to document issues that people will actually turn to docs for?

    1. Tom Johnson

      Good point. This problem doesn’t have an easy solution. Many different types of users turn to help for different purposes. The quick reference may be a good instructional source for new users unfamiliar with the app; once they become comfortable with the app, the quick reference is no longer useful, but the comprehensive documentation is. In contrast, the comprehensive documentation isn’t relevant to new users — it’s too intimidating and detailed. But the quick reference does serve their needs.

  9. Craig

    I ran into this problem a few years ago. I solved it mostly by listening to the SMEs. If they seemed concerned about an issue that arose, I put it in the guide. If the issue arose but quickly sank again, I didn’t put it in. If I was confused, then I asked. If I put something in, and the guide didn’t seem better for it, then I removed it. It’s a judgment call, really.

  10. Ben

    Really interesting article. I’m facing a similar issue of needing to rewrite help documentation. My first version was pretty comprehensive and written in a bit of a rush to hit the project launch, so I knew at the time I needed to revisit and ‘finish’ it.

    Now I’m getting around to revisiting it I’m finding that I don’t need to ‘finish’ it at all, I need to work out what to get rid of so people actually read what’s left.

    There was a really good webinar ‘The Need to Weed’, hosted by Gerry Mcgovern, focussing on a a clean up exercise that Microsoft went through for their online Office help content. They found they had to archive and merge pages that were annoying more people than they were helping. The webinar is still available at http://www.customercarewords.com/webinar-previous.html – The Need to Weed: Microsoft Office Online.

    1. Rebecca O'Connell

      As a person who is often the oddball user looking for arcane information about an edge case, I usually find it is by doing a Google search, which generally points me to either a user forum or (less frequently) a Knowledge Base article. Perhaps the formal documentation should center around reasonably detailed information about how to do the major tasks performed with the program, and leave the more arcane information out. This would, I think, be sufficient for less advanced users who turned to the help because they couldn’t do something.

      That more arcane information could then be contained in well-tagged FAQ or Knowledge Base articles that aren’t part of what shows up in the help’s table of contents (the material that would appear in a printed manual). Alternately, the company could (as some do) have employees that monitor their forums, and contribute to some threads. I think having someone like that who can jump in and say, for instance, that you can’t find a way to do this arcane thing because it can’t be done in the program is very helpful, both to the current user and to other “long-tailers.”

      1. Tom Johnson

        Rebecca, excellent point about including topics that state what users can’t do. I covered this strategy in my Known Limitations post. I am for it, though many product managers may be uncomfortable stating so many product limitations to customers.

    2. Tom Johnson

      Thanks for the webinar recommend. I like Gerry McGovern’s philosophy. However, with a product like Microsoft Word, they hit the findability threshold long ago. Removing some topics may not help out too much. Still, this question of whether to add or remove information is fascinating. Does adding more information actually obscure other information, hence reducing the value of adding information? When you add more information, are you really subtracting information, since you remove focus from the core tasks and place it on the less important information? Perhaps with the right findability techniques, this paradox isn’t such an issue.

  11. Richard Sanford

    Most studies show that users have a high resistance to consulting extrinsic documentation (Help systems) and a nearly absolute resistance to reading external documentation (user guides). I don’t think the solution is to reduce the amount of information, but rather to blur the distinction between the user interface and the user assistance (context-sensitive Help), provide rich content on the index tab, and since we’ve all been Googlized, ensure that the system integrates with superior search. Rengaraman made the same point earlier. In my Help I try to cover all program controls and features (users get annoyed when they can’t find labels they see in dialogs) and reasonably likely usage scenarios. I also like to include a “Top 10 How To’s” and keep in mind John Carroll’s principle of minimalism focused on error recognition and recovery.

  12. Rob Crowther

    I think the dichotomy you’re proposing is the cause of the issue you’re seeing:

    “you can either have a 5 page quick reference guide, or you can have a 500 page reference manual. But you cannot have both. Which do you choose?”

    I know you said the situation was hypothetical, but you frequently see the same issue in real life – books that are trying to be everything to everyone, beginners guides that also allegedly serve as complete references. The answer to me as a user seems obvious – I want both. There’s no way a single document can serve both needs well at the same time.

    1. Tom Johnson

      Good analysis. I always try to include both a quick reference and a longer guide. Providing both does help solve many problems. Still, when you look at findability in the longer guide, you still run into issues because this is where you cross the findability threshold. My point in posing the scenario was to point out that short guides don’t have a findability problem; long guides do. In this scenario, the user would probably look first at the short guide and quickly conclude that the answers aren’t there. Then the user would turn to the long guide and expend a lot of effort trying to find the topic. Both formats have limitations — the short guide has findable topics but lacks much information. The long guide doesn’t have very findable topics but contains a lot of information. Giving both formats to the user helps but doesn’t completely solve the problem.

  13. Diane Borgwardt

    I agree with the point of this article. To compound the issue, however, are the studies that show college students are not able to determine when they have found information they are looking for — online, in help systems, etc.

    If we have the time to document everything, future users of the software will not even know they’ve found the answer if they do actually hit upon it.

  14. Jonatan Lundin

    One of the most challenging tasks for tech communicators is to identify what content to include in a manual and how to organize it to make the manual searchable.

    I’ve been working on a concept called SeSAM since 2006. SeSAM has several purposes, but the most important is to provide technical communicators with a framework that helps them identify and organize content to make it searchable.

    SeSAM start off from several view points, and the most fundamental is that users are searching and that they are goal driven. The minimalist philosophy states that the user is active when using a product. Users are using the product based on prior background knowledge (“hey, if I try to do like this the product might work”). Users are also lazy in a sense where they want to put in as little energy as possible to reach a goal.

    When the prior knowledge is not sufficient, a user tries to find information elsewhere and at a certain point a manual is used. Based on this assumption you can elaborate search situations. SeSAM includes eight search situations (see http://www.sesam-info.net) and they are developed to be generic, meaning that they are not targeting a specific type of technical product.

    The idea is that a user located in a search situation shall be able to identify the situation in the manual and be guided to the part in the manual that gives the answers. Content is built around “Meaningful result statements (MRS).

    When a user is searching for something there are several facts to consider: First the user might be helped if s/he understands what type of content is available in the manual. Then given that the user has understood what is available, s/he wants to know where in the manual a specific info is located. This assumes that the user can express what s/he is looking for.

    Thus one of the hypotheses with SeSAM is that it becomes much easier to find information if the user is aware and understand the logic and principles behind what type of content is available and how it is organized. Sort of standardizing the content architecture. SeSAM provides a mechanism to standardize the architecture.

  15. Graeme Forbes

    The paradox will be removed
    If the quality of indexing is improved.
    The comprehensive can be retrievable;
    Good metadata makes it achievable.

Comments are closed.