Here’s what the Yahoo Pipes feed looks like at the moment.

Writer River Yahoo Pipes feed

It’s simple compared to other Yahoo Pipes feeds. Basically the pipe takes RSS feeds from as many sources as I add here, sorts the posts by the date published, filters out any duplicate titles, and then merges all the information into one RSS feed. Writer River then displays this RSS feed on its home page. When you subscribe to the Writer River RSS feed (or when you subscribe to Writer River’s email delivery or Twitter updates), you’re also subscribing to this same Yahoo Pipes feed.

I’m convinced that human-assisted aggregation and filtering, with the help of such tools as Yahoo Pipes, is the trend for managing the deluge of information online. Since everyone is an author, publishing on separate sites, RSS is the only way to keep up. And people are publishing like mad, pushing out about a million posts a day.

Post titles are often hit and miss in terms of quality, so some human filtering is necessary. We need people to pick and choose the good content from the poor. People are naturally doing this all the time. I’m just trying to leverage those efforts in an effortless way to pull all of this good information into one running feed. This is what Writer River is all about. It attempts to gather all of this worthwhile content and help you find better information more quickly. If enough people participate, the quality of content flowing through Writer River could easily surpass the quality of any print publication.

Here’s an example. I like UXMatters, but I missed the latest articles published on it because I have hundreds of feeds in my feedreader and I don’t sit there watching feeds all day. However, Alistair Christie saw an interesting UX Matters article by Mike Hughes and posted briefly about it in his What I’m Reading category. I saw it on Writer River because Alistair told me about his What I’m Reading feed, and I added it to the Yahoo Pipe that’s feeding Writer River. I checked out the article tonight and immediately felt it was a valuable post. Without this human filtering and aggregation, I would have missed the post.

Now imagine if not just one or two people submitted similar What I’m Reading or What I’m Bookmarking feeds to Writer River, but dozens, even 100 people. It would be like having 100 researchers scouring the Internet for you, looking for the best posts available.

Here’s a little more math. Let’s say on average, the 100 researchers post one article a day to their What I’m Reading feed — one article a day they feel is worthwhile. In one month, that would be 3,000 articles.

Now of course not everyone has the same interests and tastes as you, so let’s say that only about 10% of these “worthwhile” articles are actually interesting to you. That still means that in one month, you’ll have 300 worthwhile articles to read.

Compare that to static print publications like the Tech Comm Journal, Intercom, the Communicator, or other print publications, which only have about 10 articles per issue, and you begin to see how valuable and powerful human aggregated content can be. This is the rationale behind Writer River. We now need more people to add feeds to it.

The manual method of going to the Writer River site and publishing a link to your post is somewhat archaic. It takes time and is slow. It takes effort. But the RSS feed doesn’t take effort. It only asks that you share your what-I’m-reading RSS feed with the Yahoo Pipe (by sending it to me, so I can add it), and then you don’t ever have to return to the site again. Content will just flow through the feed, however you choose to subscribe to it.

It makes sense to somehow mark or tag or bookmark or post or share or tweet good content that you read, right? You want to hang on to that article somehow so that you can find it later. That’s the nature of reading. But for online content, you need a method for keeping track of it, because the World Wide Web is too deep and wide and slippery to find something again after letting it go.

For those people who don’t have a blog or Delicious account, or Identi.ca or some other way of posting or marking content, I recommend starting one. One of the easiest ways to keep track of your good reads is through a WordPress.com blog, which is free, requires no maintenance, and provides you with an easy-posting bookmarklet that allows you to quickly add a link in two clicks from any page you’re reading.

Let’s pull together these efforts. Rather than having everyone run in their own direction, which accomplishes little, let’s harness all these individual efforts (which people are already doing) and turn them into a massive collective effort that dwarfs anything one simple person can do alone. Send me your category-specific RSS feed or links page and we’ll build an information machine that churns out the best content of the web without requiring you to do much at all to find it.

Blog Sponsors

• Webworks ePublisher
• Scriptorium
• Help Generator help authoring software
• Southern Polytechnic: Information Design and Communication
• Simplified English
• MindTouch
• Madcap Software
• Dr.Explain
• Adobe Technical Communication Suite
• Congree

Download MP3
Duration: 27 min.

In this podcast, I talk with Bill Albing, founder of KeyContent.org, about the impact of social media on technical communication. Bill talks about different ways social media helps audiences interconnect and interact. Good social media technologies enable professionals to collaborate easily, without being encumbered by complicated technology or even burdened by managing and filtering feeds.

Bill explains that the web is more than just a venue for publication — it’s a medium that allows people to interconnect and work/collaborate with information. This is the direction we’re moving towards, and technical communicators are starting to integrate social media, such as user forums, directly into their help.

Resources Mentioned in the Podcast

• Keycontent.org
• KeyContent blog
• Naymz
• Bill on Naymz
• Facebook
• Linked in
• Content Wrangler LinkedIn
• Last.fm
• Myspace
• Amazon
• Tikiwiki
• Flickr
• Secondlife
• Flare’s online help file with Feedback server
• My instructions for installing mediawiki
• Yahoo pipes
• Joann hackos’ blog
• Ann rockley’s blog
• Neil perlin’s blog
• Alan houser’s blog
• Bill’s slideshares on social media
• Slideshare

Podcast Sponsors

MadCap Flare is the most versatile XML-based Help authoring tool on the market, with thousands of customers using MadCap products including Microsoft, Google, HP, GE, yahoo and the list goes on. Check out Flare version 3.1 and a host of other new tools at at madcapsoftware.com.

Lunar Pages offers basic web hosting starting at $6.95. When you sign up for a basic hosting account, you get 350 GB of storage, 3500 GB of bandwidth per month, free tech support, Fantastico, and and dozens of other tools. If you’ve been thinking about starting your own self-hosted blog, contact Lunarpages.com to set it up.

Adobe – The Technical Communication Suite software offers a complete solution for authoring, managing, and publishing interactive instructional information from technical documents and books to online help systems, knowledge bases, interactive training, and eLearning content in multiple formats and languages. Learn more here.

Yahoo Pipes has received a lot of praise. In its debut, Tim O’Reilly said:

Yahoo!’s new Pipes service is a milestone in the history of the internet. It’s a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant.” While it’s still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.

A few examples will bring this tool’s capability into focus. Suppose you want to combine your favorite feeds into one feed. Yahoo Pipes allows you to aggregate the feeds, choose a sorting order, define filters, truncations, keywords, and so on. The pipe then generates a single RSS feed containing this information.

In another example, let’s say you want to monitor the web for keyword searches on “technical writing.” Traditionally, you could perform individual searches on various search engines, even subscribing to RSS feeds for each engine’s search results. But Yahoo Pipes allows you to define different data sources and then run your keyword search across all the data sources at once.

The Aggregated News Alerts Pipe (shown visually by the image on the left) takes the keywords you input and searches for the across the following search engines: www.blogpulse.com, news.search.yahoo.com, technorati.com, blogsearch.google.com, icerocket.com, findory.com, yahoo.com, www.bloglines.com, search.yahoo.com, api.technorati.com, www.icerocket.com, bloglines.com, search.live.com, news.google.com, rss.findory.com, blogpulse.com, google.com

After you type a keyword and click “Run this pipe,” the pipe retrieves the data and allows you to subscribe to the results as an RSS feed. This is an excellent way to keep your eye on the blogosphere for particular topics.

The BI Jobs Pipe, another piping application, allows you to search various sources for jobs, after inputting a location and job title.

Yahoo Pipes is not immediately intuitive (but here’s a good tutorial for doing the simplest mashup). Essentially, the pipe-building is a bit like creating an advanced search with different criteria. You can define operators, filters, number of returns, dates, locations, and other parameters for the search returns. You also define the data sources. The pipes can be robust and intricate, or they can be simple. What’s totally new is the visual drag-and-drop pipe interface.

Yahoo Pipes also allows you to clone well-built pipes and tweak them to suit your own purposes. Because of this, Yahoo Pipes is very web 2.0. You can browse the most commonly run and cloned pipes by clicking “Browse Pipes” from the Pipes home page.

Although I think the product is unique, innovative, and powerful, I was little disappointed in Yahoo’s documentation. I may be mistaken, but it appears Yahoo launched its innovative product with little instruction. Their home page says,

How do I use pipes?

Please check back, in a few days we’ll have online tutorials demoing how to get the most out of Pipes.

The product was launched more than a month ago. I guess if the product is cool enough, users will write the help.

The drag-and-drop interface, with connecting pipes all over the place, make it visually appealing and fun. In the near future, I’d like to create technical writing pipes that would be useful to our community. I created a simple pipe that aggregates 10 different tech writing feeds, but for some reason when I subscribe to the pipe’s RSS feed in IE or FeedDemon, it strips the feed of its formatting. Another limitation is that you can’t see the source each feed is coming from (although Lifehacker apparently has a workaround).

In using Yahoo Pipes, it’s probably easiest to find successful and interesting pipes, clone them, and then tweak them to suit your purpose.

Update: I found some excellent video tutorials for using Yahoo Pipes at http://usefulvideo.blogspot.com. There’s also a written tutorial here.