Yahoo Pipes and the Mashed Up World of Aggregated, Filtered, Blended Information

Yahoo Pipes is a new online tool that allows you to blend, manipulate, and combine feeds from various data sources to create a streamlined, single feed of information. Essentially Yahoo Pipes allows you to create feed mashups of different data sources without having a knowledge of programming.Yahoo Pipes

 

Yahoo Pipes has received a lot of praise. In its debut, Tim O’Reilly said:

Yahoo!’s new Pipes service is a milestone in the history of the internet. It’s a service that generalizes the idea of the mashup, providing a drag and drop editor that allows you to connect internet data sources, process them, and redirect the output. Yahoo! describes it as “an interactive feed aggregator and manipulator” that allows you to “create feeds that are more powerful, useful and relevant.” While it’s still a bit rough around the edges, it has enormous promise in turning the web into a programmable environment for everyone.

A few examples will bring this tool’s capability into focus. Suppose you want to combine your favorite feeds into one feed. Yahoo Pipes allows you to aggregate the feeds, choose a sorting order, define filters, truncations, keywords, and so on. The pipe then generates a single RSS feed containing this information.

Yahoo PipesIn another example, let’s say you want to monitor the web for keyword searches on “technical writing.” Traditionally, you could perform individual searches on various search engines, even subscribing to RSS feeds for each engine’s search results. But Yahoo Pipes allows you to define different data sources and then run your keyword search across all the data sources at once.

The Aggregated News Alerts Pipe (shown visually by the image on the left) takes the keywords you input and searches for the across the following search engines: www.blogpulse.com, news.search.yahoo.com, technorati.com, blogsearch.google.com, icerocket.com, findory.com, yahoo.com, www.bloglines.com, search.yahoo.com, api.technorati.com, www.icerocket.com, bloglines.com, search.live.com, news.google.com, rss.findory.com, blogpulse.com, google.com

After you type a keyword and click “Run this pipe,” the pipe retrieves the data and allows you to subscribe to the results as an RSS feed. This is an excellent way to keep your eye on the blogosphere for particular topics.

The BI Jobs Pipe, another piping application, allows you to search various sources for jobs, after inputting a location and job title.

Yahoo Pipes is not immediately intuitive (but here’s a good tutorial for doing the simplest mashup). Essentially, the pipe-building is a bit like creating an advanced search with different criteria. You can define operators, filters, number of returns, dates, locations, and other parameters for the search returns. You also define the data sources. The pipes can be robust and intricate, or they can be simple. What’s totally new is the visual drag-and-drop pipe interface.

Yahoo Pipes also allows you to clone well-built pipes and tweak them to suit your own purposes. Because of this, Yahoo Pipes is very web 2.0. You can browse the most commonly run and cloned pipes by clicking “Browse Pipes” from the Pipes home page.

Although I think the product is unique, innovative, and powerful, I was little disappointed in Yahoo’s documentation. I may be mistaken, but it appears Yahoo launched its innovative product with little instruction. Their home page says,

How do I use pipes?

Please check back, in a few days we’ll have online tutorials demoing how to get the most out of Pipes.

The product was launched more than a month ago. I guess if the product is cool enough, users will write the help.

The drag-and-drop interface, with connecting pipes all over the place, make it visually appealing and fun. In the near future, I’d like to create technical writing pipes that would be useful to our community. I created a simple pipe that aggregates 10 different tech writing feeds, but for some reason when I subscribe to the pipe’s RSS feed in IE or FeedDemon, it strips the feed of its formatting. Another limitation is that you can’t see the source each feed is coming from (although Lifehacker apparently has a workaround).

In using Yahoo Pipes, it’s probably easiest to find successful and interesting pipes, clone them, and then tweak them to suit your purpose.

Update: I found some excellent video tutorials for using Yahoo Pipes at http://usefulvideo.blogspot.com. There’s also a written tutorial here.

Adobe Robohelp Madcap Flare

This entry was posted in general, web 2.0, web design on by .

By Tom Johnson

I'm a technical writer working for The 41st Parameter in San Jose, California. I'm primarily interested in topics related to technical writing, such as visual communication (video tutorials, illustrations), findability (organization, information architecture), API documentation (code examples, programming), and web publishing (web platforms, interactivity) -- pretty much everything related to technical writing. If you're trying to keep up to date about the field of technical communication, subscribe to my blog either by RSS or by email. To learn more about me, see my About page. You can also contact me if you have questions.

5 thoughts on “Yahoo Pipes and the Mashed Up World of Aggregated, Filtered, Blended Information

  1. Pingback: MiramarMike

  2. Pingback: Linkjockey - The Best Site for Cool Daily Links

  3. Pingback: Core Dump

  4. Pingback: polyphonic ringtones for sony ericsson

  5. automated blog

    I think what you composed was actually very reasonable.
    However, consider this, suppose you composed a catchier title?
    I mean, I don’t want to tell you how to run your blog, but suppose you added a headline that grabbed a person’s attention?
    I mean Yahoo Pipes and the Mashed Up World of Aggregated,
    Filtered, Blended Information | I’d Rather Be Writing is a little vanilla. You should peek at Yahoo’s front page and watch how they create news titles to get people interested.
    You might try adding a video or a picture or two to get people excited about what you’ve got to say. Just my opinion, it could make your website a little livelier.

Comments are closed.