Merge & filter news feeds with Yahoo Pipes

In the previous post, I talked about EventBox and how it can make your feed-loaded life easier. But here’s the thing: It’s again skimming through 100s of feeds. Could I be lazier?!

Well, there is a way that you can merge all (or a particular group of them) together and filter them against some keywords.

So what? Then you have a SINGLE feed filtered based on the topic you want to read about. Imagine you are interested about what is happening to GM and you check CNN, BBC, MSNBC, and millions of other feeds everyday. What if you could just check one single feed?

Getting Started

All you need is a Yahoo account. Yahoo has a brilliant tool, Yahoo! Pipes, that will let us create what we want. Actually Pipes can do much more than what we need and you can check it out if you are interested.

OK just login to Pipes page at pipes.yahoo.com to start creating your first pipe. There are useful demos and documentation for you to get into the details.

Click on “Create a pipe” to go to the design environment.

It might look a bit scary for newbies but it’s extremely easy to use.

Start with dragging a “Fetch Feed” component from the right sidebar and dropping it to the work area.

Suppose you want to focus on the “Large Hadron Collider” that CERN built to find out about the begining of the cosmos.

So we need to search in news and scintifics sites. If you don’t have any particular sources in mind, just create a related Google News feed and add it to the “Fetch Feed” component.

I added 2 more feeds (you can add as much as you want) from Reuters and Science Magazine to enrich my search scope. Try adding more sources yourslef.

Note: Don’t forget to save your work regularly or you lose your beloved pipe!

Now drag a “Filter” component from the sidebar and drop it below “Fetch Feed”. It’s in the “Operators” section.

Well, now to make the work flow to make sense, try to connect the output of the “Fetch Feed” (the circle on the bottom of the dialog box) to the input of the “Filter” (the circle on the top of the dialog box).

We need to put something in the “Filter”. Try “Large Hadron Collider” for the title of the feed. To make sure we won’t miss anything, enter “LHC” for feed title as well as “Large Hadron Collider” for the description.

don’t forget to change “Block” to “Permit” AND “all” to “any” at the top of the dialog box, otherwise you’ll see everything but what you were looking for!

Now it’s time for the output. Just connect the output of the “Filter” to in put of the “Pipe Out” and click on the “Pipe Out” itself (or click on “Refresh” in the debugger area down below the screen) to see the results of your very first pipe.

There’s one more step to go. To use the output of the pipe you need to add its RSS feed to you favorite news reader. Click on “Run the pipe…” on top of the screen. A new window/tab opens that has loads of options and tools like publishing tools and etc. All we care is to get the output as RSS. By clciking on “Get as RSS” you can add the feed to your reader and save time by checking ONLY one feed about a topic.

Just note that this pipe is very basic and you can fine tune it with other tools like Regular Exressions and etc.