Yahoo! Pipes

Introducing Yahoo! Pipes

Yahoo! Pipes translates the Unix pipe concept to the Web. The last few years have seen an explosion in data available on the Internet. In the early days of the Web, HTML was the rule—great for displaying in a browser, but not good for much else. Now we're in an era of structured data: XML, RSS, Atom, and JSON, as well as web APIs, which are programming tools that extract and manipulate data from sources like Google, Yahoo!, Amazon, and eBay.

All of these sources create structured data. Yahoo! Pipes gives programmers (and intrepid nonprogrammers) a set of general software tools to merge, filter, slice, and dice this wealth of data.

What's an RSS Feed?

The Web has evolved. In its early days you might find a dozen sites you enjoyed. You'd bookmark them and spend an hour each morning navigating from site to site looking for new stuff. Today, most of those sites have an RSS news feed. This is a simple text file they keep on their server. When they update content—add a news story or a blog entry—they also update this RSS file.

The RSS file isn't intended for human consumption. It's designed to be read by a computer program called a feed reader. You use the feed reader to subscribe to each of those dozen sites' news feed, i.e., that RSS file.

Now, instead of navigating to a dozen different sites, you fire up your feed reader and see what's new at a glance. And many feed readers have advanced features that let you filter and search for just the content you're looking ...