Yahoo Pipes
Two people sent me links to Yahoo's new “Pipes.” Pipes is a very neat concept and some of the technology behind it seems pretty cutting edge.
What is Pipes?
Pipes allows you to mix and match various data sources such as RSS feeds and create something totally new. In it's very simplest form you can splice together two RSS feeds and create a single aggregate.
But, there are operators... filters... loops, content analysis, location extraction, language translation, and more.
You wire the various input sources into these operators and push them to an output which you can ultimately subscribe to. Outputs are RSS, Atom, and JSON (JSON is a data format that lets JavaScript get values without needing to parse XML).
Here is an iconic representation of my sample pipe, OozeTest1. This pipe aggregates my Blog's RSS with a flickr feed on keyword beach. Oh, and not any beach pictures, but beach pictures by my hometown in 11793 zip.
What people are saying about Pipes
This blog entry does a pretty good job describing the new Pipes service and offers some thoughtful commentary on the mixed reception it will likely receive.
Along with the good a service like this can do, content publishers may be scared of the service because it can be used to modify their feeds in arbitrary ways... like removing ads. Publishers can block pipes using various web tricks (blocking the “Yahoo Pipes” User-Agent, sending Yahoo! an e-mail asking to have your site blacklisted, etc.)
All the while, Yahoo! will now be the destination and penultimate aggregator of feeds, which seems like a pretty good business based on how I see FeedBurner expanding and expanding. I suppose there is no reason why just about all of FeedBurner's features could not be duplicated using Pipes. Plus, users will have arbitrary control on how the features work, which would be a step above FeedBurner's capabilities.
How did they do it?
Pipes seems to have been built using Perl and the Yahoo UI JavaScript library. I'm actually pretty amazed at what I see in the implementation. Although the GUI is a bit sluggish on my BlackBook, it's pretty darn pretty.
Plus, they draw these shiny pipes that aren't images and don't seem to be flash elements. I brought up “Page Info” in Firefox and looked at the media assets on the page... no pipe images. I right-clicked on the layout canvast (all parts of it) and didn't see any Adobe Flash player menus.
View source revealed a neatly implemented page, only two screenfulls tall. At the top are a bunch of links to javascript imports:
<script type=“text/javascript” src=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/utils/2/yahoo-dom-event_2.1.2.js”></script>
<script type=“text/javascript” src=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/utils/2/connection_2.1.2.js”></script>
<script type=“text/javascript” src=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/utils/2/animation_2.1.2.js”></script>
<script type=“text/javascript” src=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/utils/2/dragdrop_2.1.2.js”></script>
<script type=“text/javascript” src=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/widgets/2/logger/logger_2.1.2.js”></script>
<link rel=“stylesheet” type=“text/css” href=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/css/reset-fonts-grids_2.1.2.css”/>
<link rel=“stylesheet” type=“text/css” href=“http://us.js2.yimg.com/us.js.yimg.com/lib/common/widgets/2/logger/css/logger_2.1.2.css”/>
I loaded up some of the files and see that the Yahoo UI library has some cool animation functions, some with names like YAHOO.util.Bezier. I'm still now sure how things are drawn but I'll continue to look into it since it seems like a neat technology.
There is a Yahoo UI blog, which I just subscribed to. Lots of videos in that feed, which could of course be spliced into your “Master Media Pipe” which you can create yourself.