bulknews.typepad.com

Dapper, Pipes and Plagger: meta-Mashup

The launch of Yahoo! Pipes raised a lot of discussions of "similar services / products" and apparently a lot of people have found the similarity between Pipes and Dapper or Plagger.

We can say something about pros and cons for these products, because they're built on top of different architectures and implemented differently. But instead of just comparing what-can-do or cant-do, I'd suggest doing some "mash-up" of these services. Let's say it's a meta-mashup.

Pipes accepts RSS/Atom feeds as an input, and doesn't have an ability to detect site updates that don't come with feeds. Dapper and Plagger are good at it. So you use Dapper's cool UI to analyze and generate API for any web pages, then consume that generated feeds as an input to Pipes, and create a flow to generate remixed feeds.

Dapper and Pipes are both hosted on the server and hence there's no way to interact with local devices or personally authenticated services, like storing updated feeds to iPod, or notify via MacGrowl or post the updates to Twitter using your account, which Plagger is really good at.

So you can mash up these remix sites to create a fully GUI controlled API creator (Dapper) and Pipes-flow programming (Pipes) with your own publisher/notification engine (Plagger).

Yahoo! Pipes = Dressed up Plagger, but the dress is nice

Plagger is a perl-based, open-source feed routing system, or so called "mash-up creator." A couple of interesting sites, like CDTube (a mash-up of Count Down TV and YouTube) are powered by Plagger and it has been driving geeks to do some interesting bits with RSS/Atom/iCal feeds.

The most annoying thing about Plagger development, for me and for end users as well, is the lack of the document, and the lack of pretty interface. And it requires half of CPAN to run it. (Well, not really, because most of CPAN modules are required by "plugins" and you don't actually need to install all plugins. They're all optional.) These barriers have been sort of intentionally made there, to reduce the S/N ratio in the community, and looks like it's been working well, but it caused the other problems ("Plagger is HARD to install!").

But anyway, the most frequently asked question in the conferences have been: "Are there any hosted version of Plagger that I can use, without installing it by myself?" and I always answered "Yes, there could be, but if you're asking me to do it, I won't. Why? I don't need it :) Plagger is licensed under Artistic/GPL and there's no way for me to stop someone to create the hosted Plagger."

So now, Yahoo! Pipes could be the answer for these people. I'm not saying "Y! Pipes ripped off my idea!" since there are other services already like Dapper or xFruit, but having that cool IDE and the feature to share the "pipe" is really awesome. Congrats to the Pipes team.

It also makes me grin that they named the service "Pipes," and follks like Jeremy or Tim O'Reilly are saying "RSS is the pipes for Internet!", which coincidentally matches with the Plagger's tagline "the UNIX pipe programming for Web 2.0." As you can see from the slides (esp. in YAPC::Europe and XML Developer's Day), I've been saying "Plagger allows you to mash up feeds just like Unix pipe/filters." Thanks to Yahoo! for proving that my vision's been correct :)