30boxes.pl: how I can end up living in the shell

Even in the web 2.0 world, geeks like me still tend to do lots of tasks on the UNIX shell. Recently Best Pracital released their command line tool todo.pl for their awesome task management service Hiveminder. Jesse gave me a preview release for it a couple of weeks before the public announcement and I've been finding it practically useful.

So I created a similar command line tool for my favorite calendar service 30boxes. There you go, 30boxes.pl. The usage of the command is quite similar to todo.pl (or the original todo.sh):

30boxes.pl list

List all events in your calendar starting from today to 3 months later

30boxes.pl --from "2 weeks ago" -to today

List all events in your calendar starting from 2 weeks ago to today

30boxes.pl --month "October 2006"

List all events in your calendar scheduled on October 2006

30boxes.pl --date tomorrow

List all events scheduled tomorrow.

30boxes.pl add meeting with Brad tomorrow 3pm

Add new event title "meeting with Brad" on 3pm tomorrow, using 30boxes One box input.

30boxes.pl del 100

Delete the event with id 100.

That's it. Pretty simple and usable. You need some Perl modules installed from CPAN, and most of them are quite popular and easy to install.

I like to add 'update' command to this shell, to add the ability to change title or starting date from command line also, but looks like 30boxes events.Update API is kind of borked, or just unfriendly to use. When you call method=events.Update&summary=foo, it'll clear out all the fields other than summary, which to me is very intuitive.

I'll update the code to retrieve the original data and send back if there're popular demands for it. But for now you can just use web interface to update.

Enjoy!

Summary support in Plagger 0.8

From my email sent to plagger-dev list:

I've been working on summary support in hackathon-summary branch at http://plagger.org/trac/browser/branches/hackathon-summary

The point of summary feature is:

  • summary, title, body and author has "is_html" or "is_text" method,
    to determine if the data it holds is HTML or plaintext.
  • Plagger::Util::strip_html($html) will render HTML into text using
    HTML::FormatText (Making it pluggable is left TODO)
  • $entry->summary->text and $entry->body->html does what you mean
  • summary is automatically extracted from feed metadata ala RSS description or Atom:summary field. If there's not one, Summary::Auto plugin will auto-generate using Plagger::Util::summarize($entry->body)
  • actually how to generate summary from body HTML is pluggable. There're couple of Summary::* plugins already checked in in the branch. http://plagger.org/trac/browser/branches/hackathon-summary/plagger/lib/Plagger/Plugin/Summary

The final thing left undone is how to declare which field to use in each Notify/Publish plugin without updating the templates. I'd like to say, "send full-content HTML mail to my gmail account, but plaintext of summary to my mobile. Use summary as HTML in Publish::Planet." with
a single config, not requiring to update the templates.

The syntax would be something like:

- module: Publish::Gmail
   config:
     mailto: ****@gmail.com
   override:
     body: $args->{entry}->body->html

- module: Publish::Gmail
   config:
     mailto: ****@mobile.example.com
   override:
     body: $args->{entry}->summary->text

- module: Publish::Planet
   config:
     mailto: ****@mobile.example.com
   override:
     body: $args->{entry}->summary->html

But not sure what would be the best syntax.

I'd like to apply the override/localize methodology to link(s) as well, ala:

- module: Notify::IM
   override:
     link: $args->{entry}->alternate_link('tinyurl')

to display link in TinyURLed (using WWW::Shorten), for instance.

Any feedbacks would be welcome on this.

Plagger Hackathon 2 in Cybozu Labs

This weekend we had our 2nd Hackathon dedicated to Plagger in Cybozu Labs, Akasaka Tokyo.

The biggest event during this Hackathon was TestAThon. Most of our committers have went wild to create unit test files for plugins without test. Now # of .t files jumped up from 32 to 102, which is amazing. Kudos to our testathoners team: youpy, takesako, 33rpm, tomi-ru, hakobe & charsbar (on-site) and mizzy, hsbt & drawnbody for off-site remote testing. Thank you ALL!

Thanks to the unit testing effort, I've been going far to merge the work done in Hackathon-MT branch, and do lots of core changes without worrying about the backward incompatiblity, since whenever I broke the code, those tests will notify me. Great.

So I've been working on: JSON dumper, Feed serializer, Notify::Audio and enclosure integration, decoding YAML config as UTF-8 by default, summary and pluggbale summarizer support, rewriting OPML parser using LibXML SAX, refactored XML::Feed parser into Plagger::FeedParser so plugins can use. Hmm there's a lot.

Summary and summarizer stuff is being done in hackathon-summary branch and will be merged to trunk soon.

Otsune has been working on document improvement and Nagayaman was doing the great website redesign, which will be online soon. Yappo was doing Senna hacking and per-plugin storage, and Search::Rast improvement.

Trac the changes during the Hackathon to be amazed more.

SAKK Plagger Hackathon #1: MT meets Plagger!

Mtplagger

Today we had a Plagger Hackathon in Six Apart KK. We had most of MT and TypePad engineers coming around the couch and hacked together to build a Movable Type plugin for Plagger and couple more search extensions. You're curious why Vox engineers are not there. Yeah, they are all in San Francisco now :)

今日は赤坂の Six Apart KK オフィスで Plagger Hackathon。MT/TypePad のエンジニアがカウチに集まって MT-Plagger プラグインと Search 系のエンハンスメントをハック。Vox エンジニアはいま San Francisco にいってるので不参加です :)

So let's see MT-Plagger demo. The basic idea of the plugin is so simple. When you create or update a new post on Movable Type CMS, new MT::Entry object will be transformed to Plagger::Feed (where MT::Blog corresponds to Feed and MT::Entry to Plagger::Entry) and it will bootstrap the Plagger context to run the later pharses than Filter.

まずは MT-Plagger のデモをどうぞ。このプラグインがやっていることはシンプルで、MT で新しいエントリをつくったり、更新したりしたときに、MT::Entry をベースに Plagger::Feed と Plagger::Entry をつくって Plagger プロセス(Filter:: 以降のフェーズ) を bootstrap します。

You can see in the demo how it works with Publish::Gmail to notify email, and Notify::IRC to do realtime notification to the IRC channel. Integration with Search::* to build a better search engine for MT would be a nice hack. Making it work with new Comment (rather than Entry) would be cool for pluggable Comment notification. Using Publish::Delicious and you'll always get your entry bookmarked on del.icio.us first :)

デモでは Publish::Gmail をつくってメールを送信したり、Notify::IRC をつかってリアルタイムに IRC アナウンスしたりしてます。Search::* と連動して MT 検索を改善してみたり、エントリだけでなくてコメントがきたときに Notify にとばすなんてのも面白そう。Publish::HatenaBookmark でつねにセルクマ 1 get とかもできますね。夢が広がりんぐ。

MT Plagger code is now in hackathon-mt branch and will be merged down to the trunk soon.

Plagger competitive services

Planet and newspipe are examples of the Plagger competitive "software". What about services? Well, there're a couple of competitive web 2.0 sites that do things pretty close to what Plagger does.

Feed Rinse: "Feed Rinse is an easy to use tool that lets you automatically filter out syndicated content that you aren't interested in. It's like a spam filter for your RSS subscriptions"

Touchstone: "Subscribe to changes on your favourite website. Set rules for what’s important. Have alerts appear on your desktop while you work." They call subscription/notify/publish plugins as Input/Output Adapters.

Dapper: "Dapper’s mission is to allow you to use any web based content in any way you can imagine. And by use, we mean going beyond just reading or viewing a webpage."

xFruits: "XFruits makes possible the Mashup RSS creation in a very simple way thanks to the Composer. You can assemble the bricks together so as to build your own feed-based service.
"xFruiter" service's users are referenced. "

OSCON: My Plagger talk went well

So my first talk in OSCON is over. Check out the slides on Plagger website if you missed it. Basically I got mostly positive feedbacks "Hey that's cool!". I'm pretty happy. And now Junior (Mark Smith) from Six Apart wrote a really nice writeup about the session. Thanks Junior!

I updated the slides after I woke up on 5am this morning and did some reharsal in the room. The main problem I found is that it took longer than 45 minutes, and it was actually 60 minutes. So I decided to speak faster than usual. Another issue was the order of the slides, since it started with the killer app before the quick guide of plugins, it got boring towards the end, and at the same time it gets somewhat awful to explain which plugin it actually uses.

So I decided to switch so I introduce the killer apps after the plugins tour. It worked well, but it might be boring for some people during the 1st half. I don't know how to deal with that in YAPC::EU again. Feedbacks are welcome.

BTW the highlight plugin this time is Notify::Pizza. That was Notify::Eject in YAPC::Asia, and Publish::Excel in YAPC::NA. I wonder what is gonna be in YAPC::EU :)

Now that my talk is over and I'm relaxed. Looking forward to attending to other sessions in the conference, and party!

Getting Ready for OSCON 2006

I'm here in San Francisco, Six Apart office now. This evening I'll fly from OAK to PDX for OSCON 2006 and I'm really excited about it.

This is my first OSCON ever (either as an attendee or a speaker), and I will give a 45 minutes talk about Plagger. The talk will be mostly the same with the one that I gave in YAPC::NA, but it will reflect the feedbacks I got in YAPC, and will have the latest updates in the project, namely the 0.7.4 release and upcoming 0.7.5 release with lots of Plaggnet goodness.

I'm really looking forward to the parties at night, too. See you in Portland!

Plagger vs. newspipe

Meta: I've been lazy posting here on this blog. If you need more geeky stuff, check out my use Perl blog and want to check non-geeky stuff, go to my shiny Vox blog. Thanks,

Plagger is written in Perl and the competition always comes from Python world. For instance, Planet is a good software written in Python to aggregate posts from lots of blogs into a single HTML and our friend Dave Cross wrote a very good post how easy it is to build Planet site using Plagger.

Today I found another competition: newspipe, which is also written in python. The site has a very good summary of what it does and how you use it. And the list of the features makes me feel that the author has the very similar strategy to how to deal with configuration and site scraping stuff, and what's actually needed for email notification/archiving. Very good stuff.

So, let's compare the things newspipe claims to do, with Plagger (Subscription::OPML + Publish::Gmail).

Supports RSS and Atom feeds through Mark Pilgrim's Universal Feed Parser

Yeah, Plagger uses XML::Feed and XML::Liberal to deal with various versions of feeds, even if they're somewhat broken.

feeds are listed in an OPML file:

Yes, you can write a list of feeds in an OPML file using Subscription::OPML. Optionally you can take the list from XOXO file, Database, your Bloglines account etc. with the power of plugins.

feed options can be set individually for each feed or for a group of feeds

No feed options support yet when you use Subscription::OPML. I think this is a good idea.

supports screen-scraping scripts via an internal pipe:// URI schema

Supports screen-scraping with the usual HTML page URL (written in htmlUrl if you use OPML) and register CustomFeed:: for the URL. (Plan: I'll rename the CustomFeed:: to Scraping:: or something) CustomFeed::Pipe plugin actually calls another process to get the data from various sources including screen scraping.

the OPML file can reside locally (in your hard disk) or remotely (in a web server).

Yes. The OPML file can be local or remote one.

sends news items via SMTP to a designated e-mail address

Yes. Plagger supports SMTP, Sendmail and SMTP-TLS. There's a patch to do SMTP-AUTH as well.

messages can be in HTML/multipart MIME format or plaintext

Yes. The Emails are now sent in HTML. No built-in plaintext format support yet but there're plugins to do that.

multiple items from a feed can be grouped and sent in a digest message

Yes. This is the default behavior of Plagger Publish::Gmail.

updated news items are detected and re-sent with additions and deletions highlighted

No, we don't do this but leave it for Gmail's diff functionality.

images linked from the feed items are downloaded and included as inline images inside the mail message (great for archiving purposes).

Not by default but easily can be done with:

- module: Filter::FindEnclosures
- module: Filter::FetchEnclosure
- module: Publish::Gmail
  config:
    attach_enclosures: 1

a mobile (text-only) view of a feed can be sent to a secondary e-mail address (to read on a PDA or an MMS-enabled mobile phone)

Once we've done refactoring on email notification, it'd be pretty simple to enable another Notify::Email directive with plaintext mode on.

full support for HTTP optimizations like gzip compression, If-Modified-Since and If-None-Match headers. Feeds and image files will only be downloaded when they have changed.

Yes, with URI::Fetch.

E-mail "threading" based on previously sent RSS items.

No, it doesn't do and leave it for Gmail's "conversations" feature. Adding X-Item-* headers and References: (or In-Reply-To:) would be a good idea.

That's it. Obviously, we are better in some places and can learn from them in others, too.

Control Google Maps via ThinkPad tilt

Another day, another hack. Here's a demo that controls Google Maps using ThinkPad Hard Drive Active Protection sensor. You tilt your ThinkPad and Google Maps roll over. Pretty cool.

というわけで ThinkPad ハックづいてるこの頃(といっても2日ですが)、また面白いハックを1つ。Google Maps の地図を、ThinkPad を傾けてナビゲートします。まずはデモをどうぞ。

My recommendation is to choose Satellite mode, with the 3rd Zoom level. It makes me feel like flying in the sky, just as birds. Because of Google Maps JS library prefetching images, sometimes you have a delay (latency) moving, but other than that, it is quite fantastic.

サテライトでズームは最大から3つ目ぐらいが最適。空を飛んでいる鳥の気分になれます。ちょっと Gmaps のローディングで遅延があったりしますが、それ以外は快適の一言。

The code is available here in my svn repository. Note that because it uses ActiveX, you need to download the gmaps.html locally and open it with Windows IE. The app is written in JavaScript (using Google Maps API) and ActivePerl's PerlScript inside Windows IE. You need to install Win32API-File CPAN module (use ppm to install) before running it.

コードは こちら からダウンロードできます。ActiveX を使うので、ブラウザ上でそのまま動かすことはできません。gmaps.html をダウンロードして WinIE で開いてください。ドライブプロテクションのついた ThinkPad, ActivePerl (PerlScript が有効になっている必要あり) に Win32API-File が必要です (ppm でインストールできます)。IE で <script language="PerlScript"> の中から ShockMgr を開いて、JavaScript に返して GMaps を pan してます。IE だからできるワザ。

Enjoy!

Turn your ThinkPad into lightsaber

Recent IBM/Lenovo ThinkPad models have HDAPS (Hard Drive Active Protection System) support. HDAP's ShockMgr API is accessible to developers, by reverse engineering. For scripting languages like Perl, it's pretty straightforward to read the Accelerometer data from ShockMgr device.

So here it comes: ThinkPad Saber. It would be familiar as a ThinkPad clone of MacSaber if you're a Mac Geek.

The code is written in Perl, and is available at my public svn repository. You can also download the self executable package (of course for Win32) from releases directory.

My Photo

del.icio.us/miyagawa

Flickr

  • www.flickr.com

My Last FM

Blog powered by TypePad

Blog Search

Lingr