bulknews.typepad.com

Tatsumaki is now on CPAN

Tatsuhiko Miyagawa / Tatsumaki-0.1000 - search.cpan.org

Tatsumaki, my tiny Perl port of Tornado using Plack and AnyEvent originally started as a proof of concept to test and demonstrate PSGI's streaming interface but its chat app using a simple long-poll Comet got a lot of attentions from people: 

Audrey Tang used the MessageQueue and comet interface to build multi-person SocialCalc, Daisuke Maki packaged the chat app, gugod is building an iPhone IRC client and jshirley is doing some interesting to stream content with YUI.

Actually, by "tiny port" I mean Tatsumaki does most of the things Tornado does, but in a very limited but simple way: simple embedded Perl templates, non-blocking I/O for everything including built-in HTTP clients, long-poll Comet ready MessageQueue API, simple server side push (aka firehose) and Digg's multipart XHR push as a Comet replacement. Yes, it is really simple but exciting :)

I really didn't want to increase the number of modules/applications i maintain but now it has proven very useful when last night I wrote a new PubSubHubbub hub Subfeedr using Tatsumaki, Plack, AnyEvent and Redis in 3 hours: it's a polling PuSH proxy and everything works in the single process in the event loop non-blockingly (so nice).

So anyway, it's now on CPAN as well as on github , which makes it a little harder for me to change APIs and what not, but no worries, everything is undocumented, so you still have to peek at the code to do what you want to do :)

I/O bound HTTP clients in web frameworks

If the server environment supports it the client will automatically run asynchronous and allow the server to handle more requests while waiting for the client. In environments such as CGI and FastCGI the client will simply fallback to blocking until it is finished. Multiple client requests will always be handled parallel though and never block each other.

via labs.kraih.com

Congrats sri on his progress on Mojo project. I'm really glad my Tatsumaki framework still has a relevant competitor :)

Here's the code to do that non-blocking proxy request in his blog post, as a web handler in Tatsumaki.

It's built on top of Plack and AnyEvent, so it handles multiple requests in parallel non-blockingly and you can use whatever Plack middleware components and AnyEvent aware modules to do non-blocking work, and Tatsumaki also falls back to the blocking mode in Apache, FastCGI and CGI if it makes sense.

You can see more examples of doing I/O bound response, server push (streaming or Multipart XHR) and Comet long-poll in Tatsumaki's eg/ directory.

Historic Indie anime on Nico Nico

via www.nicovideo.jp

A-Amazing indie Anime!

AnyEvent::Redis

miyagawa's AnyEvent-Redis at master - GitHub

Last night I hacked a little bit on a non-blocking Perl client for Redis (#nosql Key-value store) using AnyEvent. It's now available on github and CPAN.

Six Apart HQ

via www.flickr.com

Cool that Google Maps now displays the building names on their maps. It's kind of odd our HQ is on the street though :)

3D Dot Game Heroes

via blog.livedoor.jp

Collection of user-generated three-dimension dot game characters. Love it!

Far Away

I love this Miku song by kz livetune.

Simon Cozens | I finally get PSGI and Plack!

For the past few months I've been meaning to get around to understanding PSGI and Plack; for various reasons, I guess. First, because it's always good to keep abreast of what's going on in the programming world; second, because they're by Miyagawa, and really, anything by Miyagawa is worth looking into; third, because I've been writing a bunch of different web applications recently and wanted to know what the state of the art was.

via www.simon-cozens.org

Great post about PSGI and Plack by Simon Cozens. Must read.

HOW TO: embed nicovideo.jp videos on your site

Nico Nico Douga updated their site to version "(9)" a couple of days ago and apparently they got rid of the annoying restriction of the referral domain to embed their videos with the flash.

Here's the tag to embed their video in any site (you should replace the sm123456789 with the actual Video ID:

<script type="text/javascript"
 src="http://ext.nicovideo.jp/thumb_watch/sm123456789"></script>

Testing NicoVideo embed

via www.nicovideo.jp

Great OP from Bakemonogatari.

Updated the blog design

via www.flickr.com

Yeah, the new theme rocks.

The banner photo was taken in Koh Samui in 2006 with my pretty old Canon point-and-shoot camera. The photo is obviously retouched a little bit with Picasa to look better, though :)

perldoc-search

Joshua ben Jore / App-Perldoc-Search-0.03 - search.cpan.org

I've been always having problems locating the right POD files when I deal with a big module/software/framework in Perl, for instance, Moose. They have a fairly good amount of document, but merely doing `perldoc Moose` doesn't give me what I want to know: I usually need to do `perldoc Moose::Manual` and then `perldoc Moose::Cookbook` to scan for the things by typing i want to know. It's tedious.

perldoc-search is awesome: it gives you an ability to search your local POD files without building index like Spotlight. For instance, when I need to know how to mangle new() args in Moose, i can just type: `perldoc-search BUILDARGS -G Moose` (-G is a regular expression to match filenames to look for) and it gives you a fairly small amount of files, including Moose::Cookbook::Basics::Recipe10 which exactly tells me what I need to know.

install App::Perldoc::Search now and enjoy.

New IRC channel for PSGI and Plack

We've been using #http-engine IRC channel for the convenience but we agree that PSGI/Plack talk invades the whole channel and is confusing to newcomers: so we made a new IRC channel dedicated for PSGI and plack talk (yes, another channel!)

#plack on irc.perl.org

Join now and let's chat!

Try::Tiny

Try::Tiny rocks.

Every perl programmer should start using it. Especially if you're a CPAN module author and wants to eval { } inside your module (often to check if an user of your module has some optional module installed, in which case you optimize stuff for it etc.), you're strongly suggested to use this module instead of your own eval { } stuff. 

Why? If you do eval { } in your module that clobbers the global $@, and to avoid that you need to local $@, and then if you want to die (rethrow) from local $@ then the exception gets cleared etc... Try::Tiny solves this endless yak shaving and there're no dependencies at all and works with perl back to 5.005x.

Anyway, i started using Try::Tiny (when nothingmuch himself added this :)) in Plack and Tatsumaki with so much fun, but last night I experienced one minor pitfall of this module that I might want to share with you.

I had this code:

sub write {
    my $self = shift;
    my $writer = $self->writer || $self->init_writer;
    $writer->write(@_);
}

I wanted to catch Broken pipe exception from ->write, so I changed to do this:

sub write {
    my $self = shift;
    my $writer = $self->writer || $self->init_writer;
    try {
        $writer->write(@_);
    } catch {
        /Broken pipe/ and $self->{client_disconnected} = 1;
    };
}

It might be obvious to you but for me it wasn't at first, and I wasted about 15 minutes to figure out what's wrong with that code.

Yeah, the use of @_ in the try { } clause. it's internally implemented as try sub { ... } so the content of @_ is changed when it's executed.

Lots of clever modules on CPAN that exports DSL sugars (including my own Web::Scraper) shares the same problem. At least you have to be careful not to use @_ in the block, by assigning the @_ content into some lexical variables before you construct the block,

(Well maybe Try::Tiny can add a workaround to save the @_ in the caller stack but i'm not sure if it's a doable without black magic like DB::args and whether it's right thing to do anyway)

Anyways, Try::Tiny is a great module if you care about this tiny thing. Enjoy.

UPDATE: this is now mentioned in Try::Tiny POD docs.

Task::Plack now added git_clone

cpanf Task::Plack (install App::CPAN::Fresh if you haven't :)) and now you can call perl -MTask::Plack -e git_clone on your shell, which clones all Plack/PSGI relevant development git repositories for you. This is quite handy now that we split the Plack repo into multiple projects.

mod_reproxy for Apache and lighttpd

typester's mod_reproxy at master - GitHub

Kazuho made mod_reproxy for Apache2 and typester made one for lighttpd.

They add a support for X-Reproxy-URL header, which is used in perlbal often in combination with mogilefs, so your web app can internally redirect static files to other static web server after the authentication etc. is done. It's more flexible than X-Sendfile since the file doesn't really have to live on the frontend proxy, which is not often the case in the user uploaded content like photos or videos.

Good stuff.

PSGI/Plack streaming is now complete

In the last couple of (or even more :) days Yuval and I have been endlessly discussing what the asynchronous response API would look like in PSGI applications. And that was also becoming one of the most frequently asked questions on my side, since many people from AnyEvent, POE and perlbal (Danga::Socket) land are curious how to port their non-blocking application to let them run on PSGI servers.

Thursday while having frodwith on board in the discussion (that was really helpful since he has a third person view as a POE developer) and we sorted out the nicely done middle ground, which is actually the revive of our original start_response callback we abandoned in favor of IO::Writer-like abstraction.

So, basically the idea is the same as the original Python WSGI's start_response but this callback is NOT an optional parameter to the app because that stands in the way of everybody in the chain including middleware and that sucks. Instead, an app can optionally return a callback that accepts another callback to which you can return the response array ref (code, headers and body) if you want to delay your response.

my $app = sub {
  my $env = shift;
  return sub {
    my $respond = shift;
    # do some event stuff
    $event->callback(sub { $respod->([ $code, $headers, $body ]) });
  };
};

If you also want to delay the content body delivery as well (i.e. streaming) you can omit the body, in which case you'll get the writer object that has write(), close() and poll_cb(). 

my $app = sub {
  my $env = shift;
  return sub {
    my $respond = shift;
    my $w = $respond->([ 200, [ 'Content-Type' => 'text/plain' ]]); # no $body here
    # do more event stuff
    $event->callback(sub { $w->write($body) });
    $event->done_callback(sub { $w->close });
  };
};

I plan to update the PSGI specification to address this optional response style in a few days. Yuval also has a nicely summarized blog post on this, and we already added this callback style to our AnyEvent, Danga::Socket, Coro and POE backend as well as most middlewares, including automatic chunking middleware (supposedly used by servers, not apps) which I line-by-line cloned from Rack :)

Meanwhile Yuval is working on his data-pimping REST machine Hydrant and I've been working on Real-Time-optimized web framework Tatsumaki, and both will be built on top of PSGI/Plack from the ground up. This will definitely be how web frameworks in Perl would look like in 2010.

Friday night I was at my friend's party and five people talked to me how awesome Plack is and it's changing the world. (Disclosure: it was a party full of my Six Apart co-workers and alumni like Artur, Randy and Simon, so that's not that surprising :))

Plack and PSGI uploaded to CPAN

Tatsuhiko Miyagawa / Plack-0.9000 - search.cpan.org

Some yak have been shaved and the last minute bikeshed of renaming things in Plack::Builder was done, and now Plack and PSGI are uploaded to CPAN!

Some server backends like AnyEvent, FCGI::EV, Coro, Danga::Socket, ReverseHTTP and ServerSimple are removed from the core distribution and uploaded to CPAN separately. Actually there's no reason those implementations need to be named Plack::Server::* anymore since it doesn't inherit anything. I have more to write about naming your stuff on CPAN, hopefully soon to be posted here.

I also uploaded Catalyst::Engine::PSGI but i keep other framework adapters like Maypole, Mason, Mojo, CGI::Application and Squatting in my github repos for now, since some of the implementations are pretty hacky, and I'm not confident nor comfortable maintaining those adapters when bugs are found etc. Anyone from the dev team of these frameworks want to take care of actually releasing this to CPAN or porting to the core? :)

Anyway, this is a big milestone. Thanks everybody for helping me this past month and let's celebrate the initial release, but let's continue hacking!

PlebGUI: WebGUI Meets Plack

The idea goes like this: Web applications, when all is said and done, are really just on about sending three pieces of information to web browsers: a HTTP status code, a list of HTTP headers, and some content (a file or some text, normally HTML).

via blog.patspam.com

Patrick from WebGUI has posted a wonderful introductory article on PSGI and Plack, and he actually shows his working code to run WebGUI framework on top of Plack stack. This is way too awesome to believe.

Devel::StackTrace::AsHTML

via www.flickr.com

I combined all of the great works done by tokuhirom, kazuho, nipotan and Sartak: render Devel::StackTrace object as an HTML. As a bonus, if you render WithLexicals object it will display the lexical variables with JS toggle button.

Plack middleware and plackup has now been converted to use this new StackTrace screen. Pretty slick!

Available at github and CPAN shortly.