bulknews.typepad.com

Tatsumaki at Shibuya.pm Tech Talks #12

via vimeo.com

The speech is in Japanese. You can also check out the slides at Slideshare.

Tatsumaki

View more presentations from Tatsuhiko Miyagawa.

Wanted: CPAN Changelog diff service

Something I've been thinking of implementing myself, but i have other exciting things to work on these days, so I just free up this idea to the lazyweb:

WANTED

We really need a web API service that gives you the changelog entry, given a distribution name and version(s). Say the service lives on changes.cpan.org: (of course it could be something else)

API: http://changes.cpan.org/changes.json?distribution=Plack&from=0.9010&to=0.9015 ("to" can be omitted, in which case the latest CPAN version is used)

Response: JSON with the following keys:

• from, to, distribution: echo the request
• changes: plain text of what's new, extracted from Changes file

That's it. Extracting the changes as a data structure, like identifying RT ticket numbers, would be neat but that could be built as a separate service or a module, so i'd rather want it just serve a plain text view.

HOW

Daisuke Murase actually created CPAN Recent Feed for this, but since his server died a couple of weeks ago, the service went down. (Alas, the source code also is temporary unavailable)

We could probably use search.cpan.org's not-very-well-known Diff tool to extract changes, or manually download two tar-balls to peek at Changes file to diff. The other approach would be to parse Changes file to recognize versions. Not all dists use the same formats or filenames to log changes, so we may need some heuristics to recognize it. Some module authors use formats like YAML to log changes, in which case the parsing would be easier.

WHY

It would be obvious why this should be useful, but just in case: once we build this available as a web API, we could:

• write a CPAN.pm (or CPANPLUS) plugin to display changes when you upgrade modules
• write a command line tool to check the installed module version and display changes, so you can determine whether you should upgrade
• CPAN realtime updates can now add changes as a comment to the module update entry
• etc. etc..

So, who wants to write this? :) 

Plack and PSGI screencast and feedbacks

via vimeo.com

Recorded screencast and the rest of the talk in tonight's SF.pm talk. The talk went well with lots of questions and feedbacks!

Also, if you come to London Perl Worlshop in two weeks, you probably should avoid watching this movie. It's a spoiler! :)

(Note that I was misusing the apachebench command line option to test Catalyst on Plack. The right use will give me 200 QPS rather than 30, which is quite fast)

Comet servers/frameworks collection

Just a note for myself, but could be useful:

• Orbited Python, long-poll, XMPP, STOMP, *MQ  
• nginx_push_module nginx module, long-poll, Basic HTTP Push Relay Protocol
• cometd java, python, perl, Bayeux protocol
• PoCo-Server-Bayeux perl, POE, Bayeux 
• nodecast javascript, v8, node.js, long-poll
• webstreamr python, twisted, long-poll
• Compp Ruby, XMPP
• Shooting Star Ruby, Rails
• Stardust Perl, Coro, long-poll
• Tatsumaki Perl, Plack, AnyEvent, long-poll, MXHR
• Meteor Perl, long-poll

Man to marry his video game girlfriend this Sunday

Last month, I wrote about a Japanese husband who confessed to his wife that he had a virtual girlfriend, a character from an addictive Nintendo DS game called Love Plus. Now, another man is planning to hold a wedding ceremony with his Love Plus girlfriend this coming Sunday. The man, who calls himself SAL9000, was so in love with Nene Anegasaki that he decided to marry her and take her on a honeymoon to Guam. Of course, this means that he literally just took his Nintendo DS to Guam... while there, he took photos, livecast their adventures on popular video-sharing site Nico Nico Douga, and documented their adventures using the augmented reality iPhone app Sekai Camera. In any case, the guy plans on having a public reception in Tokyo this Sunday. It will be livecast on Nico Nico Douga, but in case you miss it, we'll be bringing you an update early next week. Stay tuned!

via www.boingboing.net

This crazy Love Plus phenomenon we've been enjoying for a few months is now being discovered by the internet at large ... i feel it's really embarrassing for some reason i don't know. Disclaimer: I don't have this game, though many of my friends do :)

snark's Plack-Middleware-Rewrite at master - GitHub

snark's Plack-Middleware-Rewrite at master - GitHub

Steve has been working on porting Rack's Rack-rewrite over to Plack. Give it a shot, any feedbacks will be welcome on #plack IRC channel on irc.perl.org.

London Perl Workshop

The London Perl Workshop is getting closer. It's on Saturday 5th December at the University of Westminster's Cavendish Street Campus (the same place it's been for the last few years).The schedule was announced a couple of days ago and, at always, it looks like a great line-up.

via perlhacks.com

It's official - I'm flying to London to attend London Perl Workshop and speak about Plack and PSGI. The line-up of the talks looks so great and I can't wait to be there.

Purchased this: GF1

via www.amazon.co.jp

Panasonic DMC-GF1!

It's never been easily available in the North America and the price has been up and down of this couple of days in Japan but amazon.co.jp finally came up with the "Last 2 items" stock a couple of hours ago with the pretty good 65000 JPY deal.

I just one-click purchased so I can get it when I'm back there in Japan next week. Exciting!

Love SQ available on iTunes US

via itunes.apple.com

THANK YOU Square Enix for making the awesome Love SQ compilation (collection of remixes of tracks from Final Fantasy, Chrono Trigger and Romancing Saga) available on iTunes store, first on United States with no freaking DRM!

I'm so enjoying this: livetune's to Far Away Times, Pe'z FF Main Theme and De De Mouse's Eternal Wind is so great.

Shibuya.pm Tech Talks #12

RDBMS 以外の non-relational なデータベースとして KVS の枠にとどまらない Not only SQL を指向する OSS の実装と運用の実際について語り合います。 一人5分間の持ち時間でライトニングトーク的なプレゼンを行なった後、 パネラーの方から10分間のツッコミを行ないます。 最後に総論として会場からの質疑応答を受け付けます。 （※発表タイトル・順・時間はすべて仮です）

via shibuya.pm.org

I'm going to Japan next week around Thanksgiving holidays (haven't yet booked the flight but I'm 90% positive :)) and asked Takesako-san to setup awesome Shibuya.pm meetup. A day later and he's come up with an awesome lineup of talks: Themed #nosql #nokvs #noperl.

I'll talk about Tatsumaki, the I/O bound web frameworks built on top of Plack and AnyEvent. Other speakers include Mikio Hirabayashi the author of TokyoCabinet and TokyoTyrant, and we'll talk about lots of cool stuff like golang, kumofs, memcached, Redis and RDBMS sharding.

Can't wait to be there!

Reminder: Plack at SF.pm is one week away

Plack is a set of PSGI reference server implementations and helper utilities for Web application frameworks, exactly like Ruby's Rack. It supports Standalone, CGI, FCGI, Apache, AnyEvent, Coro, Danga::Socket and many other server environments.

via www.meetup.com

I'll be talking about PSGI and Plack at SF.pm meetup. Will start with the general introduction but will include lots of updates since YAPC::Asia and Six Apart engineering tech talk. I'll also mention a little about my event-driven web framework Tatsumaki.

Come to the meetup if you're in the bay area.

Dynamically enabling Plack middleware

When you use middleware components in your PSGI app with Plack, you often want to enable them conditionally based on runtime environments. 

For example, some administrative middleware like debug-toolbar or stacktrace should be enabled if it's accessed from your office network, and js/css-minifying middleware and image-resizing transcoder should be enabled if the requesting client is the mobile device or from known-to-be-slow network, etc.

I've looked at WSGI and Rack middleware and couldn't see a generic way of handling this: most middleware components have their own configuration options, for instance the rack-bug has ip_masks options which checks the request's address to activate the behavior not. I don't like to see individual middleware component implement this kind of options since that'd be very inconsistent and confusing. 

So I thought that it's easy to make a wrapping middleware that conditionally runs the wrapped middleware or passes through, based on the runtime environments.

So there you go: Plack::Middleware::Conditional and enable_if DSL sugar. Here are some examples from the docs:

# Minify JavaScript if the browser is Firefox
enable_if { $_[0]->{HTTP_USER_AGENT} =~ /Firefox/ } 'JavaScriptMinifier';

# Enable Stacktrace when being accessed from the local network
enable_if { $_[0]->{REMOTE_ADDR} =~ /^10\.0\.1\.*/ } 'StackTrace';

# Work with other conditional setter middleware:
# Transcode Jpeg on the fly for mobile clients
builder {
  enable 'MobileDetector';
  enable_if { $_[0]->{'psgix.mobile_detected'} }
    'TranscodeJpeg', max_size => 30_000;
  $app;
};

The nice thing about it is that now those conditional routine can be implemented as a piece of middleware like scook's mobiledetector and can be used in other middleware's enable_if clause and that's really nice. Note that MobileDetector should come first because it's a pre-run condition check which runs from outer to inner, which is from the top to the bottom with Builder DSL.

You can of course still use enable ... if ... to enable middleware statically in the server boot time rather than runtime. enable_if dynamic builder is like adding a hole in the onion layers.

This is now implemented as Plack 0.9.12 and is available on CPAN via cpanf :)

YouTube goes 1080p

via www.youtube.com

Stunning.

Snow Leopard / MBP 13" checklist

My old macbook at work just died and now I'm with the new MacBook Pro 13" with Snow Leopard on it.

I just used TimeMachine backup and the Migration Assistant, and it worked pretty nice (amazingly): here're some migration list.

• Switch perl local::lib to rebuild with perl5.10 binaries. (DONE)
• To do that, i need to install the developer tools. (DONE)
• SIMBL - upgraded to the Snow Leopard compatible version (DONE)
• TerminalCopyOnSelect - seems incompatible with 10.6.2
• 1password - Upgraded to the 1password 3 beta (DONE)
• Macbook Pro multitouch touchpad: Hard to get used to it but should use a thumb (not forefinger) to click it, or always use tap-to-click :/
• Rebuild Remedie and Github Growler to run on perl 5.10: Remedie is done, but GG has problems when running as .app, works with 5.8.9 from the command line though.
• Leech - Doesn't work. Seems like a problem with SIMBL.
• Keywurl - Doesn't work, but Glims can replace the functionality. (DONE)
• GreaseKit - Doesn't work? Not sure how to run SIMBL on Safari without making it run in 32-bit mode.
• (continues while the migration goes on)

Amazing Miku

via www.nicovideo.jp

Wow Miku with glasses (around 1:26) is soooo incredibly cute.

Article on moe politics on io9 (NSFW)

via www.tokyomango.com

Brilliant.

Q;indivi starring Rin Oikawa €ŽWinter Celebration

via www.youtube.com

I must admit q;indivi starring Rin Oikawa covers great classics in a hip electro remix and I love it, but meh, this is getting almost enough. I really want their new originals.

Tatsumaki 0.1001

Tatsumaki 0.1001 is on CPAN.

I deprecated the over Moosification, especially the use of MooseX::NonMoose in Tatsumaki::Request and Response, made Tatsumaki roll much faster, giving 1300 QPS (which was 900 QPS) on my MacBook. Also switched to use Any::Moose, so all dependencies are now pure perl by default with XS for speedups. It's a big step for the micro framework we could target next, much like Sinatra for Ruby but with no real external dependencies. (I don't count pure perl dependencies as external, since we can always bundle them in extlib like I do for Remedie)

Tatsumaki is now on CPAN

Tatsuhiko Miyagawa / Tatsumaki-0.1000 - search.cpan.org

Tatsumaki, my tiny Perl port of Tornado using Plack and AnyEvent originally started as a proof of concept to test and demonstrate PSGI's streaming interface but its chat app using a simple long-poll Comet got a lot of attentions from people: 

Audrey Tang used the MessageQueue and comet interface to build multi-person SocialCalc, Daisuke Maki packaged the chat app, gugod is building an iPhone IRC client and jshirley is doing some interesting to stream content with YUI.

Actually, by "tiny port" I mean Tatsumaki does most of the things Tornado does, but in a very limited but simple way: simple embedded Perl templates, non-blocking I/O for everything including built-in HTTP clients, long-poll Comet ready MessageQueue API, simple server side push (aka firehose) and Digg's multipart XHR push as a Comet replacement. Yes, it is really simple but exciting :)

I really didn't want to increase the number of modules/applications i maintain but now it has proven very useful when last night I wrote a new PubSubHubbub hub Subfeedr using Tatsumaki, Plack, AnyEvent and Redis in 3 hours: it's a polling PuSH proxy and everything works in the single process in the event loop non-blockingly (so nice).

So anyway, it's now on CPAN as well as on github , which makes it a little harder for me to change APIs and what not, but no worries, everything is undocumented, so you still have to peek at the code to do what you want to do :)

I/O bound HTTP clients in web frameworks

If the server environment supports it the client will automatically run asynchronous and allow the server to handle more requests while waiting for the client. In environments such as CGI and FastCGI the client will simply fallback to blocking until it is finished. Multiple client requests will always be handled parallel though and never block each other.

via labs.kraih.com

Congrats sri on his progress on Mojo project. I'm really glad my Tatsumaki framework still has a relevant competitor :)

Here's the code to do that non-blocking proxy request in his blog post, as a web handler in Tatsumaki.

It's built on top of Plack and AnyEvent, so it handles multiple requests in parallel non-blockingly and you can use whatever Plack middleware components and AnyEvent aware modules to do non-blocking work, and Tatsumaki also falls back to the blocking mode in Apache, FastCGI and CGI if it makes sense.

You can see more examples of doing I/O bound response, server push (streaming or Multipart XHR) and Comet long-poll in Tatsumaki's eg/ directory.

Historic Indie anime on Nico Nico

via www.nicovideo.jp

A-Amazing indie Anime!

AnyEvent::Redis

miyagawa's AnyEvent-Redis at master - GitHub

Last night I hacked a little bit on a non-blocking Perl client for Redis (#nosql Key-value store) using AnyEvent. It's now available on github and CPAN.

Six Apart HQ

via www.flickr.com

Cool that Google Maps now displays the building names on their maps. It's kind of odd our HQ is on the street though :)

3D Dot Game Heroes

via blog.livedoor.jp

Collection of user-generated three-dimension dot game characters. Love it!

Far Away

I love this Miku song by kz livetune.

Simon Cozens | I finally get PSGI and Plack!

For the past few months I've been meaning to get around to understanding PSGI and Plack; for various reasons, I guess. First, because it's always good to keep abreast of what's going on in the programming world; second, because they're by Miyagawa, and really, anything by Miyagawa is worth looking into; third, because I've been writing a bunch of different web applications recently and wanted to know what the state of the art was.

via www.simon-cozens.org

Great post about PSGI and Plack by Simon Cozens. Must read.

HOW TO: embed nicovideo.jp videos on your site

Nico Nico Douga updated their site to version "(9)" a couple of days ago and apparently they got rid of the annoying restriction of the referral domain to embed their videos with the flash.

Here's the tag to embed their video in any site (you should replace the sm123456789 with the actual Video ID:

<script type="text/javascript"
 src="http://ext.nicovideo.jp/thumb_watch/sm123456789"></script>

Testing NicoVideo embed

via www.nicovideo.jp

Great OP from Bakemonogatari.