bulknews.typepad.com

Beautiful Moon over the bay bridge

via www.flickr.com

Taken with GF1 with 6 sec exposure.

Plack 0.99* (and 1.0) milestones

I am pleased to announce that i've just shipped Plack 0.99_01 to CPAN, the first dev release towards Plack 1.0.

What's New?

There's been lots of refactorings, renames and changes but first of all, this 1.0 release should be 99.9% transparent to the end users, and they don't need to change anything wrt how to use Plack. I carefully added backward compatibility code here and there when I rename things.

What this release would affect most are developers: middleware developers, server developers and framework developers. For the first two, I think I've communicated very well, and I'm already working with them on github, to keep their code up to date with the changes, while the changes required are so minimal, or even better, in most cases, none.

Plack is now officially PSGI "toolkit"

There's been confusions about what Plack is: Is it a middleware library, a mini framework or a web server. Though there's a google bomb campaign to link Plack with Perl Web Server (thanks everyone! :D) Plack is not technically a web server:

• Is Plack a framework? - No. It's a toolkit for frameworks
• Is Plack a web server? - Not technically, but has lots of adapters and handlers for web servers. It also bundles the reference pure perl web server: HTTP::Server::PSGI
• Is Plack a middleware library? - Yes, but something more than that.

So, PSGI is an interface and Plack is an API and tools around it, that you can actually run your server or application on. I hope it clarifies things more.

Plack::Request changes

(If you're a lucky framework developer that doesn't use Plack::Request like CGI::Application, Mojo, Dancer or Catalyst, then you can skip the entire section :))

The biggest change that would impact framework developers is that Plack::Request is now in core distribution. It is a good utility for middleware developers and framework developers, but it has a nature of a port from HTTP::Engine::Request which is a port from Catalyst::Request. That means some methods do not make much sense, or it has something applications or frameworks should do, rather than Plack should do. 

Plack::Request internals have been rewritten and refactored a lot, so it's now designed to work in middleware components safely, by caching the parsed parameters as well as the post body in the env buffer or a temporary file, and you can instantiate Plack::Request as many times as you want in the middleware chain and it just works. 

There are small incompatibilities: Some methods, such as parameters, path, cookies still work but behave now a bit differently, but for good purpose. It is to remove the oddness of parameters(), fixes the code/doc mismatch in the path handling and says good bye to undocumented weird Cookie serializations, originating from CGI.pm. We discussed these things on #plack IRC channel and it seemed everyone agreed that this change is a good change. (Or, whoever opposed to the changes were just silent :))

I apologize in advance that if these incompatible changes cause any problem or unexpected behavior on your application, and am happy to help if you have questions etc.

More changes before 1.0

I can't promise anything but I'm expecting 2 weeks or more before making Plack 1.0 release, the changes planned before that happens would include:

• Dechunk HTTP requests and HTTP::Server::PSGI uses the temp buffer to implement psgi.input (being worked on topic/chunked-input branch)
• Some PSGI extension to allow non-blocking read from psgi.input and/or raw socket access, to make things like WebSocket possible
• charset decoding support in Plack::Request and Response
• Rename Plack::Server::* to their own namespaces

Keep questions coming in the comment or on the IRC!

Japan: Stop the ridiculous "experiments"

　総務省は２８日、若い女性らに人気がある「ケータイ小説」で、作者に自分の作品を評価してもらう実証実験を行うと発表した。表現の自由を確保しつつ、ネット上の有害情報の拡大を食い止める手法として注目されている「セルフレイティング」の普及が狙いだ。

via www.asahi.com

　パソコンや携帯電話、専用端末への雑誌記事の有料配信を目指す、共同サイトの実証実験「ｐａｒａｒａ（パララ）」が２８日、始まった。日本雑誌協会（雑協、東京都千代田区）内の「雑誌コンテンツデジタル推進コンソーシアム（共同事業体）」が進めるもので、講談社や小学館など、５８の出版社の雑誌９１誌が参加している。

via www.asahi.com

"Japan: Ministry of Internal Affairs begins experiments to let authors self-rate their work based on violence, sexuality and recommended ages for reading."

"Japanese Magazine Content Digital Promotion Consortium begins experiments of digital downloads of magazine articles to computers, cellphone and other devices."

Stop that crappy experiment (実証実験).

The reason they do "experiments" is obvious; they don't want to take a risk, and they don't want to break the implicit "traditional way" in the circle. The only possible way to make the things happen, for them, is to ask "consortium" or "the ministry of whatever" to say "okay, we're going to do this just for the experiment so we can stop at any time once it gets some troubles, right?"

That way you'll never, ever succeed. It's just a waste of time. Just remove the ridiculous restriction and update the laws so any startups (even funded by some major publishers/labels) to build business as quickly as possible. If you can't take the risk, you're guaranteed to fail.

Utada at The Fillmore

via www.flickr.com (Not my photo :P)

Utada was so freaking awesome! I was so excited when she played a couple of old songs like Automatic and First Love, like everyone else in the venue. It was a bit crap that absolutely no photos were allowed (and my camera was taken away!) but I really, really enjoyed the show.

I found the set list from LA/Seattle show on the wonderful internet, which looked like almost the same but i changed the order a little bit to fit with my memory, and also searched on Twitter timeline to make it complete.

1. On and On (This is the One)
2. Merry Christmas Mr. Lawrence - FYI (This is the One)
3. Poppin (This is the One)
4. This One (Cryling Like A Child) (This is the One)
5. Passion [J] (Ultra Blue)
6. Sakura drops [J] (Deep River)
7. Stay Gold [J] (Heart Station)
8. Devil Inside (Exodus)
9. Kremlin Dusk (Exodus)
10. You Make Me Want To Be A Man (Exodus)
11. The Bitter End (Placebo cover) 
12. Apple and Cinnamon (This is the One)
13. Come Back To Me (This is the One)
14. First Love [J] (First Love)
15. Can You Keep A Secret? [J] (Distance)
16. Automatic [J] (First Love)
17. Dirty Desire (This is the One)
(Encore)
18. Simple And Clean (Deep River)
19. Me Muer (This is the One)

As you can see from the list, the most exciting moment came along where she played First Love :) She performed many songs from her latest, and then did some talk about Asian population and Japanese fans in SF, and asked "Do you want to hear some Japanese songs?" which psyched everyone, but the songs she sang were Passion and Sakura Drops, which were great but not that "Japanese songs" we all wanted to hear, right? :D

She also played a couple of songs from Exodus, which, er, is not the greatest success to put mildly. Then suddenly she played First Love after Come Back To Me, which made everyone just go crazy (AWESOME video - goose bumps guaranteed if you were there!). She also went on with Can You Keep A Secret? and Automatic... I just hoped the live last forever.

I've seen many japanese bands and artists show in San Francisco such as Puffy Amiyumi and Cornelius but Utada is definitely the best. Actually this is one of the most epic live i've ever been to in my entire life. I'm updating this post on Tuesday, 2 days after the show but i still feel like i'm hungover from her awesomeness.

If you're in Las Vegas, Chicago, NY or London and have a chance to get to see her show (though i know most tickets are sold out) i highly, highly recommend it. This is like once in a lifetime experience that you can see her live in such a small "live house" type of venues, which is impossible in Japan anyways.

Perl Oasis

I'm now in Orlando, Florida attending "Perl Oasis" Orlando Perl Workshop 2010.

It's a shame that I forgot to pack my camera so there are no photos to show but Orlando is really warm and humid and beautiful out there. I hope I can see and share photos from Karen and mdk who had excellent Canon cameras.

I did the Plack presentation again, with a lot of updates reflecting the recent changes and plans we've been discussing. I think it went well and Plack was mentioned in other talks such as Stevan's Ox::Applicaiton talk, mdk's keynote and I got mst's Catalyst book copy as his "thank-you for Plack so we can remove Catalyst::Engine that I hated" reward :) Oh, as I mentioned in the talk, i hope Plack should show up in the top position or near when you search Perl Web Server on google, which currently shows some outdated miserable results. May ask you for some google bomb? :)

Some of the talks I attended: Stevan was explaining his recent Plack-ful project called Ox::Application. Shawn showcased his excellent TAEB nethack bot. Cory (gphat) told us how Solr rocks and how they migrate that to magazines.com search engine. Lucas is a spy from PHP community and told us how PHP sucks which I already knew. dhoss told us his experience with Google Summer of code which was very interesting. Marty explained the functional programming which reminds me again of the goody of PSGI; since PSGI app and middleware are code references, the traditional functional oriented programming methodologies like composing functions just work. We actually have the same code Marty showed, in our Plack::Runner middleware compose sub :) mst told us his tale on his new DBIx::Data::Store.

After the conference we went out for drinking, where we had pretty good conversation with awwaiid and Sartak, and then at the lobby we had an extended hallway session till midnight, like 3, with Karen, Marty, Casey and Shawn.

Today is the hackathon day; plenty of things in my mind but not much time!

Thanks Chris (perigrin) for organizing this great mini-conference and Shawn (sartak) for sharing his room.

Is Plack middleware? Is it library or is it a server?

Short answer if you come from Google: Plack is a toolkit containing middleware, helpers and web server adapters to run PSGI. Plack is NOT a web server but it's rather a "web server interface" which means if you want to run you PSGI application on any web servers, look at Plack to find handlers.

The namespace "Plack::Server::*" to implement PSGI web server was abandoned, and they're renamed to their own namespaces (like Starman, Twiggy, Corona, POE::Component::Server::PSGI or HTTP::Server::PSGI) and we now have Plack::Handler::* as thin adapters to connect Plack to those web servers.

--

There's always been a confusion on what Plack really is. 

First, there were people who thinks "Plack is a framework." This is probably because we have Plack::Request as a separate utility module on CPAN to write "mini-frameworks". Our current plan to kill this confusion is to merge Plack::Request to core as a middleware library and release the current Plack::Request under a new name, kind of like Python's WebOb.py.

Then today I had the following conversations on IRC with Marcus and Theory. They say different things on Plack, and the problem is that they're mostly correct, but the view where they look at Plack from makes them say different things.

Most people on the application developer side thinks "speak PSGI" and "use Plack" mean the same thing. That is totally fine. Using tools like Plack middleware and Plack::Test doesn't add any dependency for particular Plack server environments.

What's confusing you and some of us internally, is that this "Server" means different things: 1) Standalone Web server that speaks PSGI 2) Web server adapters for CGI and FastCGI and 3) plackup and the daemon process. etc. 

There's a couple of PSGI servers out there, and most of them are implemented as a Standalone HTTP server and has Plack::Server::* namespace, while other web server extensions are called like mod_psgi, evpsgi or Perlbal::Plugin::PSGI.

I've been thinking this is a bit confusing, but didn't have a tuit to rename it since that might cause compatibility issues for early adaptors. Today, however, we chatted on #plack and decided to make some changes. But no worries, the changes are 99% transparent unless you're one of the very few server developers (and all of them are involved in this discussion already).

Plack::Server stays is gone, now we have Plack::Handler and they are adapters

Plack::Server namespace will be gone and renamed to Plack::Handler, and plackup -s BlahBlah autoloads the server backend appropriately. That'll be the same and nobody should ever change any code around it. The only changes you need is when you have a hard-coded CGI scripts that has "Plack::Server::CGI->new->run" in it. That will continue to work for a certain amount of period but you're recommended to rewrite it with Plack::Handler, or even better, use Plack::Loader to autoload, without hardcoding the class names.

I already started this process on the branch, and it's now more straightforward. Server::Standalone is now renamed to Handler::Standalone, and it's now an adapter to dispatch the standalone PSGI server implementation whether single proc or preforking one. We had a huge bikeshedding discussion about its names, but we agreed HTTP::Server::PSGI is the most appropriate and intuitive name to upload to CPAN, assuming it's a "reference implementation".

We potentially have another "application" like HTTP server (think of Ruby web servers such as Thin, Mongrel, Rainbows! and such) with a different name but uses HTTP::Server::PSGI and its Prefork class as a base class, and we can continue to have Plack::Handler namespace for adapters/bridges to those new web servers. 

Similarly, Plack::Server::POE would be renamed to POE::Component::Server::PSGI, so does Plack::Server::AnyEvent be AnyEvent::HTTPD::PSGI, etc. etc. 

Well, these bridge classes are called "handlers" in Rack and wsgiref anyway. We always thought it's kind of confusing but we now figure out why they do this, because it's really confusing to name those adapters "Servers" :)

Anyway, I'm all for having top level namespaces to have some software that stands out, but i don't disagree to having descriptive names like POE::Component::Server::*, assuming that plays well with the standard CPANisms that is already out there.

I hope this is one of the last very few things to shave before making Plack 1.0 release... :)

Plack: streaming can be blocking? Hell yes.

It's been a week break since I finished the very well received Plack Advent Calendar. And today i'm back on track on Plack development to get ready for the next week Perl Oasis in Orlando.

Streaming != Non-blocking

A few months ago we agreed on PSGI streaming interface for non-blocking applications and it's now implemented in most non-blocking backends like AnyEvent, Coro, Danga::Socket and POE. This is extremely useful to write non-blocking applications in an event driven callback style like Tatsumaki does.

Well, is feels a little weird that the extension is called "psgi.streaming" when we talk about non-blocking applications, doesn't it? At that time we kind of thought the name is confusing, but today it turns out limiting the streaming interface to "non-blocking" is what is confusing, and names are just correct.

Streaming interface is just an interface, and sure, CGI and mod_perl can also stream content if you call print to STDOUT or $r->print repeatedly, right? Streaming does not have anything to do with non-blocking event loop, and PSGI spec is ready for that: psgi.streaming can be true while psgi.nonblocking is false.

Of course, psgi.streaming is still really important for non-blocking apps to implement "delayed response" as well as "server push" using the event loop callback, but there's no reason streaming can not be done in blocking servers as well. 

Do we really need it though?

Originally we didn't think about implementing the streaming interface in a blocking server backend, and a quick look of existing frameworks such as CGI::Application or Jifty don't seem to support that kind of non-buffering write. So our idea was that you should change the existing code to do IO-like object that gets a callback if you want to do streaming write. But well that is pull rather than push, so it needs to revert the control of the code, and the existing CGI apps or Catalyst apps that does non-buffering write from their controller code does not really work.

rafl, the Catalyst developer, showed up on our IRC channel #plack today, with the goal to kill Engine from Catalyst in his mind (w00t!), and asked if that kind of non-buffering write is possible without changing the existing users code, and we agreed that it's impossible unless we add psgi.streaming to blocking servers (such as Standalone, CGI and mod_perl) as well. 

We already have a Writer middleware that does some kind of fallback, but that does a buffering write and that's not what we want here, so we started implementing non-buffering writes for the blocking servers and that is not particularly difficult.

There are some applications such as Tatsumaki or Plack::App::Proxy that use psgi.streaming and psgi.nonblocking extensions, and it's quite possible they confuse one from the other, and if psgi.streaming is available on most servers, their code can be much simplified. (If you use AnyEvent, just create a condvar and then ->recv on it if it's running in a blocking server)

I'm currently thinking of renaming Middleware::Writer to Middleware::BufferedStreaming so the users can use that middleware as a last resort when the server does not support streaming to run their application, though the ->write might be buffered, and probably promote the "MAY support psgi.streaming" to SHOULD.