bulknews.typepad.com

One Blog.

OK, I just rebooted my Japanese blog and only a few days later now, it just feels stupid to maintain two blogs, just for the sake of separating English and Japanese audience.

There's Google Translate if you like to read my thoughts in whichever language, and the blog posts will mostly be a short link blog posts so there won't be a huge barrier, and I will post in whichever appropriate language considering which audience will be interested, and maybe in both when I feel necessary.

That's still easier than maintaining two blogs, and because I will feed the updates into Twitter, Facebook and Google+ updates, separating the audience won't make sense anyway!

Anyway, subscribe to my new feed and you'll see more posts showing up there.

My new Japanese blog (weblog.bulknews.net)

weblog.bulknews.net

I just revived my Japanese blog, which has more than 10K subscribers on livedoor/Google but has been abandoned for more than 2 years. It's now up on Tumblr.

I've always been a fan of TypePad, which I am proud of having worked on at Six Apart/SAY, but for this particular blog, it just strikes me that Tumblr is easier to customize and do some wacky things like inserting Ads. (I also considered github's Jeckyl as well, since it does the ultimate flexibility in doing crazy things :-))

Anyway, Japanese posts will go there, but I keep blogging in English here, so no worries.

Read Later apps on Kindle

I've been toying with Read Later applications on Kindle Touch and am trying to find the best tools out there. It is sad to tell you that none of the applications score perfect. 

Here's the complete list of things I want for read later apps:

• Can add articles on Chrome with a keyboard shortcut, not clicking bookmarklets
• Can add articles from Google Reader with a keyboard shortcut
• Can add articles from iOS apps like Echofon and Flipboard
• Can add artciels from Android with "Share URL" intent
• Deliver unread articles to my Kindle every day, on preferrably configurable time
• Kindle version should support chapters for each page
• Kindle version should support inline images

Read It Later scores almost perfect on the adding articles side. The bookmarklet works with keyboard shortcut using Chromac extension, and keyboard shortcut with Google Reader using Obvious Interest, supported by Echofon and Flipboard. Android Share intent works nice because it uses "A" with "Add to Read It Later", and share intent is sorted by alphabet - it's a nice hack.

Read It Later does not support sending articles to Kindle, so I did some hack using RSS subscription on Google Reader to send it to Klip.me. It works mostly well, except that neither Read It Later API nor Klip.me document supports inline images. Also, Klip.me does not display URL nor author of the page until the very end of the article, which is a bit annoying when reading a summary list.

Instapaper is also pretty good on the adding side, but for some reason I could not get the bookmarklet to work with keyboard shortcuts. That means I always have to move the mouse over there and click the bookmarklet, which is annoying. However Instachrome extension seems to work and allows configuring keyboard shortcuts. It is good that the bookmarklet natively supports Google Reader (currently selected items are automatically detected), and lots of iOS apps support Instapaper and there's a Share intent app available on Android, although it's mediocre.

Kindle documents generated by Instapaper do not seem to support images either, but it looks sligthly better than the one created by Klip.me. I could use Instascriber to synchronize my Read It Later items over to Instapaper automatically, although it probably means i have to manage unread items in two places.

Readability has a nice Chrome extension that supports keyboard shortcuts. I can't yet see Google Reader integration that works with keyboards and iOS apps support is a bit weak since its subscription service is new. The Kindle documents sent by Readability look the best out of three, and seems to avoid the Kindle Software bug that you can't archive items on the device.

So far, I'm trying to move over to Instapaper, but that requires me to fiddle with lots of input channels to be updated.

UPDATE: I completely switched to Instapaper. The reading app on iPad is much better than that of ReadItLater. The Kindle reading experience is nice but it also doesn't support inline images, and there's the other Kindle software bug (maybe) that does not recognize the beginning of the book correctly. It always opens the first page of the last chapter, instead of the first, when I first open the book or select "Beginning" from Go to menu.

UPDATE 2: Instapaper officially recognizes that there's a software bug in Kindle Touch that it has some issues with Instapaper generated Kindle docs.

Today I switched to Readability's Premium plan since I support their idea of giving back the revenue to the publishers. (Wow I switch apps like every day now) The problem with Readability is it's lack of universal "Add to..." support on iOS and Android applications. And also there's no keyboard shortcut to add from Google Reader. To solve this, I set up two ifttt tasks, to import from Google Reader starred items and Instapaper personalized RSS. So now I just star items on Google Reader, and keep adding articles to Instapaper from Android and iOS apps, and ifttt takes care of syncing articles to Readability, and it now sends me the documents to my Kindle every day.

Kindle Touch

So the next day of Kindle Fire delivery, I got Kindle Touch delivered.

tl;dr - I like it.

Unlike Kindle Fire, it is a single-purpose device that does one things: reading an ebook (and personal documents and all that).

Hardware

The hardware is pretty light and easy to hold. Because it is a touch interface that accepts taps to go through pages, it is possible (not that easy though)  to hold it in one-hand and turn pages. It would be great for people who want to read in a train while standing (not me :D). It also supports swiping left and right turn pages.

Speaking of swipe, you can swipe up and down to go to previous/enxt chapters, which is handy when reading articles and newspapers I think.

Touchscreen

I was a bit curious about the performance of touch screen of e-Ink. It works reasonably well. Because I haven't had any other Kindle devices I can't compare, but to me it is responsive enough. Ability to select a word to lookup on built-in dictionary and highlight for notes is something you'll greatly miss if you have a non-Touch Kindle, in which case you have to use D-Pad to select words.

Cut and scanned PDFs

Because Kindle hasn't launched in Japan yet, it is still popular to create your own eBooks using a cut-and-scan technique. I have a bunch of these PDFs, and sending them to this Kindle doesn't really make a readable experience. You have to optimize the PDF bu cutting surrounding spaces, and make it a monochrome rather than grayscale, etc. 

I scanned most of the books I had in Japan by BookSCAN, and they have a service to optimize the PDFs for various Kindle devices. I tried that, and it makes somewhat readable PDF, but it still hurts my eyes. Can't wait for the launch of Amazon.co.jp Kindle store!

Read Later

This has little to do with the new Kindle Touch, but because this is my first real Kindle device, I toyed with a little bit about Reading apps, sending articles to Kindle automatically.

There are many, many applications available in this field: InstaPaper, Read It Later and now Readability and Evernote.

For my iPad and Android reading app, I've been using Read It Later - because Instapaper doesn't have a decent Android apps (well, there are many third party apps, but none of the ones I tried matched what I liked and wanted to use). 

But Read It Later doesn't support Kindle output, at least not natively.

I've searched through alternatives to find out something that supports Kindle, but because I already integrate RIL to my browsers and Google Reader shortcut extensions, i don't like to change all of them. The options I have is to sync the RIL to some other services. Yeah, all of this feels a lot like Plagger or Pipes recipes now :)

I ended up Klip.me. It supports adding via bookmarklet and browser extensions much like RIL, but also supports fetching feeds from Google Reader. The generated articles have chapters for each web page, and works perfect on my Kindle.

So what should I do? I should just convert ReadItLater unread articles as a fulltext RSS feed, then subscribe to it on my Google Reader, and then tell Klip.me to fetch that from my Google Reader (with a special folder for Klip.me).

First i tried ReadItLater feed app, but it doesn't generate a fulltext feed for me, so I created my own little app that does it, and host it on dotcloud.

It worked. Beautifully. I get all the unread articles on ReadItLater on my Kindle, automatically delivered once a day. Hooray.

Conclusion

Overall I'm pretty happy with Kindle Touch. It is reponsive, easy to use in one-hand, and ability to use touchscreen to turn pages and select words is super. 

CPAN dependencies are more fun to manage than ever

At London Perl Workshop 2011 there were many talks about CPAN modules and dependencies. 

When people talk about "CPAN dependencies", it usually used to come with the phrase "Hell". Not anymore. Everybody talked about more fun things to do with CPAN dependencies, which is probably inspired by the toolchain updates and my own cpanminus.

Especially at lightning talks, Tim Bunce explained his dist-surveyor tool that scans the whole library path and queries metacpan to figure out which distributions you have to need, when you migrate perl to a newer version. It also rebuilds the CPAN mirror structure you can then feed into cpanm's --mirror option (like Carton does).

JJ introduced his own cpackage module, which is extremely similar to what we do in carton or OrePAN do, i.e. rebuilds the mirror index based on the archives you downloaded with cpanminus, and then bundle cpanm itself into a self-contained perl script you can execute on a remote server.

Penfold gave a quick tour for "best practice" of getting your own Perl with perlbrew, install modules with cpanm and deploy with carton.

In addition to all of this, I presented the proposal on cpanfile, which is an extracted version of Module::Install's DSL to represent CPAN::Meta::Prereqs. This is still a proposal, but it will be trivial to implement on upcoming cpanm and Carton releases, and this would make declaring and managing CPAN dependencies in your Perl application much easier.

Kindle Fire Review

I was originally thinking of skipping Kindle Fire because I'm happy with the original iPad, but when I visited my parents back in October, Dad said he was considering of buying an Android tablet. I thought it was a terrible idea, given the lineup of available options of shitty Android tablets in the market. I offered to give him my original iPad, but he wanted something lighter, like 7 inch. That made me buy this Kindle Fire, hoping this could be an actually "usable" Android tablet for him.

The gadget arrived earlier today, and I played with it for 2 hours now. Here's my review.

(tl;dr - go to the Conclusion in the bottom :D)

Hardware

As a 7 inch tablet, I was expecting something light. It is definitely lighter than iPad, but it is yet over 400g, and it feels heavy. I can carry it in one hand with no issues, but reading or watching something on the device with holding in one hand is definitely a torture.

It comes with a wall-to-micro-USB plug, and no standard USB-microUSB, which I can't understand why.

There's only one button - power button. And there's no volume up/down or an orientation lock hardware button. It is easy to access them from any apps using the power widget on the right top navigation bar (or some video/music apps display them inside the app), but hardware keys for these functions would be greatly missed.

The power button itself is hard to locate and touch. Actually, it is hard to push when I need to wake up the device, yet is easy enough to accidentally push when reading a book.

The size is a bit wide for my one hand to hold, yet narrow for two hands to hold. I know I'm a bit biased and unfair when the original iPad I use always wears the Apple soft case.

Unboxing

Unboxing experience is pretty smooth. Like other Kindle devices, Fire is already pre-configured with your account. I just selected Wi-Fi SSID and entered WiFi password, and then it recognizes as my account, and started downloading the software update. Once it is done, it's preloaded with bunch of books I purchased on Kindle, and some apps I purchased through Amazon appstore and Amazon MP3 (see below). The whole out of the box experience is pretty good.

OS

Kindle Fire is based on Android OS. As a heavy Android user (using 2.3 Gingerbread OS with MIUI custom ROM on a rooted phone), it gives a mixed feeling from "Yeah this is how Android works" to "Where is the screen for this and that?".

It feels clever that notifications bar now has two modes, one for notifications (left top) and power widget (right top), but could take a while to get used to.

The usual Android's Home/Search/Menu/Back capacitive buttons are gone, and they show up depending on the app's state, and for some fullscreen apps you have to tap the bottom key (the shape of "^") to pop up the menu.

Some Android apps tend to "hide" advanced menus under the menu key, which this new button UI would be really confusing for, though I guess that's common for the stock Android OS as well once they remove these capacitive buttons and go software button only in ICS 4.0.

There are some tweaks in the OS itself - for example, the configuration to turn off WiFi when screen goes off is enabled by default, and I can't yet find a screen to disable it. Amazon probably doesn't want to give users a control on this, and for the most use case of this device, such as Video, Music, Games and Books, the screen should be always on, and yet Amazon wants to give this device a better battery life when not used. Otherwise, Android's battery won't get as long as iPad when WiFi is on.

While charging, there's no actual sign of percentage of the battery. You have to go to the power notification, click "More", and then "Device" to see the battery percentage. It is ridiculous.

The OS crashed once for me, once I applied the software update and tried to remove "IMDB" from favorites tab. It was a sudden death, and screen went black, fiddling with power button did not do anything. The solution was to long hold the power key for so long, like 20 seconds, and then leave it and push again. First, the crash was bad, but second, this recovery process was really confusing.

Performance

I haven't much experienced the lag or slowness that I usually have with Android phones or tablets. The screen scrolls pretty fast and rendering on reading apps and videos are really smooth. With the help of server side rendering with Silk (see below), i see almost no issues with performance on this device.

Video

I am a Amazon Prime customer and now can access lots of streaming video content. It works really well, the streaming quality is pretty good and the screen is beautiful. Yet, without a stand it would be a torture to watch a 2 hour movie on this device.

Other video apps like Netflix and Hulu Plus can be downloaded from Amazon appstore for free, and they work really well as well.

Reading

Reading books on Kindle Fire is exactly like reading on iPad. It is not e-Ink, so it might not be the best for your eyes, but the reading app is snappy and dictionary works pretty well.

When I compare the original iPad and Fire side by side, Fire's screen and resolution looks slightly better.

Music

I buy music from time to time on iTunes and Amazon MP3 but these days almost converted to use Rdio for streaming. I only briefly tested Amazon MP3 Cloud Drive thing when it was out, and have almost no music in there. Google Music and iTunes match aren't perfect either, but Amazon MP3 uploader still leaves a lot to improve, including its complicated checkout process and confusing pricing model.

Pandora and Rdio music app are available on Appstore for free again, and they work really well. It is interesting though that the built-in Music app has a play/pause button in the power notification bar (right top) but other music app tends to have "Ongoing" notifications in the notifications area (left top).

Documents

As requested by my Japanese friends, I tried to copy some scanned PDF files, one for manga and another for paperback novel scans. Manga PDF file looked okay, while a novel PDF looked really horrible. It may probably be because the file was just scanned in a low-res format, and I haven't applied any optimization for Kindle.

Later I found out that Adobe Reader app on appstore can display the same PDF in much more readable way. Adobe probably has a much better PDF browsing functionality than Kindle's own software.

Photos

Remembering that my dad wanted to use the tablet to browse his photos, Kindle Fire's pictures functionality is so poor. It has the same-old Android stock Gallery app, which can only see photos stored in the device. Given that the device doesn't have a camera, it sounds like a joke. It also doesn't support Amazon Cloud Drive's Pictures folder - I uploaded some jpeg files to the folder via web browser(!), and nothing shows up.

There may be third party apps that supports viewing Picasa or Flickr albums.

Silk

I haven't fully tested Silk enough to review its functionality, but the browsing experience looked fast - much faster than the original iPad, and most javascript enabled sites work without any issues.

Other apps

"Contacts" app looks like a joke.

"Facebook" app is just a bookmark for the website, and opens in Silk browser.

Japanese support

Obviously the device can display Japanese web pages, but because it uses the Android's stock CJK fonts, some characters are unified with Chinese version of the glyph, and it looks weird.

You can't input Japanese texts with the stock Kindle keyboard. You have to install third party IME from the market. I haven't even tried it yet, and am not sure how the OS allows it.

Third party apps

Third party apps are available through Amazon appstore. It allows side-loading applications (.apks), so if you're geek enough to root your phone and extract .apks out there, you can probably install more apps that are not available on Amazon store.

Because this is an Android software that isn't authorized by Google, it doesn't come with Google Auth Framework. It means most apps that requires the framework doesn't just work, even if you install the APKs.

Conclusion

Just as this device is designed, Kindle Fire is best to consume media provided by Amazon, especially on demand videos, books and music through Amazon app store. the 16-9 screen makes a perfect fit for video apps including netflix and Hulu.

In my opinion the device is still better than most of Android tablets out there in the market, and $199 price tag can't be beaten so easily - yet, it is far, far behind iPad, even the original iPad.

I will see if my Dad still likes to try it out, but given the lack of integrated Photo browsing experience and Japanese input support, he will probably not like it, in which case I will return it.

Kindle Touch, just for reading books, will arrive tomorrow :)

Status of cpanmetadb

Google AppEngine went out of beta and changed its pricing model today.

cpanmetadb the backend database that cpanm uses, is affected by this change. Basically because it heavily uses GAE datastore, it begins to throw OverQuotaErrors this morning.

I've made a change to it to use MetaCPAN API instead of data store. It seems to work well for most of the modules, but because MetaCPAN handles modules and scripts a little differently from just parsing PAUSE 02packages file, there are some mismatches for some queries. Namely, there are some modules that do not return expected results, such as version or warnings returning the perl-5.14.2 as its distribution, etc.

I dont' have time to fully migrate this over to full ElasticSearch based query, or to have another cpan packages DB running on some other host - so far cpanminus has an option to fallback to search.cpan.org screen scraping, which used to work fine before our switch to cpanmetadb.

I will keep you posted as I make more updates to stablize everything.

UPDATE: Nov 9th 3pm PST: I set up a new instance of cpanmetadb using Perl, Tatsumaki and Twiggy and on-memory database (aka perl hash :-)) and run it on my linode instance (might take a bit while before DNS update gets propagated), and switched cpanmetadb to redirect all the requests to the instance.

This seems to be working well for most clients including cpanm. If your client doesn't handle redirections automatically, you might have to change that to follow redirects (like -L option for curl). Let me know if you have any troubles with this switch.