somewhere to talk about random ideas and projects like everyone else

stuff

#offline

This site is going down tomorrow in protest of SOPA/PIPA 17 January 2012

Not that anyone is going to particularly miss my blog (most of my other web pages will still be up because I’m far too lazy to go through all of them in order to make a slightly less futile protest). But this is one of the most interesting internetwide events that I’ve ever encountered, and I should be shamed to not take part in it. Copyright and its ongoing war with libertarian and anarchist mentalities has fascinated me ever since squinting through Free Culture on my iPhone more than five year ago. Going to Jonathan Zittrain’s Free Culture X keynote a few years ago was fun too.

I’m sure most would feel uncomfortable with my characterization of this as some cyberlibertarian movement, but I think it’s an entirely excusable position for someone living through a period of adolescence characterized by rebellion and Ayn Rand. It’s certainly not the only perspective, but it seems the most poignant and consistent, not that consistency is necessarily a goal of any legislative body. Legislative bodies are meant, to borrow from programming jargon, to monkey-patch a framework that ill-approximates the societal expectation of government.

Now excuse me while I contrive a metaphor that relates to something that I don’t know much about and will seem cringe-worthy to anyone knowledgeable in the subject. Lets say that the Earth is the shape of what the ideal government is. Not ideal in a sense that it’s perfect and elegant, but in the sense that it is empirically derived through the scientific process of experiment. But in large part, this mass conforms to a very general principle (maximizing individual freedoms). I would characterize this is approximating the earth as a sphere. For nearly all applications, this rather crude geometric model works astonishingly well, leading some to even believe that this is what the world actually is. That everything else, the obsession with finding imperfection is actually a delusion and everything comes from measurement error (well, that’s when this metaphor breaks down because nobody actually does this). Legislators shape the body of law to conform with the needs of society by poking and adding ridges to where the overlay is incongruous. They do the passive activity of fixing the model, the geoid, to incredible precision. But there’s a more radical way, molding the earth into what our model was. We can forget about the lapses, and soon enough the surface of hte earth will erode down into that perfect sphere of ideology.

There. I think I’ve fulfilled my obligation to write some words with some ostensible meaning with regard to something that pertains rather dearly to my life. If this passes and we descend into a dystopian nightmare reminiscent of 1984, at least I’ll have something to look back on. An old man filled with regret waiting to die alone. Or at least without an internet. I did my best (but still far from enough, as is anything besides martyrdom) to preserve crowd immunity to hazardous legislation, but alas we were stricken.


Offline Wiki Redux 30 December 2011

There’s just something incredibly alluring about the concept of holding the sum of human knowledge with you at all times. While near-ubiquitous connectivity alleviates this to a certain extent, the momentary lapses of networking are incredibly corrosive to an information dependent mentality. Wikipedia never ceases to amaze me and, while I’ve tried in the past to encapsulate part of its sheer awesomeness, this marks a much more significant attempt.

The differences start even before the data gets to the application. The preprocessing toolchain was entirely rewritten for a multitude of reasons. First of all, it compresses not the entireity, but rather the most popular subset of the English Wikipedia. Two dumps are distributed at time of writing, the top 1000 articles and the top 300,000 requiring approximately 10MB and 1GB, respectively. While ostensibly, the mere top 300k articles is far too narrow to delve deep into the long tail, the breadth of the meager 1/25th of articles consistently surprises me in its depth. The advantage is that at 1GB, it’s relatively easy to fit into any system. The algorithm which strips extraneous content has been made far more sophisticated than the original series of regular expressions. This enables greater compression and less accidentally omitted content.

On the application end, the application has switched from a GWT-compiled LZMA SDK to a speedy, pure javascript decoder. This makes page loads significantly speedier and allows greater compression ratios, for individual blocks can be made larger (256KB instead of 100KB). It also now uses WebGL Typed Arrays to further speed things up, such as sending data to and from the WebWorker thread.

The interface was redesigned with CSS media queries to dynamically transition between different modes in response to different viewing environments. The interface consists of two regions: the fixed position recessed left panel which holds the page title, a search bar, controls and the page outline. This collapses down to a toolbar header automatically when the screen estate is limited. It uses an Apple-esque noise texture background.

Downloads happen in little units called chunks (they’re half a megabyte for the dump file and about four kilobytes for the index). The local file can be built up out of order. While online, all storage operations check the virtual file, indexed db, or web sql database. If it’s not there, it transparently uses an XMLHttpRequest in order to fulfill the request and caches it to disk in the respective persistence mechanism. A bitset is used to keep track of which chunks are already downloaded and which need to be downloaded.

http://offline-wiki.googlecode.com/git/app.html


Samsung Series 5 Chromebook 23 June 2011

On 5:04pm EST June 21st, 2011, I got an email inviting me to the super exclusive cool club: the Chromebook guru program. Today, June 23rd, 2011 8:05am I woke up to the sound of my iPhone 3G ringing to a certain 800 number. I have this paranoid tendency to ignore most calls, and my half awake zombie state didn’t exactly help my judgement. Fast forward two minutes, I checked my inbox and there was a terrifying Google Voice transcription sitting there in my inbox. That took two minutes because of how glacially slow it is to get anywhere on an iPhone 3G after the deadly error of updating to iOS 4.

Billable to sign for the shipment. You can go to Fedex dot com To track your package is Status, and determine if it is elligible to be held for pick up at a convenient FedEx location yo repeat this information, press any key yank you from Fedex goodbye. It felt oddly incomplete, because it turns out, it was. I listened to the (less disjointed) real message, and it was clear that the first forty minutes of the message was probably omitted, and Google Voice only picked up the small section after the beep. Apparently robots aren’t very good at talking to other robots.

I ran downstairs and checked the door. Nothing was there. I opened the blinds and waited. A few hours later (I managed to build a <canvas> pinball game in the time in between), I heard a sharp knock on the door. I’m not exactly sure of the time because I recorded it all on my iPad and apparently Apple feels that nobody ever cares about when the picture/video was taken and makes it virtually impossible to get that information.

Delivery

So here it was, a brown box. Delivered by FedEx (Which is what’s called a “syllabic abbreviation” of Federal Express as opposed to the boring initialism UPS which stands for United Parcel Services, Inc.). Just because I can, I’ll tell you that the box was about 16.25x16.25x5.5 inches (my portable tape measure is only customary, no metric love). Inside is yet another box (boxes inside boxes are awesome!), but unlike the Cr-48, it’s not a nice friendly brown box with a jetpack-wearing labrat diagram. It’s a gray box with a picture of what’s supposed to be inside the box. The outer box’s packaging was just crumpled paper, which doesn’t look nearly as nice as the other stuff (I think the Cr-48 came with awesome little packaging peanuts).

So here’s the unboxing. Wait, the box was already opened. And what’s with the small empty green speech bubble sticker on the side? Anyway, the lack of tape probably means that some time travelling ninjas hijacked the FedEx plane in an attempt to rip out the TPM module chip in my Chromebook in order to infect the kernel with a keylogger/filter which replaces all references to time travelling ninjas with time travelling ninjas.

Inside is some nice white packaging foam. It’s neatly packed, and pretty cool. Sadly, there’s only one component wrapped in bubble wrap, the rather useless VGA adapter. Random side note: I think it’s rather interesting that the Chrome OS people decided that somehow a VGA adapter was somehow more important than an ethernet port. I’ve never plugged in a laptop to a larger display, and I don’t imagine that being a primary use case. But it would make sense to enable a web-oriented device to have faster web access.

Setup

So I turn it on, and it opens to something about reformatting the stateful partition because of those time travelling ninjas. It doesn’t really bother me, it reboots, I ignore some legalese and click buttons. It updates (which, by the way, took forever), I login the first time and it asks me to take a picture. The camera’s actually a bit nicer and the video isn’t laggy for some reason. This device is noticably faster than the Cr-48. I decide against taking a picture and just select the little erlenmyer flask with bubbling green liquid (presumably this is what gives the time travelling ninjas super powers).

Once again, it reboots. I login, it says wrong password, I try again, realize that somehow it forgot my wifi password, login, and it still says it’s wrong because it’s not done connecting to the router literally three feet from me, and I type my google acounts password again and press enter where it still fails yet again because I’m way too fast at typing, or at least I’ll say that because truthfully I’m not really fast at typing but it wouldn’t do any harm pretending I am because I really feel stupid not thinking about waiting for the wifi network to connect first, and so I stare at that little icon on the top right and then it clicks solid. I login.

I’m greeted a page that tells me how to use a trackpad. I go through the exercises to test my ability to do some rather advanced and intellectually challenging tasks such as “Click the circle” and “Move the circle”. I realized that it was probably just a distraction so that Chrome has some time to load all my apps and extensions by the time I’m done with these challenges.

Testing out the speed of this thing, I clicked Angry Birds, which to my surprise actually worked. Though I have to admit that trackpads aren’t great for these kinds of pointer-driven games. I would much rather play Angry Birds on my iPad.

But I guess as a product reviewer, I should probably focus on the hardware first in order to provide a vague semblance of structure and order to this review.

Hardware

At a glance, the Chromebook is thin (But I lied, it’s not really that thin). Still, it’s quite heavy. It feels heavy and has a really solid build. And it comes with tons and tons of stickers, and that’s pretty spammy, but I guess it has this partially glossy finish that needs protection from ninja fingerprints (oh wait, that’s an oxymoron).

Oddly enough, among the first few things I noticed was that the hinge is a bit weaker than the Cr-48’s. Or at least, when you hold up the device, the lid will sort of collapse on itself under the force of gravity. I think this also happens with Macbooks as well, but it’s a little weird.

Second, is the little chrome logo on the surface has a sort of texture which feels pretty cool. The chrome logo also really bothers me because the colors just feel slightly off. Also, speaking of weird logos, the Samsung logo doesn’t have a (looks up typography terms diagram) crossbar on the “A”, which looks really weird. So it’s more like schevronsung or S^msung or something.

The display doesn’t go as far back as I would like, and with the absence of a protracter, I’ll use my powers of eighth grade geometry (oh wait, no I mean the eighth grade launch of Wolfram Alpha, the last time I ever needed to do math) to determine that the maximum angle is 127.5 degrees (approximately, or 2.226 radians or 4.452 tau-ians if tau day is your kind of thing).

The body of the Chromebook is nice plastic, it’s smooth and pretty hand friendly. The lid is a little weird though, the rim is actually slightly sharp. Not sharp enough to function as a type of improvised knife for murdering people nearby, which a Macbook would suffice at (Anecdote: My leg once started bleeding a lot because it rubbed a little against the sharp part of a macbook pro). But it’s still sharp enough that it feels inconsistent with the rest of the device and to make it feel weird opening and closing the lid. Also, on the lid is this huge terrible shiny bezel. It’s sort of cool for a while when you think it’s sort of cool that you can look at the movement of your fingers while you’re typing a blog post. But it very quickly starts getting annoying and makes the device look cheap. It simply doesn’t feel right in combination with the matte display and the soft diffuse black plastic body.

Sticking cables like USB and the power adapters makes a somewhat loud click, and rather annoyingly it’s nigh impossible to yank the cable out. Unlike the Cr-48 or any Mac, whose cables pop out fairly easily, this device seems to grab hold of the cables and never wants to let go.

I like how the new Chromebook is sleeker and looks more solid. It’s less bland (Once I actually mistook a random black paper folder/portfolio for a chromebook). But it also is less of a total Macbook clone. And when making something less of a Macbook clone basically means adding a glarey bezel and a cheap looking lid, sometimes the blatant clone is better.

Keyboard/Touchpad

I’ve actually never noticed the special browser function keys on the chromebook aside from the volume, brightness and power buttons. I’ve basically never used the full screen button, which I just pressed a second ago and I think this is actually pretty cool. Now I see why OS X Lion has that full screen emphasis. Though for some reason, I can’t leave one page full screened and Alt+Tab over to another non-fullscreen window.

The window switching button, which is more like workspace switching since you only ever have one window open at a time, is more accurately referred to as the “window jiggly button”. Because that’s exactly what it does when you’re on one window, as I’m always on. I guess it’s main purpose is to facilitate those politically incorrect image macros with bad taste about holding F11 in order to make a picture of Haiti (or Japan) shake. It would be a lot more useful if it was a tab switching button instead.

I’ve never used the refresh button, because it’s always easier to hit Ctrl+R and likewise for forward and back, it’s easier to hit Alt+Left or Alt+Right. Same with the search button, I just hit Ctrl+T.

Another weird thing is the placement of the Alt and Control keys. I generally never use the alt keys on my desktop computer, but I happen to use it a lot more often here (mainly for forward/back). But it’s also annoying because it still has a mac feel so I want to pretend that Alt is the same thing as the Command button, and then everything’s weird. I liked the Mac Home/End buttons, which I think were Cmd+Option or something. Anyway, I would really want Ctrl+Alt+Left/Ctrl+Alt+Right to work as Home/End.

Just on looks, the Series 5 keyboard looks a bit weird. The letters on the keys feel slightly off center and the words are printed in a much lighter shade of white/gray. I guess this would help the problems I sometimes have with finding the right keys at night (but I haven’t had this long enough to encounter nightime, in fact, it’s still just past noon). I’ve never noticed that the shift key has a sort of connected “ft” arm (I sure hope I’m using these typographic terms properly, and yes, I did just set the word typographic in comic sans ms).

The touchpad is better than the Cr-48’s but it’s still really quite lacking in comparsion to the ones on all the Macbooks. Maybe it’s just software, because I’m so used to three finger swipe for navigating forward/back, and Chrome on Mac’s Tabpose feature is genuinely magical. Also, two finger scrolling should be kinetic, it’s just that much more natural of an experience and makes the device more intimate.

Tablets

The idea of a Chromebook is very similar to that of a tablet (such as the iPad or a future Chromepad). Tablets are very web, or at least web information-oriented, much like how the Chromebook intends to.

Google has included a rather nice physical keyboard in the device, which shows that they view the keyboard as a superior (and necessary) system for interacting with the web. It’s pretty obvious that the keyboard is great for writing long blog posts, but that’s really not that common of an exercise. Google has to demonstrate that not only is the keyboard useful in certain fringe circumstances, but an everyday useful component.

Google needs to show that the keyboard isn’t just something that gets in the way of interacting with the web, but a useful aid. An emphasis on search; tab search, page search, or web search could do that (and it would be a great use for that Search key). Firefox has a great feature called Type Ahead Find (There’s a chrome extension that tries to do the same thing, but it’s buggy, and sadly Chrome doesn’t sync localStorage state) where you can just type to navigate and click items.

And I don’t think they’ve properly done that. The web is currently still very much a pointer driven world, and the Chromebook touchpad is quite lacking.

Offline

Chrome OS is actually surprisingly useless in situations where the user is offline. You might find the adverb “surprisingly” a little confusing, because almost all the other reviewers seem to bring the notion that being useless while offline is somehow intrinsic to the concept of the platform. Like that it’s obvious that anything built on internet connectivity will always be useless offline.

Every Chromebook out there, to my knowledge has a sixteen gigabyte SSD. Sure a gigabyte or two is necessary for the operating system’s function, the kernel, and the other kernel for that fast background feature. Fourteen gigabytes is plenty of space for a cache. Absolutely plenty.

Chrome has the opportunity to basically cache everything it encounters (and the cache itself is already sufficient for offline browsing if it were accessible), and you can load everything from the cache when the user’s offline. Firefox does this, and I have no idea why Chrome doesn’t.

As for Google’s own applications, it’s rather disappointing how long it’s taking them to add offline. The Gears API isn’t too much different from AppCache, and it’s unreasonable to take over a year (basically centuries in chrometime) to port that feature over. But at least there’s an expected date (summer, which, come to think of it is actually pretty soon).

Software

I don’t know why I’m even writing this section. But since I am, let me first write a disclaimer. Chrome progresses fast. A major release every six weeks. Things get fixed quickly. I remember (half a century ago in chrometime) either Larry or Sergey said something about how Chrome/OS is really about a radical shift that rather than having your software get slower (due to bloat, etc) over time, it actually gets faster. Everything here will probably be irrelevant very soon.

The bundled version, which I’m not using since I almost immediately switched to the always-better unstable ones, had some weird properties. Hovering over the wrench icon would give this horribly hideous black-gray gradient background (and really, that’s all I noticed). Unstable doesn’t have any of that.

Also, I was disconnected from my Wifi network three times in the process of writing this. It may be my wifi network’s problem, but it’s never happened to me before. Music Beta is acting buggy and sometimes stops after opening and closing the lid.

Conclusion

The very first impression is always from the hardware. When it’s covered with layers of stickers, that really does sort of subtract from it’s beauty. I don’t want to spend time peeling off six layers of stickers on a laptop already in a bag inside a foam cover inside a box inside packaging inside another box. The hardware underneath all those stickers is pretty nice, with exception to the lid which has a somewhat sharp edge.

The second impression comes from the setup of the software. I guess legalese is fairly standard, so I can’t take points off for that. Updating right then and there hurts the user experience. You can do that in the background. That’s the point. There’s probably some security rationale, but that initial feeling has a big impact on what users feel.

Once that’s done, the user learns that the device is practically useless offline.

Chrome OS feels incomplete. It’s probably deliberate.

Chrome OS is visionary, and part of the idea is that software can improve over time, rather than getting worse over time. Starting with a flawless experience means that there’s only one way to go: down (That’s why if you’ve owned a Mac for any amount of time, the weakest Macbook Air at the Apple Store feels so much blazingly faster). Starting with a terrible experience gives profound opportunity for a great anagnorisis, which wikipedia defines as

a moment in a play or other work when a character makes a critical discovery. Anagnorisis originally meant recognition in its Greek context, not only of a person but also of what that person stood for. This first batch of Chrome OS devices represents the beginning of Google’s great plan, that of instituting the new paradigm of progressive enhancement rather than regression. The deliberately sour experience gradually and noticably improves every six weeks. It’s the equivalent of waking up one day and seeing that your toaster now makes coffee.

This is what Chrome OS represents. It’s not the web as a platform, because any platform can run applications. It’s about what the web represents, a continuous online system where things improve every day, without notice. Change just happens. Updates are silent and computing becomes alive.

Or at least, I want more free hardware.


Offline Wiki Chrome Webstore App 23 April 2011

Offline Wiki for Chrome is now on the Web Store. The app that started a few months ago and took forms as Offline Dictionary. The chrome app version is much more refined, aesthetically and functionally. The search bar and autocomplete are much less obtrusive, and there’s gradients and box shadows, a sure indication of progress :)

Description

The most significant difference, is that it includes the entire English Wikipedia, a pretty close approximation of the sum of all human knowledge. Uncompressed, it’s something like 30 gigabytes of raw text, and the compressed version that I’ve compiled for this app clocks in at around 3.7 gigabytes (3.4GB for the actual compressed dump and 0.3 for the search index file). Hosting and serving up all those gigabytes does cost money, so that’s why the app isn’t free. But it’s still a pretty good value considering the equivalent apps for iOS are twice the price.

It includes an index which lists every single article in Wikipedia, paginated and navigable through a scroll bar. There are something like 50,000 pages of the index (the exact number depends on the size of your screen), and articles are divided into columns. There’s a button which sends you to a random article, and a search bar.

However the notion of this app is pretty strange: A webapp which only serves its function when offline. However, the ease of installation and use of this app is somewhat unparalleled on the desktop space. This app allows the browsing of the database instantly after the download has begun, rather than nearly all other such apps which require the entire dump to be downloaded first. There’s no conversion process. Everything pretty much hopefully just works.

HTML5 is awesome.

The interface is done entirely with CSS and HTML, no images except the obligatory xkcd reference. The toolbar is a css3 gradient and enclosed with HTML5 tags like <article>, <header> and <footer>. The download progress is indicated by a native <progress> bar element. The index is navigable through a slider bar created using <input type=range> and all the page titles are put automatically into columns by the css3 column layout properties.

The really cool stuff is what’s in the javascript. Probably the most significant development which has enabled this app is an awesome thing called the FileSystem API which exposes a special persistent sandboxed read/write directory which can be modified through the File API. When the page is first loaded, the process of downloading begins, and it uses the responseType attribute of the XMLHttpRequest to get the downloaded chunk (it just sets the request header to specify a range of one megabyte) as an array buffer. Array Buffers are a part of the WebGL specification which enables the storage and manipulation of binary data. It then uses BlobBuilder to convert that array buffer into Blob which can then be written using the File API to the hard disk.

To search, it reads a small portion of the index file by using a particular slice of the file, which is the equivalent of seeking on a disk. It implements binary search to quickly locate a certain article on the disk and then reads a chunk from the dump in much the same way. Then it uses a javascript implementation of the LZMA compression algorithm to decode a certain hundred-kilobyte block into raw WikiText which is then parsed into HTML.

Finally, the pushState and replaceState methods from the History API are used to handle the navigation of pages without reloading,.

Get it now

Offline Wiki for Chrome

 

 

 

 


MP3 Player 26 March 2011

This is my take at an MP3 player. I would consider this one of my better designs (I’m no designer or artist, and that sort of shows). It’s fairly minimalistic, a product both of my design and the fact it was created a few hours. But this isn’t about design. No, it’s a music player that operates entirely in your browser with files stored on your hard drive.

Here’s basically how it works. There’s an <input type=file webkitdirectory> so you can go and select your music folder. It gets a list of all files, reads the first 128 kilobytes of each mp3 file and parses it for ID3 tags, constructs a library and makes it searchable.

The interface is composed of four main items: a huge search bar, the music library, a playlist and the audio controls. It’s pretty self explanatory.

Here it is: http://antimatter15.github.com/player/player.html

It should work on Chrome, it might work on Safari, and will partially work on Firefox 4 (No directory select, no MP3). Definitely won’t work on IE9 and probably won’t work on Opera.


HTML5 Offline Wikipedia Reader and Help me win a Cr-48 04 February 2011

690 (1)

So, lets begin with a plead for you to rate my submission (ideally favorably) for the LucidChart chrome notebook contest, partly because it would be awesome, and I need to test the aforementioned HTML5 offline Wikipedia dump reader.

Wikipedia is awesome, and on my service-less iPhone 2G, one of the most useful apps is one with an outdated (by something like three years) Wikipedia dump reader. There’s something very much like the Hitchhiker’s Guide in carrying the “sum of all human knowledge” in one’s pocket. Wikipedia grows really quickly and the English language dump (as of last month at time of writing) is over six gigs of bzipped XML.

Another reason for building an offline Wikipedia dump reader is simply that I can. There’s a lot of cool and cutting edge stuff in this. It uses the file input JS API, FileReader, Blobs, WebWorkers, fast Javascript to handle a pure-JS implementation of the LZMA compression algorithm, and the FileSystem API.

All this stuff has only been possible for a very short amount of time. In fact, the FileSystem API only arrived with the release of Chrome 9 today. It’s pretty awesome to imagine that a web page, HTML and Javascript is processing multiple gigabytes of data at real time. At the current unoptimized state, the search results and the article shows up instantly within 100ms of a key press event (However, I admit it has yet to be tested on the actual English Wikipedia).

This app is almost ideally suited for Chrome OS. A hundred megabytes worth of data in a month (with the Verizon deal, not to mention that it’s limited to two years) isn’t very much and it would be a pity for the sixteen gigabytes of SSD space on a Cr-48 to be wasted. One of the dumps that I’ve compiled is a subset of the English Wiktionary which makes a great lightweight (8MB) dictionary with over sixty five thousand words. Wiktionary is fairly large at 217MB, and the Simple English Wikipedia is 51MB.

Assuming you can somehow access the dumps, I probably won’t post them on this server because of the massive bandwidth usage, someone else could, the process for using it is very simple. Download the file to your computer through HTTP, a torrent, or create it yourself. You might have noticed that the favicon is a picture of a carrot because that’s the current codename. Well, not really a codename, but I was scrolling through some icon lists and saw one for a carrot on the first page. I don’t even like carrots, but I digress. The dump uploading screen has a picture of a bunny biting a file upload carrot that you click on to select the file. Select the file, and it automagically copies it over to a persistent folder. Then you’re off to searching and reading articles and definitions :)

I’m not done yet, or at least, I don’t feel like packaging it up and submitting it to the chrome webstore yet because I need to figure out a good way to host database dumps.