This is my submission to blogdom’s burgeoning class of eternally “work in progress” sites. I’ve been working on this blog reboot for nearly a year at this point (rest assured, I haven’t worked on it for any appreciable fraction of that time— but it’s nonetheless traumatizing for me as “blog posts written” is my primary metric for personal productivity).

There’s still a lot left to be done, but right now it should be in a more or less functional and navigable state. I’ve just added image thumbnailing (so this homepage shouldn’t take a triassic aeon to load anymore), which seems like an addition substantial-enough to warrant some new words adorning the featured post callout.

Most importantly though, it’s now a hip static site hosted on Github Pages (though I might move to S3), so the Turkish phishers last remaining venue for hijacking this site may be sending a pull request.

Getting Hacked

I unwittingly invited some consultants work on the new blog design, but all they could come up with was clearly somewhat unoriginal…

I don’t generally pay a lot of attention to this blog— especially during the beginning of every month. I used to generally hold myself to a rule that I would post something new at least once a month, but that policy kept slipping and then there was a mad dash to finish some blog post the first or second day of each new month to be retroactively timestamped.

Anyway, point is that I didn’t notice when the site got hacked for a few days. Unfortunately I don’t have any screenshots of what the site looked like when it was hacked. And at this point I can’t promise you that my mental model of what it looked like is anything more than a hallucination.

It’s a bit of an interesting chapter in my blog’s history, so I’ve spun off this section into its own post.

First Sketches

I guess it makes a reasonable amount of sense to write about how this site works. This is the first incarnation of the blog which is to any appreciable extent aesthetically original. It isn’t the default theme of JupiterCMS, PHPFusion, nor is it a free template for Wordpress.

It’s a pretty simple design, and it’s pretty nifty that in this day and age, a design which is sparse by ignorance is indestinguishable from a good design which is sparse by (uh) design. There isn’t much besides the simple arrangement of rectangles and lines and it relies pretty heavily on typography and whitespace to delimit sections.

At around the same time my blog stopped working, I came into the posession of a Moleskine. On June 14th, apparently, I started my first sketch of what my blog might be like.

In retrospect, it didn’t really look very coherent and I have literally no idea what I was trying to do at the time. But one of the motivations behind the entire endeavor was to strike a sort of balance between a blog and a portfolio. I like the idea of documenting the process of things and writing some part of the though process a somewhat non-abbreviated form. But one thing that I noticed is that a blog is a terrible means of surfacing older content, and I think a lot of the interesting ideas I’ve explored are the ones which I played around with several years ago.

On the other spectrum, I didn’t want to go too far in the direction of summary. I didn’t quite want it to be simply a résumé where I condense every endeavor into two buzzword-packed sentences which summarize the little iota of cleverness imbued in the project.

And so the motivation of the design was finding some way to hybridize the two goals— summary and process.

I found another sketch in my Moleskine dated June 24th, which seems to be a bit more coherent. On the right you can see some blocks and lines which, while not having much bearing on the current site design, at least seem like they might pass a Turing test criterion for sufficiently-websiteish website.

One thing that I noticed is that the sequence of my projects tends to be pretty structured. Over the years, I tend to explore little ideas which tend to culminate into singular projects— or alternatively, I build a larger project and spin off smaller components. All of this tends to happen over a relatively short time period (a few months to a year).

So in terms of a projects list, this means that I can introduce a bit of an aesthetic cadence where a series of minor projects are occasionally punctuated by larger projects. And because of this natural faux-hierarchy there’s a natural clustering which is generally both temporal and subjective. The short projects have single-sentence summaries, whereas the larger projects can have a paragraph worth of elaboration.

Naturally, all the project descriptions are hyperlinked and lead to a project page which includes all the blog posts documenting the process of creating that particular project.

New Tools

At some point I helped design a mobile app and ended up getting reasonably familiar with Adobe Illustrator.

It was also at this time that I started using tools that designers use— things like Adobe Illustrator and Bohemian Coding’s Sketch.

Work expands so as to fill the time available for its completion.

— Parkinson’s Law

For the past four or so months, I’ve been working on just one major project. It’s rather depressing to think that I built a reasonably impressive initial prototype over the course of about a dozen sleep-deprived hours, and that all I’ve accomplished since then is minor polish. Technically, there have been at least two rewrites, completely new capabilities and substantially improved technology, but none of that really matters when it comes to describing the project. A project is what it is, at 80% completion, at 95% and 99%.

Second semester at school has started, and that means that Pass/No Record isn’t a thing anymore, and everyone else is adjusting their behavior appropriately. Problem is, I haven’t changed. It turns out that an entire semester of racing toward the bottom is unsustainable when suddenly the floor has been adjusted to above your current position. But the more important point is that it’s leaving less time, and in particular, less contiguous time to work on anything.

Last month, I was working on a port of the Telea inpainting algorithm. Inpainting refers to any kind of image processing operation which attempts to intelligently fill in certain regions of an image with other content. Perhaps the most famous implementation of this is Photoshop’s Content Aware Fill feature, which uses a texture-synthesis and patch-based system, which enables them to copy over things like patterns, and textures, filling in vast contiguous regions of missing content. The problem is patch-based inpainting is almost always quite slow, in spite of its high quality results. There are simpler algorithms based on the Fast Marching Method like Telea or Navier-Stokes which use diffusion models and the ilk in order to propagate the solid colors of the bordering region of some inpainting mask inwards. I’ll write an actual blog post about this once I package it up and build a nifty demo for it.

Last year, Ben Kamens of Khan Academy posted about reverse engineering the flyout menu in Amazon, which suffered from the rather odd behavior of acting exactly how it should. You totally should be able to navigate to a submenu without worrying about accidentally mousing over something else. I looked around for a context menu library which could actually do this, but for some reason I couldn’t find one, so I decided to make my own.

If the frequency of these progress reports is to say anything, it’s that I’m terrible at sticking to a schedule and even worse at estimating progress. I was hoping that I’d be able to use the Thanksgiving break in order to be productive and get ahead of my various projects– in particular, getting the text-detection and OCR extension to a state complete enough to publish.

For the past few days I’ve been playing with writing a minimal cryptocurrency which incorporates the central innovation of Bitcoin— the hashcash based transaction chain. It’s a simple enough concept that I could manage to implement the core functionality in about a hundred lines of Coffeescript.

I’m also working on a cute little alarm clock which incorporates a pressure sensor underneath my bed in lieu of a snooze button. The goal is to use it as a framework for novel approaches to the tried and true problem of getting undergrads to wake up at ungodly hours (anything before noon). It’s powered by an Arduino and I2C ChronoDot and a large 7-segment display.

I’ve laid a little ethernet cable from the side of my bed, along the ceiling and to the door which contains a bunch of little EL wire inverter boxes (I’m smothering the transformers with Sugru in order to hopefully dampen the onset of my acoustically-induced insanity).

Screenshot

MusicAlpha works again, and this wasn’t some minor update that fixed a small hole, the entire app has been rewritten. There’s a new user interface which has really quite beautiful animated polar progress bars, but the most significant part is that the mystical protobufs-over-https protocol has been decoded and documented.

It really started on January 30th, when Simon Weber contacted me for information on the upload process because his Unofficial-Google-Music-API received various user requests to implement uploading. I remembered a message sent a little while earlier offering help decoding the protobufs and tried out the method described, which involved using a disassembler to patch the curl_easy_setopt method to disable checking of SSL certificates. This turned out to be the trick and I used Fiddler to collect a few dumps of data for subsequent analysis. That was really quite a magical moment because everything afterwards was relatively straightforward. The huge stumbling block which had prohibited progress for these past months had been lifted and the task which was left, of reverse engineering the protocol, seemed eminently viable. With only minor hiccups over the next few days, I built a python prototype of the program. I created a .proto file which was somewhat human readable with field names based on guesses.We realized some pretty cool things or at least got a much better understanding of how Google Music seems to work (This blog post was started in the early part of February, but I’m finishing this up halfway through March).

For instance, the reason Music Manager doesn’t work on virtual machines is because they use the computer’s MAC address as a system for identifying individual devices. They have a cap on the number of devices which can be authorized to manage your music, which is currently set to 10. This caused a bit of confusion when I hit that device cap while testing (At one point I assumed that the MAC-esque field was actually a random string, so after fifteen or so tests everything started silently failing). However, it occurs to me that it isn’t particularly interesting to read the narrative of how exactly the current version of the application has come to exist. The technical overview of how the actual protocol works can be found on the google-music-protocol project page on Github. It includes a brief overview of the mechanism as well as sections which detail the specific messages which are sent to the server and also includes a simple reference implementation. Hopefully it should be enough information for anyone who is actually interested in doing something which relates to it (though I am curious if anyone could possibly create something interesting out of it, please contact me if you decide to do anything at all with it). So at this point, I’d like to skip over to the actual application-specific aspect: MusicAlpha itself. This is the story of how the rewrite came to be, and the rather lengthy time it took to to write this blog post describing it.

One of the first things I realized was that the entire application would essentially need to be rewritten. The code dealt primarily with altering the attributes of a song by mimicking the web client’s interface, something which wouldn’t be very useful now that all the metadata is sent via the magical protobufs scheme. Since that code gets scrapped, I figured why not scrap the rest. I wrote the first version while experimenting with darker color schemes and obscenely prominent border radii and gradients. I’m not sure that I can say it’s objectively better, but it does look a bit more interesting, especially that big two-ringed polar chart.

There’s some text to the right side, but it’s not particularly of interest. There’s a big file selection button, but that’s pretty uninteresting. Embroidered on the solid gray backdrop is the logo, a simple and unchanged music symbol- unchanged from the last version. I used, for the first time, custom web typography. Just some fancy looking sans-serif fonts I picked from Google’s Font Library. On the old application, I had two progress bars to show the upload progress, one is for individual songs and the other is the overall progress. I wanted to portray upload progress in a manner which was somewhat smoother and more aesthetically alluring than a mere linear bar.

In trying to create that better progress bar, I created a polar progress bar in canvas. All updates are smoothly interpolated and watching the bars spin around and invert is somewhat reminiscent to staring at an old record spin on a turntable, something which I’ve only seen a few times in my (rather short, at this moment) life, but magical enough to still inspire a sense of nostalgia.

Central to making an aesthetically appealing polar chart is making the animation smooth. This might be doable through various esoteric CSS transforms, but I’m not particularly knowledgeable in that area, and I’m not a huge fan of crafting elaborate animations with CSS. It lacks a certain amount of control (which manually updating and redrawing affords) and tends to feel extremely sluggish on computers which lack extremely modern graphics cards (such as mine). Since things like uploading a file involves a lot of guesswork and various aspects of the upload process are very much stochastic (ie. waiting for the server to respond), the smoothing system needs to be able to create a nice smooth animation out of highly erratic information. But it also needs to simultaneously relay information without having an excessive amount of lag.

In order to strike a sort of balance, it uses an interpolation algorithm which was employed in Steve Wittens’ js1k entry and you can see my code here. In order to minimize the CPU tax of running such an animation, it uses the HTML5 requestAnimationFrame function. This has the nice side effect that when you leave it uploading in the background for a while, it automatically stops updating the animation and quickly slides into position when you return.

There are two rings, the outer one being dark gray and the inner one being a somewhat lighter shade of gray (fitting into the mostly unsaturated theme of the interface). The outer ring represents the total upload progress while the inner ring represents the progress of an individual file. The outer ring “spins” in a sense, in a clockwise manner while the inner ring rotates in the opposite direction. Every time a ring completes a revolution, it “inverts”, for instance, when the outer ring fills up and becomes a solid circle, the ring begins to open from the opposite direction so that the imaginary end of the rotation continues unimpeded. The animation is somewhat difficult to describe in words so here’s a demo of the progress animation.

When the page first loads, the polar plot exists in the background, behind the prominent logo. It represents the song quota, the percentage out of the 20,000 song quota which Google Music imposes.

While implementing the algorithm, I had a little trouble because of SSL certificates. The domain which accepts all the protobufs encoded information has a certificate signed for the wrong domain, and browsers have a tendency to care a lot about certificate validity. To get around it, those requests are proxied by a simple server which is made with Google App Engine. All communication with that server itself is signed as well, and it doesn’t do any logging.

When I was about to publish this post and first announced that it works again, there was a bit of trouble which people had getting it to work. This was actually because Google Music allows each MAC address to be associated with a few accounts but forbids any additional accounts. The first released version would use a single mac address for all clients, but it has since been replaced with a sort of unique identifier which is stored in localStorage.

Apparently a browser based uploader may be something coming soon to Google Music. Enjoy this app while it lasts.