392722_2010404474880_380090264_n

You might have noticed that I haven’t written much for this blog in the past few months. In truth, it’s because of school work, which has never really been something of an issue before. This is, quite probably the least productive stretch of time in my life thus far. I have a suspicion that this issue stems more psychologically than due to some radical increase in work load, but I haven’t looked in to testing that hypothesis (I’ve been collecting data hour-by-hour about what I’ve been doing in the past two years, so I could probably look into it if I were actually interested in that matter). But school’s nearing a close, and hopefully I can get back to a more productive lifestyle, maintaining my blog and most importantly, trying out cool things. I have a few things which I am working on at the moment which should be completed in the coming weeks (though I make no assurances). But since I have an internal goal for writing one blog post per month, I’m going to recycle a project from December of 2011.

Nearly every day, I inevitably end up glancing at my Facebook “buddy list” of sorts, wondering how many people are online. It’s a figure which almost always seems to depend on the time of day, and behaves almost like clockwork, there’s always a massive swarm of people online around 10-11pm, and hardly anyone is ever online at 4 in the morning. I guess the problem with drawing any conclusions from this in particular is how specific a group this graphic represents. It constitutes my friends, and in particular, my Facebook friends. Essentially all of them are people I’ve encountered in real life, and may or may not actually find interest in. But the thing that unites just about everyone is that they’re generally high school aged.

Before going on discussing how pretty of a chart this is, I think it’s worth going through what this chart actually represents. It’s quite easy to tell that this is in fact a polar chart, and on the inner circle, you can tell that it’s a 24 hour clock. Each of the rings represents a friend, and the rings are sorted by the total amount of time spent on Facebook in the given period. So you can see that toward the middle, the graph is almost opaque at every time, whereas on the fringes, the online activity is quite erratic and infrequent.

So, where does this data come from? It’s actually quite simple to get from the Facebook API. I have a cron job which runs every minute to run a FQL request and save the results to a specific log file.

The actual FQL which runs in order to retrieve the list of online users is

SELECT uid, name, online_presence FROM user WHERE online_presence IN (‘active’, ‘idle’) AND uid IN (SELECT uid2 FROM friend WHERE uid1 = me())

Basically, get the User ID, the name, and their online presence state for friends who are either active or idle in the list of the logged-in user. Since Facebook is an OAuth2-type API, you need an access token in order to do anything cool. I just use the Facebook Graph API Explorer to generate my access tokens. Just go press “Get Access Token”, and select (at minimum) the permissions “user_online_presence”, “friends_online_presence” and “offline_access”. Then copy and paste the revealed token into some authkey.txt and you should be set.

I have a python script to go through the log file and to render it as the polar chart which is depicted on the top of the page. The code used for that is frankly atrocious and the output is even more so. Python Imaging Library is used, which is a lovely library, except not for drawing graphics. There isn’t any smoothing or anti-aliasing on the arcs drawn by PIL and they all look hideous. So I render the chart at some absurdly high resolution and down-resize it in GIMP while adding layering, blurs and opacity in order to make the picture somewhat less atrocious. Also, it does’t support restricting the app to drawing a specific day of the week, even though it might be interesting to see the how the trend differs on a weekday versus weekend.

Something interesting about the appearance of the polar graph is that it almost resembles something of a digital fingerprint, and that brings up some interesting privacy considerations. Inside that graphic are the Facebook browsing habits of some two hundred people. There’s the question of how much this changes day by day for users, and to what extent this can be used to identify people. And even if a single ring doesn’t unambiguously represent a single person, the two hundred or so rings of their friends probably goes pretty far into identifying people. There’s also a striking amount of uniformity that says a lot about the type of people who I tend to associate with. Just at a glance, one can tell that there are very few people I’m friends with on Facebook who live in different timezones. Maybe what’s more dangerous than being able to identify a person is to be able to identify what kind of groups that person belongs to. And over the course of a day, just about everyone checks Facebook a few times.

Schedule Compare

Posting this here is almost certainly useless. I assume very very few people who read this blog tend to be in the target 13-18 year old facebook-using high school student demographic. By the unlikely chance you are (if you aren’t, you can forward this to the nearest person who fits into this demographic, as I will add later, that I’m desperately looking for users).

It’s that time of year. The brief window where summer vacation isn’t technically over yet as school hasn’t started but you still know your classes for next year. You’re frantically attempting to complete those long procrastinated summer assignments, or like me, you’re desperately trying to avoid them by giving a yourself a false sense of productivity by building random apps.

My first foray into the realm of creating Facebook applications is fairly simple. It compares class schedules. In truth, the reason I made this was probably not the fact that I enjoy making useful tools, but more likely residual bitterness of rejection by a sci/tech high school over three years ago which has a school-specific schedule comparing app. Nonetheless, a neat side effect of this attempt is that it does happen to be quite cool.

This is also my first published app which is written in the CoffeeScript language. For those of you unaware, CoffeeScript is a language which is syntactically similar to Python but compiles into Javascript. It’s not a nasty GWT-esque compilation, but a relatively clean one (barring the underscores that result when you try doing comprehensions and the really cool stuff). I’ve always meant to write stuff in CoffeeScript, as it has quite a few awesome features. Most importantly is probably the ability to declare a function with two characters (->) rather than a massive “function(){}” and the array comprehensions.

Compiled (or should I say Transpiled?) languages have odd a few annoying properties, especially with debugging. The biggest issue was probably setting up everything: running a script which uses inotifywait to automatically compile your CoffeeScript once you hit “Save” on your editor of choice (gedit just because it works and comes with Ubuntu). Then when errors happen, your line numbers don’t match up and that’s also annoying.

The Facebook API is actually pretty good. My app reads Facebook schedules from your friends’ statuses. It’s not quite as easy as it should be. I could search the user’s news feed and that would be trivial except that it only gives me a subset of the statues that I want to be able to process. When using FQL (which I ended up pronouncing Feequel which sounds a bit like Fecal because it’s a SQL derivative, even though you’re not supposed to pronounce it “sequel”) it would only return/search the most recent status. I ended up doing a FQL request for each and every friend that the current user has, which is a on average a pretty big number. Fortunately it doesn’t seem like Faecbook has any API limits. Awesome.

For the longest time I was confused because my app inexplicably only worked for me. It turns out that my queries returned blank results for everyone else because I didn’t request the right permissions. That’s terrible. Absolutely terrible. First of all, the developer shouldn’t be entitled to have those magical privileges that the end users can’t have. It’s insanely confusing. And don’t just silently return no results and make the developer question his own sanity.

But it was a permissions issue - a one line fix in the end.

It’s also quite depressing that nobody’s using it. It’s pretty server intensive at the moment and it’s running on Google App Engine, which has that new pricing which means I should have my free quota expire after something like a meager 100 users. But I haven’t really come close to that. Why? I guess I have little influence over friends.

http://schedule-compare.appspot.com/

Can you tell that Wave is the backend? I think not.

Just to prove that it

Can you tell that the screenshot on the left is actually an embedded wave? Probably not.

First, a little overview on what this is. A lot of people are interested in using Google Wave as a commenting system, however the official embed api at the current state suffers from several shortfalls. It’s tedious to operate a wave-integrated commenting system. As an author, you have to manually create a wave for every post, copy the content over to the wave, get the wave ID, and paste it into your template. Every time you change your post, you have to yet again manually sync the page with wave. Once you’ve gotten the wave into the page, the embedded wave itself has a few issues. You can’t style it using CSS, and you can only change the background color and font. The width can’t adjust to the width of your page after the initial load. There’s no way to get rid of or change the style of the huge blue border around it. It forces it’s own scroll bar which looks nothing like anything else on the page and prevents intuitive navigation of comments using the same scrollbar as the rest of the page. Lastly, there’s no way for anonymous readers to comment on the post.

For each of these issues, the new w2_embed api offers a solution. Instead of using iframes to hack another page onto the current one, this one is built into the page, injecting elements directly onto the page’s DOM. These elements can be styled in nearly any imaginable way through CSS and HTML. The sandboxed and namespaced single-function API allows for multiple waves on one page with less than 10KB of JS overhead (Compare to about 1.4 megabytes). Instead of relying on a special Wave ID, the app uses a centralized and remotely stored database which associates any identifier you provide with functional wave IDs. That means, that instead of using a string like googlewave.com!w+Blsm0RMW_A, you have the URL of your blog, say 2010/06/wave-embed-api. Not only that, but it can automatically pull the title of your wave, the authors, the permalink and tags from the post, automatically create a wave from that with the same tags and title with the content automatically set as the root blip as well as a permalink back to the blog post. When viewed from wave, the user can follow, unfollow, search and find waves using the exact same familiar interface. On your blog, you can make the wave backend entirely seamless and subtle. Since it’s built on top of Anonybot, people can reply blips without having google wave accounts right from your blog.

It’s all part of a new embed API built upon Anony-bot (which itself is built upon microwave, which is built on wave reader, which is built upon the prototype python desktop client… Someone should make a wordpress plugin for this to make this family history even longer).

The last post noted the beginning of this project, and here is part two. The API has been cleaned up a lot. The code has been simplified. Features added and design changed.

The major change is a total decoupling of it and the mainline Anonybot codebase. There are some major implications of the change. In the original prototype of the API, it still had the iconic blip context menu inherited from microwave. That’s gone. So is the participants bar, link to real wave client, blip html, blip content, reply box, and everything. There are now no required or automatically added html/dom elements. Everything is specified through a simple HTML templating engine. Since the code isn’t shared, changes from each individual project now on probably won’t be instantly applied to both. This has advantages and disadvantages. On the plus side, is that changes to anonybot won’t break the embed api. However, bug fixes on one may not propagate to the other. The code can be much smaller since features like search, which was previously hidden, can now be totally removed. Also, since it’s been rewritten from scratch as an embed api, it no longer exposes any extra globals and multiple embeds can be done for each page.

The biggest change is the templating engine. Before, it only worked with the blip design, but now it’s totally integral to every aspect of the interface. Lets walk through the basics of the API, starting with a minimal example.

Here you see a link and a button. The link has been configured to prompt the user for a participant ID to add when clicked, and the button was set to prompt the user for a tag to add when clicked. There are several such actions.

• addparticipant Prompt the suer for a wave address to add to the wave
• reload Reload the current wave
• addtag Prompt the user for a tag to add

• setname Prompt the user to set a username for posting There is also the user list. It comes in two flavors, {{participantlist}} and {{contributorlist}}. The latter only works when inside a blip scope (the stuff inside wavebody). They’re special because they’re actually interactive. When the list is long, it shortens it and creates a link to expand.

• participantlist A list made from wave.participants

• contributorlist Only works when in wavebody, a simple users list made from blip.contributors Now, is finally the stuff that goes in the blip. Like wave it extends some wave api features, so I’ll start with pasting what google has documented.

• blipId contains the ID of blip in which the event occurred.

• childBlipIds contains an array of blip IDs for each of the blip’s children.
• contributors denotes participants who have contributed to the state of this blip.
• creator denotes the participant who created this blip.
• lastModifiedTime denotes the Unix time at which this blip was last modified by any participant.
• content contains the textual content of this blip.
• version contains the version of this blip. Each atomic operation on a blip increases this version number.
• waveId contains the Wave ID associated with this blip.

• waveletId contains the wavelet ID associated with this blip. Note that for waves which contain only one wavelet (that don’t have private conversations, in other words), this wavelet ID is usually of the form conv+root indicating that the wavelet is identical to the conversation root, the root wave. There are a few additional methods which, again are non-standard.

• permalink contains a link to the waveref which points to the specific blip on the real wave client

• html contains a string of the rendered (with formatting and annotations applied) content There are also actions, similar to the global addparticipant, reload and setname actions. They can still be used within wavebody but there are some actions which can only work in the blip template.

• edit Add an edit box immediately below the blip, pre-populated with the text content of the blip

• remove Remove the blip, it will first issue a confirmation prompt to the user

• reply Add a reply box immediately below the blip. If there is already a blip below, the reply box is indented. Below is a little example showing how to use the blip object. Lastly is the JS part. Without it, nothing would ever show up. You need to include the currently 6.5KB JS library from http://anony-bot.appspot.com/assets/embed2.mini.js. After that, you need a separate script tag anywhere after the big hidden super element. It must be outside the super element (unimaginable evils will happen if it isn’t). Inside that script tag needs to be a function call to w2_embed. The first (and only) argument has to be a JS object with at least the element key pointing to a DOM element which contains an element with the class wavebody. There are lots of other config options, and here they are.

• element (Required) a dom element which contains a div with the “wavebody” class

• identifier The identifier for the wave that the server will use to generate a wave ID from, default: location.pathname
• root_title If the identifier has not previously been associated with a wave ID, the wave will be created with the specified title. default: (no title)
• root_content If the identifier has not previously been associated with a wave ID, the wave will be created with the specified content as the root blip. default: (no content)
• participants The participants the wave should be created with if not previously associated with a wave ID, default: [], suggested: [‘public@a.gwave.com’]
• hideroot Boolean whether or not to not render the root blip
• edit A reference to a DOM element which can be templated into the edit box, should contain handlers for the actions {{submit}} and {{cancel}} as well as a textarea with the class “wavetext”
• tags Set of tags to be added to the wave if the identifier has not previously been associated with a wave and must be created.
• api_root The domain containing the server component (proxy and dictionary server). Default: anony-bot.appspot.com
• gadgets Enable native gadgets (does not work)
• render_state Display the user a gadget state
• chronological Render blips chronologically (see Microwave’s “Classic Forum Layout” option)
• json_url Location of JSON implementation to be loaded dynamically in case it’s not native to the browser default: http://anony-bot.appspot.com/assets/json2.mini.js
• waveid Skip the identifier lookup and directly reference a wave by it’s id  

You can see that the function call was saved to a variable named demo. That’s useful because you can use that to reference later to manipulate things. The return of a w2_embed call is a JS object with the following keys.

• config A reference to the initial config object
• reply(text) Create a root-level reply
• reload() Reload the wave. So lets try putting everything together.

Afterthoughts: Just a little thing to add at the end. You can can create a a div with the class waveedit as used above. It’s not strictly necessary, but it lets you customize things a bit more. You also need a CSS style for .wavethread with a padding-left (or margin-left) in order for threads to show up properly. Blips are automatically given a .waveblip class.

That screenshot on the top of the post, if you want to try it out. It’s here.

I’m not making any money off of this, so there’s no reason for me to hide the fallibilities of this solution from anyone. This, like any other solution is not perfect. There is no true wave-style real-time editing, it uses polling. It does however update every 10 seconds if it’s configured to do so using the poll_updates: true configuration option. It can’t ever really do much better, since app engine doesn’t yet support pushing out real time data to attached clients, even though they announced the neat Channel API (still waiting to use it!) at I/O. Either way, that probably won’t help too much as w2_embed uses JSONP for client/server communication across multiple domains, and it’s unlikely that the channel api will allow for that. Also, since there’s no way to fetch a single blip, the user must request the entire wave to check the tiny 32 bit version integer if it’s been incremented, and if it has, then the whole wave has to be fetched again. It’s by no means efficient, and so my server would be very very sad if the polling interval was significantly increased. It’s not very SEO friendly. Though it’s not always that people want the comments to be indexed by search engines, so some people may care less. Accessibility-wise, it shouldn’t be too horrible if the screen reader/browser supports javascript and hugely depends on the HTML being used in conjunction with the script.

As of date, this was my longest blog post ever, and I would be sad if nobody commented (ironically using my non-wave-based commenting system). And I’m not posting my source code unless someone comments, so, an incentive :P

The problem with the implementation of lots of things like VectorEditor, SVG-edit and the Ajax Animator is that something i’m going to randomly call “echo”.

Basically, you might delete something on your client, and right after that, since the result isn’t real time, the app adds the shape back, and it gets deleted again. With multiple users, it gets more confusing, and there’s a random chance that it might not even delete.

In wave2 which is now a really complicated and massively slow library filled with tons of inelegant crap spanning almost 600 lines, the way it’s resolved (somewhat) is by not truly deleting things, but rather replacing it with DELETED/TIMESTAMP.

I was working on a new little Wave State API update. It supports lists in the form of subkeys, and something very much like hierarchy and events on specific sub-nodes. This way you don’t have everything update (someone’s edits on frame 36 doesn’t matter if you’re on frame 42).

Everything is namespaced under a singleton global, wave2. It includes functions like

• listen (execute a callback when something beginning with the prefix is changed)
• ignore (un-listen),
• keys (shortcut for wave.getState().getKeys()),
• subkeys (get keys which begin with a certain prefix, important to the pseudo hierarchy),
• set (submitdelta with first arg as name and second as value),
• get (shortcut for wave.getState().get()
• reset_gadget (a simple way to empty all the data in the store) And since it’s quite short, I’m putting it under public domain at http://antimatter15.com/misc/wave/wave2.js