Uncategorized

SockJS, multiple channels, and why I dumped socket.io

At Hubdoc we have a section of our application which requires real-time interactive feedback – when users add billers we tell them what stage we are at in fetching their documents. After hearing how awesome it was I set out to use socket.io, and indeed it worked great in development on chrome with a nicely open network. The API was also extremely easy to work with.

But we would get sporadic bug reports of it just hanging, and we had no idea why, no errors in our logs, and no way to find out exactly what was going on.

For a while I tried to evaluate engine.io and the newer 1.0 version of socket.io. Version 1.0 (vs 0.9 we had in production) uses a different method of establishing communication channels – polling first, then upgrading to websockets/flashsockets if it can. I found very little documentation and some confusing reports about hangs and problems getting it to work, so I did further research. It turned out that the Meteor.js project had switched to SockJS for similar reasons, so I decided to bite the bullet and make the switch.

One thing that I found I had to do was figure out a way to establish multiple different communication channels over the same sockjs connection. There is a module called websocket-multiplex which attempts to provide this, but unfortunately it requires you establish the channels on the server ahead of time. This wasn’t an option – I just wanted arbitrary multiplexed channels established whenever I wanted them.

The solution was to communicate using a uuid with each message, and for the server to keep track of those uuids for the lifetime of the connection using a simple closure.

The next problem was to provide an EventEmitter type interface for these channels. Creating an EventEmitter in Node is really easy, so the code for that was just:

function SockJSEmitter (conn, uuid) {
    this.conn = conn;
    this.uuid = uuid;
}

var EventEmitter = require('events').EventEmitter;
util.inherits(SockJSEmitter, EventEmitter);

// "emit" an event over the SockJS connection
SockJSEmitter.prototype.emit = function (event, data) {
    if (event === 'newListener') return;
    this.conn.write(JSON.stringify({event: event, uuid: this.uuid, data: data}));
}

// call this when we receive an event from the remote end
SockJSEmitter.prototype.emit_event = function (event, data) {
    EventEmitter.prototype.emit.call(this, event, data);
}

With that module, I can route data to the right SockJSEmitter with the following code:

sockjs_server.on('connection', function (conn) {
    var sockjs_uuid_map = {};
    conn.on('data', function (message) {
        var msg = JSON.parse(message);
        if (!(msg.event && msg.uuid)) throw "Invalid message format: " + message;

        // StartRobot is the first message we get from the browser
        // -- we use it to setup the SockJSEmitter and associate with a uuid
        if (msg.event === 'StartRobot') {
            var sockjs_emitter = new SockJSEmitter(conn, msg.uuid);
            var status = run_robot(msg.data, sockjs_emitter);
            if (status.status === "Started") {
                sockjs_uuid_map[msg.uuid] = sockjs_emitter;
            }
            sockjs_emitter.emit("start_status", status);
        }
        else {
            // For every other message we route to the right
            // sockjs_emitter and emit_event on it
            var sockjs_emitter = sockjs_uuid_map[msg.uuid];
            sockjs_emitter.emit_event(msg.event, msg.data);
        }
    });
})

Now we need to implement the flip side for the client – always sending JSON containing at least {event: e, uuid: u} when sending messages back and forth. This looks a little more complex as it deals with waiting for the connection to be established, and maintaining a queue of clients to be sent once the SockJS connection is established. This just uses a status variable with three states: “disconnected”, “connecting” and “connected”.

var uuid_map = {};
var sockjs = null;
var sockjs_status = 'disconnected';
var pending = [];
Robot.prototype.init = function () {
  uuid_map[self.uuid] = self;
  if (sockjs_status === 'disconnected') {
    sockjs_status = 'connecting';

    sockjs = new SockJS(app.robotServer + "/_sockjs");
    sockjs.onmessage = function (e) {
      if (e.type != "message") return;
      var msg = JSON.parse(e.data);
      var self = uuid_map[msg.uuid];
      if (!self) throw "No such uuid";
      var method = self['event_' + msg.event];
      if (!method) throw "No such event: " + msg.event;
      method.call(self, msg.data);
    }
    sockjs.onopen = function () {
      sockjs_status = 'connected';
      for (var i=0; i<pending.length; i++) {
        pending[i].run();
      }
      pending = [];
    }
    sockjs.onclose = function () {
      sockjs_status = 'disconnected';
    }
    pending.push(self);
  }
  else if (sockjs_status === 'connecting') {
    pending.push(self);
  }
  else if (sockjs_status === 'connected') {
    self.run();
  }
}

Robot.prototype.run = function () {
  this.sockjs_send("StartRobot", this.robot_data);
}

Robot.prototype.sockjs_send = function (event, data) {
  sockjs.send(JSON.stringify({event: event, uuid: this.uuid, data: data});  
}

// Now implement Robot.prototype.event_* = function (data)
// -- these will catch your remote events specific to this instance of Robot.

So far I’m happy with the transition – XHR polling is working well from IE, and WebSockets are working from browsers that support them, and hopefully now we’ll get less bug reports about mysterious things just not working. Will keep you posted.

Standard

8 thoughts on “SockJS, multiple channels, and why I dumped socket.io

  1. Pingback: Web terminal built in sockjs, BDD in Nodejs with websockets

  2. Hey Matt,

    Interesting article.. Any updates on how SockJS has worked out for you? I’m trying to choose between SockJS and socket.io at the moment.

    Thanks,
    Ian

    • It has worked out perfectly. We’ve had zero problems or invisible errors since converting. Honestly I can’t recommend socket.io at all, and tell everyone to use SockJS instead.

      • Remmurd says:

        What about the usability and learnability? I am pretty new to advanced javascript and plan to implent a real-time notification system, socket.io is well documented and has a huge community. So would you recommend using sock.js for a advanced js newbe too ? ;)

      • I think it’s worth checking out Socket.IO now they have changed the connection model. It may be much more stable now than this article points out.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s