Node.js – it’s not just for the web

Having been in the Node community for about 2 months now it’s very clear that 99% of users of node are using it to build some sort of web site or framework. This is obviously one of Node’s strong points, due to the stellar HTTP support built into the API.

But I’m not a web guy. I haven’t done much coding for the web in the last 10 years (what I have done is small UIs for our anti-spam systems at work). And back then using Javascript was considered a no-go area, and so prior to getting into Node I hadn’t done any Javascript coding beyond cut-and-paste into some little web pages.

So then I built Haraka, a very capable SMTP server written in Node that uses plugins for all of its functionality. Those who have used Qpsmtpd or Lamson will find this concept familiar. The key idea with these is to be a front-end to an email delivery system such as Postfix or Qmail, and provide easier configuration and hackability in a much easier to code language.

Node’s strong point is network servers. Unfortunately that does mean it’s weak point is command line tools. There’s no “getoptions” parsing provided in the Node API which is a big downer, and furthermore there’s the fact that a number of libraries use console.log() for debug information, which by default on Node goes to stdout, which is bad if you’re piping tools together and you get unexpected output on stdout. There’s been various discussions recently about this on the mailing list. My basic feeling on this is that libraries should do no logging by default, but have a way to turn debugging on, and that should go to stderr (probably via util.debug()). If you find any libraries that don’t do this then patch them.

There’s another problem I’m coming across just now – there’s no fsync exposed in Node. There’s callbacks to write() which get called “when your data is written” according to the docs, but I’m pretty sure that just means it’s in the hardware buffers, not synced to disk. There’s a callback to close() which *might* be enough, but I remain unconvinced. This stuff is important when writing an SMTP server as you can’t return “250 Queued” to the sender until you can guarantee that the data is on the disk.

I guess overall this post was meant as a callout to anyone looking at writing network-y type stuff regardless of whether that’s for the web or not. Node.js is great for writing web stuff, but it’s also pretty damn good at writing non-web stuff too. More importantly it’s really fast for a dynamic language. Current benchmarks seem to show it is second only to Lua (with JIT) (and maybe PyPy, but that only seems faster in microbenchmarks). But nobody seems to write anything major with Lua, apart from using it as a language to embed in some application.


5 thoughts on “Node.js – it’s not just for the web

    • libeio (the async I/O module Node uses) already has an eio_fsync call in it, so there’s no reason it couldn’t be exposed in node. I’m guessing Windows compatibility worries are why it isn’t. Though an async link() is exposed and I’m not even sure that has an equivalent in Windows-land. Maybe I should just patch Node and submit a pull request.

  1. Hello71 says:

    Can’t you use fs.watchFile? Also, write(2) says that “A successful return from write() does not make any guarantee that data has been committed to disk. In fact, on some buggy implementations, it does not even guarantee that space has successfully been reserved for the data. The only way to be sure is to call fsync(2) after you are done writing all your data. “.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s