A while ago someone linked me to this post regarding the performance of Apache James vs Haraka. Of course all benchmarks are a bit fuzzy, because you never know the exact setup, but it seemed clear to me that the biggest flaw in the Haraka part was not adding in “nodes=cpus” in the configuration, forcing the 8 CPU test box to only use one CPU.
So much against my better judgement I installed Java on my Mac (that’s a joke btw – in case any Apache James hackers come and read this), and re-did a similar benchmark, trying to be as fair as possible to each.
My thought was to use a fairly default James install, and add a user, thus having the mail written to disk. In order to be fair with Haraka, the main disk-writing component is the outbound queue, so to make it queue everything to disk, and NOT attempt to deliver it, I just do “echo 1 > config/outbound.disabled”.
Also in order to get similar log levels I set Haraka loglevel to LOGNOTICE, as by default it logs the entire SMTP transaction. At LOGNOTICE levels you get enough logging for statistical purposes, without too much noise. It’s a sane default for most production installs.
The entire process for James was reasonably simple. For 2.3.2 you install the binary, chmod a+x bin/*.sh, and then run bin/run.sh. After that you telnet on port 4555 and add a user. For 3.0.0 you need to use the bin/james-cli.sh script, but it’s just as simple.
Once I verified that emails were being saved by both Haraka and James, I ran smtp-source from the postfix distribution to see which performed best.
This was a simple test. I’m happy to do more. It really does show that Haraka and Node.js offer stunning SMTP mail server performance.