Creating nice looking Slack bot Messages with Hubot

At Ideal Candidate we finally had enough of HipChat. After several days of messages not being sent, which was really affecting our ability to communicate (even though we sit next to each other – chat is a better interface for keeping everyone involved).

We spent a week evaluating Slack, which many of my friends/colleagues around the world had recommended. Despite their operational status boards looking very similar, we had zero issues with slack messages getting delayed, and the people doing the eval preferred the interface.

We had a few things we needed from Slack which we wanted to keep from the HipChat days:

  • Jira Integration (this is what I’ll talk most about here).
  • Posting to channels a few bits of data from our application (customer signup, customer login).
  • Rainforest messages about running tests.
  • A few other “fun” plugins

Fun plugins were easy. I’ll pass on that here for now.

Rainforest support was a supported plugin. Done.

Posting to channels required a bit of custom code, which we had already written for HipChat, but our experience (below) made us improve what we had.

So let’s get to Jira integration with Slack.

Jira Slack Integration

Let’s be honest about this: The default Jira-Slack integration is total shit.

With HipChat we could control what types of events could generate a message to a channel. The Slack built-in Jira integration only sends a message for the following events:

  • An issue is created or re-opened
  • An issue is closed (unless it was previously resolved)
  • An issue is updated to done or resolved
  • An issue is updated to in progress or to do after previously being marked as done

We wanted much more nuance than that:

  • A bug is created on the current sprint
  • An issue changes status
  • An issue goes from “Ready For Test” back to “ToDo”, we want the last comment (i.e. the reason)
  • An issue is marked “Done”, we want the last comment
  • An issue is mentioned in the channel, we want the summary

We found a number of Hubot Jira plugins on npm that we hoped could work, but none really fit the bill, so we ended up writing our own. The main hubot-jira plugin linked above just dumps plain text into the channel, and we knew we could do better.

So we set about seeing how to create good looking messages with Hubot. The thing to understand about Hubot is it is designed for multiple systems (HipChat, Slack, IRC, etc), so we kind of have to be a bit devious to create a nice looking message. Slack supports a way to do this, calling them attachments (which is a stupid name – but all software has historical shit). First we forked the hubot-jira plugin to get issues in a pretty format. The hubot-jira plugin only responds to mentions of tickets (doing an API query based on the regexp /\b(\w+-\d+)\b/), so first step was to format that nicely.

The code in hubot-jira does a plain msg.send, which just sends a plain text message. To get a nicely formatted message we constructed an attachment object:

              attachment =
                title: "[#{key}] #{json.fields.summary}"
                title_link: jiraUrl + "/browse/#{key}"
                fields: [
                    title: "Status"
                    value: json.fields.status.name
                    short: true
                color: "#003366"
                author_icon: "https://developer.atlassian.com/imgs/jira.png"

And then can post it using the a bit of a hack on hubot directly accessing the Slack object:

                channel: msg.envelope.room
                username: msg.robot.name
                attachments: [attachment]

This created a message in Slack that looks like this:


Note: There are other bits of code to add in certain fields there. See the Slack Attachment docs for more info.

Finally we adding in webhook support, which allowed us to support all sorts of status transitions we wanted:

Screen Shot 2015-11-19 at 21.16.09


We added various other things to our Slack account later, such as posting when new customers sign up, using similar code, but in all it has been a good transition. We haven’t had any times with Slack yet where messages don’t make it through. We hope that continues.



Announcing Haraka v2.7.0

After several months of frenzied development work I’m extremely proud to announce Haraka v2.7.0.

This release consists of 455 commits, 232 files changed, and contributions from 15 people. We also added a new core committer to the team: Josef Fröhle – welcome to the team! Josef has contributed significantly to this release and we look forward to his contributions in the future.

One other change – we moved the repository on github from baudehlo/Haraka to haraka/Haraka – please update your links – thought github will issue redirects on any older links.

Full list of changes in this release:

New Features

  • SPF bounce check
  • rspamd plugin (@fatalbanana)
  • watch plugin
  • limit plugin (connection concurrency, errors, unrecognized commands)
  • plugins can now be npm packages (see also #946)
  • built-in HTTP server (Express backed)
  • ESETS AV plugin
  • DCC plugin (incomplete)
  • Add LOGIN support to XCLIENT
  • backscatterer plugin
  • full IPv4 & IPv6 compatibility inbound #1120, #1123, #1154 (@Dexus)
  • Early talker #1075 (@smfreegard, @msimerson)
  • permit loading of plugins in node_modules #1056 (@msimerson)


  • Fix anti_spoof by use config #1171
  • Add license clause #1170
  • package.json dependencies and travis update #1147, #1168 (@Dexus)
  • logging: remove node-syslog and strong-fork-syslog with modern-syslog #1145 (@Dexus)
  • aliases: support for email, user and host aliases #1149 (@Dexus)
  • add docs for use private key with TLS #1130 (@Dexus)
  • outbound: ENOENT on dotfile – compatibility for windows #1129 (@Dexus)
  • plugin/attachment: block more attachment file types #1191 (@Dexus)
  • remove double functions #1126 (@Dexus)
  • Outbound Bounce messages according to RFC3464 #1189 (@hatsebutz)
  • toobusy: only run checks if toobusy.js installed and loads
  • HAProxy: set local_ip, local_port and remote_port
  • save auth pass/fail/user to result_store
  • ini files no longer require values (useful for storing lists)
  • connection: add MAIL and RCPT to results
  • results_store: enable ’emit’ feature for .push()
  • add support for custom Outbound Received header value (@zombified)
  • save smtp_forward result to result_store
  • auth_base: permit a return message (@DarkSorrow)
  • add DSN.create() and RFC 4954 support
  • enhanced pipelining support
  • added config/access.domains with some tips (@EyePulp)
  • Add SSL detection over plain-text socket
  • earlytalker: store results
  • bounce: make it safe to check non_local_msgid
  • AVG: store results, added defer options
  • tls: change createCredentials to tls.createSecureContext (@DarkSorrow)
  • update dependency versions (esp async 0.2.9 -> 1.0.0)
  • ASN docs: add FTP download note for routeviews
  • karma: removed concurrency limits (see limit plugin) and penalty feature
  • added utils.elapsed()
  • deny message includes hostname
  • Add Fisher-Yates shuffle to randomize lookup order in data.uribl
  • change default message size limit to 25mb
  • auth_base: save auth results
  • upgrade toobusy plugin to toobusy-js (@alexkavon)
  • configfile: permit / char in ini keys
  • added utils.node_min()
  • added result_store.get_all()
  • updated ubuntu upstart script
  • plugin/rate_limit: return in no custom default is set 0 = unlimited #1186, #1185
  • Outbound.send_email: added dot-stuffing #1176, #1165 (@hatsebutz)
  • make sure server object is availabe to plugins loaded from node_modules #1162 (@bmonty)
  • Net_utils.get_ips_by_host #1160 (@msimerson)
  • fcrdns: don’t log error for ENODATA #1140 (@msimerson)
  • improve MUA detection #1137 (@msimerson)
  • tls: tmp disable for hosts that fail STARTTLS #1136 (@msimerson)
  • karma: skip deny on outbound hooks #1100 (@msimerson)
  • Store HAProxy IP in connection object #1097 (@smfreegard)
  • Remove UUID from queued message #1092 (@smfreegard)

Bug Fixes

  • fix windows build and test failures #1076 (@msimerson)
  • Fix plugin ordering #1081 (@smfreegard)
  • Fix distance reporting to X-Haraka-GeoIP for geoip-lite #1086 (@smfreegard)
  • uribl: prevent calling next() more than 1x #1138 (@msimerson)
  • Fix so constants are imported when plugin is loaded from node_modules. #1133 (@bmonty)
  • Include STMP-code in bounce-reason string for upstream 5XX responses #1117 (@hatsebutz)
  • TLS fixes: add timed_out flag and karma should not run deny hook on it. #1109 (@smfreegard)
  • Fix port to number instead of string for HAProxy #1108 (@DarkSorrow)
  • Plugin dcc: fixed syntax error #1164 (@hatsebutz)
  • config: fix flat files if \r\n lines #1187 (@Dexus)
  • corrected hook_rcpt log code hook_rcpt_ok returns CONT
  • fix crash bug when loglevel = LOGDEBUG
  • corrected pathname in rcpt.ldap plugin (@abhas)
  • added helo.checks boolean for proto_mismatch
  • make rate_limit redis keys always expire @celesteking
  • dkim_sign: Buffer.concat expects an array of buffers
  • transaction: check discard_data before adding line end (@DarkSorrow)
  • fix 8-bit msg not displayed properly in gmail
  • fcrdns: always init results
  • TLS timer on error
  • dkim_verify: fixed timeout issue
  • smtp_[proxy|forward]: correct authentication example
  • Fork child workers after init_master hook
  • connection: return 450/550 for plugin DENY* (was 452/552)
  • spamassassin: don’t call next() when transaction gone
  • outbound: fix crash when sending bounce mail
  • auth_base: fix bad protocol in auth_base.js #1121 (@Dexus)
  • outbound: Fix HELO/rDNS issue while using multiple outbound ip #1128 (@Dexus)
  • connection: Fix bug when client disconnect after sending data #1193
  • Fix connect.geoip bug #1144 (@smfreegard)
  • Fix tiny bug in messagesniffer #1198 (@smfreegard)

When upgrading, please ensure to copy over the new public suffix lists from the installation directory.


SublimeGit or GitGutter stopped working? Here’s why.

This week my dev productivity dropped for a day. And I hated it.

Two of my favourite plugins for Sublime Text are GitGutter and SublimeGit. The former lets me see what I’ve changed in the current branch easily, and the latter lets me easily check in code, run “blame”, etc.

They suddenly stopped working. I had no idea why. GitGutter stopped showing anything, and SublimeGit told me I wasn’t even in a folder managed by git.

Yet command line git worked just fine. Infuriating.

I re-installed both. No joy.

I re-installed SublimeText. No joy.

I downgraded from Sublime Text 3 (beta) to v2. No joy.

I was seriously tearing my hair out.

I finally came across the solution and reason purely by accident. I had to upgrade a module that required a compiler and it failed because I had a new version of xcode and the command line tools – they needed me to accept the license via the command:

sudo xcodebuild -license

Once run, that fixed GitGutter and SublimeGit.

The reason? I had git from homebrew installed. The plugins in Sublime Text used the OS X version of git instead, which wouldn’t run without accepting the license.

I’m not sure this is entirely OK. Should I have to accept a license for non-Apple software when re-installing the command line tools? It seems wrong to me. Thoughts?


The Brain Is Not Setup for the Netflix Model

I love Netflix. For my $9 a month I get access to great content, without ads, whenever I want it. I’m happy to admit that watching TV isn’t the greatest thing to do with my time, but frankly screw you – I enjoy it. A bit of mindless entertainment after a day of thinking about Angular Controllers and SQL calls is a nice break for me.

But I’m not convinced Netflix’s model for new TV shows is right.

Netflix is now producing some truly fantastic new content, only available to subscribers (modulo piracy):

  • Orange is the New Black
  • House of Cards
  • Narcos
  • Daredevil
  • and many more

They release it under the same model they release all their TV content: All episodes available at once.

I’ve discussed with coworkers and friends the first problem with this: It prevents the watercooler discussions. It’s impossible to know what episode someone is on without asking them, and so assuming you’re a polite person you don’t want to introduce spoilers into the discussion, so you just don’t talk about it until everyone is done the entire season. But regardless, that’s not what I want to discuss.

What I think is interesting, is if you ask a coworker what their favourite episode or favourite scene in Game of Thrones was, it is easy for them to answer (assuming they watched it). If you ask the same coworker what their favourite episode/scene was in Narcos (which was an amazing series, I highly recommend it), it’s a much more difficult question to answer.

I wonder if this is related to the fact that the brain creates long term memories in very strange ways, and because binge watching a TV show creates so many short term memories muddled into one, we can’t generally remember specific episodes or scenes very well – it becomes one big muddle.

I still hope that some day Netflix experiments with releasing a “Netflix Original” the old-school way. Just to see if people prefer it on some level.


The Problem With Ahmed’s Clock

Hopefully most readers of this blog will know of the case of Ahmed Mohamed who brought a home-made clock to school, and was taken into custody and interrogated for the appearance of this electronic device.

This has caused uproar in the geek community, and even resulted in Cmdr Hadfield tweeting an invite to a talk in Toronto, and even an invite from President Obama to visit the whitehouse.

Now, let’s first visit the issue at hand – at first I was outraged at the situation, then I saw the clock, and thought “Well that does kinda look dodgy”. Let’s see the picture the police posted:


I mean, it does look like a bomb you might see in a movie. But then you look closer, and realise that a) the LED display points outwards, and b) it’s actually a pencil case, not a metal suitcase:


This softened my feelings significantly. I mean, it’s a freaking pencil case, not a metal suitcase. I’m pretty sure I had 25 pencil cases in school that I hacked up into all kinds of messed up things for other uses. Sorry, Mum.

But it’s all beside the point. The problem comes down to one word alone:


I don’t really know if it’s since 9/11, or if that’s just when my awareness of this change started to peak, but there’s a huge problem in our feelings that needs to be addressed.

Adults in the western world now fear everything. And it has to fucking stop.

We coddle children, and it’s stupid. When I was 7 years old I was riding my bike to school, about a mile away. Alone. Nothing ever happened to me. By 11 I was cycling to school 4 miles away. I thank my parents for that freedom so much – they absolutely did the right thing letting me do that.

Nothing happens to children today, either. We hear about extremely rare cases of child abduction (which are almost never conducted by strangers) far more often than we did in years ago due to 24 hour news channels and the internet. Crime continues to decline year on year, and yet our fear of it continues to rise.


The fear needs to stop. It’s just so illogical and dumb.

Ahmed is not alone

Ahmed is far from alone in the U.S. Many have made this a race issue, but while it may have had some racial aspects due to the “bomb” issue, it is really a problem with how the people are afraid, and pass far too many issues off to law enforcement these days. Hundreds of children in North America are now taken into custody by police every year.

I suspect, if I were in school today, I would also be taken into custody. Hell, I remember blowing up capacitors (after being demonstrated how to do it by our “cool” electronics teacher) in the science lab, and I suspect that today would be an offence.

Adults everywhere need to stop, and take stock of what they have done to themselves. These are kids – they are going to push boundaries, play with things, explore, hurt themselves, challenge your rules, break stupid laws, and scare you half to death. But they are kids. They are our future. Let them explore!

Think about how you played as you grew up – you did probably much worse. And here you are.


The Sales Process is Hard

I was recently involved in a sales call at Ideal Candidate with a company run by a couple of old friends of mine. While I was comfortable asking their help to setup the initial call, I am not the kind of person to abuse that relationship to push a sale – I’d much rather their company see the value proposition in our product.

Unfortunately on the sales call I probably talked too much – I am after all, a technologist. I am hugely enthusiastic about our technology and our science. But that doesn’t explain the actual value our our product.

I’ve been thinking about this since the call. We are basically a recruiter. But we really have two major differences:

  1. We’re massively cheaper. For less than 1/5 of the price of a recruiter for a single hire, you can hire as many people as you need, albeit you have to pay us every month (*).
  2. Rather than scanning resumes/CVs for a match, we use proven science in the field of sales to find people who will actually succeed in your company, from a database of salespeople we have already built, and who have already taken our personality assessment (used in the matching process). This is really hard to explain in either a blog post or a 30 minute call, but there is plenty of well proven research out there showing this science works.

Unfortunately I focussed on the science behind point 2, and missed telling the person on the call about value – the fact that we provide the candidates to them, and these are candidates which are already scientifically assessed to perform well in the department being hired for.

What I’m astonished by at Ideal Candidate is the amount of work we can do for a company for this price. We provide pre-matched candidates. We phone screen for you. We set up and book interviews for you. If I had this service for this price (about double the cost of a job post on LinkedIn), for hiring developers, I would love it. Unfortunately the science works differently for developers (but some day we’ll probably work on it).

But damn, it’s hard to persuade people that the technology is sound, and we’re actually going to save you money. Our sales guys have to be really persistent, and I’m very thankful that they are. Good luck to them!

* – if you’re hiring in sales, and the salesperson performs, then the more salespeople you hire, the more money your company makes, so even if this seems a little odd for hiring in a development (which most of my followers are), in sales it makes a lot of sense for sales.


Announcing Haraka 2.6.1

The Haraka development team is proud to announce the release of v2.6.1 of the popular mail server, Haraka.

This release adds some small new features, but is mostly a bug fix release for the 2.6 series. In particular a bug in mail_from.is_resolvable was fixed which can cause Haraka to crash.

Other changes in this release:

  • added sedation timers for config file re-reading
  • Add AUTH support to outbound
  • tests/spf: quiet excessive DEBUG noise
  • allow domains with underscore
  • correct name of domains config file in access
  • Fix SMTP AUTH in smtp_forward/proxy and add docs
  • Fix opts not being passed to HMailItem _bounce function
  • log.syslog will try strong-fork-syslog (for node 0.12 compat)
  • improvements to Plugin docs
  • rename net_utils.is_rfc1918 -> is_private_ip
    • IPv6 compat
    • test coverage
    • add IPv6 unique local fc00::/7
  • pre-populated config/plugins
  • added utils.extend, copies props onto objects

Upgrading for most users should be as simple as running “npm install -g Haraka”, and copying over the files “extra-tlds”, “top-level-tlds”, “three-level-tlds”, “two-level-tlds” and “public-suffix-list”.

Enjoy, and for any issues please use github issues or join us on #haraka on irc.freenode.net