In Depth Tutorial on Writing a Slackbot

(This is a repost of an article I wrote on Monsoon’s blog prior to Capital One acquiring us.)

At Monsoon (my employer), we are avid users of Slack. It’s a great collaboration tool in addition to adding a new social dimension to the office. We just crossed 500k messages sent over the platform and we’ve only been on it for a few months!

We recently held a 4-hour slackathon at Monsoon where people were tasked with writing the most useful Slack bot they could think up. The winner was a secret polling script that we use to vote on controversial topics such as what to name our teams or who the coolest person in the office is. We chose to do our event using Hubot, a popular open source bot framework written by Github. Half of the participants were mobile developers, so JavaScript, the hubot scripting language, was foreign to them. We spent a few minutes prior to the event training everybody on how to write scripts to ensure an even playing field.

I’d like to share our Slack tutorial with the rest of the community.

Setting it up takes 5 minutes

To get started with the tutorial, you’ll need to setup your machine by installing Hubot. The most important step is the first two: clone the repo and then run:

$ npm install

Once you’ve done that, you’re ready to write scripts! In the scripts folder you’ll find a file called slackbot-examples.coffee. This example script is a more feature-rich set of examples than the default that comes with Hubot. We’ll go over these examples in greater detail below.

The first thing to notice is that this is a “coffee” file. CoffeeScript is a language that compiles into JavaScript. It is popular with some communities due to its terseness compared to JavaScript. If you don’t like it, you’re welcome to write scripts in JavaScript by naming your file .js instead of .coffee.

Talking to the Bot – robot.respond

In the next step, we start working with a bot.  All bot behaviors start with a listener. The first one we’ll review listens for messages directed at the bot.

When you mention the bot directly in a room via @botname, followed by the command, the bot will execute the above block of code. In this case, it will look for the text “@botname sleep it off.” This behavior will also trigger if you privately message the bot with the text “sleep it off.”

Either of these will trigger the bot to run the command msg.send ‘zzz…’

Making the Bot Say Stuff – msg.send

Now that the bot is listening for messages directed at it, let’s see if we can get it to talk back.  The msg.send command tells the bot to send out a message to the current chat room (that told it to “sleep it off”). In this case, the bot will say, “zzz…” publicly. msg.send always replies in the same channel it detects the original message.

It Sees Everything – robot.hear

We’ve already programmed a bot to respond to messages directed at it, but you can program a bot to listen to conversation anywhere in the office, and respond to a specific word or phrase.  The second type of listener is robot.hear, a blanket listener that reacts to a phrase regardless of who it is directed at.

In this example, we are using the regular expression looking for the word (and not just a phrase containing) “up.” If anybody says “up,” this block will trigger. It will also trigger if you direct “up” at the bot; both of these would trigger the behavior:

Michi: up
Michi: @botname up

It Can Remember – robot.brain

Bots can also store information for retrieval later.  In the “up” example above, the robot initializes and/or increments a value which we save as everything_uppity_count. It does nothing more. In this case, a user can say, “up” all they want and nothing will seemingly happen while the counter increases. This is done through the “brain,” which is a simple key-value store.

Note that the “brain” uses Redis to store its contents. This way, if the bot restarts, the data is preserved. If Redis is not running, the bot will still function, but all data is lost next time the bot restarts.

In the second example of robot.hear, the bot retrieves the current value of everything_uppity_count and displays it via msg.send. As a reminder, this means the robot would just reply in the chat room that it heard the “are we up?” statement.

Calling People Out – msg.reply

Bots can add tailored prefixes to their responses. You can use the command msg.reply for this. msg.reply probably does *not* do what you think it does. Rather, it acts similarly to msg.send except that it prefixes whoever authored the original message it is replying to.

In the above example, the script will simply reply to the original sender as illustrated in the following theoretical exchange:

Michi: What’s up!
Bot: @Michi What’s up!

Note that the reply is in the channel where you sent the original message. If this was a private chat room between you and the bot, the reply would have appeared there.

Replying to Private Messages – Advanced msg.send

Handling private messages is a little more tricky. This is because Hubot doesn’t treat private messages differently from any other types of message. Instead, you have to examine the room that the message is sent in.

In order to reply to a private message, you need to check if the room shares the name with the bot. If the channel names are the same, it is implied that the channel is a private channel. We’ve provided the helper methods to accomplish this:

Starting New Private Conversations – robot.messageRoom

Sending unsolicited private messages is more straightforward. Just remember that private messages are just another room named after a user. To accomplish this, simply tell the bot to message a room:

Notice that an error can be thrown if the channel is invalid. In that case, it’s a good idea to catch the error so that the bot does not crash.

More Examples!

The example script file also includes an example of how to run a web service in the bot to listen for external data sources (such as a github webhook) and how to trigger/watch custom events. Take a look – and when you’ve finished, hopefully you’ll have as much fun designing bots and expanding your office interactions and conversations as we have had here at Monsoon!

Adding BasicAuth to the Kue Dashboard in an Express App

Do you use Kue and want to put it in its own folder that is password protected using BasicAuth or some other type of mechanism?

I know I’m not alone on this problem, yet nobody seems to have posted the concise answer. Here’s the solution.

Explanation

Modules are their own little world

In Express (Node), there’s a couple of important points to know:

  1. Route declaration order matters
  2. Auth strategies are attached globally or to individual routes
  3. Modules tend to come with their own routes (Kue certainly does)

If we want to attach an auth strategy to Kue, you’d naturally want to attach an auth strategy to its entire scope. First, I tried locking down the entire module:

module.exports = function (app, config, passport) {
  var kue = require("kue");
  var auth = express.basicAuth(function(user, pass, callback) {
    var result = (user === 'username' && pass === 'password');
    callback(null /* error */, result);
  });
  // any kue related settings can go here
  kue.app.set('title', 'Jobs');
  // create a wrapper to add auth on since without it we can't globally wrap kue's paths
  kue.app.use(auth)
  // bind the subApp to the desired path
  app.use('/secret_location/kue', kue.app)
};

Global auth can’t hook onto Kue

This fails to prompt for authentication. Why? Honestly, somebody better at Express can explain. I believe it has something to do with the way Kue is written since app settings are first-come first-serve.

Sub app it

In the gist above, you can see I create a sub app by passing in Express and creating an instance inside the file. I then apply the global auth settings (“subApp.use(auth)”) to this app before wrapping it. See the relevant code here:

  var subApp = express()
  // add authentication to the entire sub app
  subApp.use(auth)
  // re-add kue.app (but dont put it in its own folder)
  subApp.use('', kue.app)
  // bind the subApp to the desired path
  app.use('/secret_location/kue', subApp)

Notice that I add the kue.app using a blank string as a first argument. That tells Express to put this sub-app in the same folder path as the app. Then, I bind that app to “/secret_location/kue.”

This works.

Adding a Centralized Event Dispatcher on Backbone.js

This article contains my solution to adding a simple global event dispatcher to Backbone. It should also help noobies (myself included) understand how Backbone events work (which has one big gotcha).

Sharing Events = Global Dispatcher

Today, I came across a fairly mundane – and probably common – problem: I have two views that need to talk to each other. An example for this would be when you have a status bar in one corner that gets updated when a user completes an action in another area. It turns out that Backbone (as far as I can tell) does not support this use case very well out of the box. Well, it does: it expects you to build that yourself.

Backbone supports a model where you publish and subscribe to things happening in your views/models (commonly known as an “event dispatcher“). For example, you can make your application do stuff when a user clicks on something or if a model attribute changes. The native solution localizes these events (probably a good thing) to the model/view that you’re working with. This works for simple stuff, but when you need it in other parts of your application, things break down. The short answer is that you need to build a global event dispatcher A.K.A. global event “hub.” Everything publishes (“trigger”) and listens (“bind”) to events from this object.

This is a common pattern. In fact, I found two solid articles on this topic. The first explains in detail why you need an event dispatcher. The second shows you how to make the dispatcher natively accessible by all of your Backbone classes.

Event Dispatcher Gotcha: Native Events

However, I felt both solutions weren’t quite there. The first forces you to pass around an Event object everywhere you need it. This is nice from a decoupling standpoint, but highly error prone. The second solution is nice, but makes the caller oblivious to the global event namespace they are triggering/binding to. This is dangerous because Backbone has native generic events that are fired when certain things happen. For example, when you edit a Model, it automatically fires a “change” event into its event dispatcher (which bubbles up to its Collection too). This means if that dispatcher was global, every object would see a “change” firing every time every Model changed. Not good.

Goals

I came up with my own solution that accomplishes three main goals:

  1. Retain native Backbone event triggering/binding
  2. Allow the developer to trigger/bind to global events
  3. Not require the developer to pass things around

Solution Code

(If you can’t read CoffeeScript, just use this converter to convert it back to JavaScript)

The following (very simple) code modifies your Backbone definition to attach the global dispatcher to all objects. The dispatcher is easily accessible from any Model, Route, View, or  Collection through the global_dispatcher parameter. I thought about having it trigger events to the local dispatcher as well, but I decided that keeping them fully separate was in everybody’s best interest (to prevent events from accidentally colliding):

Example Code

The next section is a simple class I wrote that will demonstrate how these events work. I am defining a simple Collection and a Model. Note that I named the global and local events the SAME. This was to help demonstrate that the namespaces are in fact completely separate (as in, just because they are named the same doesn’t mean a local event will ever trigger a global event with the same name).

The following snippet illustrates triggering the events attached to the Collection. Note that we add a Model into this collection (which has no impact on the output). The trigger_stuff method triggers both a local and global event (in that order). The output shows that the events were picked up in the order fired (local, then global). Note that the Collection also listens to the “model_custom_action” event, which coincides with an event the Model triggers. This is very important.

The order of operations looks like this:

  1. Fire a local event
  2. The Collection picks it up
  3. END OF FIRST EVENT
  4. Fire global event
  5. The Collection’s globally attached event handler picks it up
  6. END OF SECOND EVENT

The next snippet shows what happens when you fire an event on a Model inside a Collection. It is also why I decided to keep things separate. This is a little hairy, so pay special attention.

The order of operations looks like this:

  1. Fire a local event
  2. The Model picks it up
  3. The Collection picks it up since all events in children bubble up
  4. END OF FIRST EVENT
  5. Fire global event
  6. The Collection’s globally attached event handler picks it up
  7. The Model’s globally attached event handler picks it up
  8. END OF SECOND EVENT

Notice that in step #6, the Collection’s binding fires BEFORE the Model’s. This is key.

Step #3 fires because there is a LOCAL binding to the “model_custom_action” event in the Collection. The local binding is reacting to an event triggered in the child Model’s local event dispatcher. In other words, any event triggered from the local event dispatcher in a Model will bubble up to the Collection’s local event dispatcher and be otherwise indistinguishable from events triggered directly from that Collection.

Step #6, however, is different. That event did NOT originate from the child model’s local event dispatcher. Instead, it is reacting to the global bindings, which happen to share the same name as the event we saw earlier in step #3. Because it didn’t bubble up, the events are being processed in the order they were bound (via the bind() function). The Collection was defined before the Model, thus, the Collection’s global binding fires first.

In this last snippet, we demonstrate how Models behave when not inside a Collection.

  1. Fire a local event
  2. The Model picks it up
  3. END OF FIRST EVENT
  4. Fire global event
  5. The Model’s globally attached event handler picks it up
  6. END OF SECOND EVENT

This is very straight forward if you managed to follow the last example. To get additional clarity, you may want to try renaming the events in the above class and re-run the examples.

By having the global dispatcher separate, you can consciously decide when an event should be “public,” as well as not clobber any existing Backbone functionality. Backbone is still really young (pre 1.0!), so I wanted to avoid using any solution that might break if they changed the internals. Also, completely preserving the behavior of event bubbling for Model-Collections is important to future proof my hack.

I hope this is useful for all of your Google-visitors!

Q: Hiding JS Files? A: Impossible

In my popular post about hiding your Word Press folder, a reader asked:

Hi Michi, can you help me with this, in the head section i wrote this:

<script src="/style/js/somescripts3.js” type=”text/javascript” charset=”utf-8″>

and when we go to the webpage then right click, it will show:

<script src="content/themes/exampletheme/exampletheme/style/js/somescripts3.js”
  type=”text/javascript” charset=”utf-8″>

can you teach me or show me how to do that, any help highly appreciated, And im so sorry if my english not good.

This question was complicated enough where I thought a new post might make sense.

For clarification, I believe he is asking if it’s possible to put one thing in the source and another that the browser sees. This is impossible. Anything that the browser can see, the user can see. There is no way to “show” something different in the source of an HTML file versus what the browser sees (except through obfuscation); however, you can forward things along behind the scenes. You want to create an htaccess rule that will redirect your requests.

RewriteRule /path/to/thejsfileyouwanttoshow\.js /path/to/real/js/file.js [L]

Let me reiterate that you *cannot* hide the content of the JS file. However, you *can* hide the true folder structure of the web server. If you desire to hide your JS contents, the better solution is a JS minifier.

Alternatively, if your goal is to somehow make it harder for somebody to steal your code and you don’t want ot use a JS minifier, you could write the JavaScript tag dynamically using another piece of JavaScript. However, ultimately, that level of weak obfuscation won’t protect you from anything since Firebug will quickly expose what’s really going on.

I hope that answers your question.

Rainbow Google and Annoying Google

I launched two more Google parodies: Rainbow Google and Annoying Google!

Rainbow Google is just a pretty demonstration of dynamic stylesheet modification. It was actually extremely hard to code — it took me about 8 hours of JavaScript hell. On another day, I’ll go over how I did it. This site adds a colorful spray of colors to the text on the page (see screenshot).

rainbowgoogle.com
RainbowGoogle.com

Annoying Google was about 10 minutes of work since it was just a super simple version of Rainbow Google’s code. 🙂 Search queries and results are jumbled so that their letters are in random capitalized states… LiKe ThiS.

annoyinggoogle.com
AnnoyingGoogle.com

If you have suggestions or ideas, please let me know!

Getting Around Overwriting form.submit()

Since my dear reader Sameer requested it, I’m here making an update. I’ve got a cool JavaScript fix for everybody! I mentioned in a post a long time ago, but JavaScript has this semi-unexpected “feature” where you can accidentally overwrite the submit() function from a form. As in:

<form id=”myform”>
<input name=”submit” value=”submit me” type=”submit” />
</form>
<script>
document.getElementById(‘myform’).submit(); // THIS FAILS – Object not a method
</script>

Apparently, by creating a form element called “submit” you overwrite the native function that exists in every form element in JavaScript. Because it’s native, it also means you can’t just willy-nilly redefine it. And to make things worse, you cannot (at least not in a cross browser manner), successfully re-assign the submit() method because some browsers will disregard any attempt to reassign its value. As in:

<script>
document.getElementById(‘myform’).submit = ‘This gets ignored’;
</script>

Fortunately, there is a fix. This fix requires modifying the actual DOM. Because this tends to be inconsistent across browsers, I’m doing this fix in MooTools (which is my JS library of choice). However, the fix is fairly straight forward and can easily be done with (or without) any JS framework, as you will see. The steps are:

  1. REMOVE the form element in question. This is an absolute requirement to make the solution cross browser compatible. This can be skipped, but it will cause quirks. However, the good news is that we can assume that 99.99% of all form elements named “submit” are due to designers being ignorant — thus, such cases are exclusive to submit buttons. Luckily, these are almost NEVER needed in the server side code and really just act as wall flowers.
  2. Check if step #1 completed successfully
  3. If it did not, create a new Form element and copy its submit function over
  4. Submit

The code looks like this:

<script>
var formObject = document.getElementById(‘myform’);
// Removes the node
formObject.submit.remove();
// Functions don’t have tagName defined

if(‘undefined’ == (typeof formObject.submit.tagName)) {
    // create a form and assign its submit function
    formObject.submit = new Element(‘form’).submit;
}
formObject.submit()
</script>

Let me know if you encounter any problems.

Improving Your JavaScript Load Time

On our production website at work, I noticed that there was considerable lag time when loading the page due to a high number of JavaScript files. For those of you who don’t know, when a JavaScript file is loaded into a page, the rest of the page will hang until that file is completely downloaded. So unlike an image on a page, a slow JavaScript file can completely bog down your page. This is similar to an issue I noticed many months ago with FeedBurner. Each JavaScript file that is pulled requires the full overhead of firing up Apache and serving an HTTP request. This can be slower for you and painful for the web server if you have a high traffic website.

Additionally, once the JavaScript files load, if the code is full of asynchronous snippets (AJAX, event handlers, etc), the pieces can load in the wrong order! This has caused me headaches when unexplainable and random JavaScript errors began popping up (undefined variables and functions that are clearly defined in another file that should have loaded prior). This issue became increasingly common as the number of files being loaded increased. While I admit I don’t understand browser physiology enough to explain why this problem is more common with more files, I concluded there is some correlation that likely is attributed to the rendering order.

So after some thought, I came up with a solution. The goal was simple: decrease the number of web calls and try to make the JavaScript code render in 100% reliable and linear matter. Additionally, the hack would need to be easy to implement and take issues such as caching into account. The solution is a PHP file that looks like this:

/*
 * This file compiles a collection of JS files and then dumps them collectively
 * to the page, thereby reducing overall request overhead
 * Copyright 2007 Michi Kono (www.michikono.com)
 * Feel free to modify this however you want.
 */
header("content-type: application/x-javascript");
foreach(explode(",", $_GET["files"]) as $filename) {
   /*
    * prevent malicious attacks, only allow JS files
    */
   $filename = basename(trim($filename), ".js") . ".js";
   if($filename && file_exists($filename)) {
       $handle = fopen($filename, "r");
       fpassthru($handle);
       /*
        * in case there is no trailing ;
        */
       echo ";";
   }
}

Put this file in your JavaScript folder next to the rest of your JavaScript files. I called mine render.php.

Then, you put this in your HTML:

<script type="text/javascript" src="/javascript/render.php?files=firstfile.js, secondfile.js, thirdfile.js, etc.js"></script>

Ta-da! Faster JavaScript loading for everybody. 🙂 Oh, and the issue of caching? Just put a timestamp on the end of the JavaScript URL string (like, "&<?php echo substr(time(), 0, -2) ?>" — cache changes every 100 seconds)

Other useful ideas: JS code could be compressed during this step, with comments and extra spaces being removed. Because code is being run through PHP, server side macros are now possible, rather than relying on cryptic JavaScript functions (such as for date management or database integration).

IE Redirect Bug with Dynamic Location Hash

I discovered the most obscure bug today in IE. For those of you paying attention, this bug is the reason I haven’t been updating — it ate up all my god-damned time. People who aren’t programmers can stop reading here.

What Happens

The browser is redirected when it shouldn’t be after modifying the URL hash (the stuff after #).

Scope

The bug exists on IE7, possibly 6 (why not, right?).

Steps to Reproduce

Assume you are on page A and want to redirect to page B.

  • Go to page A.
  • From page A, do a header redirect to page B in PHP/ASP/whatever. As in, header(‘location: $pageB’);
  • On page B, using JavaScript, modify the document.location.hash variable.

What Should Happen

The anchor text in the address bar should change. As in, http://www.michiknows.com/#someanchor changes to http://www.michiknows.com/#newanchor. This should happen without the page refreshing.

What Actually Happens

The browser refreshes. @#%!*(&$!

Solution / Fixes

On page A, rather than redirect using headers, use JavaScript:

<script>
// if page A was http://www.michiknows.com
document.location.href = 'http://www.michiknows.com/';
</script>
<a href="http://www.michiknows.com">go to page A</a>

For some dumb reason, this fixes the problem.

Damn you, Microsoft!

The Truth Behind Giving IE7 Standards Compliance Updates

I just realized a funny irony about Microsoft competing with Google. See, Google has some really advanced JavaScript when it comes to its Adsense scripts (look to the left of or below this article). If you’ve never thought about this process, here’s your chance. The JavaScript that makes these ad unit boxes must:

  • Scan the page and send the contents to Google for analysis, get the response, and serve an ad accordingly (AJAX)
  • Build an entire block of HTML from scratch (DOM manipulation – very annoying)
  • Fire after the page loads, to ensure all of the content is in place (event handling)
  • Override any CSS or other page altering scripts to ensure people can’t be tricked into clicking (CSS hacking!)
  • Look and function exactly the same in all browsers

Well, the stuff I listed up there involves some of the most annoying aspects of programming in JavaScript (believe me). AJAX, Event handling, DOM manipulation, and CSS pretty much sum up the four pillars of “the most inconsistent things in JavaScript.” Thanks to IE, these tasks are a pain in the ass, when they really shouldn’t be.

Microsoft has to ensure their ads appear in all browsers, especially Firefox (#2 browser). You can’t sell your services to advertisers if it’s known that your ads break, and possibly misfire, when the wrong browser hits it. That means they’re forced to use a standards compliant implementation to do their ads.

This, of course, means they’re taking their own medicine and finding out what a horrible pain it is to support Firefox 1.5, Firefox 2.0, IE5, IE6, IE7, Opera, Safari, etc. It’s mostly a pain because of IE and its inconsistent and buggy support for the established JavaScript and CSS standards.

Maybe this is why they appear to be supporting some standards in IE7. Or, maybe they’re just building in the ones they use. Either way, this explains a lot. 😛