Home / Blog / growl /

Loris: Autotest for Javascript

November 6th, 2009

I’ve [previously written a number of posts on javascript and autotest](http://monket.net/blog/tag/autotest/). Explaining how to integrate javascript lint, unit tests, and growl with the ruby [Autotest](http://ph7spot.com/articles/getting_started_with_autotest) project.

While this all worked, it felt a little clunky as Autotest doesn’t natively support the idea of running multiple tasks one after the other. Rather than hack at the Autotest codebase, I thought I’d get some ruby experience by rolling my own autotest-style framework. Not great for reuse of code, but a great way for me to learn :)

## Loris

Loris will monitor your project and run [Javascript Lint](http://www.javascriptlint.com/) and [JS Test Driver](http://code.google.com/p/js-test-driver/) whenever a file changes, it will report the results to the command line and using [Growl](http://growl.info/). If required, Loris will automatically start the JS Test Driver server and register your default browser with it.

### Installing

Loris is hosted on [Gemcutter](http://gemcutter.org/), so you need to install their gem if you haven’t already.

sudo gem update --system
sudo gem install gemcutter
gem tumble

Then to install Loris, just run the following:

sudo gem install loris

Loris has no command line options, and no configuration file (at the moment). It looks for configurations files to decide which tasks to run.

### Configuring Javascript Lint

To enable Javascript Lint, create a `jsl.conf` file in the folder where you run Loris. This should be a standard Javascript Lint config file. If you need here is an [example Javascript Lint config file](http://mike.kruckenberg.com/archives/2009/03/configuration-options-for-javascript-lint.html)

You just need to specify which files Javascipt Lint should process. For example:

### Files
# Specify which files to lint
# Use "+recurse" to enable recursion (disabled by default).
# To add a set of files, use "+process FileName", "+process Folder\Path\*.js",
# or "+process Folder\Path\*.htm".
#
+process src/js/*.js
+process tests/js/*.js

If no `jsl.conf` file is found, the Javascript Lint task is silently skipped.

### Configuring JS Test Driver

To enable JS Test Driver, create a `jsTestDriver.conf` file in the folder where you run Loris. This should be a standard [JS Test Driver config file](http://code.google.com/p/js-test-driver/wiki/ConfigurationFile).

This should specify which files JS Test Driver should process, and how it connects to the JS Test Driver server. For example:

server: http://localhost:9876

load:
  - tests/qunit/equiv.js
  - tests/qunit/QUnitAdapter.js
 
  - src/js/*.js
  - tests/js/*.js

If no `jsTestDriver.conf` file is found, the JS Test Driver task is silently skipped.

To make it really simple to run JS Test Driver tests, if the server is set to run on `localhost`, and Loris doesn’t detect one running, it will automatically start one, and register your default browser with it.

This makes it a one step process to get automated tests up and running.

### Running

To run, open a command line window, navigate to the root folder of your project, and run:

loris

Loris will run Javascript Lint, and JS Test Driver tasks (if it finds their configuration files), and will output the results on the command line.

#### Example output

Javascript Lint
success
All files are clean
 
0 error(s), 0 warning(s)
JS Test Driver
success
All tests pass
[PASSED] GreeterTest.testGreet
  [LOG] JsTestDriver Hello World!
[PASSED] GreeterTest.testGoodbye
[PASSED] GreeterTest.testSetName
[PASSED] GreeterTest.testSetNameAndNameParamter
[PASSED] Asserts.test OK true succeeds
[PASSED] Asserts.test Equals succeeds
  [LOG] about to call assertEquals
[PASSED] Asserts.test Same assert succeeds
[PASSED] Lifecycle.test Setup and Teardown are run, and can contain assertions
Total 8 tests (Passed: 8; Fails: 0; Errors: 0) (3.00 ms)
  Firefox 1.9.1.4 MacIntel: Run 8 tests (Passed: 8; Fails: 0; Errors 0) (3.00 ms)

Every time you make a change to a Javascript file, or a configuration file, Loris will automatically re-run Javascript Lint and JS Test Driver. So you can instant feedback on your changes.

Loris will clear the command line when re-running tasks. So the latest run is always at the top of you command line.

Loris will also report a summary of each task using [Growl](http://growl.info/) (if it is installed). This allows you to get quick feedback without needing to refer back to the command line on every change.

### Requirements

JS Test Driver is written in Java, so you will need to have Java installed to run it.

To get Growl notifications, you will need to install either [Growl for OSX](http://growl.info/) or [Growl for Windows](http://www.growlforwindows.com/). Growl for Windows requires the [.NET Framework 2.0+](http://www.microsoft.com/downloads/details.aspx?FamilyID=0856EACB-4362-4B0D-8EDD-AAB15C5E04F5&displaylang=en).

### Caveat

Loris is pretty limited at the moment, I just wired up the basics to get it running for a work project.

It doesn’t have any configuration options at the moment, so you have to follow it’s assumptions for now. I’m happy to add configuration options for any element as required.

Loris only comes with a few tasks (Javascript Lint, JS Test Driver, JSpec, and RSpec), but I hope to allow it have new tasks added via new gems (kind of similar to Autotest).

It comes packaged with a version of Javascript Lint, and JS Test Driver, and will use it’s own versions. It only includes the OSX and Windows versions of Javascript Lint.

If you want to modify the code, just fork the [Loris github project](http://github.com/karl/loris)

In a previous post I detailed changes I made to add Growl for Windows support to Autotest Growl.

These changes have now been integrated back into the autotest-growl gem. Along with some major improvements to the analysis of test results, and the details in the notifications. Thanks to Svoop, the creator of autotest-growl.

This gem isn’t yet available on Rubyforge, but you can grab a copy from my fork on GitHub until it is:

gem install karl-autotest-growl --source http://gems.github.com

This should be a drop in replacement for the previous version of karl-autotest-growl, the only thing you should notice more detail from the growl notifications!

I’ll post again once the official gem is available on Rubyforge.

I previously posted an update that enables Growl notifications for Autotest in Windows.

Which is nice, but what about other Ruby programs, they might want to send Growl notifications too. To this end Vision Media have produced a Ruby Growl gem which makes it easy for any Ruby program to send Growl notifications.

But the visionmedia-growl gem only works on OSX.

Now With Added Windows Support

So I had a go at adding Windows support. The short version is that it works, but the code is seriously ugly and not well tested (you have been warned!).

Check out my visionmedia-growl fork. Or just install the gem:

sudo gem install karl-growl --source http://gems.github.com

Packaging growlnotify

First up I decided to package both the OSX and Windows versions of growlnotify with the gem.

This is a departure from the existing gem, which requires that you have installed growlnotify yourself. I wanted to be able to include this gem in new projects without having to bother users to download extra dependencies.

Choose the Right growlnotify

I decide which growlnotify to use by checking which platform we are running on:

51
52
53
54
55
56
57
58
59
60
61
62
63
64
  def self.is_windows?
    processor, platform, *rest = RUBY_PLATFORM.split("-")
    platform == 'mswin32'
  end
 
  ##
  # Execute +args+ against the binary.
 
  def self.exec *args
    bin = PACKAGED_BIN
    bin += '.com' if is_windows?
 
    Kernel.system bin, *args
  end

Different Switches on Different Operating Systems

The biggest headache with adding Windows support was adding the ability to generate Windows style command line arguments.

Example OSX command line:

growlnotify --message llamas --title Llama! --image images/llama.png

Example Windows command line:

growlnotify.com llamas /t:Llama! /i:images/llama.png

Note that in the Windows command line, switches are specified with /x: prefix, and no space. The names of the switches don’t correspont exactly with the OSX ones.

Also, with the Windows version of growlnotify you don’t give an switch for the message body, just include it as the first argument.

Adding Support for Windows Style Switches

The first part of adding this support was to map the OSX switches to their Windows counterparts (where they existed).

204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
    switch :title,      :t
    switch :sticky,     :s
    switch :priority,   :p
    switch :host,       :host
    switch :password,   :pass
    switch :port,       :port
 
    switch :name,       :a
    switch :message,    :EMPTY
    switch :image,      :i
    switch :identifier, :id
 
    switch :iconpath,   nil
    switch :appIcon,    nil
    switch :icon,       nil
 
    switch :udp,        nil
    switch :auth,       :hash
    switch :crypt,      nil

As you can see, not all OSX switches are available in the Windows version of growlnotify. I dealt with this at the moment, by just stripping out any switches that won’t work in Windows.

Finally I altered (hacked?) the Growl run method, to produce parameter strings for either OSX or Windows.

153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
    def run
      raise Error, 'message required' unless message
      self.class.switches.each do |name, win_name|
        if send(:"#{name}?")
          value = send(name).to_s if send(name) && !(TrueClass === send(name))
          if is_windows?
            next if win_name.nil?
 
            switch = (win_name == :EMPTY) ? "" : "/#{win_name}:"
            args << switch + value
          else
            args << "--#{name}"
            args << value
          end
        end
      end
      Growl.exec *args
    end

The main points of interest here are the The windows switches don’t have any space between the switch name and the value. So instead they are concatenated together into a single string.

Finally we deal with the special case where using the Windows version of growlnotify we need to add the message parameter without a switch, so we use the special token :EMPTY to deal with this.

The Result

The results of this is a Ruby gem that supports Growl notifications in Windows. The gem can be installed with the following commands:

sudo gem install karl-growl --source http://gems.github.com

And used in your Ruby program as follows:

Growl.notify {
    self.message = 'Hello World'
    self.image = File.join 'path', 'to', 'image.png'
  }

For more usage examples see the karl-growl site.

Known Issues

There are a few known issues. The biggest of which no support for normalised icons.

To support OSX and Windows always give a path to an image, e.g.

  :image => 'path/to/image.png'

There are a number of switches that only work in OSX, and are ignored in Windows. Unsupported switches are:

  :iconpath
  :appIcon
  :icon
  :udp
  :crypt

The gem has only received the most cursory of testing, so there may be a whole host of other issues, be warned!

The Future

I’m treating the current version of this gem, as a design spike, a proof of concept that shows we can have cross Operating System support. The code is a real mess, and has no unit tests, but I’m releasing it here to follow the ‘better now beats best later’ rule.

I hope to refactor the code into something more production worthy when I get a chance.

Autotest Growl for Windows

July 10th, 2009

In a previous blog post I detailed how to get Autotest Growl notifications on Windows using Snarl. But, there now exists an official Growl client for Windows.

So how about we update autotest-growl to support Windows (on top of it’s existing support for OSX).

If you don’t want to know all the gory details, you can just install my fork of the autotest-growl gem, that includes native Growl Windows support:

gem install karl-autotest-growl --source http://gems.github.com

Obviously, you’ll need to have Growl for Windows installed as well!

Updating Autotest Growl to support Windows

First up download and install Growl for Windows (if you haven’t already).

Updating Autotest Growl turns out to be pretty simple. We need to download growlnotify for Windows, and add it to the growl/ directory.

Then we can update the autotest-growl code to decide which growlnotify to use depending on the Operating System.

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
  def self.is_windows?
    processor, platform, *rest = RUBY_PLATFORM.split("-")
    platform == 'mswin32'
  end
 
  ##
  # Display a message through Growl.
  def self.growl title, message, icon, priority=0, stick=""
    growl = File.join(GEM_PATH, 'growl', 'growlnotify')
    image = File.join(ENV['HOME'], '.autotest-growl', "#{icon}.png")
    image = File.join(GEM_PATH, 'img', "#{icon}.png") unless File.exists?(image)
 
    if is_windows?
      growl += '.com'
      cmd = "#{growl} #{message.inspect} /a:\"autotest\" /r:\"Autotest\" /n:\"Autotest\" /i:\"#{image}\" /p:#{priority} /t:\"#{title}\""
    else 
      if @@remote_notification
        cmd = "#{growl} -H localhost -n autotest --image '#{image}' -p #{priority} -m #{message.inspect} '#{title}' #{stick}"
      else
        cmd = "#{growl} -n autotest --image '#{image}' -p #{priority} -m #{message.inspect} '#{title}' #{stick}"
      end
    end
 
  system cmd
  end

You’ll notice that the command line flags are slightly different for the Windows version of growlnotify, the using the /x: format, rather than the OSX -x format.

The Windows version also adds the flat /r:"Autotest" to ensure the Autotest application is registered the first time it is run.

There is no need to deal with the @@remote_notification settings, as Growl for Windows does not have the same problem with intermittent notification loss that OSX does.

Installing updated Autotest Growl gem

I have rolled all these changes into my fork of autotest-growl. To switch over to this fork, open a Windows command prompt and enter:

gem uninstall autotest-growl
gem install karl-autotest-growl --source http://gems.github.com

And then watch the notifications roll in!

Growl and Autotest work brilliantly together. Autotest runs all your tests in the background every time a file changes, giving you extremely fast feedback on your test driven development. And Growl notifications save you from needing to flip back to the Terminal to see the result of each test run, you get an unobtrusive popup in the corner of your screen showing the success or failure of the tests.

But what about those users working on the Windows platform?

Snarl is a windows counterpart to Growl. Providing much of the same functionality.

And there is a Snarl Ruby gem allowing us to create Snarl notifications from Ruby.

So lets convert our Growl calls to Snarl calls, and get Autotest notifications under Windows.

Installing Autotest, Snarl, and the Growl to Snarl converter

If you haven’t already download and install Ruby.

First up we need to download and install Snarl.

Then we need to install Autotest and the autotest-growl gem (that we are later going to override). Open a command prompt, and type the following (Autotest is part of the ZenTest gem):

gem install ZenTest
gem install ruby-snarl
gem install autotest-growl

Then save the code below to lib/autotest/growl-to-snarl.rb within your project. This code converts any Growl calls to equivalent Snarl calls.

growl-to-snarl.rb

require 'snarl'
module Autotest::Growl
 
  # Display a message through Snarl.
  def self.growl title, message, icon, priority=0, stick=""
    image = File.join(ENV['HOME'], '.autotest-growl', "#{icon}.png")
    image = File.join(GEM_PATH, 'img', "#{icon}.png") unless File.exists?(image)
 
	Snarl.show_message(title, message.inspect, image)
  end
 
end

Next we need to update our .autotest configuration file to include the autotest-growl gem, and the Growl to Snarl converter. Add the following to your .autotest file in the project root directory.

require 'autotest-growl'
require 'snarl'
require 'lib/autotest/growl-to-snarl'

Running Autotest with Snarl support

First up ensure Snarl is running (check for the icon in the system tray).

Autotest will fail to run on Windows if a HOME environment variable doesn’t exist, so we need to create one before we run (I’ve also noticed that the Ruby gem command will fail to run if the HOME evironment variable does exist, which is frustrating!).

Open a command prompt, navigate to the project root directory. Then enter the following to set the HOME environment variable and run Autotest.

set HOME="C:\Documents and Settings\username"
autotest

The results of your test runs should now display as Snarl notifications.

As an example of getting Autotest, Cucumber, and Growl up and running I’ve created a super simple test project. You can download the example project, or just create it from the code on this post.

Install Ruby, Growl, and Gems

First up we need to ensure that we have all our dependancies installed.

If you haven’t already, download and install Ruby.

Then we need to install the Autotest, Cucumber, and Growl Ruby gems. We can do this using the gem command that comes packaged with Ruby.

# Autotest test is part of the ZenTest gem
sudo gem install ZenTest
sudo gem install cucumber
# Use my modified autotest-growl gem (until the changes are merged into the official gem)
sudo gem install karl-autotest-growl --source http://gems.github.com

Next we need to ensure that the Growl application itself is installed. If not download and install Growl.

Create Project

Now we are ready to create our project. The either download the example project or create the file heirarchy below:

  • [dir] Autotest-Cucumber

    • [file] .autotest
    • [dir] features
      • [file] test.feature

Copy the code below into the .autotest file:

.autotest

require 'autotest/growl'

And the copy the following into the test.feature file:

test.feature

Feature:
 
    Scenario:
        Given I save 1
        Then I have 1

Set Environment

Lastly we need to set the AUTOFEATURE environment variable to true, so that autotest will run the Cucumber tests automatically:

AUTOFEATURE=true

Run Autotest!

Finally we can run autotest, and watch as it picks up the Cucumber tests, runs them, and notifies us via Growl. Open terminal and navigate to the project directory. Then run autotest:

autotest

Autotest will initiate a Cucumber run. Cucumber will pick up the test.feature file (because it looks for a features folder by default). The Cucumber run will show that you have 1 undefined scenario (and be kind enough to give you the code for your undefined steps). And finally Growl will display a notification that you have 1 undefined scenario, yay!

c:/ruby/bin/ruby c:/ruby/lib/ruby/gems/1.8/gems/aslakhellesoy-cucumber-0.3.11.3/bin/cucumber --format progress --format rerun --out C:/Temp/autotest-cucumber.17824.1 features
UU
 
1 scenario (1 undefined)
2 steps (2 undefined)
0m0.000s
 
You can implement step definitions for undefined steps with these snippets:
 
Given /^I save 1$/ do
  pending
end
 
Then /^I have 1$/ do
  pending
end

Following on to my previous post on Autotest, Cucumber, and Growl, I have forked the autotest-growl repository, and applied my fix for Cucumber notifications.

You find my patched version of autotest-growl on GitHub.

You can switch to this version of autotest-growl by uninstalling any existing version, and then installing from my fork on GitHub:

sudo gem uninstall autotest-growl
sudo gem install karl-autotest-growl --source http://gems.github.com

As well as using Autotest to run Cucumber scenarios I have also been looking into integrating lower level test into the Autotest cycle.

My first attempt at this is a small module to run Javascript Lint on all the javascript files within a project any time any file changes.

The module below hooks in to Autotest just before the tests are normally run. It runs javascript lint over all the *.js files in the project, outputs the results to the autotest results object and the standard output, and finally fires a new ran_javascript_lint hook

Errors and warnings found by Javascript Lint will also be notified through Growl (if Growl and autotest-growl are installed). If there are no errors or warnings than no Growl notification is shown. This keeps distracting popups to a minimum.

Installing Autotest Javascript Lint

First up, download Javascript Lint. Extract the jsl executable to lib/autotest/ within your project.

Copy the code below to lib/autotest/javascript-lint.rb within your project (the same directory where you have the jsl execuatable).

javascript-lint.rb

# Run Javascript Lint as part of autotest
# Supports Growl notifications if using autotest-growl
#
# Version 1.0
 
require 'autotest'
 
module Autotest::JavascriptLint
    @@js_dir = ''
    @@jsl_dir = File.dirname(__FILE__) + '/'
    @@config_file = ''
 
    def self.js_dir=(string)
        @@js_dir = string
    end
 
    def self.jsl_dir=(string)
        @@jsl_dir = string
    end
 
    def self.config_file=(string)
        @@config_file = string
    end
 
    Autotest.add_hook :run_command do |at|
 
        # run javascript lint
        results = `#{@@jsl_dir}jsl -conf "#{@@config_file}"  -process "#{@@js_dir}*.js" +recurse`
        puts results
 
        at.results = [] if at.results.nil?
        at.results.concat(results.split("\n"))
 
        at.hook :ran_javascript_lint
    end
 
end
 
module Autotest::Growl
 
    @@show_js_lint_success = false
 
    def self.show_js_lint_success=(bool)
        @@show_js_lint_success=bool
    end
 
    # Growl results of Javscript Lint
    Autotest.add_hook :ran_javascript_lint do |autotest|
        gist = autotest.results.grep(/\d+\s+error.*,\s+\d+\s+warning.*/).join(" / ").strip()
 
        if gist == ''
            growl @label + 'Cannot run javascript lint.', '', 'error'
        else
            if gist =~ /[1-9]\d*\s+(error)/
                growl @label + 'Lint: Some files have errors.', gist, 'failed'
            elsif gist =~ /[1-9]\d*\s+(warning)/
                growl @label + 'Lint: Some files have warnings.', gist, 'pending'
            elsif @@show_js_lint_success
                growl @label + 'Lint: All files are clean.', gist, 'passed'
            end
        end
        false
    end
 
end

Then add Autotest Javascript Lint to your .autotest configuration file within the base of your project.

require 'lib/autotest/javascript-lint'

To Do

This could be packaged as a gem for easy installation, and it could possibly be modified to only run over changed files.

Autotest is a great Ruby tool to speed up test driven development by automatically running your tests every time a file is saved.

Cucumber is an awesome tool for doing behavioural driven development. It allows you to write plain text automated acceptance tests.

Autotest and Cucumber work together seamlessly, you just need to set the AUTOFEATURE environment variable to true:

$ AUTOFEATURE=true

Growl is an excellent notification system for OSX that allows applications to popup unobtrusive messages on the users desktop. (Snarl is the equivalent for windows)

You can add Growl support to Autotest (using the autotest-growl gem) so that you get popup notifications of test results.

But the current version of autotest-growl doesn’t provide notifications for the result of Autotest Cucumber runs.

Cucumber Growl support in Autotest

So I’ve added support for notification of Autotest Cucumber results using growl. It’s super simple at the moment as I’m still pretty new at Ruby.

You need to have Autotest (part of the ZenTest gem), Autotest-Growl, and Growl already installed.

Save the code below to the file growl-cucumber.rb.

growl-cucumber.rb

require 'autotest'
 
module Autotest::Growl
 
  # Growl results of Cucumber
  Autotest.add_hook :ran_features do |autotest|
 
	gist = autotest.results.grep(/\d+\s+scenario.*\)/).join(" / ").strip()
	if gist == ''
	  growl @label + 'Cannot run features.', '', 'error'
	else
	  if gist =~ /[1-9]\d*\s+(failed)/
	    growl @label + 'Some features have failed.', gist, 'failed'
	  elsif gist =~ /pending/
	    growl @label + 'Some features are skipped.', gist, 'skipped'
	  else
	    growl @label + 'All features have passed.', gist, 'passed'
	  end
	end
    false
  end
 
end

Then update your .autotest configuration file to include the new growl-cucumber file:

.autotest

# Add the growl and growl-cucumber requires to your .autotest config file 
require 'autotest/growl'
require 'autotest/growl-cucumber'

To Do

I need to see if this can be integrated with the existing autotest-growl gem.