In a previous post I detailed changes I made to add Growl for Windows support to Autotest Growl.

These changes have now been integrated back into the autotest-growl gem. Along with some major improvements to the analysis of test results, and the details in the notifications. Thanks to Svoop, the creator of autotest-growl.

This gem isn’t yet available on Rubyforge, but you can grab a copy from my fork on GitHub until it is:

gem install karl-autotest-growl --source http://gems.github.com

This should be a drop in replacement for the previous version of karl-autotest-growl, the only thing you should notice more detail from the growl notifications!

I’ll post again once the official gem is available on Rubyforge.

I previously posted an update that enables Growl notifications for Autotest in Windows.

Which is nice, but what about other Ruby programs, they might want to send Growl notifications too. To this end Vision Media have produced a Ruby Growl gem which makes it easy for any Ruby program to send Growl notifications.

But the visionmedia-growl gem only works on OSX.

Now With Added Windows Support

So I had a go at adding Windows support. The short version is that it works, but the code is seriously ugly and not well tested (you have been warned!).

Check out my visionmedia-growl fork. Or just install the gem:

sudo gem install karl-growl --source http://gems.github.com

Packaging growlnotify

First up I decided to package both the OSX and Windows versions of growlnotify with the gem.

This is a departure from the existing gem, which requires that you have installed growlnotify yourself. I wanted to be able to include this gem in new projects without having to bother users to download extra dependencies.

Choose the Right growlnotify

I decide which growlnotify to use by checking which platform we are running on:

51
52
53
54
55
56
57
58
59
60
61
62
63
64

  def self.is_windows?
    processor, platform, *rest = RUBY_PLATFORM.split("-")
    platform == 'mswin32'
  end
 
  ##
  # Execute +args+ against the binary.
 
  def self.exec *args
    bin = PACKAGED_BIN
    bin += '.com' if is_windows?
 
    Kernel.system bin, *args
  end

Different Switches on Different Operating Systems

The biggest headache with adding Windows support was adding the ability to generate Windows style command line arguments.

Example OSX command line:

growlnotify --message llamas --title Llama! --image images/llama.png

Example Windows command line:

growlnotify.com llamas /t:Llama! /i:images/llama.png

Note that in the Windows command line, switches are specified with /x: prefix, and no space. The names of the switches don’t correspont exactly with the OSX ones.

Also, with the Windows version of growlnotify you don’t give an switch for the message body, just include it as the first argument.

Adding Support for Windows Style Switches

The first part of adding this support was to map the OSX switches to their Windows counterparts (where they existed).

204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222

    switch :title,      :t
    switch :sticky,     :s
    switch :priority,   :p
    switch :host,       :host
    switch :password,   :pass
    switch :port,       :port
 
    switch :name,       :a
    switch :message,    :EMPTY
    switch :image,      :i
    switch :identifier, :id
 
    switch :iconpath,   nil
    switch :appIcon,    nil
    switch :icon,       nil
 
    switch :udp,        nil
    switch :auth,       :hash
    switch :crypt,      nil

As you can see, not all OSX switches are available in the Windows version of growlnotify. I dealt with this at the moment, by just stripping out any switches that won’t work in Windows.

Finally I altered (hacked?) the Growl run method, to produce parameter strings for either OSX or Windows.

153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170

    def run
      raise Error, 'message required' unless message
      self.class.switches.each do |name, win_name|
        if send(:"#{name}?")
          value = send(name).to_s if send(name) && !(TrueClass === send(name))
          if is_windows?
            next if win_name.nil?
 
            switch = (win_name == :EMPTY) ? "" : "/#{win_name}:"
            args << switch + value
          else
            args << "--#{name}"
            args << value
          end
        end
      end
      Growl.exec *args
    end

The main points of interest here are the The windows switches don’t have any space between the switch name and the value. So instead they are concatenated together into a single string.

Finally we deal with the special case where using the Windows version of growlnotify we need to add the message parameter without a switch, so we use the special token :EMPTY to deal with this.

The Result

The results of this is a Ruby gem that supports Growl notifications in Windows. The gem can be installed with the following commands:

sudo gem install karl-growl --source http://gems.github.com

And used in your Ruby program as follows:

Growl.notify {
    self.message = 'Hello World'
    self.image = File.join 'path', 'to', 'image.png'
  }

For more usage examples see the karl-growl site.

Known Issues

There are a few known issues. The biggest of which no support for normalised icons.

To support OSX and Windows always give a path to an image, e.g.

  :image => 'path/to/image.png'

There are a number of switches that only work in OSX, and are ignored in Windows. Unsupported switches are:

  :iconpath
  :appIcon
  :icon
  :udp
  :crypt

The gem has only received the most cursory of testing, so there may be a whole host of other issues, be warned!

The Future

I’m treating the current version of this gem, as a design spike, a proof of concept that shows we can have cross Operating System support. The code is a real mess, and has no unit tests, but I’m releasing it here to follow the ‘better now beats best later’ rule.

I hope to refactor the code into something more production worthy when I get a chance.

In a previous blog post I detailed how to get Autotest Growl notifications on Windows using Snarl. But, there now exists an official Growl client for Windows.

So how about we update autotest-growl to support Windows (on top of it’s existing support for OSX).

If you don’t want to know all the gory details, you can just install my fork of the autotest-growl gem, that includes native Growl Windows support:

gem install karl-autotest-growl --source http://gems.github.com

Obviously, you’ll need to have Growl for Windows installed as well!

Updating Autotest Growl to support Windows

First up download and install Growl for Windows (if you haven’t already).

Updating Autotest Growl turns out to be pretty simple. We need to download growlnotify for Windows, and add it to the growl/ directory.

Then we can update the autotest-growl code to decide which growlnotify to use depending on the Operating System.

50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74

  def self.is_windows?
    processor, platform, *rest = RUBY_PLATFORM.split("-")
    platform == 'mswin32'
  end
 
  ##
  # Display a message through Growl.
  def self.growl title, message, icon, priority=0, stick=""
    growl = File.join(GEM_PATH, 'growl', 'growlnotify')
    image = File.join(ENV['HOME'], '.autotest-growl', "#{icon}.png")
    image = File.join(GEM_PATH, 'img', "#{icon}.png") unless File.exists?(image)
 
    if is_windows?
      growl += '.com'
      cmd = "#{growl} #{message.inspect} /a:\"autotest\" /r:\"Autotest\" /n:\"Autotest\" /i:\"#{image}\" /p:#{priority} /t:\"#{title}\""
    else 
      if @@remote_notification
        cmd = "#{growl} -H localhost -n autotest --image '#{image}' -p #{priority} -m #{message.inspect} '#{title}' #{stick}"
      else
        cmd = "#{growl} -n autotest --image '#{image}' -p #{priority} -m #{message.inspect} '#{title}' #{stick}"
      end
    end
 
  system cmd
  end

You’ll notice that the command line flags are slightly different for the Windows version of growlnotify, the using the /x: format, rather than the OSX -x format.

The Windows version also adds the flat /r:"Autotest" to ensure the Autotest application is registered the first time it is run.

There is no need to deal with the @@remote_notification settings, as Growl for Windows does not have the same problem with intermittent notification loss that OSX does.

Installing updated Autotest Growl gem

I have rolled all these changes into my fork of autotest-growl. To switch over to this fork, open a Windows command prompt and enter:

gem uninstall autotest-growl
gem install karl-autotest-growl --source http://gems.github.com

And then watch the notifications roll in!

I’ve made a small update to the QUnit Adapter for JS Test Driver.

This fixes a bug where the ok() assertion was much stricter than the qunit equivalent. It was only succeeding when passed a boolean true value.

Now it behaves the same as the qunit version, and succeeds with all values other than 0, false, or null.

You can get the new 1.0.1 verison of the QUnit Adapter from Google Code.

As part of my experimenting with JS Test Driver, I have been porting over a suite of about 300 existing tests.

Some of these tests deal with time based logic, such as delayed callbacks or time limited user interaction. Currently these tests are run in a HTML page using qunit.

To deal with the time based logic I use the jsUnit’s Mock Timeout. See Simulating Time in jsUnit Tests on the Google Testing Blog.

jsUnit’s Mock Timeout overrides the built in setTimeout, clearTimeout, setInterval, and clearInterval methods, and provides a Clock object that allows you to control the passage of time.

Problem

This works fine in qunit, but it fails catastrophically in JS Test Driver. This is because JS Test Driver uses the setTimeout and setInterval methods to control it’s own behaviour. Once these have been overridden with the mock versions JS Test Driver no longer works.

As a temporary fix for this I have created a patched version of JS Test Driver, which creates it’s own copies of the original setTimeout, clearTimeout, setInterval, and clearInterval methods, before loading any code. This ensures it works well with the jsUnit Mock Timeout.

Patched JS Test Driver

Download a patched JsTestDriver with no-confilict timeouts.

Just use this jar in place of the original JsTestDriver-1.0b.jar, and you can include jsUnitMockTimeout.js in your tests without breaking JS Test Driver!

You may also want to download jsUnitMockTimeout.js (to save you finding it within the jsUnit source).

The Future

I see this patched JS Test Driver as very much a temporary stepping stone, while the correct solution is worked out. Beware of relying on it too heavily!

In my last post I introduced a QUnit to JS Test Driver adapter, a small adapter which converts QUnit tests into native JS Test Driver tests.

This adapter is now part of the JS Test Driver project.

There is now a QUnit Adapter wiki page on the JS Test Driver project site. This will be updated to keep up with any changes to the adapter.

The code is now stored within the JS Test Driver subversion repository. You can always download the latest version of the QUnit Adapter using the Google Code web interface.

Thanks to Jeremie Lenfant-engelmann for deeming QUnit Adapter worthy of official regonition :)

In my previous post on QUnit and JS Test Driver I showed how to run your qunit tests with JS Test Driver.

The technique used was to run the tests with qunit, and report either their success or failure to JS Test Driver. This works, but you miss out on the important feedback of exactly which assertions have failed, and why.

This problem has led me to taking a new approach to running qunit tests with JS Test Driver, where I don’t use any of the existing qunit code, and instead just create an interface wrapper that converts qunit style tests and assertions directly into JS Test Driver tests and assertions.

This gives assertion level error reporting, making it much easier to write and debug tests. Essentially this adapter allows you to write native JS Test Driver tests, but using the less verbose qunit syntax.

The new approach also means that qunit lifecycles (setup and teardown) work.

Installing the QUnit Adapter

First up download the equiv.js file, which is required for the qunit same assertion.

Then download the QUnitAdapter.js file (or copy the code below).

QUnitAdapter.js

(function() {
 
    window.module = function(name, lifecycle) {
        QUnitTestCase = TestCase(name);
 
        if (lifecycle) {
            QUnitTestCase.prototype.setUp = lifecycle.setup;
            QUnitTestCase.prototype.tearDown = lifecycle.teardown;
        }
    };
 
    window.test = function(name, test) {
        QUnitTestCase.prototype['test ' + name] = test;
    };
 
    window.expect = function(count) {
        expectAsserts(count);
    };
 
    window.ok = function(actual, msg) {
        assertTrue(msg ? msg : '', actual);
    };
 
    window.equals = function(a, b, msg) {
        assertEquals(msg ? msg : '', b, a);
    };
 
    window.start = window.stop = function() {
        fail('start and stop methods are not available when using JS Test Driver.\n' +
            'Use jsUnit Clock object to deal with timeouts and intervals:\n' + 
            'http://googletesting.blogspot.com/2007/03/javascript-simulating-time-in-jsunit.html.');
    };
 
    window.same = function(a, b, msg) {
        assertTrue(msg ? msg : '', window.equiv(b, a));
    };
 
    window.reset = function() {
    	fail('reset method is not available when using JS Test Driver');
    };
 
    window.isLocal = function() {
    	return false;
    };
 
    window.QUnit = {
    	equiv: window.equiv,
    	ok: window.ok
    };
 
})();

Save both these files to your project (for example tests/qunit/).

Configuring JS Test Driver

To run your qunit tests in JS Test Driver you need to configure it to load the adapter before your qunit tests.

Update your jsTestDriver.conf to load the files:

server: http://localhost:9876
 
load:
  # Add these lines to load the equiv function and adapter in order, before the tests
  # (assuming they are saved to tests/qunit/)
  - tests/qunit/equiv.js
  - tests/qunit/QUnitAdapter.js
 
  # This is where we load the qunit tests
  - tests/js/*.js
 
  # And this loads the source files we are testing
  - src/js/*.js

Running JS Test Driver with qunit tests

Now we can run JS Test Driver and watch as it runs all our qunit tests!

The tests will run as individual JS Test Driver tests, with the format Module Name.Test Name.

Example output:

[PASSED] Module 1.test Test 1
[PASSED] Module 1.test Test 2
[PASSED] Module 2.test Test 1
Total 3 tests (Passed: 3; Fails: 0; Errors: 0) (1.00 ms)
  Safari 530.18: Run 3 tests (Passed: 3; Fails: 0; Errors 0) (1.00 ms)

Limitations

There are a few limitations on which qunit tests will successfully be converted.

The tests must run synchronously (which means no use of the qunit stop and start methods).

If you need to test timeouts, intervals, or other asynchronous sections of code, consider using the jsUnit Clock object to deal with timeouts and intervals.

QUnit DOM support is not included. Consider avoiding interacting directly with the browser within your unit tests. But if you do need to, you’ll need to create and remove the DOM objects yourself with each test, or the setup and teardown methods.

I was very impressed by the new Google JS Test Driver project, which provides a blisteringly fast, and easily automated way of running your Javascript unit tests. See this introduction to JS Test Driver by Miško Hevery for a great overview.

I previously described how to run JS Test Driver automatically with Autotest.

But I have an existing project that uses the jQuery testing framework qunit for testing. I didn’t really fancy rewriting 300+ tests just so I could use the JS Test Driver framework.

So I wrote a converter that automatically converts qunit modules and tests into JS Test Driver TestCases and test methods.

Download Converter and Patched Testrunner

In order to convert from qunit tests I’ve had to add a few extra hooks into the qunit testrunner.js file.

Either download the patched testrunner.js file, or just add the 3 lines below:

234
235
236
237
238
239
240
241
242
243
244
245

    QUnit: {
        // Add the following 3 lines 
        runTest: runTest,
        config: config,
        validTest: validTest,	
 
        // This is existing code	
        equiv: equiv,
        ok: ok,
        done: function(failures, total){},
        log: function(result, message){}
    },

Next download QUnitToTestCases.js and save it to the same folder as testrunner.js. This is the file which converts the qunit tests into TestCases that JS Test Driver understands.

It works by overriding the qunit test() function, and rather than adding the test to qunit, it creates a test method on a TestCase object which, when called by JS Test Driver adds the test to qunit and runs it.

Configuring JS Test Driver

Once you have the patched testrunner.js and QUnitToTestCases.js, you just need to let JS Test Driver know to load them before your qunit tests. They need to be loaded in order, with testrunner.js first, followed by QUnitTiTestCases.js, as the converter modifies some of the testrunner methods.

Update your jsTestDriver.conf to load the files:

server: http://localhost:9876
 
load:
  # Add these lines to load the testrunner and converter in order, before the tests
  # (assuming the files are saved to tests/qunit/)
  - tests/qunit/testrunner.js
  - tests/qunit/QUnitToTestCases.js
 
  # This is where we load the qunit tests
  - tests/js/*.js
 
  # And this loads the source files we are testing
  - src/js/*.js

Running JS Test Driver with qunit tests

Now we can run JS Test Driver and watch as it runs all our qunit tests!

The tests will run as individual JS Test Driver tests, with the format Module Name.Test Name.

Example output:

[PASSED] Module 1.test Test 1
[PASSED] Module 1.test Test 2
[PASSED] Module 2.test Test 1
Total 3 tests (Passed: 3; Fails: 0; Errors: 0) (1.00 ms)
  Safari 530.18: Run 3 tests (Passed: 3; Fails: 0; Errors 0) (1.00 ms)

Limitations

There are a few limitations on which qunit tests will successfully be converted.

The tests must run synchronously (which means no use of the qunit stop and start methods).

Module lifecycles are ignored at the moment, which means setup and teardown functions are not called.

Growl and Autotest work brilliantly together. Autotest runs all your tests in the background every time a file changes, giving you extremely fast feedback on your test driven development. And Growl notifications save you from needing to flip back to the Terminal to see the result of each test run, you get an unobtrusive popup in the corner of your screen showing the success or failure of the tests.

But what about those users working on the Windows platform?

Snarl is a windows counterpart to Growl. Providing much of the same functionality.

And there is a Snarl Ruby gem allowing us to create Snarl notifications from Ruby.

So lets convert our Growl calls to Snarl calls, and get Autotest notifications under Windows.

Installing Autotest, Snarl, and the Growl to Snarl converter

If you haven’t already download and install Ruby.

First up we need to download and install Snarl.

Then we need to install Autotest and the autotest-growl gem (that we are later going to override). Open a command prompt, and type the following (Autotest is part of the ZenTest gem):

gem install ZenTest
gem install ruby-snarl
gem install autotest-growl

Then save the code below to lib/autotest/growl-to-snarl.rb within your project. This code converts any Growl calls to equivalent Snarl calls.

growl-to-snarl.rb

require 'snarl'
module Autotest::Growl
 
  # Display a message through Snarl.
  def self.growl title, message, icon, priority=0, stick=""
    image = File.join(ENV['HOME'], '.autotest-growl', "#{icon}.png")
    image = File.join(GEM_PATH, 'img', "#{icon}.png") unless File.exists?(image)
 
	Snarl.show_message(title, message.inspect, image)
  end
 
end

Next we need to update our .autotest configuration file to include the autotest-growl gem, and the Growl to Snarl converter. Add the following to your .autotest file in the project root directory.

require 'autotest-growl'
require 'snarl'
require 'lib/autotest/growl-to-snarl'

Running Autotest with Snarl support

First up ensure Snarl is running (check for the icon in the system tray).

Autotest will fail to run on Windows if a HOME environment variable doesn’t exist, so we need to create one before we run (I’ve also noticed that the Ruby gem command will fail to run if the HOME evironment variable does exist, which is frustrating!).

Open a command prompt, navigate to the project root directory. Then enter the following to set the HOME environment variable and run Autotest.

set HOME="C:\Documents and Settings\username"
autotest

The results of your test runs should now display as Snarl notifications.

Google recently released a new Javascript testing framework, JS Test Driver. It provides incredibly fast execution for Javascript unit tests, and can be run from the command line without the need for manual control of browsers. Check out this introduction to JS Test Driver by Miško Hevery.

Fast test execution and the ability to be run from the command line make it a perfect fit to integrate into the Autotest test cycle. So I have.

The module below hooks into Autotest just before the normal tests are run. It runs JS Test Driver over all the tests in the project, outputs the results, and finally fires off a :ran_js_test_driver hook.

Errors and failed tests will automatically be notified through Growl (if Growl and autotest-growl are installed). By default successful tests runs are not notified through Growl, in order to keep distracting popups to a minimum.

Installing Autotest JS Test Driver

First you need to download a copy of JS Test Driver.

Save the JS Test Driver jar file to the lib/ directory within your project.

Then copy the code below to lib/autotest/js-test-driver.rb

js-test-driver.rb

# Run JS Test Driver as part of autotest
# Supports Growl notifications if using autotest-growl
 
require 'autotest'
 
module Autotest::JsTestDriver
 
    @@jar = File.dirname(__FILE__) + '/JsTestDriver-1.0b.jar'
    @@config_file = 'jsTestDriver.conf'
 
    def self.jar=(string)
        @@jar = string
    end
 
    def self.config_file=(string)
        @@config_file = string
    end
 
    def self.show_success=(bool)
        @@show_success=bool
    end
 
 
    Autotest.add_hook :run_command do |at|
        # run js test driver
        results = 'JS Test Driver:'
        results += `java -jar "#{@@jar}" --config "#{@@config_file}" --tests all --verbose`
        puts results
 
        at.results = [] if at.results.nil?
        at.results.concat(results.split("\n"))
 
        at.hook :ran_js_test_driver
 
    end
 
end
 
module Autotest::Growl
 
    @@show_js_test_success = false
 
    def self.show_js_test_success=(bool)
        @@show_js_test_success=bool
    end
 
  # Growl results of JS Test Driver
  Autotest.add_hook :ran_js_test_driver do |autotest|
 
    gist = autotest.results.grep( /Total\s+\d+\s+tests/ ).join(" / ").strip()
 
    if gist == ''
      growl @label + 'Cannot run JS Test Driver.', gist, 'error'
    else
      if gist =~ /Errors:\s+[1-9]\d*/
        growl @label + 'Error running some JS tests.', gist, 'failed'
      elsif gist =~ /Fails:\s+[1-9]\d*/
        growl @label + 'JS Test: Some tests failed.', gist, 'failed'
      elsif @@show_js_test_success
        growl @label + 'JS Test: All files are clean.', gist, 'passed'
      end
    end
    false
  end
 
end

Configuring Autotest and JS Test Driver

JS Test Driver uses a configuration file to connect with the JS Test Driver server, and to decide which javscript files to load.

Create a jsTestDriver.conf file in the project root directory as follows.

server: http://localhost:9876
 
load:
  - src/js/*.js
  - tests/js/*.js

This assumes that you have our javascript source files will be in the src/js/ directory, and our javascript test files will be in the src/js/ directory. We will create a test file, and associated code later.

The server: lets JS Test Driver know we will be connecting to a server on our local machine, on port 9876. We’ll get this server running later.

Next we need to configure Autotest to run JS Test Driver, by requiring the module and specifying the location of the JS Test Driver jar.

# Require the JS Test Driver module
require 'lib/autotest/js-test-driver'
 
# Set the location of the JS Test Driver jar
Autotest::JsTestDriver::jar = './lib/jsTestDriver-1.0b.jar'

You can also configure the location of the JS Test Driver config file, and whether or not to show successful test runs.

# Uncomment this if you have autotest-growl, and Growl installed
# And want to have notifications of JS Test Driver runs
# require 'autotest/growl'
 
# Uncomment this to change the location of the JS Test Driver config file
# By default we look for a jsTestDriver.conf file in the directory autotest is run from 
# Autotest::JsTestDriver::config_file = './jsTestDriver.conf'
 
# Uncomment this to show successful test runs
# Autotest::Growl::show_js_test_success = true

Now that all the installation and configuration is done, let get everything running.

Running Autotest with JS Test Driver

First up we need to get our JS Test Driver server up and running. Open a Terminal, and navigate to the directory containing the JS Test Driver jar. Run the following to start a server:

java -jar JsTestDriver-1.0b.jar --port 9876

Now we need to capture a browser to use for testing. Open a browser and automatically capture it for use with JS Test Driver by going to the following URL:

http://localhost:9876/capture

Now we can run autotest, and watch as it runs JS Test Driver and reports the results to us on every file change:

autotest

JS Test Driver will probably report that no tests were run, as we haven’t written any tests yet. Tests are written using the TestCase object, which exposes JUnit style functionality.

Writing Some Tests

Here is an example test file, and the production code it tests:

GreeterTest.js

GreeterTest = TestCase("GreeterTest");
 
GreeterTest.prototype.testGreet = function() {
  var greeter = new myapp.Greeter();
  assertEquals("Hello World.", greeter.greet("World"));
};

Greeter.js

myapp = {};
 
myapp.Greeter = function() { };
 
myapp.Greeter.prototype.greet = function(name) {
  return "Hello " + name + "!";
};

If you copy these to your tests/js/ and src/js/ directories respectively, Autotest should pick up the new files, run the tests and notify you that there is an error. See if you can spot it :P

To Do

This would be nice packaged up as a gem. It would also be nice if failed Javascript tests could stop further tests being run.