Functional testing for Rijksmuseum.nl
The codebase for Rijksmuseum.nl has become quite large through the years. I mainly work on the frontend, but that alone demands quite a lot of attention to expand and improve while not breaking things along the way. We’ve had some tests to cover the most essential parts of the frontend, but frankly, they were hard to read, write and debug when something broke. This resulted in not writing them. Luckily I now have some time to clean this up. Little did I know…
Previously, on Casper
We’ve been writing functional tests in CasperJs. This is a utility that uses PhantomJs to navigate through and interact with pages, asserting stuff along the way. Writing tests with Casper looks a little bit like this:
casper.test.begin('Google search retrieves 10 or more results', 5, function suite(test) {
casper.start("http://www.google.fr/", function() {
test.assertTitle("Google", "google homepage title is the one expected");
test.assertExists('form[action="/search"]', "main form is found");
this.fill('form[action="/search"]', {
q: "casperjs"
}, true);
});
casper.then(function() {
test.assertTitle("casperjs - Recherche Google", "google title is ok");
test.assertUrlMatch(/q=casperjs/, "search term has been submitted");
test.assertEval(function() {
return __utils__.findAll("h3.r").length >= 10;
}, "google search for "casperjs" retrieves 10 or more results");
});
casper.run(function() {
test.done();
});
});
Call me spoiled, but this looks cumbersome. We had some utility methods to make it a bit easier, but at the same time they made the suite more complex and the output pretty damn noisy. We really need some kind of framework that provides a clear, concise structure and makes writing tests fun again.
Let’s do this
As I’m used to RSpec, I was intrigued by Jasmine. Writing tests with Jasmine goes a little bit like this:
describe("A suite", function() {
it("contains spec with an expectation", function() {
expect(true).toBe(true);
});
});
describe, it, expect, yes please! We just need to find a way to slap a browser.visit(url) in there somehow and let’s get cracking, right? Wrong.
Welcome to the jungle
To visit a URL and do some tests on it, we’ll at least need a browser to run them with. I have nothing against PhantomJs, so I started looking for a way to combine this with Jasmine. Turns out, PhantomJs is not node, so the only way to drive Phantom from a Node process is through one of the node packages: phantom, phantomjs, node-phantom, phantom-proxy, just to name a few. There’s even a promising phantom-jasmine! I’ve been told that being lazy is a good thing in programming, so I started with the latter. The disappointment, however, was that this is not meant to visit a URL and sprinkle some Jasmine on it. This is a package to run unit tests with Phantom, which involves pointing it to a HTML file that contains references to Jasmine-specs. In fact, most things involving testing in Javascript deal only with unit testing.
Alright, back to driving Phantom from Node. We’ll need phantom and jasmine-node so we can do something like this:
var phantom = require('phantom');
describe("A suite", function() {
it("should open google.com", function(done) {
phantom.create(function(ph) {
ph.createPage(function(page) {
page.open("http://www.google.com", function(status) {
expect(status).toBe("success");
page.evaluate(function() {
return document.title;
}, function(result) {
expect(result).toBe("Google");
ph.exit();
done();
});
});
});
});
});
});
Yeah, I know, this isn’t any better than Casper. You could write a wrapper to fix this, but that’s the least of your problems. The thing is that the only way to interact with a Phantom-page is to write evaluations like this:
page.open('http://google.com', function(status) {
var title = page.evaluate(function(s) {
return document.querySelector(s).innerText;
}, 'title');
console.log(title);
phantom.exit();
});
This stuff will rapidly turn your tests into an unreadable mess. You could expand your wrapper with commonly used evaluations (like I did) but now you’re rewriting Casper.
When in doubt, unleash zombies
After trashing this approach, I thought I had seen the light when I came across this article about using ZombieJS. The idea is to use ZombieJs instead of Phantom and use jasmine-node to run the tests. Yeah! I got this up and running in no time. Champagne was falling from the heavens and I was ready to write my first real test.
But it didn’t work. I’ve spent hours on filling in a form and clicking a button, but it did not work.
Up your karma
Further on in my quest, I came across this pretty thorough tutorial. It uses Karma, which seems to be quite complex to setup, but pretty damn awesome indeed.
I got really excited when I saw this example code:
describe("Integration/E2E Testing", function() {
// start at root before every test is run
beforeEach(function() {
browser().navigateTo('/');
});
// test default route
it('should jump to the /home path when / is accessed', function() {
browser().navigateTo('#/');
expect(browser().location().path()).toBe("/login");
});
});
browser().navigateTo()! Yes! Please! Salvation is near!
Nope, you can only do this when you’re testing an Angular app.
Enough with the drama
It seems that all roads lead to CasperJs. And let’s be fair, for a good reason: it uses PhantomJs (which is fine) and provides a workable API to query the DOM. It’s just the testing part that we’re trying to replace here. So maybe we just have to hook up Jasmine with Casper, instead of letting Jasmine flirt with Phantom directly!
Behold: the package that was foretold in the prophecy. The hero, the one, the Keanu Reeves of NPM: mocha-casperjs
Let’s back up for a sec. Mocha? Mocha is a lot like Jasmine. It’s a testing framework that provides a way to write your tests in BDD style (describe(), it(), before(), etc) or TDD style (suite(), test(), etc). It also renders a very nice breakdown of your results, even with some comprehensive diffs to help you find out what went wrong when something fails.
Anyway, let’s get to an example:
describe('Google searching', function() {
before(function() {
casper.start('http://www.google.fr/')
})
it('should retrieve 10 or more results', function() {
casper.then(function() {
'Google'.should.matchTitle
'form[action="/search"]'.should.be.inDOM.and.be.visible
this.fill('form[action="/search"]', {
q: 'casperjs'
}, true)
})
})
})
I like this a lot. Bundle tests and spec out what your site should be doing. You can also nest the describe blocks and the output will reflect this structure.
Noticed the assertions? Those aren’t actually from casper, mocha or casper-mocha, that’s Chai. Better yet, it’s casper-chai. There are a lot of descriptive ways to assert things and it’ll make your tests pretty damn readable.
Grunt your heart out
We use Grunt for our frontend tasks, so something like grunt test seemed like a good fit. Well, if you think you have read about a lot of node-packages in this article, there’s a grunt-* package for every single one of them. grunt-mocha-casperjs did not work for reasons I can’t remember, so we ended up using grunt-exec. This turned out to be a better approach anyway, because all this had to run on both OS X and Windows (and I had to do some custom stuff to make that work).
This is how we do it
I’ve migrated the good ol’ test suite (which took a lot less time than I thought) and I really like this setup. I actually started to love Casper because of it! And the combination with Mocha is gold.
var utils = require(’./helpers/utils’);
describe(‘Agenda’, function() {
before(function() {
casper.start(utils.baseUrl + 'nl/agenda’).thenAssertStatus(200);
});
describe('datepicker', function() {
it('opens', function() {
casper.then(function() { '[data-role~=date-navigation-calendar]'.should.be.not.visible })
.wait(150)
.thenClick('[data-role~=date-navigation-value]')
.wait(150)
.then(function() { '[data-role~=date-navigation-calendar]'.should.be.visible })
});
it('closes', function() {
casper.then(function() { '[data-role~=date-navigation-calendar]'.should.be.visible })
.wait(150)
.thenClick('body')
.wait(150)
.then(function() { '[data-role~=date-navigation-calendar]'.should.be.not.visible })
});
it('navigates dates', function() {
var todaysUrl = casper.getCurrentUrl();
casper.thenClick('[data-role~=date-navigation-value]')
.thenClick('[data-handler=selectDay] a:not(.ui-state-active)')
.then(function() { this.getCurrentUrl().should.not.equal(todaysUrl) })
.then(function() { '[data-role~=date-navigation-calendar]'.should.be.not.visible })
});
});
describe('arrows’, function() {
it('navigates dates’, function() {
var todaysUrl = casper.getCurrentUrl();
casper.thenClick('.button-icon.button-forward')
.then(function() { this.getCurrentUrl().should.not.equal(todaysUrl) })
.thenClick('.button-icon.button-back')
.then(function() { this.getCurrentUrl().should.equal(todaysUrl) })
});
});
Yep, I know, the .wait() calls are nasty. They need to be there because of something I want to improve in the actual code. Testing already paid off :)
In retrospect
As much as I love this new way of testing, I’ve got a chip on my shoulder about last week. I can’t count the number of fancy called packages and accompanying documentation, issues, cognitive load and misconceptions. We ended up using casperjs, phantomjs, mocha, mocha-casperjs, chai, casper-chai, grunt and grunt-exec. Remember, this is only for testing and it’s just a fraction of what’s out there. It’s kinda insane if you think about it.
Also, the stack of readme documents I’ve read with lines like “just install this”, “simply compile that” and “the only thing you’ve got to do is build a spaceship” blows my mind. When someone is reading documentation it’s because he or she didn’t understand how something should work. It doesn’t help anyone to act like everything is simple. Sometimes, it just isn’t. Let’s quit this kind of language, alright?
Yeah, ok, so much for the depressing stuff. Happy testing, happy coding! :)