e2e testing with AngularDart.

So this weekend I spent a grueling 12 hours trying to get an e2e test harness going for a personal project of mine I built in AngularDart.

I will spare you the obscenity-laden recap and share with you the minimal amount of code needed to get an e2e testing solution going using NodeJS.

Before you ask "why not Protractor?" let me explain: it doesn't work with AngularDart. I looked into the codebase and it relies on AngularJS's internals -- I had always assumed that AngularJS just emitted DOM events as the integration point for Protractor, but it turns out there's some private services in there like $browser that are accessed directly. Since I'm not using AngularJS, that immediately removes Protractor from the running.

If you don't know what e2e is, it stands for end-to-end testing. Basically, you can unit test the hell out of your codebase, but at some point you have to make sure that based on user interaction everything works in harmony. It catches problems in your markup like not sending the right message when a user clicks an element, and other integration issues.

I'm going to be sampling files straight from my repository, so the usual caveats apply: I am a foul-mouthed asshole and have no regrets about that, I might eat your baby, etc.

1.) First off, let's get some necessary packages installed:


jasmine-node: our test framework. This allows you to use Jasmine in Node, which is great, because Jasmine is great.

coffee-script: CoffeeScript makes for very fluent, very readable test code. Your mileage my vary, but I use CoffeeScript where-ever I'd normally use JavaScript.

webdriverjs: this is our API for interacting with Selenium. Despite the name, it is not the official Selenium WebDriverJS package; it is a more fluent, node-like API. It's got some minor issues: failures from Selenium don't bubble up, so you have to check the Selenium output to see what went wrong, and why.

selenium-standalone: this package gets selenium server and has a helper executable that starts running the Selenium Standalone Server, which is required. It comes with Chrome driver.

gulp: task runner. It's simple but undocumented; I will probably be migrating to Grunt in the near future. Dart has 'Hop' but that seems just as bare documentation-wise, and it's also more complicated to get going with.

Running `npm install` (and optionally `npm install -g` so the executables packaged with those packages are available globally) gets us ready.

2.) Now let's setup our Gulpfile. Typing 'jasmine-node spec/' gets pretty tiresome after awhile.




Nothing too complicated, just some tasks to run tests. I prefer CoffeeScript, so Gulpfile.js just executes the CoffeeScript version of a Gulpfile.

Just typing `gulp` in the terminal will start executing all of our specs. I keep my specs in the subdirectory 'spec/' instead of 'test/' because why not.

Now, since this ended up being a total nightmare to get going -- I went through three or four more high-level APIs before I discovered what exactly was going wrong -- I ended up writing some "sanity checks." Here they are:




Typing 'gulp selenium' in one terminal, and then 'gulp' in another should result in 2 passing tests.

Great! Now let's setup a basic config.coffee file to store our settings for an actual, but extremely simple, e2e test for our app. I'm not promising anything about the accuracy of the comments in this file.



The timeout is super large because it takes `pub serve` forever and a day to compile an AngularDart application, and I'm still not sure if that has any influence on how long webdriverjs / Selenium waits until it tries to interact with a page.

Now I wrote a small 'sanity check' test for the application itself, which basically ensured that the page loads properly:



This test just ensure that the page actually loads in the browser.

You'll need to run `pub serve` first. The first time I ran this test it took about 60 seconds for dart2js to finish compiling on a Macbook Pro.



Here is the test that gave me the most trouble: it's where I exercised the Selenium drivers beyond a simple sanity check and encountered the dozens of problems that had me pulling my hair out.

Chrome wouldn't find any elements on the page, Safari worked fine (I discovered by accident) except that the webdriver won't interact with file inputs properly, Firefox crapped out with a bug in the shadow-dom polyfill, and PhantomJS was just... I don't know, there were too many errors with PhantomJS for me to even bother with.

In the end, to get this test passing, I choose Firefox as the browser and commented out the shadow dom poly fill ("shadow_dim.min.js" in your AngularDart index.html file, probably), which caused some rendering errors but otherwise allowed the functional parts of the test to pass... but only in Firefox. I also removed some expected user behaviors and jumped straight to populating the file input.

You'll notice I have some helper jQuery statements: normally in your stylish app, the file input is hidden. You trigger it when the user clicks on another element, like a button labeled "Upload File."

However, you can't do that in Selenium. Since it's running a real browser, clicking on "Upload File" blocks the process since the OS then presents its file picker dialog. So what you have to do is use Selenium's API (webdriverjs calls this 'chooseFile'), which then simulates the process of attaching a file via a file input.

Of course, since your stylish application has hidden the default ugly file input, there's nothing for Selenium to "click" on, so the `showInputs` makes the element visible for testing purposes.

Is the `selectFile` script necessary? I don't know. It was there when I finally got everything working, and after 12 hours of hacking away trying to get a basic e2e test going it was 3AM and I wasn't about to try messing with it right then.

Anyway, that's what I learned about e2e testing AngulartDart applications.