Selenium as part of the build

We are using Selenium as our automated user acceptance test runner at work. I wanted to include it in our continuous integration script. I couldn't find any information online as to how best do this, so here are my solution:-

We are using CruiseControl.Net with NAnt on our continuous integration server.

Selenium is an automated user acceptance test (UAT) framework written in JavaScript. It interprets tests (in the form of HTML) into commands that it uses to drive and validate a website that it has open in an iframe. To run selenium you open up a browser pointing at your selenium test runner page. Our URL is something like this (but on one line obviously)...

http://localhost/selenium/TestRunner.html?
test=http://localhost/acceptanceTests/TestSuite.aspx&
auto=true&
resultsUrl=http://localhost/acceptanceTests/PostResults.aspx

Parameters

• 'test' - specifies the test suite to run
• 'auto=true' - run as soon as the page loads
• 'resultsUrl' - tells selenium a page it can post the results to
  

The problems I faced were as follows:

• Run the selenium test from NAnt
  
• Detect when the tests have finished running
• Cause the build to fail if any tests failed
  

I chose to write a custom NAnt task to help achieve this. My NAnt task fires up Internet Explorer and tells it to navigate to the aforementioned URL. It then listens to the 'load complete' events in the browser until it gets told the final 'PostResults.aspx' page has been loaded.
The PostResults.aspx is a simple ASP.Net page that saves the passed in test results out to a file in a well known location. The NAnt task then checks this file to see if any tests failed. If they did then it throws a Build Failure exception. If not it just completes successfully.

This worked really well.

Here is an excerpt from our NAnt script:


<target name="--runSelenium">
<trycatch>
<try>
<selenium
seleniumUrl="http://localhost/selenium/TestRunner.html"
testsuiteUrl="http://localhost/acceptanceTests/TestSuite.aspx"
resultsUrl="http://localhost/acceptanceTests/PostResults.aspx"
errorLogFile="C:\temp\Log\error-results.html"/>
</try>
<finally>
<exec program="taskkill" commandline='/FI "windowtitle eq Selenium*"' failonerror="false" />
</finally>
</trycatch>
</target>



I have been thinking of making the 'PostResults.aspx' page take the 'errorlogfile' uri as a parameter. Currently this filename is duplication. I would also like to allow the NAnt file to specify a timeout for the selenium process. I could also move the 'taskkill' logic into the NAnt task.

Here, you have a go:

• Download the project from here .. (Update: Sorry this file was lost a couple of server moves ago. If anyone fancies giving writing it a go, let me know so I can reference it. It wasn't hard to write.)


• Drop the Selenium.Tasks.dll into your NAnt/bin folder.


• Put the above NAnt code in your NAnt script..


• Write yourself a PostResults page (check out the selenium site for details)
• Away you go.
  

When I get time I'll post about how we are generating out selenium tests from C#, which is allowing us to remove duplication and make really succinct tests.