Running TestSwarm in WebMatrix

For last few weeks I have been exploring a subject of JavaScript Unit Testing. I looked at testing frameworks, runners, IDE and CI integration. One of the last items on my list of topics to explore was to find production ready solutions for continuous integration scenarios. Because I work in Microsoft .NET environment CI integration seemed to be a rather uncharted territory. This post describes my experience with getting TestSwarm to run in windows centric settings.

The next two paragraphs describe what TestSwarm is and why it might be an important to learn about it, a rationale behind the whole effort.

TestSwarm, John Resig’s open source project for supporting distributed continuous integration testing for JavaScript, while still in an alpha development stage, generates a lot of interest among web UI developers and project managers. As UI development shifts towards client side technologies, particularly JavaScript, the need for the same level of support that has been in place, for many years already, for server side technologies is now expected for client side development as well.

JavaScript unit testing has been used for quite some time now. There are more than a few established unit testing frameworks and IDE support for some of them is either available or in works. Missing though is support for running JavaScript unit tests in continuous integration scenarios. Before the TestSwarm the most popular option for handling CI scenarios seemed to be using JSTesDriver from Google. One inconvenience of using JSTestDriver is the need for developing an adapter to integrate a unit testing framework and JSTestDriver runner. When a framework and a runner evolve, quite a bit of maintenance overhead is being added for keeping them both in synch (updating an adapter). In contrast, TestSwarm integration is framework independent. The only moving part is a script that needs to understand test results. Thus, it seems that TestSwarm should be considered a serious player in the area of JavaScript Unit Testing and Continuous Integration.

Back to technology then, these are the parts that will be needed:

• TestSwarm
• WebMatrix
• WordPress it’s an easy way of getting PHP and MySQL on windows machine and you will have some PHP code to look at or compare if needed.
• MySQL Workbench if you need GUI for MySQL management tasks.
• IIS URL Rewrite Module to get import Apache .htaccess url rewriting rules into IIS Express .
• QUnitfor creating demo JavaScript unit testing project.

Getting it all working:

Step 1.

In WebMatrix open directory with TestSwarm files downloaded from git repository. Read TestSwarm configuration instructions either in ReadMefile or in Wiki. I used WebMatrix to run the TestSwarm which means that IIS Express was used as a web server and that TestSwarm ran in the root of the site (on random port).
In testswarm.json I needed to modify contextpath“: “/testswarm” to contextpath“: “/“. Otherwise, images and css would not load correctly. TestSwarm application depends on url rewriting rules in .htaccess file. Use IIS URL Rewrite extension to Import those rules to IIS. Just copy the results from the Import screen and add them to your IIS Express configuration file. Delete this: RewriteEngine On RewriteBase / from Rewrite Rules box to make the error seen in Converted Rules box disappear.

Import Rewrite Rules

Add the ruele to IIS Express configuration file (applicationhost.config) which should be located in C:\Users\yourUserName\Documents\IISExpress\config folder. The section shown below should be added at the end of that configuration file.

<location path="testswarm">
        <system.webServer>

<rewrite>
            <rules>
                <rule name="Imported Rule 1" stopProcessing="true">
                    <match url="^" ignoreCase="false" />
                    <conditions logicalGrouping="MatchAny">
                        <add input="{REQUEST_FILENAME}" matchType="IsFile" ignoreCase="false" />
                        <add input="{REQUEST_FILENAME}" matchType="IsDirectory" ignoreCase="false" />
                    </conditions>
                    <action type="None" />
                </rule>
                <rule name="Imported Rule 2">
                    <match url="^(.*)/$" ignoreCase="false" />
                    <action type="Redirect" url="{R:1}" appendQueryString="true" redirectType="Permanent" />
                </rule>
                <rule name="Imported Rule 3">
                    <match url="^([a-z]*)$" ignoreCase="false" />
                    <action type="Rewrite" url="index.php?action={R:1}" appendQueryString="true" />
                </rule>
                <rule name="Imported Rule 4">
                    <match url="^([a-z]*)/(.*)$" ignoreCase="false" />
                    <action type="Rewrite" url="index.php?action={R:1}&amp;item={R:2}" appendQueryString="true" />
                </rule>
            </rules>
        </rewrite>

        </system.webServer>
    </location>

At this point TestSwarm should be looking like working. It should load pages and images. But, you still would not be able to add jobs or run any tests.

Step 2.

Go to http://localhost:51336/signup to create a user account (just notice 51336 is a random port number, yours will be different).
Your account should get created and when you open MySQL Workbench and navigate to a table you should see an entry in a users table.
The important thing to note about the user data is to copy the value from “auth” column. It will be needed for crating jobs. If you do not see a value of that field but just a “BLOB” word, go to Edit –> Preferences an open SQL Editor Tab and check “Treat BINARY/VARBINARY as nonbinary character string“.

SQL Editor

Now, you should be ready for adding jobs.

Step 3.
Before you can actually add a job you need to have a JavaScript code to test. So, create a web application and add a JavaScript file to it. Next, decide which JavaScript unit testing framework you are going to use. My recommendation is to use QUnit as it’s one with greatest support in TestSwarm. For start, you just want to get things running. Next, create a unit test for your script. Verify that the tests passes.

Copy a inject.js from a js folder in TestSwarm to your demo application and make sure to add a link to it in a QUnit test html file.

The head tag in a test html file should look like this:

<head>
    <meta http-equiv="cache-control" content="no-cache">
    <meta http-equiv="X-UA-Compatible" content="IE=Edge">

    <title>QUnit Testing</title>
    <link href="qunit.css" rel="stylesheet" type="text/css" />
    <script src="qunit.js"></script>
    <script src="inject.js"></script>
</head>

You are still not ready, at least very likely, for adding jobs though. Go to the next step to see what else might be missing.

Step 4.

On my installation I ran into a problem. Adding jobs failed because a results column in run_client table would not allow NULL and somehow a NULL was being inserted. Results column is a TEXT field and a default value of empty string cannot be set on it. It’s probably an issue with MySQL version. So, I modified the column and made it nullable.

Results Column

Step 4.

Go to http://localhost:51336/addjob page and enter the required information. For start, I would suggest to check desktop browsers option, it will be easier to analyze the data changes in a database in a response to TestSwarm UI actions. Also, enter only one run. It will be easier to troubleshoot if things do not go right. In the URL field for the run enter the URL of the test html page from your demo web app.

All should work now and you should see this confirmation message:

Add Job

Step 5.

You ready to add agents (browsers) for running jobs now. Navigate to http://localhost:51336/run/testuser2 (testuser2 needs to be a username that you created in step 2) in your choice of the browser and you should see the message that a job run. Do the same for other browsers and you should similar messages.

Now if you go to http://localhost:51336/job/6 (6 is my job number, yours might be different of course, it’s just RESTful Url) you will see a page looking like this:

Job Status

In this particular case you can see that runs in IE, Opera and Safari succeeded and that other agents are probably not available or have not been run yet. You can click on a note icon to see the details of the run. The details will be just html form a QUnit test page, see example below.

Run Details

Read this if things still do not work

If it looks like you still cannot run jobs. Here is another hint, something that I did to make attaching clients work on my system. I had a problem with a call to gzencode (line 57 in SeverunAction.php file.

ServerunAction.php page

I modified that line (57) to be like 58.

57. // $results = gzencode( $request->getVal( "results", "" ) );
58. $results = $request->getVal( "results", "" );

and to make things symmetrical also had to modify (comment out) a line 37

36. header( "Content-Type: text/html; charset=utf-8" );
37. // header( "Content-Encoding: gzip" );

in ResultsPage.php

Results Page

In the end all troubles with a NULL in a Results table and trouble with gzip encoding might have been a LINUX vs Windows issue (I did not investigate a root cause, just tried to get things running).

Summary
The goal of this blog post was to show how to setup a TestSwarm with WebMatrix. Setting up TestSwarm in CI scenario would require more effort but steps would be very similar. Also, one easy step to take would be to use SQL Express in place of MySQL. Converting PHP code to MVC Razor syntax would be probably more difficult but also very likely some fun.