Integrating Javascript Unit Tests with Visual Studio – Wrapping Up

Welcome to the grand finale of this four part series. So far you’ve got your javascript unit tests, you’ve got them all nicely organized and indexed using the info from the previous post, now how do I get to see those results when I’m inside Visual Studio? For that last cherry on the pie we’re going to use a browser automation tool walled Watin and a data-driven unit test in C#. This was borrowed from another blog and tweaked to use the XML indexing trick.

First things first, setting up your data driven unit test. If you are not familiar, this is a unit test in MSTest that you can pass in a dataset have rerun against all the values in that dataset. Here, our “dataset” is going to be the results of our javascript unit tests. In the class initialize of our unit test we go out and download the xml data that holds the location of all our test pages. From there, we use Watin to visit each page and parse out the results of each test. Because our unit tests run when a browser hits the test page, this is all we need to do run our unit tests. One thing you may need to tweak is the Watin dll has needs a reference to Interop.SHDocVw.dll, if you get errors loading that DLL, check it’s properties in your project reference and make sure that Embed Interop is false and Copy Local is true

Then we’re going to add a single test file called JavaScriptTests.cs to our unit test project. In the class initialize we’re going to setup the code that downloads our xml test index and parses out each test page’s results

    private static string _baseUrl = "http://localhost:8000/Test/";
    private static IE _ie;

    public static void ClassInit(TestContext context)
        var xdoc = XDocument.Load(_baseUrl + "?notransform");
        _ie = new IE();

        var resultsDoc = new XDocument(new XElement("Rows"));
        foreach (var testPage in xdoc.Descendants("Row"))
            _ie.GoTo(_baseUrl + testPage.Descendants("TestPageUrl").First().Value);
            var results = _ie.ElementsWithTag("li").Filter(x => x.Parent.Id == "qunit-tests");
            var xResults = from r in results
                           select new XElement("Row",
                               new XElement("name", GetTestName(r)),
                               new XElement("result", r.ClassName),
                               new XElement("summary", r.OuterText));

Quick summary, we load the xml index into a XDocument, parse each url, use Watin to load the page, and scrape out the result for each test on the page. This is all added to a separate XDocument and saved in the local test run folder as JavascriptTests.xml. This will be the dataset that gets passed into our data driven unit test and ends up looking something like this:

    My First Tests_FullNameTest
    My First Tests: FullNameTest (0, 1, 1)Rerun
full name built properlyExpected: "Nick Olson"
    My First Tests_capitalizeTest
    My First Tests: capitalizeTest (0, 1, 1)Rerun
capitalize worksExpected: "OLSON"
    My Second Tests_FullNameTest
    My Second Tests: FullNameTest (0, 1, 1)Rerun
full name built properlyExpected: "Brian Olson"
    My Second Tests_FullNameFailTest
    My Second Tests: FullNameFailTest (1, 0, 1)Rerun
full name built properlyExpected: "Brian Olson"
Result: "Nick Olson"
Diff: "Brian "Nick  Olson" 

This is the aggregated test results for both of our test pages. The last little bit is a simple data driven unit test that takes this xml file and basically just asserts on the result element of each Row in the xml document

    [DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\JavascriptTests.xml", "Row", DataAccessMethod.Sequential), TestMethod]
    public void JavascriptTestRunner()
        var testName = TestContext.DataRow["name"].ToString();
        var testResult = TestContext.DataRow["result"].ToString();
        var summary = TestContext.DataRow["summary"].ToString();

        TestContext.WriteLine("Testing {0} - {1}", testName,testResult);
        if (testResult != "pass")
            Assert.Fail("{0} failed: {1}", testName,summary);

That’s it! You’re all done. Now when you run your tests in Visual Studio, it will run all the C# unit tests, including your new data driven javascript test and aggregate all these into your normal Test Results window. If you looked at my sample code you may have noticed that I have a failing test in javascript. If I run my tests in Visual Studio you can it sitting, failed, next to some C# unit tests, but it only tells me my data driven test failed, no other info

But if we click on that test it will open up a detail window and give us the results for every record in the data set for that data driven test, which is a lot nicer

Double clicking on that will give us even more detail yet

So there you go, test in the browser, test in Visual Studio it’s up to you. Generally I like to work on the html test page while I’m working and writing my tests and let the MSTest integration work for me on a build server or something else when the tests are run.

Here is the complete little sample app,

Integrating Javascript Unit Tests with Visual Studio – Testing your Javascript

First, let’s pause…the point of this post series is how to integrate your javascript unit tests with Visual Studio, not to teach you how to use the frameworks I’m discussing.  The rest of this post is going to assume you’re familiar with KnockoutJS and Qunit.

Knockout is a framework in javascript that let’s you create javascript viewmodels for your html page.  When done correctly, your view model simply maintains it’s own state and is blissfully unaware of the existence of any HTML elements. This is one of the death knells of your javascript unit tests.  When your javascript knows about html structure and elements it forces you to mock your view out when unit testing otherwise you can’t properly test it.  Knockout let’s you write clean testable javascript and move the glue between your code and the HTML DOM into a declaritive data binding syntax in the HTML.  Again, not here to teach you Knockout, much better places to learn that, like so:

Likewise, Qunit is the javascript unit testing framework used by jQuery.  It seems like the modern internet wouldn’t exist without jQuery, so we might as well run with that, even though there a number of javascript unit test frameworks out there.

To integrate our tests we’re going to need some code to tests and tests to test it first, so let’s start there.  I’ve stolen the following view model from the knockout tutorial site:

function AppViewModel() {
    this.firstName = ko.observable("Nick");
    this.lastName = ko.observable("Olson");

    this.fullName = ko.computed(function() {
        return this.firstName() + " " + this.lastName();
    }, this);

    this.capitalizeLastName = function() {
        var currentVal = this.lastName();        // Read the current value
        this.lastName(currentVal.toUpperCase()); // Write back a modified value

What should we unit test? Well, we’ve got the fullName property which has some logic, and the function to capitalize the last name, so let’s start there.

$(function () {
        module("My First Tests");

        test("FullNameTest", function () {
            var model = new AppViewModel();
            equal("Nick Olson",model.fullName(), "full name built properly");

        test("capitalizeTest", function()
            var model = new AppViewModel();
            equal("OLSON",model.lastName(), "capitalize works");

To see it all in action, check it out here:



Integrating Javascript Unit Tests with Visual Studio – Intro

I’ve been working on a project over the last few months that quickly evolved from a Silverlight project to an ASP .NET MVC3. If you’re shifting from Silverlight to MVC and want to maintain that rich client interaction, it means that you’re probably going to be writing a lot of javascript code.

In fact, we quickly realized that we were really writing a javascript client application with a .NET backend. This can be a scary proposition when it’s your first real foray into heavy javascript development. It seems like the first thing to go is the logical structuring and thought that you’d put into your code if this were a strongly typed language. Something about that <script/> tag that just makes you want use it and throw thousands of lines of javascript at it.

As a result, next to go are your unit tests if you had any to begin with.  I will confess, I’ve never been a huge unit tester.  But once you start working with dynamic languages and your warm fuzzy “Build Succeeded” blanket is taken away, I’ll reach for whatever comfort I can get.

The goal of this series of posts is to walkthrough what I did to unit test my javascript and get those test results into Visual Studio’s test output.  Although I may touch briefly on how to use the frameworks I’m talking about, the focus will be gluing them together.

That said here’s what we’ll be working with:

  • MVC3 w/ Razor
  • Qunit
  • KnockoutJS
  • Watin
  • C# Data-driven unit tests
Some of these ideas were cobbled, stolen and enhanced from other places, so if you can’t wait here is where I started off:

Simplify Unit Testing with IDisposable and the Using Statement

I’m sure I’m not the first to do this or something similar, but I’ll pat myself on the back all the same. Recently, I’ve been doing a lot of IoC and unit testing on some WCF services we’re working on. The services take implementations of our IRepository for data access which I’ve mocked out for testing purposes.

Most people are familiar with the using statement and it’s purpose to assure objects that implement IDisposable are properly cleaned up. A lot of people are also aware that you can use this interface and statement to do automatic scoping of some operations in a nice consistent manner. The first time I ever saw this type of pattern was as a shortcut to set the Cursor in a WinForms app to the WaitCursor and then back to the default once some long running operation was complete.

For Example:

using (CursorScope.SetCursor(this, Cursors.WaitCursor))
    // long running operation

Where the CursorScope class looks like this:

class CursorScope : IDisposable
    private Cursor _originalCursor;
    private Control _control;

    private CursorScope(Control control, Cursor newCursor)
        _control = control;
        _originalCursor = _control.Cursor;
        _control.Cursor = newCursor;

    public static CursorScope SetCursor(Control control, Cursor newCursor)
        return new CursorScope(control, newCursor);

    public void Dispose()
        _control.Cursor = _originalCursor;   

So as we enter the using statement, the cursor will be changed for the control, and as we leave the using statement and our object is “disposed” it will be set back to the original value regardless of any exceptions or errors that may occur between. This syntax and behavior is useful in a number situations. The most recent place I’ve been using this, as the title mentions is in some of my unit tests. To test out my services, I’ve built some mock repositories for the services to call. There are two scenarios where this has come in handy.

The first is when I’m trying to test the error and exception handling in my service. To get complete code coverage you need to make sure your tests also throw exceptions where you’re expecting them to be handled otherwise your catch block will never be tested. To support this, I added a boolean flag on my mock repository called ThrowException, when this is set to true, any operation you try to take on the repository with throw an exception I could use this in one of two ways, I could try to remember always setting the flag to true and then back to false when I’m done testing it. The problem there is depending on how you’re unit tests are run and set up, if you forget to set the flag back to false, it may cause your mock repository to throw exceptions in other unit tests. To help avoid this situation I made the flag private and added a ThrowException() method to the mock repository that returns an IDisposable.

private bool AllMethodsThrowExceptions { get; set; }

public IDisposable ThrowExceptions()
    AllMethodsThrowExceptions = true;
    return new ExceptionDisposable(this);

class ExceptionDisposable : IDisposable
    public MockRepository Repository { get; set; }
    public ExceptionDisposable(MockRepository repo)
        Repository = repo;

    public void Dispose()
        Repository.AllMethodsThrowExceptions = false;    

And then in our unit test we can test the exception path like so:

public void GetUserExceptionTest()
    using (_mockUserRepo.ThrowExceptions())
        var response = _service.GetUser(new GetUserRequest());
        Assert.AreEqual(1, response.ErrorMessages.Count);

Which will cause our exceptions to be thrown so we can test to make sure our code handles them properly. Then, when we’re done with that part of our unit testing, it will reset the flag back so any other tests that run subsequently, everything will work correctly.

The other area where this comes in useful is because our mock repository uses in-memory data, and as we manipulate this could throw off our tests as well, so we implement the same pattern, but this time instead of setting a flag, we copy our repository’s data and reset it once the test is complete.

public IDisposable ChangingData()
    return new ChangingDataDisposable(this);

public class ChangingDataDisposable : IDisposable
    public MockRepository Repository { get; set; }
    private List OriginalData;
    public ChangingDataDisposable(MockRepository repo)
        Repository = repo;
        OriginalData = CloneData(Repository.Data);

    private T CloneData(T data)
        if (!typeof(T).IsSerializable)
            throw new ArgumentException("Must be serializable.", "data");

        if (Object.ReferenceEquals(data, null))
            return default(T);

        var s = new DataContractSerializer(typeof(T));
        using (var stream = new MemoryStream())
            s.WriteObject(stream, data);
            stream.Seek(0, SeekOrigin.Begin);
            return (T)s.ReadObject(stream) ;

    public void Dispose()
        Repository.Data = OriginalData;

Here when you call ChangingData() I clone the data using serialization, and again, when you’ve disposed the object it resets the data.

public void UpdateUserTest()
    using (_mockUserRepo.ChangingData()) 
        var user = _mockUserRepo.Data.FirstOrDefault(i => i.Id == 1);
        Assert.AreEqual("Nick", user.FirstName);

        user.FirstName = "Brian";

        var request = new UpdateUserRequest();
        request.User = user;

        var response = _service.UpdateUser(request);
        Assert.AreEqual("Brian", user.FirstName);
        Assert.AreEqual(1, response.EntityId);

And all the data is reset back to the original values and ready for the next test to work off it.