Unit testing with Moq - Returning different values from multiple calls to the same method in a loop, and then verifying multiple calls to another method.

I was doing some TDD the other day in a C# .net service and found myself wanting to write a loop that called a couple of methods and acted on them in different ways depending on the return values. I needed to mock the calls so they returned different data depending on how many times the methods were invoked.

This is the method under test that I ended up with after doing the TDD cycle. (I've stripped out all the exception handling and logging to keep this example clear)

using JourneyHeader.Domain.Entities;
namespace journeyMigration
{
  public class JourneyMigrator
  {
    private readonly JourneySource _journeySource;
    private readonly JourneyDestination _journeyDestination;

    public JourneyMigrator(JourneySource journeySource, JourneyDestination journeyDestination)
    {
      _journeySource = journeySource;
      _journeyDestination = journeyDestination;
    }

    public int JourneysProcessed { get; private set; }
    public int JourneysFailedProcessing { get; private set; }
    public JourneyHeader LastJourneyProcessed { get; private set; }

    public void Start()
    {
      JourneyHeader journeyHeader = _journeySource.GetNextJourney();
      while (journeyHeader != null)
      {
        _journeyDestination.Upload(journeyHeader);
        JourneysProcessed++;               
        LastJourneyProcessed = journeyHeader;   
        journeyHeader = _journeySource.GetNextJourney();
      }
    }
  }
}

The lines we really care about are 21 through 28. Notice that GetNextJourney() is getting called many times depending on the value returned last time. Also see that on line 24 the upload() method is called inside the loop, I want to verify I was passing the correct values through to it.

This is one of the tests I came up with whilst writing this code, its a good example of the 2 things I wanted to demonstrate here.

using System;
using System.Collections.Generic;
using FluentAssertions;
using JourneyHeader.Domain.Entities;
using journeyMigration;
using Moq;
using NUnit.Framework;

namespace journeyMigrationTests
{
  [TestFixture]
  public class JourneyMigratorTests
  {
    [Test]
    public void ShouldProcessTwoJourneys()
    {
      var journeyHeader1 = new JourneyHeader();
      var journeyHeader2 = new JourneyHeader();
      var queue = new Queue<JourneyHeader>(new [] {journeyHeader1, journeyHeader2, null});
      _journeyHeaderSource.Setup(x => x.GetNextJourney()).Returns(queue.Dequeue);
 
      _journeyMigrator.Start();
 
      _journeyMigrator.JourneysProcessed.Should().Be(2);
      _journeyMigrator.LastJourneyProcessed.Should().Be(journeyHeader2);
      _journeyDestination.Verify(x => x.Upload(journeyHeader1), Times.Exactly(1));
      _journeyDestination.Verify(x => x.Upload(journeyHeader2), Times.Exactly(1));
    }
  }
}

The interesting lines here are 16 through 20 where im setting up a queue that is used to return the values in the prescribed order. You have to do it this way because in Moq you cant do multiple Setups on a given classes method, the last one to be defined will win. In the following example journeyHeader2 is always returned.
_journeyHeaderSource.Setup(x => x.GetNextJourney()).Returns(journeyHeader1);
_journeyHeaderSource.Setup(x => x.GetNextJourney()).Returns(journeyHeader2);

The Returns method takes a value as above or a function that is run every time a return value is required, you could write it like this.
_journeyHeaderSource.Setup(x => x.GetNextJourney()).Returns(() => queue.Dequeue());
But the one I've used on line 20 is a lot clearer.

Finally lines 26 and 27 verify the method Upload was called correctly, once with the first journeyHeader and once with the second.

Appendix
Moq - A popular and friendly mocking framework for .NET

Visualising the Thoughtworks Go pipline using Cradiator, a build information radiator/monitor

You know the old adage, 'out of sight, out of mind'? Like it or not, sometimes the state of the build on CI is forgotten about, and if you can't see the current state without going looking for it, it can stay red for a few days before it's noticed by someone.

I've always been a big fan of information radiators, build monitors, graphs and stats that are in peoples faces, and I just wanted to share our current solution to the whole 'who cares if the CI server is not Green' problem.

We use Go (the CI server from Thoughtworks) for our build and deployment pipeline, which is great. Although it doesn't ship with a build monitor that can be installed on a machine to show the state of the build, Thoughtworks do expose an API that allows you to build your own, for example on http://Server:Port/go/cctray.xml

We found an old-ish project called Cradiator that works with cruse control (remember that old build server? it turns out that Go has the same API, all be it on a slightly different URL). The problem was that we have secured our instance of Go so that you need to be logged in to access it. This caused problems with Cradiator and so we forked it and added the ability to set your own credentials in the config. The fork can be found here.

Below are a couple of screen shots showing the Go build server and also the corresponding Cradiator screen.

Our Go pipeline for this small part of the overall system

The Cradiator build monitor screen

As you can see every stage within a Go pipeline has a corresponding line in Cradiator. To achieve this, we use the following filter in the Cradiator config file: project-regex="^.*::.*::.*"

There is a robotic voice that announces who broke what, with some good catastrophic sound effects to accompany it. It's doing a great job of focusing people on fixing things if/when they go red.

Ever since we have started using this, the average time of a red build to a check in fixing it is less than an hour. Visibility for the win.

References:

https://github.com/DamianStanger/Cradiator

https://github.com/PandaWood/Cradiator

https://cradiator.codeplex.com/

Continuous Integration performance testing. An easily customisable solution.

Using JMeter to profile the performance of your web application and visualise performance trends, all within the CI pipeline.

The full solution as outlined here can be found on my GitHub repository at https://github.com/DamianStanger/CIPerformance

Introduction

Most companies care about the performance of their web sites/web apps but often the testing of this performance is left till the last minute with the hope that the devs will have been doing good job writing well performant code for the last x months whilst developing. I don’t know why this is often the way? If performance really is a major Non Functional Requirements (NFR) then you have to test your performance as you go, you can’t leave this until the last moment just before deployment / live and then when you find that performance is not good enough just try and hack in quick fixes. This is just not good enough, you can’t just hack in performance after the fact, it can take a substantial change to the design (to do it well).

On our team we have been performance profiling each important page of our app since month 1 of the development process, we are now live and are working towards the 4th major release. I (and the team) and have found our continuous performance testing invaluable. Here is the Performance graph as it stood a few weeks ago:

The process outlined below is not a method for stress testing your app. It’s not designed to calculate the load that can be applied, instead it's used to see the trend in the performance of the app. Has a recent check-in caused the home page perform like a dog? Any N+1 DB selects or recursive functions causing trouble? It’s a method of getting quick feedback within the CI pipeline minutes after a change is checked in.

The process

1. When we check in, our CI box (TeamCity) runs the build (including javascript tests, unit tests, integration tests, functional tests, acceptance tests), if all this is successful then the performance tests are kicked off.
2. Teardown the DB and restore a new copy (so we always have the same data for every run, this DB has a decent amount of data in it simulating the data you have in live in terms of volume and content).
3. Kick the web apps to prepare them for the performance tests, this ensures IIS has started up, and the in memory caches primed.
4. Run the JMeter scripts.
a. There are numerous scripts which simulate load generated by different categories of user. For example a logged out user will have a different performance profile to a fully subscribed user.
b. We run all the scripts in serial as we want to see the performance profiles of each type of user on each different site we run.
5. The results from each run are processed by a powershell script which extracts the data from the JMeter log files (jtl) and writes the results into a sql server database (DB). There is one record per page per test run.
6. We have a custom MVC app that pulls this data from the DB (using dapper) and displays it to the team on a common monitor (using JSON and RGraph) that is always updating. We see instantly after we have checked in if we have affected performance, good or bad. We could break the build if we wanted but decided this was a step too far as sometimes it can be a day or two to fix any poorly performing aspect of the site.

A stripped down version is avaliable on my GitHub account, Running the powershell script a few times and then running the mvc app you should see something like the following:

The juicy bits (interesting bits of code and descriptions)

Powershell script (runTest.ps1)

• Calling out to JMeter from powershell on line 112
& $jmeter -n -t $test_plan -l $test_results -j $test_log

• Parse JMeter results on line 133
[System.Xml.XmlDocument] $results = new-object System.Xml.XmlDocument
$results.load($file)
$samples = $results.selectnodes("/testResults/httpSample | /testResults/sample/httpSample")

Then iterate all the samples and record all the page times and errors

• Write results to DB on line 171
$conn = New-Object System.Data.SqlClient.SqlConnection($connection_string)
$conn.Open()
foreach($pagestat in $page_statistics.GetEnumerator())
{
    $cmd = $conn.CreateCommand()
    $name = $pagestat.Name
    $stats = $pagestat.Value
    $cmd.CommandText = "INSERT Results VALUES ('$start_date_time', '$($name)',
    $($stats.AverageTime()), $($stats.Max()), $($stats.Min()), $($stats.NumberOfHits()),
    $($stats.NumberOfErrors()), $test_plan)"
    $cmd.ExecuteNonQuery()
}

JMeter scripts

You can look up JMeter yourself to find suitable examples of this. My project posted here just has a very simple demo script which hits Google and Bing over and over. You can replace this with any JMeter script you like. The DB and the web app are page and site agnostic so it should be easy to replace with your own, and it will pick up your data and just work.
I recommend testing all the critical pages in your app, but I find the graphs get too busy with more than 10 different lines (pages) on them. If you want to test more stuff just add more scripts and graphs rather than have loads of lines on one graph.
The generic solution given here has two scripts but you can actually have as many as you like.Two would be a good choice if you had a public facing site and an editor admin site which both have different performance profiles and pages. But in the end it's up to you to be creative in the use of your scripts and test what really needs testing.

The results DB

The DB is really simple. It consists of just one table which stores a record per page per test run. This DB needs creating before you run the script for the first time. The file Database.sql will create it for you in SQL server.

The MVC app

Data layer, Dapper

Using dapper (a micro ORM installed through nuget) to get the daily results is done in the resultsRepository class:

var sqlConnection = new SqlConnection("Data Source=(local); Initial Catalog=PerformanceResults; Integrated Security=SSPI");
sqlConnection.Open();
var enumerable = sqlConnection.Query(@"
SELECT Url, AVG(AverageTime) As AverageTime, CAST(RunDate as date) as RunDate FROM Results
    WHERE TestPlan = @TestPlan
    GROUP BY CAST(RunDate as date), Url
    ORDER BY CAST(RunDate as date), Url", new { TestPlan = testPlan });
sqlConnection.Close();
return enumerable;


The view, JSON and RGraph

In this sample code there are four different graphs on the page, two for Google (test plan 1), and two for Bing (test plan 2). Heartbeat data shows a data point for every performance run. It shows you instantly if there has been a bad performance run. This shows all the runs over the last two weeks. The Daily Averages show a data point per day for all the performance data on the DB.
There are four canvases that contain the graphs, these graphs are all drawn using RGraph from some JSON data populated from the data pulled off the DB. It’s the javascript function configureGraph that does this work with RGraph, for details of how to use RGraph see the appendix.
The JSON data is created from the model using LINQ in the view as such:
dailyData: [@String.Join(",", Model.Daily.Select(x => "[" + String.Join(",", x.Results.Select(y => y.AverageTimeSeconds).ToList()) + "]"))],

This will create something like the following depending on the data in your DB:
dailyData: [[4.6,5.1],[1.9,2.2],[4.0,3.9],[9.0,9.0]],
Where the inner numbers are the data points of the individual lines. So the data above is four lines each with two data points each.
Customising things for your own purposes

Customisation

So you would like to customise this whole process for your own purposes? Here are the very simple steps:

1. Edit the function CreateAllTestDefiniotions in RunTest.ps1 to add in any JMeter scripts that you want to run as new TestPlanDefinitions.
2. Change or add to the JMeter scripts (.jmx) to exercise the sites and pages that you want to test.
3. Add the plan definitions to the method CreateAllPlanDefinitions of the class PlanDefinition in the performance stats solution. This is all you need to edit for the web interface to display all your test plans. The graphs will automatically pick up the page names that have been put into the configured JMeter scripts.
4. Optionally change the yMax of each graph so that you can more easily see the performance lines to a scale that suits your performance results.

Conclusion

We as a team have found this set up very useful. It has highlighted many issues to us including: n+1 select issues, combres configuration problems, and all number of issues with business logic usually with enumerations or recursive functions.
When set up so that the page refreshes every minute, it does a really good job. It has been a constant reminder to the team to make sure they are doing a good job with regard to the NFR which is performance.

A note on live performance/ stress testing

Live performance testing is a very different beast altogether, the objective of which is to see how the system as a whole reacts under stress: To determine the maximum number of page requests that can be served simultaneously. This is different to the CI performance tests outlined above. These tests run on a dev box and are only useful as a relative measure to see how page responsiveness is changing as new functionality is added.

Appendix

JMeter - https://jmeter.apache.org/
Dapper – Installed from Nuget
RGraph - http://www.rgraph.net/
GitHub - https://github.com/DamianStanger/CIPerformance
VS2012 - https://www.microsoft.com/visualstudio/11/en-us

A PowerShell script to count your lines of source code

We have been thinking about code quality and metrics of late and since im also learning more powershell decided to write a little script to do that for me. It basically finds all the code files in the project directories and counts lines and files:

Here it is:

$files = Get-ChildItem . -Recurse | `
    where-Object {$_.Name -match "^.+\.cs$"}
$processedfiles = @();
$totalLines = 0;
foreach ($x in $files)
{
    $name= $x.Name;
    $lines= (Get-Content ($x.Fullname) | `
        Measure-Object –Line ).Lines;
    $object = New-Object Object;
    $object | Add-Member -MemberType noteproperty `
        -name Name -value $name;
    $object | Add-Member -MemberType noteproperty `
        -name Lines -value $lines;
    $processedfiles += $object;
    $totalLines += $lines;
}
$processedfiles | Where-Object {$_.Lines -gt 100} | `
    sort-object -property Lines -Descending
Write-Host ... ... ... ... ...
Write-Host Total Lines $totalLines In Files $processedfiles.count


Line 00: Will get all the .cs files from the current working folder and below.
Line 07: Uses the measure-object cmdlet to get the number of lines in the current file being processed.
Line 09: Creates an object, lines 10 and 12 dynamically adds properties to that object for the file name and the line count.
Line 11: Adds the new object to the end of the array of processed files.
Line 17: Selects all the files from the array where the line count is greater than 100 (an arbitary amount, i only care about files longer than roughly 2 screens worth of text), Then print them out in descending order of line count.

My results:
our current project has a total of 154068 lines of code in .cs files.
2559 .cs files of which 312 files have a line count greater than 100 lines.
16 files are over 400 lines in length, but none of those were in the main product (All the worst classes are test classes and helpers which are not production code).

I also wondered about the state of my views:
320 .cshtml files a total of 10958 lines, the vast majority are less than 100 and only 6 over 150.

model dynamic causing unhandled exception

We are using ASP.MVC 3 with Razor views on one of the projects in currently working on. MVC3 uses C# 4.0 and so we get access to the dynamic types, this is really convenient when creating partial views, we can create a model in one partial and pass the new dynamic model into the partial, no need for strongly typed view models everywhere.

But we have found that dynamic models can hide other errors with very time consuming consequence's.
This is the partial "_topics" that was causing us problems:

@model dynamic

<ul>
    @foreach (var topic in Model.Topics)
    {
        <li id="@topic.Id" class="@( Model.SelectedItems.Contains(topic.Id) ?  "selected" : string.Empty ) ">
            <label>@topic.Name</label>
            @if (topic.HasChildren)
        {
                @Html.Partial("_Topics", new { Topics = topic.Children, SelectedItems = Model.SelectedItems });
        }
        </li>
    }
</ul>

as you can see _topics is called recursively.

We made some changes to other areas of our project and got this error: (sorry for the very long stack trace, but i wanted to be complete)

Application information: 
    Application domain: /LM/W3SVC/3/ROOT-3-129566639908092111 
    Trust level: Full 
    Application Virtual Path: / 
    Application Path: C:\projects\Foo\Bar\Src\Editor\ 
    Machine name: AAA20711 
 
Process information: 
    Process ID: 5928 
    Process name: w3wp.exe 
    Account name: IIS APPPOOL\Editor 
 
Exception information: 
    Exception type: RuntimeBinderException 
    Exception message: 'object' does not contain a definition for 'Topics'
   at CallSite.Target(Closure , CallSite , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at ASP._Page_Views_Shared__Topics_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_Topics.cshtml:line 4
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model)
   at ASP._Page_Views_Shared__ArticleTabs_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_ArticleTabs.cshtml:line 42
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName)
   at ASP._Page_Views_ArticleCreate_Create_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\ArticleCreate\Create.cshtml:line 23
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.StartPage.RunPage()
   at System.Web.WebPages.StartPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResult(ControllerContext controllerContext, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
   at System.Web.Mvc.Controller.ExecuteCore()
   at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
   at System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
   at System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f)
   at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Request information: 
    Request URL: http://editor/article/create 
    Request path: /article/create 
    User host address: 127.0.0.1 
    User: Foo 
    Is authenticated: True 
    Authentication Type: Forms 
    Thread account name: IIS APPPOOL\Editor 
 
Thread information: 
    Thread ID: 9 
    Thread account name: IIS APPPOOL\Editor 
    Is impersonating: False 
    Stack trace:    at CallSite.Target(Closure , CallSite , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at ASP._Page_Views_Shared__Topics_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_Topics.cshtml:line 4
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model)
   at ASP._Page_Views_Shared__ArticleTabs_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_ArticleTabs.cshtml:line 42
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName)
   at ASP._Page_Views_ArticleCreate_Create_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\ArticleCreate\Create.cshtml:line 23
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.StartPage.RunPage()
   at System.Web.WebPages.StartPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResult(ControllerContext controllerContext, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
   at System.Web.Mvc.Controller.ExecuteCore()
   at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
   at System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
   at System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f)
   at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Looks like there was a problem with the dynamic model, line 4 is the for each loop on model.topics. But debugging this code showed there was no problem with the model, it was fully populated with lots of topics and the debugger could see them and access them, so what gives?

Well after investigating it turned out another partial that the razor view uses had a compilation error in it, we only found this after removing the _topics partial. Very strange that a compilation error in another partial would cause the error we got.

To stop this ever happening again we made the project pre compile the views along with the rest of the web project but putting MvcBuildViews=true the .csproj

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
  ....
    <MvcBuildViews>true</MvcBuildViews>

Now we get the real error at compile time, not an error in the dynamic model code at run time, much nicer.

Linq to Sql many to many delete

My current project is using linq2sql, we had a small problem deleting a record from a many to many relationship that used a link table which only contained 2 required foreign key columns.

product with productId
order with orderId
productOrder with 2 foreign keys productId and orderId

System.Data.Linq.DuplicateKeyException : Cannot add an entity with a key that is already in use.
System.InvalidOperationException : An attempt was made to remove a relationship between a product and a productOrder.

However, one of the relationship's foreign keys (productOrder.productId) cannot be set to null.

The trick was to add DeleteOnNull="true" to the association

to delete modify collections (many to many link tables)

<Table Name="dbo.product" Member="product">
  <Type Name="product">
    <Column Name="productId" Type="System.Int32" DbType="Int NOT NULL IDENTITY" IsPrimaryKey="true" IsDbGenerated="true" CanBeNull="false" />
    <Column Name="Title" Type="System.String" DbType="VarChar(255)" CanBeNull="true" />
    <Association Name="product_productOrder" Member="productOrders" ThisKey="productId" OtherKey="productId" Type="productOrder" DeleteOnNull="true"/>
  </Type>
</Table>

<Table Name="dbo.productOrders" Member="productOrders">
  <Type Name="productOrder">
    <Column Name="productId" Type="System.Int32" DbType="Int NOT NULL" IsPrimaryKey="true" CanBeNull="false" />
    <Column Name="orderId" Type="System.Int32" DbType="Int NOT NULL" IsPrimaryKey="true" CanBeNull="false" />
    <Association Name="product_productOrder" Member="productId" ThisKey="productId" OtherKey="productId" Type="product" IsForeignKey="true" DeleteOnNull="true"/>
    <Association Name="order_productOrder" Member="order" ThisKey="orderId" OtherKey="orderId" Type="order" IsForeignKey="true" DeleteOnNull="true"/>
  </Type>
</Table>

product.ProductOrders.Clear();
product.ProductOrders.AddRange(myNewProductOrders);

I found help on this matter here: http://blogs.msdn.com/b/bethmassi/archive/2007/10/02/linq-to-sql-and-one-to-many-relationships.aspx

Unit testing your unity IOC wiring

I'm using Unity for the IOC container at the moment which is great, injecting in dependencies to the controller makes the dev cycle so quick, without it TDDing would be significantly more difficult. we can unit test everything including the controllers, but you all know that.

There is a small downside to using unity, in that you only find out about problems to the wiring when you run your app, or your acceptance tests, and then you need to analyse the error and figure out what went wrong.

So i thought wouldn't it be nice to have a unit test to test your unity configuration :-)
Well here it is

using NUnit.Framework;
using System.Reflection;
using System.Web.Mvc;

namespace Tests.Unit.Consumer
{
    [TestFixture]
    public class ProductionWiringDefinitionTest
    {
        [Test]
        public void ProductionWiringIsComplete()
        {
            IWiringDefinition wiringDefinition = new ConsumerProductionWiringDefinition();
            WiringDefinitionAssertions.AssertWiringDefinitionAllDependenciesResolved(wiringDefinition);
        }
    }
  
    public static class WiringDefinitionAssertions
    {
        public static void AssertWiringDefinitionAllDependenciesResolved(IWiringDefinition wiringDefinition)
        {
            var unityDependencyResolver = UnityDependencyResolver.Instance;
            unityDependencyResolver.Configure(wiringDefinition);
            DependencyResolver.SetResolver(unityDependencyResolver);            
            
            var containerFieldInfo = unityDependencyResolver.GetType().GetField("container", BindingFlags.NonPublic | BindingFlags.Instance);
            var container = (Microsoft.Practices.Unity.UnityContainer) containerFieldInfo.GetValue(unityDependencyResolver);

            foreach (var registration in container.Registrations)
            {
                unityDependencyResolver.GetService(registration.RegisteredType);                   
                unityDependencyResolver.GetService(registration.MappedToType);                   
            }
        }
    }  
}

The lines that do the magic are 30 and 31 inside the for loop, basically we try and instantiate everything that unity knows about, if any dependencies are missing you will get a nice unit test failure and a very good error message telling you exactly what is wrong.

You may have to change this example a little as it uses our abstraction of IWiringDefinition because we have multiple sites all doing the same thing but with different wiring.

using Microsoft.Practices.Unity;
namespace Framework.Ioc
{
    public interface IWiringDefinition
    {
        void Configure(IUnityContainer container);
    }
}

And an example of our wiring definition

using Diagnostics;
using Microsoft.Practices.Unity;

namespace Consumer.Application
{
    public class ConsumerProductionWiringDefinition : IWiringDefinition
    {
        public void Configure(IUnityContainer container)
        {
            RegisterViewModelMappers(container);
            RegisterDatabaseContext(container);
            RegisterControllers(container);
        }

        private static void RegisterDatabaseContext(IUnityContainer container)
        {
            container.RegisterType<IDatabaseManager, DatabaseManager>();
            container.RegisterType<DatabaseManager>(new InjectionConstructor(new DbConnectionString().ForConsumer));
        }

        private static void RegisterControllers(IUnityContainer container)
        {
            container.RegisterType<DiagnosticsController>(new InjectionConstructor(new ServerDiagnosticsController()));
        }

        private static void RegisterViewModelMappers(IUnityContainer container)
        {
            container.RegisterType<ITopicWithArticleSummariesMapper, TopicWithArticleSummariesMapper>();
            container.RegisterType<IArticleSummaryMapper, ArticleSummaryMapper>();
            container.RegisterType<IToolsPageViewModelMapper, ToolsPageViewModelMapper>();
            container.RegisterType<IToolViewModelMapper, ToolViewModelMapper>();
        }
    }
}

Like i said this example above is not entirely complete, you will need a little more than this to get your wiring working, but its a good introduction to actually testing your wiring.

Good luck.

Acceptance Tests, Fluent Interfaces and a DSL

Lately ive been doing quite a lot of acceptance testing. We did give some BDD frameworks a try especially SpecFlow, but were left with a feeling that it was too complex for the benifits that it gave, was too inflexable and in the long run could turn into a bit of a mess. well that was the evaluation of the team, im sure there are good ways of doing it that is valuble but we decided not to go with a framework.

So i came up with a DSL that gives the developer a simple 'Given When Then' syntax that im quite pleased with, it allows for the most readable / maintainable acceptance test ive seen to date.

A small acceptance test follows as an example:

using NUnit.Framework;
using OpenQA.Selenium;
using AcceptanceTests.Domain;

namespace AcceptanceTests.Fixtures
{
    [TestFixture]
    public class TagTests : BaseTestFixture
    {
        [Test]
        public void ThisAndThatShouldDoSomeThings()
        {
            var resultsPage = Page.SiteABC.ResultsPage;
            given.WeAreOnThePage(resultsPage);

            when.I.Click.TheElement(By.LinkText("civil-engineering"));

            then.TheUrlShouldBe(Page.SiteABC.Article("civil-engineering"))
                .And.TheTitleShouldBe("civil-engineering articles in ABC example page");
            then.TheElement(By.ClassName("tag-wrapper")).ContainsTheText("Articles with the Tag:\r\ncivil engineering");

            when.I.Type("london").Into(By.Id("txtLocation"))
                .And.I.Click.TheElement(By.LinkText("search"));

            then ..... (You get the idea)
        }
    }
}

The base class gives the test fixtures access to the Given When Then classes similar to this:

public class BaseTestFixture
{
    protected Given given;
    protected When when;
    protected Then then;

    [TestFixtureSetUp]
    public void SetUp()
    {
        given = new Given();
        when = new When();
        then = new Then();
    }
}

The actual Given When Then classes delegate the actions to webdriver in the following manner:

public class Given
{
    private readonly WebDriver webDriver;

    public Given()
    {
        webDriver = WebDriver.GetCurrent();
    }

    public Given And
    {
        get { return this; }
    }

    public Given WeAreOnThePage(Page page)
    {
        webDriver.Driver.Navigate().GoToUrl(page.Url);
        return this;
    }
}


public class When
{
    private readonly WebDriver webDriver;

    public When()
    {
        this.webDriver = WebDriver.GetCurrent();
    }

    public When And
    {
        get { return this; }
    }

    public When I
    {
        get {return this; }
    }

    public IAction Click
    {
        get { return new Click(this); }
    }

    public TypeText Type(string text)
    {
        return new TypeText(text);
    }
}

By writing the step classes like this we can write the tests in a fluent style, making for very readable. This is important as it enables none technical (business) people to read / understand the tests and have input on the creation of the tests, although i admit it still does not really enable them to write them, as you do need to know c# and visual studio, but in my experience they never write them anyway (especially on the contact im on), its technical people who write the automated acceptance tests.

All in all im quite pleased who the suite of tests is progressing and how the DSL has evolved.

C# .net method call performance

I’ve recently been in a conversation with another dev about a code base and had an alarming exchange regarding method calls and performance.

Alarming because he claimed that the reason that the methods in the system were so long was because someone higher in the dev chain 'tech lead' or 'architect' had claimed that it was expensive to make method calls???

I’ve seen my fair share of long methods, awful things that destroy reuse, developer productivity, and introduce many, many tangled bugs. These usually evolve because the devs are under pressure to get things out of the door on tight schedules, or by inexperienced devs who unfortunately don’t know much better. Personally I like to try and keep my methods short <20 lines, to keep to the principal that a method should only do one thing, do it clearly and well (so its easy to read, understand and change), you know all the usual stuff. But I’ve never seen the justification for long methods to be because of performance...

so i thought what is this performance gain of long methods? After googling around I didn’t find anything (if you do let me know) so i wrote a program.

Essentially it consisted of 2 loops, one to do a simple calculation in line and one to call a method to do the same calculation.

for (int i = 0; i < LoopCount; i++ )
{
fakeCalculationVariable++;
}


for (int i = 0; i < LoopCount; i++ )
{
Calculate();
}

i took timings before and after each loop and recorded the results

LoopCount= 100000000
elapsed time = 234.447 milliseconds
elapsed time = 640.8218 milliseconds
Method call loop took 406.3748 millisecs longer than inline loop for 100000000 calls
Thats a cost of 0.0000041 millisecs per method call

i wasn’t sure about the results so i decided to run it again, this time with ten billion iterations of the loop, these performance penalties must show up with that many calls.

LoopCount= 1000000000
elapsed time = 2250.6912 milliseconds
elapsed time = 6376.9584 milliseconds
Method call loop took 4126.2672 millisecs longer than inline loop for 1000000000 calls
Thats a cost of 0.0000041 millisecs per method call

its quite conclusive, method calls don’t really add up to all that much overhead, as we all expected.