model dynamic causing unhandled exception

We are using ASP.MVC 3 with Razor views on one of the projects in currently working on. MVC3 uses C# 4.0 and so we get access to the dynamic types, this is really convenient when creating partial views, we can create a model in one partial and pass the new dynamic model into the partial, no need for strongly typed view models everywhere.

But we have found that dynamic models can hide other errors with very time consuming consequence's.
This is the partial "_topics" that was causing us problems:

@model dynamic

<ul>
    @foreach (var topic in Model.Topics)
    {
        <li id="@topic.Id" class="@( Model.SelectedItems.Contains(topic.Id) ?  "selected" : string.Empty ) ">
            <label>@topic.Name</label>
            @if (topic.HasChildren)
        {
                @Html.Partial("_Topics", new { Topics = topic.Children, SelectedItems = Model.SelectedItems });
        }
        </li>
    }
</ul>

as you can see _topics is called recursively.

We made some changes to other areas of our project and got this error: (sorry for the very long stack trace, but i wanted to be complete)

Application information: 
    Application domain: /LM/W3SVC/3/ROOT-3-129566639908092111 
    Trust level: Full 
    Application Virtual Path: / 
    Application Path: C:\projects\Foo\Bar\Src\Editor\ 
    Machine name: AAA20711 
 
Process information: 
    Process ID: 5928 
    Process name: w3wp.exe 
    Account name: IIS APPPOOL\Editor 
 
Exception information: 
    Exception type: RuntimeBinderException 
    Exception message: 'object' does not contain a definition for 'Topics'
   at CallSite.Target(Closure , CallSite , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at ASP._Page_Views_Shared__Topics_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_Topics.cshtml:line 4
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model)
   at ASP._Page_Views_Shared__ArticleTabs_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_ArticleTabs.cshtml:line 42
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName)
   at ASP._Page_Views_ArticleCreate_Create_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\ArticleCreate\Create.cshtml:line 23
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.StartPage.RunPage()
   at System.Web.WebPages.StartPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResult(ControllerContext controllerContext, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
   at System.Web.Mvc.Controller.ExecuteCore()
   at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
   at System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
   at System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f)
   at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Request information: 
    Request URL: http://editor/article/create 
    Request path: /article/create 
    User host address: 127.0.0.1 
    User: Foo 
    Is authenticated: True 
    Authentication Type: Forms 
    Thread account name: IIS APPPOOL\Editor 
 
Thread information: 
    Thread ID: 9 
    Thread account name: IIS APPPOOL\Editor 
    Is impersonating: False 
    Stack trace:    at CallSite.Target(Closure , CallSite , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T0 arg0)
   at ASP._Page_Views_Shared__Topics_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_Topics.cshtml:line 4
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model)
   at ASP._Page_Views_Shared__ArticleTabs_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\Shared\_ArticleTabs.cshtml:line 42
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.HtmlHelper.RenderPartialInternal(String partialViewName, ViewDataDictionary viewData, Object model, TextWriter writer, ViewEngineCollection viewEngineCollection)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName, Object model, ViewDataDictionary viewData)
   at System.Web.Mvc.Html.PartialExtensions.Partial(HtmlHelper htmlHelper, String partialViewName)
   at ASP._Page_Views_ArticleCreate_Create_cshtml.Execute() in C:\projects\Foo\Bar\Src\Editor\Views\ArticleCreate\Create.cshtml:line 23
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy()
   at System.Web.Mvc.WebViewPage.ExecutePageHierarchy()
   at System.Web.WebPages.StartPage.RunPage()
   at System.Web.WebPages.StartPage.ExecutePageHierarchy()
   at System.Web.WebPages.WebPageBase.ExecutePageHierarchy(WebPageContext pageContext, TextWriter writer, WebPageRenderingBase startPage)
   at System.Web.Mvc.RazorView.RenderView(ViewContext viewContext, TextWriter writer, Object instance)
   at System.Web.Mvc.BuildManagerCompiledView.Render(ViewContext viewContext, TextWriter writer)
   at System.Web.Mvc.ViewResultBase.ExecuteResult(ControllerContext context)
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResult(ControllerContext controllerContext, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<InvokeActionResultWithFilters>b__19()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultFilter(IResultFilter filter, ResultExecutingContext preContext, Func`1 continuation)
   at System.Web.Mvc.ControllerActionInvoker.<>c__DisplayClass1c.<>c__DisplayClass1e.<InvokeActionResultWithFilters>b__1b()
   at System.Web.Mvc.ControllerActionInvoker.InvokeActionResultWithFilters(ControllerContext controllerContext, IList`1 filters, ActionResult actionResult)
   at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
   at System.Web.Mvc.Controller.ExecuteCore()
   at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
   at System.Web.Mvc.ControllerBase.System.Web.Mvc.IController.Execute(RequestContext requestContext)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass8`1.<BeginSynchronous>b__7(IAsyncResult _)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
   at System.Web.Mvc.SecurityUtil.<GetCallInAppTrustThunk>b__0(Action f)
   at System.Web.Mvc.SecurityUtil.ProcessInApplicationTrust(Action action)
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Looks like there was a problem with the dynamic model, line 4 is the for each loop on model.topics. But debugging this code showed there was no problem with the model, it was fully populated with lots of topics and the debugger could see them and access them, so what gives?

Well after investigating it turned out another partial that the razor view uses had a compilation error in it, we only found this after removing the _topics partial. Very strange that a compilation error in another partial would cause the error we got.

To stop this ever happening again we made the project pre compile the views along with the rest of the web project but putting MvcBuildViews=true the .csproj

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
  ....
    <MvcBuildViews>true</MvcBuildViews>

Now we get the real error at compile time, not an error in the dynamic model code at run time, much nicer.

Linq to Sql many to many delete

My current project is using linq2sql, we had a small problem deleting a record from a many to many relationship that used a link table which only contained 2 required foreign key columns.

product with productId
order with orderId
productOrder with 2 foreign keys productId and orderId

System.Data.Linq.DuplicateKeyException : Cannot add an entity with a key that is already in use.
System.InvalidOperationException : An attempt was made to remove a relationship between a product and a productOrder.

However, one of the relationship's foreign keys (productOrder.productId) cannot be set to null.

The trick was to add DeleteOnNull="true" to the association

to delete modify collections (many to many link tables)

<Table Name="dbo.product" Member="product">
  <Type Name="product">
    <Column Name="productId" Type="System.Int32" DbType="Int NOT NULL IDENTITY" IsPrimaryKey="true" IsDbGenerated="true" CanBeNull="false" />
    <Column Name="Title" Type="System.String" DbType="VarChar(255)" CanBeNull="true" />
    <Association Name="product_productOrder" Member="productOrders" ThisKey="productId" OtherKey="productId" Type="productOrder" DeleteOnNull="true"/>
  </Type>
</Table>

<Table Name="dbo.productOrders" Member="productOrders">
  <Type Name="productOrder">
    <Column Name="productId" Type="System.Int32" DbType="Int NOT NULL" IsPrimaryKey="true" CanBeNull="false" />
    <Column Name="orderId" Type="System.Int32" DbType="Int NOT NULL" IsPrimaryKey="true" CanBeNull="false" />
    <Association Name="product_productOrder" Member="productId" ThisKey="productId" OtherKey="productId" Type="product" IsForeignKey="true" DeleteOnNull="true"/>
    <Association Name="order_productOrder" Member="order" ThisKey="orderId" OtherKey="orderId" Type="order" IsForeignKey="true" DeleteOnNull="true"/>
  </Type>
</Table>

product.ProductOrders.Clear();
product.ProductOrders.AddRange(myNewProductOrders);

I found help on this matter here: http://blogs.msdn.com/b/bethmassi/archive/2007/10/02/linq-to-sql-and-one-to-many-relationships.aspx

Unit testing your unity IOC wiring

I'm using Unity for the IOC container at the moment which is great, injecting in dependencies to the controller makes the dev cycle so quick, without it TDDing would be significantly more difficult. we can unit test everything including the controllers, but you all know that.

There is a small downside to using unity, in that you only find out about problems to the wiring when you run your app, or your acceptance tests, and then you need to analyse the error and figure out what went wrong.

So i thought wouldn't it be nice to have a unit test to test your unity configuration :-)
Well here it is

using NUnit.Framework;
using System.Reflection;
using System.Web.Mvc;

namespace Tests.Unit.Consumer
{
    [TestFixture]
    public class ProductionWiringDefinitionTest
    {
        [Test]
        public void ProductionWiringIsComplete()
        {
            IWiringDefinition wiringDefinition = new ConsumerProductionWiringDefinition();
            WiringDefinitionAssertions.AssertWiringDefinitionAllDependenciesResolved(wiringDefinition);
        }
    }
  
    public static class WiringDefinitionAssertions
    {
        public static void AssertWiringDefinitionAllDependenciesResolved(IWiringDefinition wiringDefinition)
        {
            var unityDependencyResolver = UnityDependencyResolver.Instance;
            unityDependencyResolver.Configure(wiringDefinition);
            DependencyResolver.SetResolver(unityDependencyResolver);            
            
            var containerFieldInfo = unityDependencyResolver.GetType().GetField("container", BindingFlags.NonPublic | BindingFlags.Instance);
            var container = (Microsoft.Practices.Unity.UnityContainer) containerFieldInfo.GetValue(unityDependencyResolver);

            foreach (var registration in container.Registrations)
            {
                unityDependencyResolver.GetService(registration.RegisteredType);                   
                unityDependencyResolver.GetService(registration.MappedToType);                   
            }
        }
    }  
}

The lines that do the magic are 30 and 31 inside the for loop, basically we try and instantiate everything that unity knows about, if any dependencies are missing you will get a nice unit test failure and a very good error message telling you exactly what is wrong.

You may have to change this example a little as it uses our abstraction of IWiringDefinition because we have multiple sites all doing the same thing but with different wiring.

using Microsoft.Practices.Unity;
namespace Framework.Ioc
{
    public interface IWiringDefinition
    {
        void Configure(IUnityContainer container);
    }
}

And an example of our wiring definition

using Diagnostics;
using Microsoft.Practices.Unity;

namespace Consumer.Application
{
    public class ConsumerProductionWiringDefinition : IWiringDefinition
    {
        public void Configure(IUnityContainer container)
        {
            RegisterViewModelMappers(container);
            RegisterDatabaseContext(container);
            RegisterControllers(container);
        }

        private static void RegisterDatabaseContext(IUnityContainer container)
        {
            container.RegisterType<IDatabaseManager, DatabaseManager>();
            container.RegisterType<DatabaseManager>(new InjectionConstructor(new DbConnectionString().ForConsumer));
        }

        private static void RegisterControllers(IUnityContainer container)
        {
            container.RegisterType<DiagnosticsController>(new InjectionConstructor(new ServerDiagnosticsController()));
        }

        private static void RegisterViewModelMappers(IUnityContainer container)
        {
            container.RegisterType<ITopicWithArticleSummariesMapper, TopicWithArticleSummariesMapper>();
            container.RegisterType<IArticleSummaryMapper, ArticleSummaryMapper>();
            container.RegisterType<IToolsPageViewModelMapper, ToolsPageViewModelMapper>();
            container.RegisterType<IToolViewModelMapper, ToolViewModelMapper>();
        }
    }
}

Like i said this example above is not entirely complete, you will need a little more than this to get your wiring working, but its a good introduction to actually testing your wiring.

Good luck.

Ruby on Rails 3 - Installing Devise authentication on Heroku

A few days ago I decided to add authentication to my Rails 3 app. I came across a few alternatives but in the end opted for Devise as this seems to be the newest and from the documents I found apparently the easiest to implement.
I needed users to be able to register new users, sign in and out and to have special admin users who could access restricted areas of the site, Devise hit all these requirements.

So I found a couple of good tutorials including https://github.com/plataformatec/devise and associated wiki https://github.com/plataformatec/devise/wiki/_pages and found these really good at getting me up and going quickly.

I got everything working locally but then when I pushed my git repo to Heroku I got the error when using the register page: We're sorry, but something went wrong. We've been notified about this issue and we'll take a look at it shortly.


so to the command line 'Heroku logs' revealed this error:
2011-03-21T14:03:09-07:00 app[web.1]: NameError (uninitialized constant Devise::Encryptors::Bcrypt):

It took me a while but after searching around any trying all sorts of things i hit the answer. and as usual it was simple, if you are deploying to heroku you need to add bcrypt-ruby to your gemfile like so:

gem 'devise', '1.1.8'
gem 'bcrypt-ruby', '2.1.4'

Also remember to generate and commit your new gemfile.lock into git and send this to heroku as well as the new gemfile or you will get the following error, as i did:

You have modified your Gemfile in development but did not check
the resulting snapshot (Gemfile.lock) into version control
You have added to the Gemfile:
* bcrypt-ruby (= 2.1.4)
FAILED: http://docs.heroku.com/bundler
Heroku push rejected, failed to install gems via Bundler

My Devise Authentication Implementation
I've added the following to the top of all my admin controller classes to ensure that only administrators can access them

before_filter :authenticate_user!
before_filter :authorise_user!

The method :authenticate_user! is a helper method built into Devise
The method :authorise_user! is my own declared in ApplicationController as follows:

def authorise_user!
  if ! current_user.try(:admin?)
    flash[:alert] = "Unauthorised"
    redirect_to ''
  end
end

where admin is a boolean field on the devise created user table. (I used option 2 from this wiki page)

For controllers which none admin users have access to I want to restrict them so that they can only see resources that they created. So for example, bob created an order-x, you don't want James to be able to access that order-x it belongs to bob not James.

For this I've created methods similar to:

def authorise_as_owner!
  order = Order.find(params[:id])
  unless current_user.try(:admin?) || order.user_id == current_user.id
    flash[:alert]  = "Not your order!"
    redirect_to ''
  end
end

The DZone effect (how blog aggregation can affect traffic)

I recently used DZone for the first time to publicise my latest blog post to a wider audience. Oh my ... it worked rather well.

So im on blogspot which gives some nice google analytics for free which i look at now and then to try and see what is popular and being read by other people.

My blog is not all that widely read to be honest, it has a fair number of hits though, approximately 250 a month mainly coming from google searches, most traffic goes to a couple of my better/more interesting posts. But then i tried out submitting a post to dzone, mainly to see what happens.

The result was instant, i got over 600 views in 2 days, for me thats a big number, was it the nature of the post (it was a little provocative) and maybe the summary on dzone made people want to see what i was on about, either way i got a large increase in my traffic %300 :-)

Im not really hung up on loads of people seeing my posts, but its nice to know your are read :-) And dzone certainly seems to deliver.

Appharbor, scripting sql server table creation using ado.net

So ive got a simple appharbor installation running, but need a database.
I was half expecting a nice API I could use similar to herokus that would let me run my scripts against the hosted DB and thus deploy my DB at the command line, but not.

What you need to do is connect to your DB through SQL server management studio (if using sql server) and do your DB work through that. All good and easy, but at work our corporate firewalls prevent us from getting out, or from external sources getting in, makes sense, and i as a lowly developer will never be able to change that, not that i would want to, it would be quite dangerous.

So what to do?
well i decided to write a page in my MVC app that (given the right credentials) will run a script that has been checked in, Here are the basics:

• Create a DB on appharbor (use the web interface to create it, really simple)
• Copy the connection settings and paste them into your web.config, here are mine (details changed to protect the innocent)


<connectionstrings>
    <add connectionstring="Server=db002.appharbor.net;Database=db1234;User ID=db1234;Password=lotsofrandomcharacters" name="mydb"></add>
</connectionstrings>



• now we need a controller with an action

public ActionResult RunTheScript(string filename)
{
    try
    {
        string connStr = ConfigurationManager.ConnectionStrings["mydb"].ToString();
        using (var conn = new SqlConnection(connStr))
        {
            conn.Open();
            string filePath = Server.MapPath("\\databasescripts\\" + filename);
            FileInfo fileInfo = new FileInfo(filePath);

            string script = fileInfo.OpenText().ReadToEnd();

            SqlCommand cmd = new SqlCommand(script, conn);
            cmd.ExecuteNonQuery();
            conn.Close();
        }

        return View();
    }
    catch (Exception e)
    {
        return View("ExceptionView", e);
    }
}

• Dont forget your routes in global.asax, Add the following before the default route.


routes.MapRoute(
    "admin_runthescript",
    "admin/runthescript/{filename}",
    new { controller = "admin", action = "runthescript" } // Parameter defaults
);


And add a couple of views (your know how to do that), one for success and one for an exception.
Yes there are better ways of doing this but its a quick and dirty demo, i expect people to use properly designed code when doing this live (Please dont mix your DB logic into your controllers people...)

A note on the exception handling, you had better do this properly too as appharbor will not give you any logging out of the box, and will not spit back helpful errors to the user, not errors that can be used to debug anyway..

• Almost there, add a script, and save it in your databasescripts folder


CREATE TABLE [dbo].[locations](
    [id] [int] IDENTITY(1,1) NOT NULL,
    [name] [nvarchar](256) NOT NULL
) ON [PRIMARY]

INSERT INTO [dbo].[locations] ([name]) VALUES ('London')
INSERT INTO [dbo].[locations] ([name]) VALUES ('Manchester')
INSERT INTO [dbo].[locations] ([name]) VALUES ('Bristol')
INSERT INTO [dbo].[locations] ([name]) VALUES ('Bath')


• Now add all the files to your git repo
• commit
• git push appharbor master
• Run the migration through your web app using the url Controller/Action/Filename (scripts/runthescript/add_location_table.sql)

Thats pretty much it, its upto you to secure the page that can be used to create and destroy your db,dont blame me if someone gets in an messes with unsecured pages,

Maybe there is something in the pipeline at app harbor that will help us out when scripting databases, but one thing is for sure using the management studio for building your applications database is not the way forward in the long run.

The true cost of TFS, is it really "free"?

You say you've got MSDN premium, so TFS is free, good for you, it is developed and supported by Microsoft, should be a great product then??

Anyway in my personal opinion, it sucks, but try telling that to none technical people, its actually really hard, and they still insist you use it because its designed for enterprise use, isn't it?

So i thought id do a 'back of the napkin' calculation of how much it actually costs.

Our dev team is apparently worth ~40k a month, consisting of 4 devs, designer, tester, db admin, tester, ba,
10 people, for arguments sake ~4 grand each per month.
So if you take a dev pair thats 8 grand a month, or 2 grand a week.

Where is the waste (source control)?
Check-ins :- We check in 7 to 10 times a day, each time it should be instant, but mostly TFS cant quite manage it, it needs user interaction to resolve the easiest of changes, some times it cant even manage to insert new lines. So estimate 3 minutes extra per check-in thats 20 minutes a day extra, or 2 hours per week. (Caveat, if you are working in the same area of code as another pair be prepared for this to at least double, ive had some awful merges in the past even when the conflicts weren't really that hard)

Get latest :- Sometimes get latest doesn't even get the latest, leaving files out or just getting it wrong?? how?? so now and then we need to sort it out, usually after wondering what on earth is going on and trying to debug your code only to find out my source is not up to date (10 - 20 mins per day on average, 1 hr per week)

Get specific :- So, often you start using get specific instead of get latest, this is slow so we try not to do it all the time, only when we think there might be a problem, the code base is big so this takes a lot longer than a get latest 2-3 minutes extra per check in, but only done 2-3 times a day (7 minutes per day, 30 mins per week)

Merge to main (feature branching) :-
Depending on how often you merge to main, and how many other teams you have also checking into main that is going to hit you quite hard. if you are a lone team with out any teams changing your code base you have it easy and can discount this step. for me its at least a 4 hour job of doing the merge to main once every month)

Where is the waste (Build management)?
MSBuild and TFS :- This is complex, significantly more complex than the same projects built with nant and cruse control, Team city, Go, et al. it sucks a lot of our time when we need to change the builds, its hard to say how much worse it is, our code base is quite old and complex, lots of build dependencies, this needs fixing, lets be conservative and say it only takes 1 hour extra per week to maintain TFS over any of the others.

Total Waste
2 hrs + 1hr + .5 hrs + 1hr = 5.5 hours of waste a week, lets call it 5 for now, I like round numbers.

So a pair of devs is on 2 grand a week, 35 hours = £60 per hour
5 hours waste = £60 * 5 = £300 per week per pair
2 pairs in our team = £600 per week * 4 = £2,400 per month on a £40,000 dev team

Thats just a 2 pair team, if you have even a moderate size team of 4 pairs you are going to get even more pain, with developers effecting each others check ins.

Oh and what about that merge to main hell we discussed (if you are feature branching), add 4 hrs * £60 per month = £240 extra.

Summary :-
your "free" version control and build agent is not looking so "free" any more.
Its costing us at least 25 grand a year... that sounds wrong? can you please check my math...

How much is it costing you?

Disclaimer:-
In my opinion I've been quite lenient in my estimates, but when you say the numbers and add it all up it sounds awful, but it is.
Also the numbers above don't take into account other teams costs, if you have 3 dev teams (like ours) all working on the same source code, merging every month (feature branch style), all having the same pain, the figures quickly become a big drain on the business.

Problems :-
The main problem I have is how to quantify the waste, often its small things here and there, they just add up, not all the time, but especially if the team is doing TDD properly and going through refactor cycles frequently, which we are.
Any ideas on how to quantify the waste better would be appreciated.


Amendments (11-03-2011)
Thanks to my reviewers (kat, skain) for pointing out my grammatical and spelling errors (now corrected). But i did just knock out this article in the gaps i had whilst waiting for TFS, i guess its good for something. Joking apart though this does lead me to something i didnt mention and that is context switching inefficiencies. So every now and then you get 5 minutes to do other tasks, email, ect, but this is a big context switch for the pair, it takes a little time to get back into the flow again, this cost is not accounted for in the above monetary costs.

Heroku - Installing your app and sqlite3

So ive just installed a dummy rails app to Heroku to test it out, its very very similar to appharbor, unsupprising since appharbor was inspired by heroku.

all went well untll i added a scaffold for my first admin page, when i tried to deploy this i got he error:


Application Error
An error occurred in the application and your page could not be served. Please try again in a few moments.
If you are the application owner, check your logs for details.

so i did check the logs 'heroku logs' (you need the logging add-on instaled for your app) and this is what i saw:

2011-03-03T01:24:00-08:00 heroku[web.1]: State changed from crashed to created
2011-03-03T01:24:00-08:00 heroku[web.1]: State changed from created to starting
2011-03-03T01:24:01-08:00 heroku[web.1]: State changed from crashed to created
2011-03-03T01:24:01-08:00 heroku[web.1]: State changed from created to starting
2011-03-03T01:24:08-08:00 app[web.1]: /usr/ruby1.8.7/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/runtime.rb:64:in `require': no such file to load
-- sqlite3 (LoadError)
2011-03-03T01:24:08-08:00 app[web.1]: from /usr/ruby1.8.7/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/runtime.rb:64:in `require'
2011-03-03T01:24:08-08:00 app[web.1]: from /usr/ruby1.8.7/lib/ruby/gems/1.8/gems/bundler-1.0.7/lib/bundler/runtime.rb:62:in `each'
...
...
2011-03-03T01:24:08-08:00 app[web.1]: from /home/heroku_rack/heroku.ru:1:in `new'
2011-03-03T01:24:08-08:00 app[web.1]: from /home/heroku_rack/heroku.ru:1
2011-03-03T01:24:08-08:00 heroku[web.1]: State changed from starting to crashed

so what went wrong? well looked like it was a problem loading sqlite3
stackoverflow to the rescue :-)

just make sure the references to sqlite3 are wrapped up as per the following line of code in your gemfile

group :development do
   gem 'sqlite3-ruby', :require => 'sqlite3'
end


and that is it, just do a 'heroku rake db:migrate' to get your database on heroku and your should be good to go.

AppHarbor - free .net hosting with continuous integration, Getting started tutorial

Ive been test driving appharbor recently and im very impressed.

Its basically a host for your .net web apps, but with a difference.
First you deploy to appharbor via a git push, as soon as you do this your app is built, your tests are run, and if all goes well (build succeeded and tests passed) your code is deployed to the hosting environment there and then.

Thats all sounds pretty simple, and is very impressive, especially since its free for a single web instance and a 20MB database.

So how do you go about doing it, how do you get a ASP.MVC 3 web app on the interweb in 15 minutes (if that) for free.

1. Create a solution with a single MVC 3 web app in it, im using VS2010 (1 minute)
2. Add a controller and a view, just static content will do (1 minute)
3. Create a git repository in the folder that contains your solution. Just run "git init" (1 minute)
4. Add your web app to the local repository "git add ." but be careful not to include the bin, obj or _resharper folders then Commit changes locally "git commit" (1-2 minutes)
5. Log into appharbor and create a new application (1-2 minutes)
6. Get your remote repository URL from the appharbor administration interface (1 minute)
7. Add remote repository to your git repo: "git remote add appharbor https://UserName@appharbor.com/AppName.git" (1 minute)
8. Push your app to git: "git push appharbor master" (1 minute)
9. Log on to appharbor and navigate to your new app, see the successful build (1-2 minutes)
10. Navigate to the live url and see the site live on the interweb (1-2 minutes)

Superb
I said 15 minutes, you can actually do it in a lot less, but you get my drift.

So what can you do :-)

• Add test projects, your code will only be deployed if all the tests pass, brilliant.
• Roll back to last deployment, very easy to do, if your not happy with the release, just click deploy on a older build and that older build gets redeployed, perfect. 
• You can add your own host name
• Add worker processors (at a cost)
• Have multiple collaborators to the same project and use appharbor as the master repo (a bit like github) its really easy to get get the source on to a different machine, use "git clone https://UserName@appharbor.com/AppName.git"

What cant it do :-(

• Acceptance tests, i use BDD style tests a lot, webdriver, selinium etc. it cant run these, and think about it, it makes sense. If a failing test stops deployment how can you run tests after you have deployed? I guess they need to develop a staging environment similar to Azure which gets deployed to first. I think that is on the pipeline of improvements but not at a high priority.
• No file storage, only what is checked into your repo, you can save things to the file system, but they will get blown away with each deployment. i guess you could use amazon EC2, google or Azure to store your files using their REST APIs but that could be slow for the users, options exist though.
• If/when tests fail you dont get a very good message, only a stack trace, you dont get the expected and actual values, not sure why this is, but its not a show stopper by any means.
• Only one web app per solution is allowed, i guess this is because appharbor uses convention over configuration, and will automatically deploy the one web app there to the hosting environment.

Acceptance Tests, Fluent Interfaces and a DSL

Lately ive been doing quite a lot of acceptance testing. We did give some BDD frameworks a try especially SpecFlow, but were left with a feeling that it was too complex for the benifits that it gave, was too inflexable and in the long run could turn into a bit of a mess. well that was the evaluation of the team, im sure there are good ways of doing it that is valuble but we decided not to go with a framework.

So i came up with a DSL that gives the developer a simple 'Given When Then' syntax that im quite pleased with, it allows for the most readable / maintainable acceptance test ive seen to date.

A small acceptance test follows as an example:

using NUnit.Framework;
using OpenQA.Selenium;
using AcceptanceTests.Domain;

namespace AcceptanceTests.Fixtures
{
    [TestFixture]
    public class TagTests : BaseTestFixture
    {
        [Test]
        public void ThisAndThatShouldDoSomeThings()
        {
            var resultsPage = Page.SiteABC.ResultsPage;
            given.WeAreOnThePage(resultsPage);

            when.I.Click.TheElement(By.LinkText("civil-engineering"));

            then.TheUrlShouldBe(Page.SiteABC.Article("civil-engineering"))
                .And.TheTitleShouldBe("civil-engineering articles in ABC example page");
            then.TheElement(By.ClassName("tag-wrapper")).ContainsTheText("Articles with the Tag:\r\ncivil engineering");

            when.I.Type("london").Into(By.Id("txtLocation"))
                .And.I.Click.TheElement(By.LinkText("search"));

            then ..... (You get the idea)
        }
    }
}

The base class gives the test fixtures access to the Given When Then classes similar to this:

public class BaseTestFixture
{
    protected Given given;
    protected When when;
    protected Then then;

    [TestFixtureSetUp]
    public void SetUp()
    {
        given = new Given();
        when = new When();
        then = new Then();
    }
}

The actual Given When Then classes delegate the actions to webdriver in the following manner:

public class Given
{
    private readonly WebDriver webDriver;

    public Given()
    {
        webDriver = WebDriver.GetCurrent();
    }

    public Given And
    {
        get { return this; }
    }

    public Given WeAreOnThePage(Page page)
    {
        webDriver.Driver.Navigate().GoToUrl(page.Url);
        return this;
    }
}


public class When
{
    private readonly WebDriver webDriver;

    public When()
    {
        this.webDriver = WebDriver.GetCurrent();
    }

    public When And
    {
        get { return this; }
    }

    public When I
    {
        get {return this; }
    }

    public IAction Click
    {
        get { return new Click(this); }
    }

    public TypeText Type(string text)
    {
        return new TypeText(text);
    }
}

By writing the step classes like this we can write the tests in a fluent style, making for very readable. This is important as it enables none technical (business) people to read / understand the tests and have input on the creation of the tests, although i admit it still does not really enable them to write them, as you do need to know c# and visual studio, but in my experience they never write them anyway (especially on the contact im on), its technical people who write the automated acceptance tests.

All in all im quite pleased who the suite of tests is progressing and how the DSL has evolved.