Deploying Azure websites and webjobs using your CI system with msbuild and azure powershell cmdlets

There are lots of tutorials detailing how to send web apps and web jobs up to azure as part of your CI pipeline but I've found them all lacking, mainly in that they require you to upload to Azure in the same command as performing your build.
Often this command looks like the following:
msbuild.exe foobar.csproj /p:DeployOnBuild=true /p:PublishProfile="azurePubProfile" /p:Configuration=Release
This simultaneously builds, configures and publishes the project to azure.
 
This is unacceptable to me as I want to promote the same built artifact through many environments ci-test-preprod-prod where the only thing to change is the config getting deployed to each. I don't want to keep building the code over and over when I deploy which has many issues ranging from changes in the code between deployments to changes in the environment of the build machine. I want to know that what is getting deployed to prod is exactly the same binaries as was deployed and tested in all previous environments. To achieve this we need to separate the building of the artifact from the deployment.

What follows is the method we are now using to build and deploy our services to Azure. It breaks down into 4 simple steps

1. Build your solution, firstly restoring any nuget packages if needed
2. Configure your service for the target environment and zip up the artifact
3. Configure the powershell session for azure
4. Publish the zipped artifact to azure

App service (websites)

Build

Firstly build your solution, if using any nuget packages restore these first
nuget restore
Perform the build on the solution
msbuild foobar.sln /t:build /p:configuration=release /p:OutDir=BUILD_OUTPUT_PATH
Grab the published website from ./foobar/BUILD_OUTPUT_PATH/_PublishedWebsites/foobar and safe it to your artifact store of choice.

Deploy

Firstly apply the correct config for your target environment for this deployment to the artifact saved from the build. there are many ways to do this so i wont go in to it here.

Next zip up the deployable artifact
Add-Type -Assembly System.IO.Compression.FileSystem
$compressionLevel = [System.IO.Compression.CompressionLevel]::Optimal
[System.IO.Compression.ZipFile]::CreateFromDirectory($sourcedir, $zipfilename, $compressionLevel, $false)
Prepare the powershell session for the upload to azure
Load in your azure cert
Import-AzurePublishSettingsFile azure-credentials.publishsettings
Select the desired subscription
Select-AzureSubscription 00000000-0000-0000-0000-000000000001
Publish the website using the azure cmdlet
Publish-AzureWebsiteProject -Package buildoutput.zip -Name foobar -Slot 'staging'

Webjobs

As above except the location of the built artifact foobarjob/BUILD_OUTPUT_PATH
And the cmdlet for creating the webjob
New-AzureWebsiteJob -Name foobar -JobName foobarjob -JobType Continuous -JobFile .\buildoutput.zip

Conclusion

It turns out its actually really easy to achieve the result we desired, mainly the repeatable deployment of a known, tested build artifact to any environment we like.

We use GoCd from thoughtworks to orchestrate the above process through the pipelines feature. When all this is combined with staging slots, blue green deployments and the like you get a really robust process, excellent auditability, a solid rollback strategy and not to mention peace of mind when pushing to prod.

Id love to hear how you achieve the same results, whether its through version control strategies, azure deployment slot strategies or something else.
Let me know.

Azure app service error 'An attempt was made to load a program with an incorrect format'

I was deploying a new website today and came across an issue running the site on azure. The site ran fine on my dev machine both running under vs2015 and when setup as a website under my local IIS.

But when running on a freshly created Azure app service i got the error:

Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.

Exception Details: System.BadImageFormatException: Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.

[BadImageFormatException: Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.] System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly ....


After digging around on the interwebs for a while it became apparent it was an issue with either the target architecture or the targetting of the dlls. My solution was built to target 'any cpu' but somehow the version of System.Web that was brought down by nuget was the 64bit version.

So i figured that changing the target architecture would have the desired effect and changed the azure platform from 32 bit (the default for new app services) to 64 bit in the application settings of the azure portal (portal.azure.com).


Problem solved. I hope this will point someone else to a quick resolution in the future.

Applying Azure resource locks to all the databases and storage accounts in a given resource group with powershell

If you have followed any of my previous blogs you will know we have tens of microservices (over 50) in our current architecture. With these microservices goes data (lots of data, valuable data). Each service has storage accounts and/or databases (which we dont really want to loose). We have been going through the process of automating the creation of these resources and in the process need to ensure they are not accidentally deleted (as we have tear down scripts, dangerous in the wrong hands).

Powershell

What follows are some powershell commands that can add resource locks to all your databases and storage accounts, they took a while to build, but are very effective, enjoy.
Write-Host -ForegroundColor Cyan "Adding a CanNotDelete lock to all databases"
Get-AzureRmResource `
 | Where-Object {$_.ResourceGroupName -eq myresourcegroupname -and `
                 $_.ResourceType -eq "Microsoft.Sql/servers/databases"} `
 | Select-Object `
     ResourceName,ResourceType, `
     @{name="name"; `
       Expression={$_.name.replace("myazuresqlservername/","")}}, `
     @{name="lockname"; `
       Expression={"lock-databases-"+$_.name.replace("myazuresqlservername/","")}} `
 | %{New-AzureRmResourceLock -ResourceGroupName myresourcegroupname`
                             -LockLevel CanNotDelete `
                             -LockNotes "Prevent accidental deletion" `
                             -LockName $_.lockname `
                             -ResourceName $_.ResourceName `
                             -ResourceType $_.ResourceType `
                             -Verbose -Force -ErrorAction Stop}
Write-Host -ForegroundColor Cyan "Adding a CanNotDelete lock to all storage accounts"
Get-AzureRmResource `
 | Where-Object {$_.ResourceGroupName -eq myresourcegroupname -and `
                 $_.ResourceType -eq "Microsoft.Storage/storageAccounts"} `
 | Select-Object ResourceName,ResourceType,Name, `
                 @{name="lockname"; `
                   Expression={"lock-storageAccounts-"+$_.name}} `
 | %{New-AzureRmResourceLock -ResourceGroupName myresourcegroupname`
                             -LockLevel CanNotDelete `
                             -LockNotes "Prevent accidental deletion" `
                             -LockName $_.lockname `
                             -ResourceName $_.ResourceName `
                             -ResourceType $_.ResourceType `
                             -Verbose -Force -ErrorAction Stop}
You can customise a bit further and replace the strings "myazuresqlservername" and "myresourcegroupname" with powershell variables and stick this straight in a powershell console or in a script.

Lock removal

As an aside, if you do subsequently want to delete the DB or storage account you first need to remove the lock like this:
Remove-AzureRmResourceLock -ResourceId /subscriptions/00000000-1111-2222-3333-444444444444/resourceGroups/myresourcegroupname/providers/Microsoft.Sql/servers/myazuresqlservername/databases/mydatabasename -LockName lock-databases-mydatabasename

Feedback

Please if you found this useful or you know a better way let me know in the comments below. cheers.

Azure Learnings

A few months ago I played with Azure, experimented with all the storage options (table, queue, blobs) web and worker roles, made things interact and generally got to know what was possible, not possible and how to do things. It was interesting, i gave a few presentations to the company I was contracted to at the time but in the end it was all just exploration, they were never going to take it up (they own lots of their own hardware, i mean Lots, and besides in my view there are issues with going to Azure as a solution for that enterprise problem), also I didn't see any use for it in my own personal projects.

I had a MSDN account which at the time entitled me to 750 compute hours a month (a compute hour is just a measure of how long your roles have been deployed in the cloud), and was careful with my deployments, always take test deployments down to minimise the total compute hours.

Anyway two problems:

1st. I didnt check my subscription(why would i?) it turned out i had the basic "starter" subscription (50 free compute hours) not the MSDN package (750 free hours).

2nd. I did a demo and someone wanted to use it afterwards so i left it up (one web role and one worker role) and then forgot about it... opps. To add to my woes my hotmail account is not my primary email account so the billing was not coming to me, i only just found it when I happened to be on hotmail.

To cut a long story short I got charged £50 + £120 + £120 for the honour of having my dummy website sat doing nothing on the azure application fabric... I;ve since talked to Microsoft and cancelled my azure account, i see no need for it, I'm not going to use it any time soon, and wanted to make double sure all my web instances were gone. but after talking to them i have got last months money back, better than nothing but still... i guess it was my fault for not checking my package, my running instances and my email, stupid boy.

Morale of the story to all you perspective Azure developers out there, check your subscriptions, and always take down your test deployments, always.