Damo's Podcast Highlights 2016 #23

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

[Mastering business analysis] Slices of Big Truths http://masteringbusinessanalysis.com/mba075-slices-big-truths/
[Agile Weekly Crew] Design Thinking with Jeff Patton http://integrumtech.com/2012/07/episode-67-design-thinking-with-jeff-patton/
[Jeff Patton] Story mapping https://www.youtube.com/watch?v=AorAgSrHjKM
Story Mapping Quick Reference - Jeff Patton & Associates http://jpattonassociates.com/story-mapping-quick-ref/ 
[Mastering business analysis] User Story Mapping with Jeff Patton http://masteringbusinessanalysis.com/episode-016-user-story-mapping-with-jeff-patton/
[Scrum master toolbox] a Systems Thinking horror story http://www.scrum-master-toolbox.com/2016/06/podcast/vasco-duarte-and-tim-bourguignon-tell-a-systems-thinking-horror-story/

[The Tim Ferriss Show] The Choice-Minimal Lifestyle: 6 Formulas for More Output and Less Overwhelm 

http://fourhourworkweek.com/2008/02/06/the-choice-minimal-lifestyle-6-formulas-for-more-output-and-less-overwhelm/

The paradox of choice

1. The more options you consider the more regret you will have (buyers regret)
2. The more options you encounter the less fulfilling the final outcome will be

The choice minimal lifestyle

1. Considering options cost attention that can't be spent on action (sense of overwhelm)
2. Attention is necessary for productivity and appreciation

Principles

1. Set rules to automate decision making
2. Don’t provoke deliberation until you can take action
3. Don’t postpone decisions just to avoid uncomfortable conversations
4. Learn to make non-fatal or non-reversible decisions quickly (Set time limits for decision and options, set financial thresholds)
1. Decision fatigue is a thing (we all have a limited amount of decision making power)
5. Don’t strive for variation and thus increased options when it is not needed (routine routine routine, daily rituals (book))
1. Embrace routine, remove decision making from what you can
2. Nutrition vs enjoying food
3. exercise vs enjoyment sport
6. Regret is past tense decision making
1. Stop making complaints (21 day no complaint experiment)
2. Remove wasted past tense deliberation

Damo's Podcast Highlights 2016 #22

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

[Domain Driven Design Europe] Vaughn Vernon: Challenges in Software Development of Today https://www.youtube.com/watch?v=4UZZjyQDgT8
[On Books] The clock of the long now http://castig.org/the-clock-of-the-long-now-by-stewart-brand/
[Continuous Discussions] CD Automation https://electric-cloud.com/blog/2016/04/continuous-discussions-c9d9-podcast-episode-39-cd-automation/
[Mastering Business Analysis] Lean Business Analysis http://masteringbusinessanalysis.com/mba074-lean-business-analysis/
Having a Lean mindset helps us to avoid some anti-patterns that lead to waste.  These eight forms of waste are:

• Unfinished work – Analysis or requirements completed but not implemented
• Unnecessary features – Features customers don’t need or want
• Silos and hand-offs – Working without a shared understanding and transitioning work items
• Unnecessary processes – Steps that don’t add value
• Context switching – Moving from one task to another without focus
• Waiting – Delays due to waiting for an answer or availability
• Rework – Work that needs to be done over
• Knowledge hording – Keeping knowledge to yourself

[Hello Tech Pros] Productivity, Getting things done http://hellotechpros.com/christian-genco-productivity/

• 3-step productivity system that gives him perfect memory and allows him to always know what to do next.

[The Tim Ferriss Show] Lazy: A Manifesto http://fourhourworkweek.com/2015/04/03/lazy-a-manifesto/

Damo's Podcast Highlights 2016 #21

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

[This Agile Life 109] Fallacies of Software Developers - http://www.thisagilelife.com/109/
[On Books] Episode 15 - How to Learn Anything Fast - http://castig.org/the-first-20-hours-by-josh-kaufman/
[On Books] Episode 18 - Growth Mindset vs fixed mindset - http://castig.org/growth-mindset-vs-fixed-mindset/
[RunAs Radio 473] Hacking a Country with Troy Hunt - http://runasradio.com/Shows/Show/473
[Scrum Master Toolbox Podcast] The role of retrospectives on the path to success - http://www.scrum-master-toolbox.com/2016/05/podcast/allison-pollard-on-the-role-of-retrospectives-to-get-us-on-the-path-to-success/
[Continuous Discussions 41] Creating an Internal Dev/Test Cloud - https://electric-cloud.com/blog/2016/05/key-takeaways-continuous-discussions-c9d9-episode-41-creating-internal-devtest-cloud/
[Freakonomics Radio] How to Be Tim Ferriss - http://freakonomics.com/podcast/tim-ferriss/
[Software Engineering Radio 23] The role of software architecture Pt1. - http://www.se-radio.net/2006/07/episode-23-architecture-pt-1/

Damo's Podcast Highlights 2016 #20

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

[Agile Weekly Crew] Episode #57 – Performance Reviews and one on one feedback
http://integrumtech.com/2012/04/episode-57-performance-reviews-with-kane-mar/

[On books] Thinking fast and slow book review https://www.acast.com/onbooks/episode-11-thinking-fast-and-slow-by-daniel-kahneman-chat-with-alex-miles-younger- and https://www.acast.com/onbooks/episode-12-thinking-fast-and-slow-part-2-on-happiness

[Enterprise devops initiatives] The Importance of Systems Thinking for Cloud Migration (focus more on AWS but still good) http://mikekavis.sys-con.com/node/3386867

[Enterprise devops initiatives] How to Succeed with DevOps? Make Room for Experimentation!
http://www.cloudtp.com/2015/08/27/how-to-succeed-with-devops-make-room-for-experimentation/

[The azure podcast] Azure functions http://azpodcast.azurewebsites.net/post/Episode-128-Azure-Functions

Damo's Podcast Highlights 2016 #19

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

Lets see if i can keep going every week shall we :-)


[Hanselminutes] An overview of containers http://www.hanselminutes.com/527/practical-containers-for-developers-with-aja-hammerly
[devchat tv] Motivation, Impact, professionalism https://devchat.tv/ruby-rogues/244-rr-program-like-you-give-a-damn-with-ara-t-howard-at-rails-remote-conf-2015
[London school of economics] Brexit, should we stay or should we go http://www.lse.ac.uk/publicEvents/events/2016/05/20160510t1830vOT.aspx
[Hello tech pros] Productivity, Motivation, Habits http://hellotechpros.com/joe-zack-productivity/

Damo's Podcast Highlights 2016 #18

I subscribe to many podcasts, you can see the list as it was in 2015 here: Developer podcasts v2 but I thought I would start to keep a weekly log of the episodes that I found interesting or useful in some way.

Lets see if i can keep going every week shall we :-)

[Software engineering radio] Unit Tests  http://www.se-radio.net/2016/05/se-radio-episode-256-jay-fields-on-working-effectively-with-unit-tests/
[Hanselminutes] Management and motivation http://www.hanselminutes.com/526/punishment-driven-development-with-louise-elliott
[Scrum master toolbox] Purpose and management 3.0 http://podplayer.net/#/?id=15224714 from http://www.scrum-master-toolbox.com/
[Devnology] Lean software development http://devnology.nl/podcasts/devnology-podcast-012-mary-and-tom-poppendieck-part-1
[Full stack radio] Git source control tips http://www.fullstackradio.com/41

P.s If you are a little confused about the number in the title its based on the week number. Anyway it looks like I've been doing this longer than i really have ;-)

Deploying Azure websites and webjobs using your CI system with msbuild and azure powershell cmdlets

There are lots of tutorials detailing how to send web apps and web jobs up to azure as part of your CI pipeline but I've found them all lacking, mainly in that they require you to upload to Azure in the same command as performing your build.
Often this command looks like the following:
msbuild.exe foobar.csproj /p:DeployOnBuild=true /p:PublishProfile="azurePubProfile" /p:Configuration=Release
This simultaneously builds, configures and publishes the project to azure.
 
This is unacceptable to me as I want to promote the same built artifact through many environments ci-test-preprod-prod where the only thing to change is the config getting deployed to each. I don't want to keep building the code over and over when I deploy which has many issues ranging from changes in the code between deployments to changes in the environment of the build machine. I want to know that what is getting deployed to prod is exactly the same binaries as was deployed and tested in all previous environments. To achieve this we need to separate the building of the artifact from the deployment.

What follows is the method we are now using to build and deploy our services to Azure. It breaks down into 4 simple steps

1. Build your solution, firstly restoring any nuget packages if needed
2. Configure your service for the target environment and zip up the artifact
3. Configure the powershell session for azure
4. Publish the zipped artifact to azure

App service (websites)

Build

Firstly build your solution, if using any nuget packages restore these first
nuget restore
Perform the build on the solution
msbuild foobar.sln /t:build /p:configuration=release /p:OutDir=BUILD_OUTPUT_PATH
Grab the published website from ./foobar/BUILD_OUTPUT_PATH/_PublishedWebsites/foobar and safe it to your artifact store of choice.

Deploy

Firstly apply the correct config for your target environment for this deployment to the artifact saved from the build. there are many ways to do this so i wont go in to it here.

Next zip up the deployable artifact
Add-Type -Assembly System.IO.Compression.FileSystem
$compressionLevel = [System.IO.Compression.CompressionLevel]::Optimal
[System.IO.Compression.ZipFile]::CreateFromDirectory($sourcedir, $zipfilename, $compressionLevel, $false)
Prepare the powershell session for the upload to azure
Load in your azure cert
Import-AzurePublishSettingsFile azure-credentials.publishsettings
Select the desired subscription
Select-AzureSubscription 00000000-0000-0000-0000-000000000001
Publish the website using the azure cmdlet
Publish-AzureWebsiteProject -Package buildoutput.zip -Name foobar -Slot 'staging'

Webjobs

As above except the location of the built artifact foobarjob/BUILD_OUTPUT_PATH
And the cmdlet for creating the webjob
New-AzureWebsiteJob -Name foobar -JobName foobarjob -JobType Continuous -JobFile .\buildoutput.zip

Conclusion

It turns out its actually really easy to achieve the result we desired, mainly the repeatable deployment of a known, tested build artifact to any environment we like.

We use GoCd from thoughtworks to orchestrate the above process through the pipelines feature. When all this is combined with staging slots, blue green deployments and the like you get a really robust process, excellent auditability, a solid rollback strategy and not to mention peace of mind when pushing to prod.

Id love to hear how you achieve the same results, whether its through version control strategies, azure deployment slot strategies or something else.
Let me know.

Azure app service error 'An attempt was made to load a program with an incorrect format'

I was deploying a new website today and came across an issue running the site on azure. The site ran fine on my dev machine both running under vs2015 and when setup as a website under my local IIS.

But when running on a freshly created Azure app service i got the error:

Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.

Exception Details: System.BadImageFormatException: Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.

[BadImageFormatException: Could not load file or assembly 'System.Web' or one of its dependencies. An attempt was made to load a program with an incorrect format.] System.Reflection.RuntimeAssembly._nLoad(AssemblyName fileName, String codeBase, Evidence assemblySecurity, RuntimeAssembly ....


After digging around on the interwebs for a while it became apparent it was an issue with either the target architecture or the targetting of the dlls. My solution was built to target 'any cpu' but somehow the version of System.Web that was brought down by nuget was the 64bit version.

So i figured that changing the target architecture would have the desired effect and changed the azure platform from 32 bit (the default for new app services) to 64 bit in the application settings of the azure portal (portal.azure.com).


Problem solved. I hope this will point someone else to a quick resolution in the future.

Output nunit2 compatable xml from the nunit3 console runner in your CI pipeline.

Upgrades

We have recently upgraded many of our .net projects to use C#6 and .net 4.6.1. At the same time we upgraded nunit to nunit3 on all our build servers. Problem was that GoCd (our CI server) doesn't recognize the new nunit3 result format, only the older more ubiquitous junit/nunit2 xml format. whilst failing tests would cause the build to break you had to dig into the console output to see the failures rather than looking in the tests tab.

Investigation

The solution to the problem was to get the nunit3 console runner to output results xml in the old nunit2 style, easier said than done. The documentation for the runner is actually quite bad in this regard. All it says is: "The --result option may use any of the following formats. nunit2" but doesn't actually show how. I ended up in the source code for the nunit3-console.tests and saw the answer burred within.

Solution

The final command i ended up with is as follows:
nunit3-console --result:"Result.xml;format=nunit2" --where "cat!=Integration" .\tests.dll
This output was then able to be imported into Go and we again get visibility of the numbers of tests run, passing and failing on the go server.

Thanks to all who responded on the google group discussion pointing me in the right direction.

Applying Azure resource locks to all the databases and storage accounts in a given resource group with powershell

If you have followed any of my previous blogs you will know we have tens of microservices (over 50) in our current architecture. With these microservices goes data (lots of data, valuable data). Each service has storage accounts and/or databases (which we dont really want to loose). We have been going through the process of automating the creation of these resources and in the process need to ensure they are not accidentally deleted (as we have tear down scripts, dangerous in the wrong hands).

Powershell

What follows are some powershell commands that can add resource locks to all your databases and storage accounts, they took a while to build, but are very effective, enjoy.
Write-Host -ForegroundColor Cyan "Adding a CanNotDelete lock to all databases"
Get-AzureRmResource `
 | Where-Object {$_.ResourceGroupName -eq myresourcegroupname -and `
                 $_.ResourceType -eq "Microsoft.Sql/servers/databases"} `
 | Select-Object `
     ResourceName,ResourceType, `
     @{name="name"; `
       Expression={$_.name.replace("myazuresqlservername/","")}}, `
     @{name="lockname"; `
       Expression={"lock-databases-"+$_.name.replace("myazuresqlservername/","")}} `
 | %{New-AzureRmResourceLock -ResourceGroupName myresourcegroupname`
                             -LockLevel CanNotDelete `
                             -LockNotes "Prevent accidental deletion" `
                             -LockName $_.lockname `
                             -ResourceName $_.ResourceName `
                             -ResourceType $_.ResourceType `
                             -Verbose -Force -ErrorAction Stop}
Write-Host -ForegroundColor Cyan "Adding a CanNotDelete lock to all storage accounts"
Get-AzureRmResource `
 | Where-Object {$_.ResourceGroupName -eq myresourcegroupname -and `
                 $_.ResourceType -eq "Microsoft.Storage/storageAccounts"} `
 | Select-Object ResourceName,ResourceType,Name, `
                 @{name="lockname"; `
                   Expression={"lock-storageAccounts-"+$_.name}} `
 | %{New-AzureRmResourceLock -ResourceGroupName myresourcegroupname`
                             -LockLevel CanNotDelete `
                             -LockNotes "Prevent accidental deletion" `
                             -LockName $_.lockname `
                             -ResourceName $_.ResourceName `
                             -ResourceType $_.ResourceType `
                             -Verbose -Force -ErrorAction Stop}
You can customise a bit further and replace the strings "myazuresqlservername" and "myresourcegroupname" with powershell variables and stick this straight in a powershell console or in a script.

Lock removal

As an aside, if you do subsequently want to delete the DB or storage account you first need to remove the lock like this:
Remove-AzureRmResourceLock -ResourceId /subscriptions/00000000-1111-2222-3333-444444444444/resourceGroups/myresourcegroupname/providers/Microsoft.Sql/servers/myazuresqlservername/databases/mydatabasename -LockName lock-databases-mydatabasename

Feedback

Please if you found this useful or you know a better way let me know in the comments below. cheers.