Script your build and deployment of android cordova apps with powershell

We are developing a new version of our customer facing solution, across web, ios and android, using cordova(phonegap).

Im a big proponent of build automation, and the classical (recommended?) way of using eclipse to build and manage the code base was getting me down, so i decided to write some scripts to build and deploy the app to either a device, an emulator or prepare for release. I also wanted any developer to be able to checkout the code and run the scripts to build the app.

I wrote the scripts in powershell (sorry!) with some batch files to make the various functions easy to run (im developing on windows 8 by the way).

You can find the scripts here: https://github.com/DamianStanger/AndroidBuildScripts

So how does it all work?

firstly it goes with out saying that you need your dev env set up for android development on the command line with cordova http://cordova.apache.org/docs/en/edge/guide_cli_index.md.html#The%20Command-Line%20Interface

As you will know (if you do cordova development) when you use the command line tools for creating cordova apps the folder created is where all your source code is placed and inside there is the www folder that is where you keep your .js and .html files. The problem is that you are keeping your source code along side the automatically built cordova files, not ideal. so I've created my own source folder that is where all the code you edit is kept. We then use powershell to copy these files to the correct places.

The development process

In a powershell (or dos cmd if you prefer)

build.bat
emulate.bat or install.bat

Line 01. run either or, depending on if you are using a real device or not

That's it. Now you might notice that build can take a wile to run because its setting up everything from scratch so i created a shortcut that will only copy your changes across.

quickCopy.bat
emulate.bat or install.bat

This is all good for general day to day dev but eventually you will want to test a production build on a real device for this use the following commands

release.bat
installRelease.bat

To make this work you must only have either a device plugged into usb or an emulator turned on (please use genymotion its so much faster than a standard emulator)
The release process signs and aligns your apk for you :-) so when you are ready you just send the apk you have tested to the play store.

The scripts

Here I'm going to show select lines of code from build.ps1

For building the app in debug and getting that on to your emulator or phone

function create()

cordova create app-cordova-android com.myapp.app myapp
...
cordova platform add android
...
cordova plugin add org.apache.cordova.device

function build()

cordova build android

function emulate()

cordova emulate android -d

function installDebug()

cordova run android -d

To build the releaseable apk and to get that onto your phone use the following:

function release()

cordova build android --release

function sign()

jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore ..\appstore\android-keystore\myapp -keypass myappKeyPassword -storepass myappStorePassword -signedjar .\platforms\android\bin\myapp-release-signed.apk .\platforms\android\bin\myapp-release-unsigned.apk myapp
zipalign -f -v 4 .\platforms\android\bin\myapp-release-signed.apk .\platforms\android\bin\myapp-release-signed-aligned.apk

function installRelease()

adb uninstall com.myapp.app
adb install .\appstore\APKs\myapp-release-signed-aligned.apk

Release versioning

When doing a release to the play store you need to make sure the version numbers are incremented each time, for this i added a helper which will update all the relevent places in the source for you.

just run:

setVersion 102 1.0.2

This will change all the files that need changing in order to properly put a new version of the app onto the play store.

Upload to play store error

Upload failed
You uploaded a debuggable APK. For security reasons you need to disable debugging before it can be published in Google Play

Make sure your manifest is set thus:

<application android:debuggable="false" android:hardwareAccelerated="true" android:icon="@drawable/icon" android:label="@string/app_name">

Resources

• https://github.com/DamianStanger/AndroidBuildScripts
• https://cordova.apache.org/docs/en/3.2.0/guide_cli_index.md.html#The%20Command-line%20Interface
• https://developer.android.com/tools/publishing/app-signing.html
• http://docs.oracle.com/javase/6/docs/technotes/tools/windows/jarsigner.html
• https://developer.android.com/tools/help/zipalign.html
• http://www.genymotion.com/

Debug your android applications by capturing/monitoring their http traffic using wireshark

I’ve always wondered what my phone is telling the outside world and recently i had the need to actually find out as I’m developing an android app for work at the moment. I needed to find out what was going over the wire as i was getting some strange problems and could not debug the traffic on the production server.

Setup

Download and install wireshark : https://wireshark.org/

Disable wifi and mobile data on the phone.

Connect your phone to your laptop/desktop via USB.

Enable internet pass though. Basically you want your phones internet to come through the USB wire, through your computer network card, which when running a wireshark capture, through wireshark.

Set up a capture filter so that you only capture the data coming to and from your phone and not data initiated from the computer itself. i pick the option to ‘create a capture with detailed options’. Set a capture filter for example ‘host 192.168.15.129’,  where 192.168.15.129 is the ip address of the phone.

Additionally (or alternatively) you can filter the traffic by ip address after capture when viewing the results “ip.src==192.168.15.129 or ip.dst==192.168.15.129” where 192.168.15.129 is the ip address of your phone. Or filter the traffic by protocol, you probably care about http traffic so filter on this by entering “http” in the filter.

Results

You can get information overload with wireshark, it takes some getting used to, but if you dig you can find everything you need. Look for the requests you care about by looking down the info column and clicking the row. This will present all the packet details where you can dig as deep as you like into the request.

I use the Hypertext Transfer Protocol section as its the level of detail i care about. From here you can see the url and the headers as well as a link to the packet that contains the response, simply perfect.

jquery promises wrapped in javascript closures, oh my..

I recently had a question from one of my fellow devs regarding a problem where the values in a loop were not what they expected. This was down to the deferred execution of the success function after a promise had been resolved.

The following code has been simplified for this explanation:

function (userArray) {
  var i;

  for (i = 0; i < userArray.length; i++) {
    var userDto = userArray[i];
    var user = new UserModel();
    user.displayName = userDto.displayName;
    var promise = service.getJSON(userDto.policies);

    promise.then(function(policyDtos){
      system.log("user.displayname : " + user.displayname);
      convertAndStore(user, policyDtos);
    });
  }
}
Input:
userArray = ['bob', 'sue', 'jon']; //* see footer
Output:
user.displayname : jon
user.displayname : jon
user.displayname : jon

service.getJSON (line 07) returns a jquery promise, the function actually makes a service call to an external API and so can take some time to resolve. Notice how the var userDto and user are declared within the loop, this is not best practice in javascript as the variables are actually hoisted up to the containing function (next to var i). To a none javascript expert it looks like the variables will be created anew inside the loop as they are in c#. In fact there is only one copy of i, user and userDto so obviously the values are overwritten within every loop iteration.

This is the fixed function using closures.

function (userArray) {
  var i,userDto, promise;

  for (i = 0; i < userArray.length; i++) {
    userDto = userArray[i];
    promise = service.getJSON(userDto.policies);
 
    (function(capturedDisplayName) {
      var user = new UserModel();
      user.displayName = capturedDisplayName;

      promise.then(function(policyDtos){
        system.log("user.displayname : " + user.displayname);
        convertAndStore(user, policyDtos);
      });
    }) (userDto.displayName);
 
  }
}
Input:
userArray = ['bob', 'sue', 'jon']; //** see footer
Output:
user.displayname : bob
user.displayname : sue
user.displayname : jon

The introduced function on line 07 creates a capture around the variable userDto.displayName, the variable is a parameter called capturedDisplayName. Now there is a copy of this variable for every iteration of the loop allowing you to use it after the promise has resolved.
You may wonder why the variable promise works as intended given the problems with user and userDto? Well that is because the object referenced within the loop iteration has the .then attached to it before it is overwritten by the next loop, remember the object itself is not overwritten or changed on line 05 only the reference to the object.


//* In reality are more complex objects than simple strings, I'm trying to keep this simple for readability.
//** The names have been changed to protect the innocent.

HTC One and Vodafone. Removing the Kikin search service

If your like me then you will hate how the phone manufacturers install all sorts of things on your phone for you, (to be fair I think its Vodafone not HTC), and I know its not as bad as the galaxy (I’ve several friends with Samsung phones) but recently my phone started popping up a search service every time I selected (long pressed) a word to copy and paste it. really really annoying.

Anyway the Kikin service (http://www.kikin.com/) was really winding me up, software i didn’t ask for, didn’t want, couldn’t remove but did get in the way every time I wanted to simply select a piece of text.

So how to remove the kikin service?

OK first the bad news, you cant, well not without rooting your phone. But you can disable it so it no longer bothers you by popping up all the time.

Settings -> Apps -> All -> Kikin

Sadly you will see no uninstall, but if you turn notifications off, force stop, then Disable. you are done :-) now the long press select text click doesn't also do a web search automatically.
And I’m not sure but since I’ve done it I have not had Kikin auto update on me, or it might just be happening behind the scenes.

Vodafone, HTC, et al. Please please stop installing useless guff on to our phones. just because you can doesn’t mean you should.

Problems connecting to a live DB in an MVC 4 web app using EF5.0

I had some real problems this week deploying my latest site live. Everything was runnig fine locally, or so i thought. connecting to a local sql2012 DB through entity framework 5.0

I was running on dev through iis express on port :41171

When i deployed to live the web server would spin and spin and then eventually produce an asp error saying [Win32Exception (0x80004005): The system cannot find the file specified]

[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)]
...
My first instincts were that SQL server was not set up right at the hosting providers end, I was using the correct connection string after all. But after some back and forth from the hosting provider (1and1) I came to realisation that the SQL instance (SQL server 2012) was probably set up right.
I finally removed this section from the web.config that entity framework adds in


<section name="entityFramework" type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection, EntityFramework, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" requirePermission="false" />
...
...
<entityframework>
<defaultconnectionfactory type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework">
<parameters>
<parameter value="Data Source=.; Integrated Security=True; MultipleActiveResultSets=True" />
</parameters>
</defaultConnectionFactory>
</entityFramework>

What this magic does? [sic]
From what i gather this is telling entity framework to auto-magically connect to a database by convention if the connection string doesn't work. I managed to ascertain that it was my code was actually trying to connect to a local sqlexpress DB. So i took it out.

After removing that section I got this error message (again after ages of trying to connect)

Server Error in '/' Application.
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)

[SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)]
System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction) +5296071
...
...
AND now dev was also broken...

At this point you may be directed here by google/bing/duckduck et al. to https://blogs.msdn.com/b/sql_protocols/archive/2007/05/13/sql-network-interfaces-error-26-error-locating-server-instance-specified.aspx
its not the problem you are looking for... No you dont need to allow UDP port 1434 or anything like that.

My local connection string was
<add name="applicationName" connectionString="server=localhost; database=localdbname;Integrated Security = true;MultipleActiveResultSets=True" providerName="System.Data.SqlClient" />
my connection string on live was
<add name="applicationName" connectionString="Server=db123456789.db.1and1.com,1433;Database=123456789;User Id=mydbuser;Password=mydbuserpassword;" providerName="System.Data.SqlClient" />
all as provided by the host.

Why was it not connecting?? i almost put that config section back but then after debugging i found that inside the the application context that was driving entity framework the connection string was not there, again it was trying to use sqlexpress??

then i stumbled upon it, my context was configured thus.


public class theappContext : DbContext
{
public theappContext() : base("databasename")
{
}
...

and in the application_start in global.asax
Database.SetInitializer<theappcontext>(null);
It turns out you need the connection strings name to be the same as the string in base("databasename"). In my instance i needed to set it to databsename not applicationName as i had previously.

So finally these are my connection strings

On dev either:
<add name="databasename" connectionString="server=localhost; database=localdbname;Integrated Security = true;MultipleActiveResultSets=True" providerName="System.Data.SqlClient" />
<add name="databasename" connectionString="server=localhost; database=localdbname;User Id=mydbuser;Password=mydbuserpassword" providerName="System.Data.SqlClient" />
On live:
<add name="databasename" connectionString="Server=db123456789.db.1and1.com,1433;Database=123456789;User Id=mydbuser;Password=mydbuserpassword;" providerName="System.Data.SqlClient" />

And thats what fixed it, simply changing the name of the connection string. So entity framework was being super cleaver and making up for bad connection strings locally, which meant that when deploying live I had no hope. Why all this auto-magic?

Using jasmine to test JQuery

Ive been doing lots of javascript work of late (mainly in a durandal SPA app) and wanted to test the following function that uses jquery.

function update () {
    errorVisibleFlag(false);
    $("input.delete:checkbox").each(function() {
        if ($(this).is(":checked")) {
            var idToDelete = $(this).attr("id"),
                status = $(this).attr("status");
            dataService.delete(idToDelete)
                .fail(function(){errorVisibleFlag(true)});
        }
    });
    get();
}
This function is part of a knockout view model within a durandal app.

I came up with the following jasmine test that uses spys to verify that the calls were working and also some DOM manipulation to allow jquery to bind the function to something. Ive found it works really well.

it('When update is called, delete is called with the correct parameters', function(){
  var inputCheckbox = '<input type="checkbox" class="delete" id="bc87d270-95bc-49bc-9cac-3e903d09590b" status="Pending">',
      inputCheckboxId = "#bc87d270-95bc-49bc-9cac-3e903d09590b";
  $("body").append(inputCheckbox);

  var spyDelete = spyOn(vm.dataService, "delete").andCallFake(getData);
  var spyGet = spyOn(vm.dataService, "get").andCallFake(getData);

  vm.update();

  expect(spyDelete).toHaveBeenCalledWith("bc87d270-95bc-49bc-9cac-3e903d09590b");
  expect(spyGet).toHaveBeenCalled();

  $(inputCheckboxId).remove();
});
Note: The viewmodel vm is injected into the test using require.js and the function dataService.delete returns a jquery promise

Format my blog post. Some online resources for formatting your code snippets

A couple of very good resources for formatting your code for inclusion into blogs

Simple and very effective
http://www.dan.co.jp/cases/javascript/encode_entities.html

Two more with some added features:
http://formatmysourcecode.blogspot.com/
http://codeformatter.blogspot.com/2009/06/about-code-formatter.html

Life, Death and Education. A couple of inspiring/thought provoking podcasts.

I don't only listen to technical podcasts. In fact there are 3 podcasts in particular that i listen to a lot; Radiolab, LES (London school of economics), and freakonomics radio. I usually post technical things on my blog but this time i thought id blog about a couple of episodes that i have listened to lately, thought provoking, inspirational, amazing, emotional.

How doctors want to die (hint: its very different from the general public)

Radiolab: The bitter end - http://www.radiolab.org/blogs/radiolab-blog/2013/jan/15/bitter-end/
Thought provoking. A difficult subject, death, but one that i think that everyone should talk about. This is a really good episode about how doctors would like to die, and contrasting that with the general public and the expectations that the 2 groups have. i would encourage everyone to listen to this and to get their loved ones to as well, its great for sparking that difficult conversation.
Extract: We turn to doctors to save our lives -- to heal us, repair us, and keep us healthy. But when it comes to the critical question of what to do when death is at hand, there seems to be a gap between what we want doctors to do for us, and what doctors want done for themselves.

Khan Academy - Reimagining Education

LES podcasts: Khan Academy - Reimagining Education - http://www2.lse.ac.uk/publicEvents/events/2013/04/20130410t1830vOT.aspx
Inspirational. Remember school? You either loved it or hated it. This podcast contains a great discussion on education of the future, some really good thinking on mastery and how education could be better in the future. I've got a little girl now and so we are thinking about her education, i hope the schools she goes to are as forward looking as these guys. I agree 100 % that the progression to mastery is definitely the way rather than the old skool approach of a set pace for all.
Extract: Salman Khan tells the inspiring story of how the Khan Academy came to be and shares his thoughts on what education could (should?) be like in the future.

When does life begin? when does it end?

Radiolab: 23 weeks 6 days - http://www.radiolab.org/2013/apr/30/
Emotional. A difficult podcast to listen to as it not only talks about life and death but relates it back to real people and in particular the story of one family, and one very premature baby. As I've said I've got a little girl now and so I related to this story in a big way. It did make me cry! very moving. But it does have a happy ending.
Extract: When Kelley Benham and her husband Tom French finally got pregnant, after many attempts and a good deal of technological help, everything was perfect. Until it wasn't. Their story raises questions that, until recently, no parent had to face… and that are still nearly impossible to answer.

Quick multi cursor support in sublime text 2

In a previous post i listed out all the podcasts i listen to. Whilst creating that list i had to do a little bit of text manipulation, i used some of the powerful multi cursor support from sublime text to do it.
I wanted all my links to be hyperlinks instead of just text.

Below is the screen cast of how i went from a long list of web addresses:
<h2>.net</h2>
http://www.hm.com/
http://www.dnr.com/
http://c9.msdn.com/
...
to a long list of hyper links:
<h2>.net</h2>
<a href="http://www.hm.com/">http://www.hm.com/</a>
<a href="http://www.dnr.com/">http://www.dnr.com/</a>
<a href="http://c9.msdn.com/">http://c9.msdn.com/</a>
...

The process

First do an incremental find (Ctrl+i)
Then select all matched (Alt+Enter)
Copy all selections into clipboard (Ctrl+c)
End
Type the end tag
Home Type the start tag
Paste the links again
Close the start tag

Sublime text 2, get to know it.

Sublime is really powerful, this is just a really simple demo of a simple real world use. ive used it to process hundreds of thousands of lines pulled from databases whilst selecting complex regular expression matches then using sort lines and unique lines to get a really good feel of the data.

I used APowersoft screen recorder to record the screen cast and Key Jedi to display the key strokes as i went.

My Collection Of Podcasts For Developers

[update "2015-11-03"]I've just posted a new version of this list to developer-podcasts-v2[update]

I'm always on the lookout for more development focused podcasts and thought id better share my current crop as i know others are also looking for more podcasts to accompany them on their daily commute.

My interests

Some background on me: Primarily a .net developer with an interest in agile, javascript, ruby, design and devops. So this list derives from these interests.

.Net

http://www.hanselminutes.com/
http://www.dotnetrocks.com/
http://deepfriedbytes.com/
http://channel9.msdn.com/

General Development

http://www.se-radio.net/
http://herdingcode.com/
http://techcast.chariotsolutions.com/
http://devnology.nl/en/podcast
http://thisdeveloperslife.com/

Javascript

http://nodeup.com/

Ruby

http://ruby5.envylabs.com/

DevOps and IT

http://www.runasradio.com/
http://foodfight.libsyn.com/
http://devopscafe.org/

Business

http://www.startupsfortherestofus.com/

Design and Development

I found this resource last month that contains lots of other great podcasts on design and development http://www.smashingmagazine.com/2013/04/19/podcasts-for-designers-developers/?utm_source=feedly