Tom Oakley

Background

Over the past year at TotallyMoney I've been focussing on the native application for iOS and Android. It's a React Native application that was released in 2017 and has seen iterative improvements since. TotallyMoney has historically focussed mainly on their website but as apps have become an increasingly large part of the tech sphere, and we've seen from our data that TotallyMoney users who use the app are more engaged than those who use the website, the app has become more and more of an important part of the company.

On the web we typically release new code to production multiple times a day and use the "build, measure, learn" methodology outlined in The Lean Startup to iteratively improve new features over time. However, this was tricky with the native app because we didn't want to be releasing new versions to the app store every day; submitting for iOS review, preparing release notes and doing some manual QA to ensure quality all took time. So when the Customer Operations team came to us and told us there was a customer experiencing a non-critical bug, it might be 1 or 2 weeks before that bug fix would be released. This was frustrating for many areas of the company, who were used to having bugs fixed on the web very quickly (usually within 24 hours, often much shorter).

Existing setup

The existing pipeline we had for the native app is common for most modern software development projects. Branch off the master git branch, make your changes, open a PR and wait for code review by your colleagues. Each push to the branch would result in a new Continous Integration pipeline being kicked off, through CircleCI. This involves unit testing (Jest), and end-to-end testing (wix/Detox). Once the tests have passed and the Pull Request has approval, it can be merged. From there, a new release would be built from master, using Fastlane and these versions would be released to TestFlight or the Google Alpha Track, where stakeholders (engineers, product managers and designers) could evaluate the changes. When we were ready to prepare a release, a release build would be approved and CircleCI would run a Fastlane pipeline to build the production version of the app, and we would then submit to the App Store.

CodePush

In about November 2019, we started to read and hear more about CodePush, which is a technology created by Microsoft to host the JavaScript bundle of a React Native app in the cloud. This allows the native app installed on a user's device to download the latest JavaScript bundle at any time (usually when the app is started).

We pretty quickly identified CodePush as a great way to iteratively push updates and be able to release whenever necessary. This potentially means that a bug that is reported could be fixed within a few hours. While this is normal for the web, this opens up new possibilities for our native app.

What we did

Alongside the iOS and Android stage builds, there's new a new job in Circle CI - CodePush stage. This runs the release-react command from the CodePush CLI, and builds and releases new JavaScript bundles to the Android and iOS stage environments within CodePush. This can then be tested by the stakeholders and if everyone is happy, the build can be promoted to the production environments - this is done through Fastlane, but is run locally. The reason for this is because currently, the label for a CodePush build can not be customised - the naming of each release is v<number>. If these could be customised, we would set it to the CircleCI pipeline number; we could then do the promotion of releases through Circle instead (because the pipeline number would be the same). When a build is promoted, the slack + fastlane integration posts a message to one of our Slack channels, so anyone who's interested knows there's a new version released.

This worked really well, and it was great to be able to release whenever we had new things to get in to user's hands. However, we found we also needed a way to be able to see what had changed between builds. So the Fastlane script now also uses the setgithubrelease integration to create a new release on Github when the CodePush stage job has completed. This is named v<native version number>/<Circle pipeline number>, and makes it easier to see what has changed between individual CodePush versions. We can then use Github's 'compare changes' feature to see changes between two releases / tags.

What's next

Our improvements to the pipeline isn't done yet; here is a list of things we'd like to do:

• Use AppCenter to host native builds for us; this is more important for Android than iOS, because we find the Google Alpha Track is pretty poor for beta testing (slow to release, difficult to find what version is installed on a specific device, difficult to manage from admin side).
• Add end-to-end tests once the stage jobs have completed, to ensure a) the CodePush build works correctly (we would still manually test, but useful to automate it too) b) check some basic stuff like our test user accounts work
• Add end-to-end tests once a Production native build has completed, so we can check the App Store account that iOS reviewers use to login with works and is in the correct state.
• Explore the possibility of switching to a service like Bitrise.io. CircleCI is quite slow for end-to-end testing a native application - it has to install and setup lots of dependencies like Xcode, detox and Cocoapods, before it can even begin running the tests
• Only do native builds when native code has changed or node modules with native module integrations have been installed/updated. Right now, the native build is done every time, which is unnecessary because the large majority of changes are JavaScript, which are now handled by CodePush.