The Librarian – A centralised server for easy build distribution

Written by Siddhartha Bokaria and Sunil Kumar, Software Engineers at MyGate

1. Introduction


The pain of creation, managing and distribution of application builds is a well known topic. As it occurs during any company’s journey, we have struggled finding the right rhythm of what we term as ‘build automation’. The core necessities remain relatively standard; the need for a developer to seamlessly distribute builds to validation engineers while operating within rigid virtual private cloud (VPC) zones.

In tightly knit independent application squads, we tend to take small steps toward harmonising an application (app) distribution strategy. Provided the product build and flavour pipelines are present, we start with the bare necessity of issuing a command that allows the package flow through proxy zones and get published for consumption.

We do not plan on reinventing the wheel but favour assembling the right solutions to manage our app distribution needs. As such, first in the list of agenda is to perform a basic survey of available options.

2. Literature survey


There are a host of projects dedicated to solve this problem. Some of the prominent ones are listed below with a brief overview of their capabilities.

Fastlane [1]
As their official phrase states, “app automation done right”. This includes a suite of pipeline centric features of extracting screenshots, beta distribution, app store deployment and code signing. Definitely what we should be setting up in the long term. However this will be an overwhelming solution for a basic app distribution need amongst squad members.

Firebase [2]
Previously part of Fabric’s Crashlytics, the upcoming Firebase app distribution which is now in beta shows the most promise and falls right within our ‘goldie lock zone’. It focuses on distributing development builds to a group of testers. It allows various approaches including console, CLI and gradle based scripts reducing friction to adopt. It promotes access to builds through a simplified portal and targeted emails. It provides a basic dashboard for central management and activity overview. It supports both iOS and Android builds. And, the entire approach falls under free tier at the moment. Adding GCP to our ecosystem is a task by itself and once allowed this is a frontrunner and a panacea to our problems.

AWS mobile hub [3]
A collection of products rather than a targeted solution. It is a feasible option and would require you to align application building activities from the beginning to account for the chosen framework. A relatively steep learning curve might be a determining factor. For an ongoing set of projects such as ours, this would require additional push for a consideration.

Apple testflight and Android test tracks [4]
This is a good platform for sharing early stage releases. It is predominantly used during beta testing. You would expect a certain level of stabilisation prior to using this facility. The portals often carry download volume quotas for your app listing. We need a simpler alternative to this.

3. Our approach


Everything said and done, we were yet to figure out the right approach that fit into our workflow. Our current goal is to reduce repetitive and time consuming activities of sharing builds with the validation team. Just when velocity of iterations were making the task of sharing builds harder, we came across a simple do it yourself (DIY) app distribution platform called Librarian [1]. This looked like a good place to start with a simple command based submission interface and light UX. Though getting this to a central location was tricky whilst navigating through layers of aforementioned VPC zones, it allowed us the flexibility of carving out our own build flow. That led to the following deliberate steps to set up our own distribution pipeline targeted at an AWS environment.

3.1 Key modules

  • A shell script to allow preparation of the environment. An Android engineer would ideally not want to be distracted with environmental setup required to support background operations. The script helps with installation of python package manager and AWS CLI. It also triggers configuration of AWS environments which means the developer should keep the access tokens ready.
  • Gradle file extension that exposes the task to the developer. A simple configuration at project level file helps identify remote resources. The app level changes allow you to link tasks to the gradle list. Now you can chain the ‘assemblePackage’ and ‘publishRelease’ tasks for seamless build distribution.
  • The final module is an example implementation of a bridge API. Librarian does not support remote submission of build currently and the accompanying NodeJS based API helps activate that feature.

3.2 Quick start (Android)


With all the prerequisites satisfied [5], set up your build environment by following these simple steps.

  1. Place the 2 files (s3Upload.gradle and publish_checkpoint.sh) in the app root where the build.gradle (for app) is present. Synchronise project and add files.
  2. Edit build.gradle (app level), put this line at EOF: apply from: ‘s3Upload.gradle’. Synchronise file.
  3. Edit build.gradle (project level), put following lines below allprojects: repositories section of the file:
    • project.ext.AWS_S3_BUCKET = ‘s3-bucket-name’
    • project.ext.BRIDGE_HOST_PATH = ‘http://localhost:3000/api/librarian’
    • Synchronise file.
  4. Refresh gradle command list.
    • The newly available commands are ready for use.
    • ./gradlew tasks –all
    • You should be able to locate the publish tasks under ‘other tasks’ group labelled as ‘app:publish-variant-ToS3’
    • ./gradlew <publish-task-name>
  5. For 1st time users the utility will setup the necessary packages (pip, aws cli installation, aws environment configuration).

3.3 Illustration


And, here is the workflow illustration to assist the speed readers.

4. Roadmap


This approach is working for us and has reduced the hassle of dealing with unorganised sharing channels. Through this post, we plan to share it with other teams and seek inputs. Some may be using alternatives easier to adopt approaches that might interest us. Others may embrace it and contribute to its development. In either case, the following items have been planned.

  • Extend approach to support iOS builds.
  • Extend support for other cloud storage options.
  • Investigate alignment to Firebase app distribution channel. The preparation script should now ideally target Firebase CLI and gradle script can be replaced by the public plugin.

Head over to our Github repository [5] and try it out yourself!

References

[1] App automation done right
https://fastlane.tools/
[2] Firebase App Distribution
https://firebase.google.com/products/app-distribution
[3] AWS Mobile Hub – Build, Test, and Monitor Mobile Applications
https://aws.amazon.com/products/mobile/
[4] Apple testflight and Android test tracks 
https://developer.apple.com/testflight/https://developer.android.com/distribute/best-practices/launch/test-tracks
[5] MyGate app build distribution utility
https://github.com/mygate-tech/mg-build-automation
[6] Librarian
https://github.com/biocross/Librarian

Admin

MyGate is building technology to simplify check-ins everywhere. Our flagship solution, launched in 2016, currently facilitates over 2 million check-in requests every day at thousands of gated premises across the 25+ cities in which we operate, and continues to show rapid growth. We’ve strengthened our commitment to information privacy by following GDPR guidelines, the global gold standard for data privacy.

Terms for MyGate Cash

The terms are applicable to all Users. Please read the terms and conditions as contained herein-below carefully and thoroughly, prior to proceeding further with MyGate Cash Services (‘Services’). By...

Cashback Offer on Maintenance Payment

Welcome to MyGate! The MyGate website app is owned and managed by Vivish Technologies Private Limited. All the services are provided by Vivish Technologies Private Limited under its brand...

Personalizing Customer Connections for Brands with Geo-Targeting

“Geo-targeting allows businesses to personalise their messagingand connect with customers on a more intimate level.”Ryan Holmes All brands are online today and reach out to their audience through online...

MyGate, convenience, and better communication

“Convenience is the catalyst that drives innovation in business.”Elon Musk Convenience is a mindful and necessary business virtue. Every brand is based on solutions to some problem. Communicating this...

A Match Made in Marketing Heaven: MyGate’s Engaged User Base

Attention brands and brand managers! It’s time to leave the old-fashioned advertising methods behind and hop on board the MyGate express! We’ve got some exciting news to share with...

Make Your Brand Unmissable On The Mygate App

“Our jobs as marketers are to understand how the customer wants to buy and help them do so.” Bryan Eisenberg, Founder & CMO at IdealSpot Brands are increasingly increasing...