Android Automation: Fastlane and Jenkins
Guide to managing time, signing process and builds
An important topic to address while developing is the ability to always deliver new features. The bigger the team, the more the features you can produce and the more you need to test features before merging them to your main branch. In order to do proper testing process without taking out of development time, companies rely on a QA Engineer to test and give feedback and deliver builds to different members of the team to test the app as well.
In order to have do the tests, the developer should make sure to manually create the build, share it with the testers and finally upload it to the store. Thus adding overhead time on each developer to deliver a test version and making sure they all have the signing key, that will be probably shared through git.
A more systematic approach would be automating the process in a way that the feature is build by the developers and the recurrent signing, building, publishing process is handled by a script minimizing the room for human errors.
This is why this article will go step by step on how to build your CI/CD system to achieve that!
If you’re interested in automating your iOS process you can check iOS Automation Guide.
First things first
You need to:
- Use a mac to complete this task.
- Have HomeBrew installed. Check the installation guide.
- Have Android SDK installed (Android studio or the SDK itself).
- Have a Google Play Console account
- Have rbenv installed due to issues on the stock ruby environment while used with bundler. Check the installation guide.
- Have a project to work with configured with git. (This article is based on a Github project)
Notes about the article:
- The usage of
${any_text}
means that you need to add the variable or value without${}
.
Author’s Advice:
- It’s better to use a text editor that highlight your code. Sublime Text would be a good option.
- The article covers the main functionality of fastlane. You can always check more capabilities of this powerful platform. Give the website a look!
- Take a look on Jenkins website to see it’s full potential!
Fastlane
Let’s start with what is fastlane
fastlane is the easiest way to automate beta deployments and releases for your iOS and Android apps. 🚀 It handles all tedious tasks, like generating screenshots, dealing with code signing, and releasing your application.
Fastlane it a set of lanes defined by the user where you call a lane and it does the action defined.
For example having a lane called release
should trigger the following actions:
- Generate a signed APK.
- Upload the app to Google Play Console.
Once these steps are defined, you will have to call fastlane release
rather than do all these steps on your own and risk missing any of them.
Fastlane will make the process better and atomic. However, it will not make it automatic. Jenkins will be our automation tool.
Setup fastlane
Install fastlane on your machine
brew install fastlane
Navigate to the project and add fastlane
fastlane init
Then Select.
4. 🛠 Manual setup — manually setup your project to automate your tasks
This step will add a fastlane
folder and the needed files including Gemfile.
How does Fastlane work?
Think of fastlane as an automation script you write, once you call the function it will execute the steps on the shell.
These automation instructions are called lanes and they’re written in Ruby. These lanes are to be written inside the automatically generated Fastfile. Once a lane is required, you simply call fastlane ${lane_name}
.
There are two types of lanes:
- Private lane:
- Public lane:
The access modifiers work just like any development language. The public lane can be called from anywhere. While, calling a private lane from the shell using fastlane ${private_lane_name}
will result in an error.
You can’t call the private lane ‘${private_lane_name}’ directly
The preferred way to use lanes is to create a wrapper for your actions in private lanes, then call the private lanes from a lane. This way you can guarantee that the actions will always be executed in sequence and you will never miss a step. Even more, you will be able to update a functionality in all the lanes by updating its private lane. Below is a quick example:
As you notice, you can pass arguments to your lane, arguments are passed in this case by using fastlane release version_name:1.0.0
. When |options|
is used to give the lane access to arguments, the order of these arguments doesn’t matter.
Building the apk:
First of all let’s get to know our building tool.
Gradle
Gradle is a build system (open source) which is used to automate building, testing, deployment etc. “Build.gradle” are scripts where one can automate the tasks. For example, the simple task to copy some files from one directory to another can be performed by Gradle build script before the actual build process happens.
Fastlane run gradle tasks without the need to write a script. In the fastlane documentation of gradle, you’ll find that you can run any gradle script simply by using the function gradle()
and specifying the needed arguments.
Downloading dependencies:
In order to download dependencies, you will be using the gradle
function which is provided by fastlane.
This lane is private since it will be called only from within the Fastfile.
Updating version name and code:
You can take multiple approaches to handle version name and version code:
- Set the version name and version code in Android Studio. All you have to do is to update the value inside build.gradle.
- Pass these variables as arguments when calling the lane. This way you will have to call
build_apk(version_name: ${Version Name}, version_code: ${Version Code}
. - Have the values read from a file.
- Fetch version code from Google Play Developer Console
google_play_track_version_codes
. You can check the documentation of this function here.
You can alway mix and match these approaches, or come up with your own approach. In this example, you will be setting the version name from within build.gradle
and reading the versoin code from a file.
Build the apk:
Now you have the dependencies. It’s time to Build the signed apk!
Release:
Now, after having the apk signed and ready to be released, it is time to do the release. You can release to an adhoc distribution service like Firebase distribution, or directly to PlayStore.
In order to release to Google Play Console, you’ll have to allow the CI server(Local machine in this example) to access your account.
Creating a service account:
Let’s start by logging in to the Google Play Console, then follow these steps:
- Expand Settings in the left side menu -> Click on API access.
- In the service account section -> Click Create new account.
- A popup will appear -> Click on Google Cloud Platform in the popup.
After the link is open you will, you will see the Project service accounts settings on Google Cloud Platform.
- Click on Create Service Account in the top.
- Fill the Service account name.
- Fill the Service account description.
- Click on Create.
After Creating the service account, the Grant this service account access to project section will expand.
- Select Role.
- In the popup that will appear, type
Service Account User
. - In the list, select
Service Account User
. - Click Continue.
- Now Click Done.
At this stage, you will see a list of all your service accounts
- Select the Service account you just created.
- In the service account Details -> Click On Keys tab.
- Click Add Key.
- Select
Create new key
option.
A popup will show up to select the key type.
- Select
Json
option. - Click on Create.
- The Created key will be downloaded.
It is important to keep this key safe. This key can be downloaded ONCE only.
Now that we created a service account, let’s give it access to the app.
Giving service account access to app
Let’s cruise back to the Google Play Console — API access section.
Now you should see your newly created account in the service accounts section.
- Click Grant Access on the account row.
This action will take you to invite user.
Under Permissions section, by default account permissions will be selected.
- Deselect all permissions.
- Switch to
App Permissions
. - Click Add app.
- Select the checkbox for your app.
- Click Apply.
After clicking Apply a popup will appear, Select the following permissions:
- Edit and delete draft apps.
- Release apps to testing tracks.
- Manage testing tracks and edit tester lists.
- Click on Apply.
Finally click Invite User. A popup will show, click Send Invite.
Now that we have a key to access the Google PlayConsole, let write our release lane.
Create the release lane:
First of all, you should create a lane that will handle uploading to Google Play Console by using upload_to_play_store
. You can see the documentation here.
The final Fastfile:
In your fast file you will be using environment variables. These variables can be set in two ways.
- Create a
.env
file inside thefastlane
directory. Then add all your variables to this file. This approach is a good approach to add non critical variable. - Add the variables to your
~/.zshrc
if you use zsh or to your~/.bash_profile
if you use bash. You can achieve this by addingexport ${variable_name}=${variable_value}
in the respective file.
I prefer using .env
for the non critical files, like bundle identifiers, workspace, schemes, google-services-info.plist path (referred to as gsp path) and etc. However I add the critical variables, such as access tokens and password, inside the shell environments. That’s to avoid pushing critical info to the git repo.
The Fastfile will look like this!
And the .env
:
Make sure that you fastlane directory is included in git.
Automate with Jenkins
In the first part of the article, we went through handling the build and push to TestFlight process without the using fastlane over the shell. Let’s automate the builds.
What is Jenkins?
The leading open source automation server, Jenkins provides hundreds of plugins to support building, deploying and automating any project.
In this article, I am setting up Jenkins on the local machine for simplicity. However, I will be collaborating with a colleague (Experienced DevOps Engineer) on an article on how to do the setup on a AWS Mac server
Define the automation process
First, you will have to create a file for Jenkins to define the automation process.
Jenkinsfile
Jenkinsfile is a text file that contains the definition of a Jenkins Pipeline and is checked into source control. Consider the following Pipeline which implements a basic three-stage continuous delivery pipeline
Jenkins will be executing shell commands and building our project by calling the lanes you already prepared in the Fastfile.
Now you should follow these steps to have your Jenkins file:
- Got to your project root directory.
- Create an empty file and name it
Jenkinsfile
. This file doesn’t have any extensions. - Open the text editor.
- On the first line, add
#! groovy
since Jenkinsfile written in groovy.
You can copy the following template to your file.
You will be doing the automation on multiple stages. These steps will be appearing on your console when building.
Stage 1: Setup
This stage makes sure that the environment is ready to use fastlane and any other gems you are using.
Stage 2: Build
In this stage you will check if the app builds
Stage 3: Publish to Google Play Console
In this stage, you will call the release lane, which will create an apk and publish it to Google Play Console.
The final Jenkins file will look like this:
Make sure that you Jenkinsfile is included in git.
Setup Jenkins
Install Jenkins on your machine
brew install jenkins
Automatically start Jenkins on mac start (Optional)
brew services start jenkins
This command will add Jenkins to the LaunchAgents which will make it run on mac start. After this, you MUST restart your mac for Jenkins to start.
Start Jenkins Manually
jenkins start
After having Jenkins running on your system, you can access it using your browser by going to 127.0.0.1:8080
or localhost:8080
.
Unlocking Jenkins
When you open Jenkins for the first time, it will ask you about the initial password to unlock.
You will be able to copy the initial Admin password using cat ~/.jenkins/secrets/initialAdminPassword | pbcopy
and then paste it in the displayed location.
Selecting plugins
Click on install suggested plugins and wait for the installation to finish.
Creating the admin user
Afterwards, Jenkins will open a panel to setup your admin account. Fill all the information and click on continue.
Instance Configuration
In this step, you can create your Jenkins URL. In this article we will stick with the localhost
.
Finally, click on Start Using Jenkins.
Project setup
Now it is time to setup your project.
After you click on New Item you will see the screen below.
- Enter the name of the project you want.
- Select Multibranche pipeline.
- Click OK.
Afterwards, the project configuration will open.
Jenkins Project Configuration
Add Source
Click on Add source and then select git.
You can connect to the git repo either using ssh
or using https
. In this example I will use https
.
Add Credential
Under credential, click add -> Jenkins. The following popup shows.
Fill the information and press Add
Branch Discovering
By default, Jenkins discovers all the branches and build the ones with JenkinsFile (will go through that in the upcoming steps).
If you want to add additional rules like filter, to filter by branch name, or add Tag discovery:
- Under Behavior section in Git, click on Add.
- A list shows and you can add the behaviors you want.
For example, you want to build only master
, develop
, and every branch starting with feature/
.
Finally, click Save in order to save the configuration.
Now Jenkins will connect to the repo and check if there are any branches to build.
Discover branches on git push
There are multiple ways to setup discovering branches on push.
Multibranch Scan Webhook Trigger is one of the easiest and best way to achieve our goal.
Install plugin to Jenkins
To install a plugin on Jenkins:
- Open Jenkins plugin manager screen:
- Go to Jenkins Home screen -> Manage Jenkins -> Manage plugins
OR
- Open Jenkins_URL/pluginManager - Select Available Tab.
- In the Search bar, type
Multibranch Scan Webhook Trigger
. - Click the box next to it.
- In the bottom of the screen, click Install without restart.
- The Plugin install screen shows, select the box
Restart Jenkins when installation is complete and no jobs are running
Add the webhook configuration
After Jenkins has restarted, Navigate to your project configuration and scroll down to Scan Multibranch Pipeline Triggers
.
Now you have a new option, Scan by webhook
. Select the box next to it and enter the token.
Configure Github webhook
After configuring Jenkins to check for build on webhook invocation, you need to configure the Github project.
- Open Github Project.
- Select Setting tab.
- Select Webhooks tab from the left menu bar.
- Click on Add webhook.
- In the Payload add:
${JENKINS_URL}/multibranch-webhook-trigger/invoke?token=${Token_specified_in_jenkins}
- Don’t add a secret.
- Select the event triggers.
- Click on Add webhook.
PS: The Jenkins URL shouldn’t be localhost
. If you are working on the local machine, you can use ngrok to expose localhost:8080
and access Jenkins.
Now you can try pushing any commit and see how Jenkins handling all the hassle!
PS: You might need to add your specify your Android SDK Path. Now you can add it to your Jenkins environment variables under the key ANDROID_HOME
.
Congratulations! Now you have successfully automated your signing and release process.
Final Note:
Now that you started benefiting for the wonders of automation. You can improve your CI/CD to handle running the tests, enforcing code style with ktLint or even taking screenshots!
You can always integrate your CI with many powerful plugins. For instance, you can inform your team of the new build using Slack Integration with fastlane, Jira integration with fastlane or Jenkins or tons of other integrations.