Automatic Signing and Publishing of Android Apps from Travis

As I discussed about preparing the apps in Play Store for automatic deployment and Google App Signing in previous blogs, in this blog, I’ll talk about how to use Travis Ci to automatically sign and publish the apps using fastlane, as well as how to upload sensitive information like signing keys and publishing JSON to the Open Source repository. This method will be used to publish the following Android Apps:

Current Project Structure

The example project I have used to set up the process has the following structure:

It’s a normal Android Project with some .travis.yml and some additional bash scripts in scripts folder. The update-apk.sh file is standard app build and repo push file found in FOSSASIA projects. The process used to develop it is documented in previous blogs. First, we’ll see how to upload our keys to the repo after encrypting them.

Encrypting keys using Travis

Travis provides a very nice documentation on encrypting files containing sensitive information, but a crucial information is buried below the page. As you’d normally want to upload two things to the repo – the app signing key, and API JSON file for release manager API of Google Play for Fastlane, you can’t do it separately by using standard file encryption command for travis as it will override the previous encrypted file’s secret. In order to do so, you need to create a tarball of all the files that need to be encrypted and encrypt that tar instead. Along with this, before you need to use the file, you’ll have to decrypt in in the travis build and also uncompress it for use.

So, first install Travis CLI tool and login using travis login (You should have right access to the repo and Travis CI in order to encrypt the files for it)

Then add the signing key and fastlane json in the scripts folder. Let’s assume the names of the files are key.jks and fastlane.json

Then, go to scripts folder and run this command to create a tar of these files:

tar cvf secrets.tar fastlane.json key.jks

 

secrets.tar will be created in the folder. Now, run this command to encrypt the file

travis encrypt-file secrets.tar

 

A new file secrets.tar.enc will be created in the folder. Now delete the original files and secrets tar so they do not get added to the repo by mistake. The output log will show the the command for decryption of the file to be added to the .travis.yml file.

Decrypting keys using Travis

But if we add it there, the keys will be decrypted for each commit on each branch. We want it to happen only for master branch as we only require publishing from that branch. So, we’ll create a bash script prep-key.sh for the task with following content

#!/bin/sh
set -e

export DEPLOY_BRANCH=${DEPLOY_BRANCH:-master}

if [ "$TRAVIS_PULL_REQUEST" != "false" -o "$TRAVIS_REPO_SLUG" != "iamareebjamal/android-test-fastlane" -o "$TRAVIS_BRANCH" != "$DEPLOY_BRANCH" ]; then
    echo "We decrypt key only for pushes to the master branch and not PRs. So, skip."
    exit 0
fi

openssl aes-256-cbc -K $encrypted_4dd7_key -iv $encrypted_4dd7_iv -in ./scripts/secrets.tar.enc -out ./scripts/secrets.tar -d
tar xvf ./scripts/secrets.tar -C scripts/

 

Of course, you’ll have to change the commands and arguments according to your need and repo. Specially, the decryption command keys ID

The script checks if the repo and branch are correct, and the commit is not of a PR, then decrypts the file and extracts them in appropriate directory

Before signing the app, you’ll need to store the keystore password, alias and key password in Travis Environment Variables. Once you have done that, you can proceed to signing the app. I’ll assume the variable names to be $STORE_PASS, $ALIAS and $KEY_PASS respectively

Signing App

Now, come to the part in upload-apk.sh script where you have the unsigned release app built. Let’s assume its name is app-release-unsigned.apk.Then run this command to sign it

cp app-release-unsigned.apk app-release-unaligned.apk
jarsigner -verbose -tsa http://timestamp.comodoca.com/rfc3161 -sigalg SHA1withRSA -digestalg SHA1 -keystore ../scripts/key.jks -storepass $STORE_PASS -keypass $KEY_PASS app-release-unaligned.apk $ALIAS

 

Then run this command to zipalign the app

${ANDROID_HOME}/build-tools/25.0.2/zipalign -v -p 4 app-release-unaligned.apk app-release.apk

 

Remember that the build tools version should be the same as the one specified in .travis.yml

This will create an apk named app-release.apk

Publishing App

This is the easiest step. First install fastlane using this command

gem install fastlane

 

Then run this command to publish the app to alpha channel on Play Store

fastlane supply --apk app-release.apk --track alpha --json_key ../scripts/fastlane.json --package_name com.iamareebjamal.fastlane

 

You can always configure the arguments according to your need. Also notice that you have to provide the package name for Fastlane to know which app to update. This can also be stored as an environment variable.

This is all for this blog, you can read more about travis CLI, fastlane features and signing process in these links below:

Automatic Signing and Publishing of Android Apps from Travis

Enabling Google App Signing for Android Project

Signing key management of Android Apps is a hectic procedure and can grow out of hand rather quickly for large organizations with several independent projects. We, at FOSSASIA also had to face similar difficulties in management of individual keys by project maintainers and wanted to gather all these Android Projects under singular key management platform:

To handle the complexities and security aspect of the process, this year Google announced App Signing optional program where Google takes your existing key’s encrypted file and stores it on their servers and asks you to create a new upload key which will be used to sign further updates of the app. It takes the certificates of your new upload key and maps it to the managed private key. Now, whenever there is a new upload of the app, it’s signing certificate is matched with the upload key certificate and after verification, the app is signed by the original private key on the server itself and delivered to the user. The advantage comes where you lose your key, its password or it is compromised. Before App Signing program, if your key got lost, you had to launch your app under a new package name, losing your existing user base. With Google managing your key, if you lose your upload key, then the account owner can request Google to reassign a new upload key as the private key is secure on their servers.

There is no difference in the delivered app from the previous one as it is still finally signed by the original private key as it was before, except that Google also optimizes the app by splitting it into multiple APKs according to hardware, demographic and other factors, resulting in a much smaller app! This blog will take you through the steps in how to enable the program for existing and new apps. A bit of a warning though, for security reasons, opting in the program is permanent and once you do it, it is not possible to back out, so think it through before committing.

For existing apps:

First you need to go to the particular app’s detail section and then into Release Management > App Releases. There you would see the Get Started button for App Signing.

The account owner must first agree to its terms and conditions and once it’s done, a page like this will be presented with information about app signing infrastructure at top.

So, as per the instructions, download the PEPK jar file to encrypt your private key. For this process, you need to have your existing private key and its alias and password. It is fine if you don’t know the key password but store password is needed to generate the encrypted file. Then execute this command in the terminal as written in Step 2 of your Play console:

java -jar pepk.jar –keystore={{keystore_path}} –alias={{alias}} –output={{encrypted_file_output_path}} –encryptionkey=eb10fe8f7c7c9df715022017b00c6471f8ba8170b13049a11e6c09ffe3056a104a3bbe4ac5a955f4ba4fe93fc8cef27558a3eb9d2a529a2092761fb833b656cd48b9de6a

You will have to change the bold text inside curly braces to the correct keystore path, alias and the output file path you want respectively.

Note: The encryption key has been same for me for 3 different Play Store accounts, but might be different for you. So please confirm in Play console first

When you execute the command, it will ask you for the keystore password, and once you enter it, the encrypted file will be generated on the path you specified. You can upload it using the button on console.

After this, you’ll need to generate a new upload key. You can do this using several methods listed here, but for demonstration we’ll be using command line to do so:

keytool -genkey -v -keystore {{keystore_path}} -alias {{alias_name}} -keyalg RSA -keysize 2048 -validity 10000

The command will ask you a couple of questions related to the passwords and signing information and then the key will be generated. This will be your public key and be used for further signing of your apps. So keep it and the password secure and handy (even if it is expendable now).

After this step, you need to create a PEM upload certificate for this key, and in order to do so, execute this command:

keytool -export -rfc -keystore {{keystore_path}} -alias {{alias_name}} -file {{upload_certificate.pem}}

After this is executed, it’ll ask you the keystore password, and once you enter it, the PEM file will be generated and you will have to upload it to the Play console.

If everything goes right, your Play console will look something like this:

 

Click enrol and you’re done! Now you can go to App Signing section of the Release Management console and see your app signing and new upload key certificates

 

You can use the SHA1 hash to confirm the keys as to which one corresponds to private and upload if ever in confusion.

For new apps:

For new apps, the process is like a walk in park. You just need to enable the App Signing, and you’ll get an option to continue, opt-out or re-use existing key.

 

If you re-use existing key, the process is finished then and there and an existing key is deployed as the upload key for this app. But if you choose to Continue, then App Signing will be enabled and Google will use an arbitrary key as private key for the app and the first app you upload will get its key registered as the upload key

 

This is the screenshot of the App Signing console when there is no first app uploaded and you can see that it still has an app signing certificate of a key which you did not upload or have access to.

If you want to know more about app signing program, check out these links:

Enabling Google App Signing for Android Project

Preparing for Automatic Publishing of Android Apps in Play Store

I spent this week searching through libraries and services which provide a way to publish built apks directly through API so that the repositories for Android apps can trigger publishing automatically after each push on master branch. The projects to be auto-deployed are:

I had eyes on fastlane for a couple of months and it came out to be the best solution for the task. The tool not only allows publishing of APK files, but also Play Store listings, screenshots, and changelogs. And that is only a subset of its capabilities bundled in a subservice supply.

There is a process before getting started to use this service, which I will go through step by step in this blog. The process is also outlined in the README of the supply project.

Enabling API Access

The first step in the process is to enable API access in your Play Store Developer account if you haven’t done so. For that, you have to open the Play Dev Console and go to Settings > Developer Account > API access.

If this is the first time you are opening it, you’ll be presented with a confirmation dialog detailing about the ramifications of the action and if you agree to do so. Read carefully about the terms and click accept if you agree with them. Once you do, you’ll be presented with a setting panel like this:

Creating Service Account

As you can see there is no registered service account here and we need to create one. So, click on CREATE SERVICE ACCOUNT button and this dialog will pop up giving you the instructions on how to do so:

So, open the highlighted link in the new tab and Google API Console will open up, which will look something like this:

Click on Create Service Account and fill in these details:

Account Name: Any name you want

Role: Project > Service Account Actor

And then, select Furnish a new private key and select JSON. Click CREATE.

A new JSON key will be created and downloaded on your device. Keep this secret as anyone with access to it can at least change play store listings of your apps if not upload new apps in place of existing ones (as they are protected by signing keys).

Granting Access

Now return to the Play Console tab (we were there in Figure 2 at the start of Creating Service Account), and click done as you have created the Service Account now. And you should see the created service account listed like this:

Now click on grant access, choose Release Manager from Role dropdown, and select these PERMISSIONS:

Of course you don’t want the fastlane API to access financial data or manage orders. Other than that it is up to you on what to allow or disallow. Same choice with expiry date as we have left it to never expire. Click on ADD USER and you’ll see the Release Manager created in the user list like below:

Now you are ready to use the fastlane service, or any other release management service for that matter.

Using fastlane

Install fastlane by

sudo gem install fastlane

Go to your project folder and run

fastlane supply init

First it will ask the location of the private key JSON file you downloaded, and then the package name of the application you are trying to initialize fastlane for.

Then it will create metadata folder with listing information excluding the images. So you’ll have to download and place the images manually for the first time

After modifying the listing, images or APK, run the command:

fastlane supply run

That’s it. Your app along with the store listing has been updated!

This is a very brief introduction to the capabilities of the supply service. All interactive options can be supplied via command line arguments, certain parts of the metadata can be omitted and alpha beta management along with release rollout can be done in steps! Make sure to check out the links below:

Preparing for Automatic Publishing of Android Apps in Play Store

Apply for Your Free Stay during the FOSSASIA Summit 2017 with our 100 #OpenTechNights Program

The FOSSASIA Summit 2017 takes place from Friday March 17 – Sunday March 19 at the Science Centre Singapore. We are now inviting Open Source contributors to apply for a free stay in a Singapore hostel and a free ticket to the event. All you have to do is convince us, that you are an awesome Open Source contributor!

The details

Developers from all over the world are joining the FOSSASIA Summit. We want to connect established and new Open Tech contributors alike. Therefore FOSSASIA is supporting the Open Source community to join the event by offering 100 free nights stay at a hostel in the centre of Singapore and a free ticket to the event. All you have to do is to fill in the form with information that convinces us that you are an awesome contributor in the Open Source community.

The Process

Step 1: Please fill in our form here before February 17 (23:00 Singapore Time).

Step 2: We will get back to you at latest within 3 days after the deadline if you are selected. But, also we are choosing very convincing applicants on an ongoing basis. So, the earlier you apply the higher your chances to get a free stay might be.

Step 3: The selected applicants will need to confirm their itinerary and tickets before March 1st to re-assure their free stay in Singapore.

Expectations of Participants – Share what you learn

1. Please support volunteers, speakers and participants at the event. Let’s bring all this good spirit of sharing Open Technologies and learning together!

2. Help to reach out to participants who cannot join us at the event. For example make some tweets, share what you learn on social media, publish photos and put up blog posts about the summit.

Our Team

Our team of “100 #OpenTechNights” – Hong Phuc Dang, Mario Behling, and Roland Turner – is excited to meet you in Singapore!

Apply Now

Apply for a free stay with #FOSSASIA #OpenTechNights and participation at the FOSSASIA Summit 2017 now here!

More Information

More updates, tickets and information on speakers also on our #OpenEvent system: https://eventyay.com/e/45da88b7/

Apply for Your Free Stay during the FOSSASIA Summit 2017 with our 100 #OpenTechNights Program

FOSSASIA Summit 2016 Science Centre Singapore – Wrap Up

FOSSASIA 2016 took place from 18th -20th March in Singapore. Hong Phuc DangMario BehlingHarish Pillay, and Roland Turner were leading the organization efforts for the 2016 summit supported by many volunteers, speakers and the community. With a good mix of 37 nationalities, we are proud to be one of most international developer events in Asia.

We would like to especialy thank our host venue and the wonderful team of the Science Centre Singapore, our partner UNESCO Youth Mobile and our sponsors Red Hat, Google, GitHub, MySQL, Hewlett-Packard Enterprise, gandi.net, General Assembly and the Internet Society Singapore for their support and participation. Thanks to everyone who helped to make FOSSASIA 2016 in Singapore possible!

FOSSASIA 2016 Group Photo at Science Centre Singapore by Michael Cannon

FOSSASIA’16 NUMBERS & FACTS

  • We reached the number of 2,917 attendees over 3 days including 230 speakers and 72 volunteers.
  • With a good mix of 37 nationalities, we are proud to be one of most international developer events in Asia.
  • There were 201 scheduled sessions and lightning talks, and more 50 exhibitors.
  • This was the first year we organised Tech Kids program with 14 hands-on workshops that covered Mobile Development, Electronics, Digital Fabrication, Pocket Science and 3D Modeling.
  • Dozens of talks are already available as videos. Thousands of photos have been uploaded to social networks. 1500+ tweets with the FOSSASIA hashtag were posted during the event.
  • A trend analysis of FOSSASIA shows that web technologies, data analytics and Internet of Things have a huge momentum. The attention of developers is also increasingly turning to open hardware.

Opening HallMario Behling the superman behind our programCat Allman

Happy Volunteers

Day 1 Opening of FOSSASIA

The first day started at the OpenTech and IoT track with a warm welcome message from Mr. Lim Tit Meng, the director of Science Centre, follow by some of our keynotes including Cat Allman with her inspiring story on Science & Education Program at Google; Harish Pillay with his intriguing title ‘A Funny Thing Happened On My Way To The Science Centre’ revealing the history of Internet and Open Source; Bernard Leong caught a huge attention on ‘Rethinking Drone Delivery with Open Source’; and Davide Storti introduced the exciting MobileYouth Program at UNESCO. The day continued with many other interesting talks/discussions and five other tracks were opened that afternoon of the same day namely Tech Kids, Hardware and IoT, DevOps, Big Data, Internet Society and Community.

More Photos: [Photo 1], [Photo 2], [Photo 3] – Tech Kids Track

Day 2 Intensive day of workshops and more discussion

Stephanie Taylor opened the second day of FOSSASIA with her informative presentation on Google Summer of Code Program and Google Code-In. Many GSoC and GCI students from Asia attended this year FOSSASIA. The day continued with series of workshops and discussions on Hardware, IoT, and DevOps. Four new tracks were added into the program including OpenTech Workshop, Python, WebTech and Databases.

Popular DevOps Track

Harish Pillay proudly presenting his first computer

Day 3 Hack Sunday and the closing notes

At the last day, we opened another three new tracks: Privacy and Security, Linux and MiniDebConf, Design VR and 3D. More hacking activities took place on Sunday. Participants formed in-depth discussion groups.

People gathering at the closing

Exhibition

More than 50 project booths and hand-on demos were set up in the Science Centre’s public space where participants could hang out, chat, discuss, share, learn, and hack.

Nanyang Polytechnic teacher and students presenting their Student Enrich ProgramExhibition hallUNESCO YouthMobile InitiativeSnapshot of Red Hat booth – Developers ChatGitHub corner

FOSSASIA – a place of friendship and joy.

As always thanks to our photographer Michael Cannon and his team for capturing some of the very best moment of us. You can search for more photos by typing #fossasia on Twitter or Flickr. If you also want to share some photos you took during FOSSASIA with us, please get in touch with me [email protected]

Excited developers from across Asia
Baby Py with her parents at the social event

What’s next in 2016?

  • FOSSASIA will again participate at Google Summer of Code
  • Call for collaboration: We welcome new contributors to FOSSASIA current projects
  • A number of new releases of FOSSASIA software projects and our event planning applications are planned. Please check out http://github.com/fossasia and http://github.com/loklak
  • Many people in the FOSSASIA community organize developer meetups throughout the year. Please join our meetups in Singapore, in Dubai and many other cities in Asia.

Blog Posts

Many participants at FOSSASIA have blogged about the event. Some links here:

Kushal Das – kushaldas.in
Fedora Community Blog – p96.io
Anwesha Das – anweshadas.in
Garvit Delhi – garvitdelhi.blogspot.com
Ankit Ashukla – ankitashukla707.wordpress.com
Sundeep Anand – sundeep.co.in
Jigyasa Grover – jigyasagrover.wordpress.com
Michael Downey – talk.openmrs.org
Owais Zahid – eleventhlane.wordpress.com
Woo Hui Ren – woohuiren.me
Daniel Pocock – danielpocock.com
Tobias Mueller – blogs.gnome.org/muelli
Menghsuan Tsai – facebook.com/notes/

Links

FOSSASIA Photos: https://www.flickr.com/photos/fossasia/

FOSSASIA Videos: Youtube FOSSASIA 

FOSSASIA on Twitter: https://twitter.com/fossasia

FOSSASIA Sg Meetup: http://www.meetup.com/FOSSASIA

FOSSASIA Summit 2016 Science Centre Singapore – Wrap Up