Lightweight Mobile DevOps on GCP (Cloud Next '18)

[MUSIC PLAYING] JOHN LABARGE: I’m going to talk about doing some Mobile DevOps on GCP It’s a bit of a unique talk that I’m using some of the tools here to do serverless builds of Android apps and distribution I actually will show some code and whatever, and if that’s what you’re here for, that’s good Might be one of the more technical talks I’m not sure So the first thing to get started is I’m going to talk about our little application I’ve built a little application that’s going to take the world by storm Not really, because I’m a developer and an ops person, not a designer Some of the GUI is not that fancy But I just built a little note-taking application And the idea is we’re going to simulate– we’re going to build this application and continually iterate on it, make changes to it, and run it through a DevOps pipeline and see what happens So a little idea This is the sad GUI that we have going on We can take the little text notes And we want to add image notes to our application, but they’re not done yet So we’re going to merge a feature and see what happens Hopefully you all can see the text well enough If you look at the bottom, there’s like a little terminal window in Android Studio So not a whole lot to see here, but what I’m going to do is try to merge a feature branch in here And just go ahead and– I want to make sure We’ve got lots of changes I guess it automerged And so I’m going to go ahead and push that code up to the repository Wi-Fi is getting slower by the minute, I guess, as people arrive But there we go So that’s all I’m going to do on the application for now I can show it to you as well It’s just a little note-taking application You can add text note, create a note, you can edit it a note Everything’s stored in Firebase And that’s how it works All right, so now let’s talk about DevOps for a minute Some lovely quotes here “DevOps is not a goal, but a never-ending process.” It is how most of us by now in the past like three or four years are used to working We want to get stuff– features in in the hands of our testers or our constituents in the company and get iteration faster, find the bugs faster, shorten the loop And so in this, what I’m going to demonstrate is a dogfooding pipeline, where basically we’re building and testing internally, developing, pushing stuff to a repository, and then sending out apps to our dogfooders What dogfood means at Google is sort of folks internally testing our applications before we release them, which I think is a fairly common term in industry And so with mobile, DevOps is a little bit funky, because unlike web, we’ve got to sign this binary We have to create the binary We have to sign the binary We have get the binary to the users, and they have to install it from somewhere And there’s lots of extra stuff going on there that’s not like web We can’t just build a new Docker container and launch it into Kubernetes We have to actually physically deliver a binary to the mobile user But we can still do that using this fun new product– old product– renamed product called Cloud Build, which was originally termed Container Builder and meant to be used to build Docker containers, and maybe deployed into Kubernetes and things like that It turns out, what we learned about Cloud Build is

that we can build anything, and the community taught us that the most There’s a Cloud Builders repository which I’ll show you that has many different types of builders for Cloud Build But back to Mobile DevOps So this is a flow for Mobile DevOps We trigger some build We test the build At least, a smoke test on the build And distribute that binary, and then maybe we monitor for crashing or other analytics So it starts with developers Hopefully it starts with a repository we’re all putting our code in I think that’s a pretty known commodity nowadays, but if you’re not using a source code repository, you probably should look into that And then we’re going to trigger a Cloud Build process And these are the repositories currently supported by Cloud Build You can use Google Cloud Source Repo, which I’m using here– GitHub, or BitBucket This is what it looks like to create a Build trigger There’s basically three steps You pick out the Source Control Repo you’re using, you select the specific repository, and then you set up your trigger When you set up your trigger, you can pick out a branch You can obviously name your trigger, pick out a branch, or a tag So you can trigger based on tag I’m using a branch here kind of a la GitFlow, but you could just as easily use a tag, which might actually be a little bit better for your specific case And this is how you trigger the build, which is what I just did So building the binary, again we were triggering a build in Container Builder– or Cloud Build, rather I’m going to mess that up a lot And that build is described by this funky magic file called cloudbuild.yaml And that is what describes every step that you want to take, just like you would in your– if you’re in Jenkins, then you have list of steps You would have that This is the description of the list of steps that the Cloud Build will take in order to build your binary and test it and do whatever else you want to do And in this case, we’re going to push source code And the steps are going to be described in the cloudbuild.yaml And the results of our Cloud Build is going to be a distribution through Fabric or Crashlytics data, which is going to end up emailing our users about the new application That slide didn’t show up right Sorry OK So the Cloud Build workflow is again, a bunch of steps that operate on your workspace So when that trigger happens, Cloud Build pulls the source code and drops in all in a workspace directory, just like Jenkins That probably sounds familiar to anybody who’s used Jenkins And then it runs a bunch of build steps The thing that’s interesting about it is that all of those build steps are containers And the commands they’re in are contained All of their dependencies are contained in Docker containers and images And so in this case, we’re doing a gsutil, copy stuff from Cloud Storage, an Android build, another test using the gcloud command, and some more Android build Each one of those steps is its own Docker container So you’re actually calling Docker Run in order to complete that step in the workspace And like I said before, there’s a rich set of those containers in the community site You can find stuff– the Android container is the one I’m discussing here, which is contributed by me But it’s open source, so if you have edits and PRs,

you can issue PRs to it But there’s, as you can see, a ton of different containers for doing builds and deployments There’s one for, for instance, create gcloud You can even create compute instances and everything like that There’s one for kubectl if you’re deploying a Kubernetes manifest, deploying a Kubernetes infrastructure And there’s one there that’s called Dataflow Python, which as you can imagine, is meant for setting up a Dataflow job So there’s just lots of possibilities So in using the Android Builder, what you’re going to do is build the container that’s going to run the Android build So it’s going to have the Gradle and Java SDK and the Android SDK And in order to build that up, what we want to do is we want to build a container that’s relatively lean So we don’t want to necessarily include every Android SDK from the beginning of time to now in that Docker container, because it would take a really long time to pull that from the registry So in order to semi-customize and package an Android builder for your own needs, you edit this packages.text file, which specifies all the packages you want in your Android Builder And every time you create a new Android Builder, you can change the packages you’re using Now the Android builder also requires you to supply your license key from your workstation So in your Android SDK home, you’re going to have a licenses directory, so you can supply your Android SDK license from there as a build substitution in Container But I’ll show you that in a minute But so each of these would have to have the EULA already accepted, and then the license key hash that you supplied to Container Builder will already know that you have accepted the license and it will can continue to build And so after that, this is the steps You clone the repo You create the packages.text file in that directory, and then you submit the Android Builder’s cloudbuild.yaml file, which is going to build a container in your Google Cloud’s Container Registry That’s going to have the Android Builder piece, and you can just go ahead and call that from your apps build, which I’ll go through So that’s the three steps And then the result is you have this Android Builder in your registry If you want more information on the Android Builder and more specifically, there’s a document drafted by me on creating this pipeline in more detail on our website, so you can hit up this document and it goes through all of the steps for setting up the Android Builder So you don’t have to remember everything that I’m saying here I’m just trying to give you an introduction But here’s some more details that you’re probably looking for So in getting more detail about what we need to do to build the mobile app, we are going to need to sign the Android application with a Keystore file, right? Everyone who is an Android developer probably is familiar with that And so the way we approach that in this, is we just go ahead and copy all of the signing information to a storage bucket And that storage bucket will only be accessible from the project that you’re using to build And you could secure that even further with key management, but that’s out of scope here You copy your Fabric secrets up to a different bucket or the same bucket And there’s an extra little optimization here called the build cache, which sort of tries to optimize the dependencies for Gradle so it’s not downloading the internet all the way from the internet, but maybe downloading the internet closer to where CloudBuild lives And then we move on for our test items The other thing to note here is the arguments in each one Those are actually arguments to the Docker run entry point

And you can change the entry point None of these actually change the entry point But you can change the entry point on these containers to just sort of execute arbitrary Bash as well and still have– for instance, let’s say in the gsutil I want to just do arbitrary Bash I can add an entry point in there, run that Bash But that Bash still has gsutil in the path for that command So that Bash command can call any of the dependencies from the gsutil container, for instance So this is sort of how you put together a build The other thing to note here is the substitutions we have Any of these shell variables with an underscore in the front is what we call a substitution in CloudBuild And what that is, is a value you supply when you configure the build trigger So you can update, you can basically parameterize your build So the signing API’s secret bucket path is where to find in Cloud Storage It’s actually a gs://uri The license is whatever you copy and paste it in for the license There’s a new version pending, by the way, where you can copy all of your Android licenses to another bucket and use a bucket reference instead That’s a PR that’s been submitted and currently pending And then another parameter to mention here is waitFor waitFor is how you wait for other things So you need your signing files, obviously, before you build the distribution version of the app And so, you’re going to want to wait for that copy to complete before you build the distribution file I’ll have to go on to the next slide to show more dependencies, but you get the idea So after we’ve built the binary, in this one we build the debug version of the binary And then we’re going to test that debug version with Firebase Test Lab, which is a service that Firebase provides that has actual physical devices in a data center that you can run both robot tests and functional tests on This pipeline specifically deals with robo tests, but you can also record and create Espresso functional tests, and run those tests on the devices in the data center So it lets you run detailed tests And you can see the results and add workflow, and et cetera Oh, yes Let’s take a look at the devices So I pulled up a list of devices recently from Test Lab So these are the devices you can actually run your functional and robo tests on There’s quite a few of them Some of them are virtual, you’ll see But most of them, the vast majority, are actual physical devices that your APK will be loaded onto The debug version of the APK will be loaded onto and tested, or the test APK as well if you’re using Espresso Let me switch And so, here’s our workflow again OK, so the Test Lab build step is interesting So here is where I had to use the Bash notion of, I’m running GCloud, but I’m going to wrap it in some Bash to make sure because the Test Lab command in GCloud doesn’t give me a good status process exit code Wouldn’t test pass or fail So I’ve added some fanciness here to check whether the test passed And there’s a corrupt statement further down But you can see more of the waitFor up here in the build So you can see some more of those waitFor arguments And this is sort of like some screenshots from a robo test I can actually flip over to the other laptop and show you some of the output of a robo test if you’re interested

So here you can see, this is some of the flow The robo test is kind of like a random touch and type test And you can see I’m using Firebase Auth I know this is a little small I can get into some screenshots But I’m using Firebase Auth here, which uses Google sign-in And actually, this thing comes up with its own Google account and signs in to the app, and then starts touching different aspects of the app just to see if anything crashes So let’s say this Create button was enabled in here Then when the robo test hits Create with an empty string, it might have caused a null pointer And if that happened, the test would have failed And you can also get a video recording of your robo test so you can go through and see what the automation did And this is actually on a Pixel So it’s getting the OAuth screen, and then saying yes And then now it’s got it open It’s a little flickery with the video, but you can see sort of what different screens it’s accessing and all of that It’s sort of like a blind person just trying to break your application, that’s the best way to describe it Going to flip back So the rest of the build is trying to distribute our– assuming the test pass, assuming there’s no failure in the robo test, we’re going to want to send it out to dogfooders for the folks internally for further usage And so, that’s what Fabric Crashlytics is used for And in this case, I’m using just a Gradle command for that You can actually use fastlane here But Android developers still like Gradle, I think Some are converting to fastlane I’m not really sure But this is, suffice to say, a Gradle implementation So a couple of things to note in this part of the build is that we’re wrapping the Gradle command with a script that does that fancy grab the cached jars from the storage bucket, and it copies the licenses over that you supply The other things you see here are substitutions So the build type is the beta build type, in this case And the email distribution group is the set of folks in Fabric where I want to email the build to And the upload distribution build type is the thing that actually uploads the APK to Fabric Crashlytics And then once that’s completely uploaded, your users that are your testers will get an email notifying them that there has been an update to the application And so, this is sort of what that tester UI looks like You can add emails for different applications you might have, and pick out which of those groups you want to email your results to And you can have different ones Maybe you have different build flavors or different build types, depending on who in the company You might have some that just go to developers, some that go to QA after that, some that, after that, might go to product managers And then, maybe, finally the whole company So you can really granularly adjust that Finally, what we do here is this is Crashlytics And this is an actual bug that I had in the code It’s hard to see stack trace, the call-in stack trace But I had a bug with my Firebase access And this was one of the notifications I got

from Crashlytics of the crash So you can– once this thing’s out in the wild or out to your users, depending on who they are, you can monitor for crashes So this is sort of like a full serverless pipeline where everything is done using managed services There’s nothing here that’s sort of an online thing you have to keep up and running There’s no Jenkins server, there’s no Kubernetes cluster to keep up and running There’s nothing to maintain It’s all sort of when you submit that new CloudBuild YAML, it boots up that container and starts running all the Docker commands So that’s sort of the use case here Just a minute All right So a few minutes ago I issued a build, and we can see the results of the build It was successful And this is sort of what the output of our pipeline looks like here You can see each command So this is running– Ubuntu is just an arbitrary shell running container So if I use Ubuntu, I can just call like mkdir, or whatever, stuff like that I’ve got a copy licenses step And I can just page through here I’m not going to go through everything The Test Lab one is interesting You can see that this one takes a bit of time, five minutes, to complete, which is why I do that at the beginning, and then wait until it finishes And you can see finally that at the end of this, there’s a whole other build And at the end, it finally sends it up to Fabrics There’s actually two Gradle builds that happen in here The first one is a debug build for the robo test And robo test takes the debug APK and does its testing on it And then finally, it’s push to Fabric And then from there I get an email, hopefully, if I can get my beach ball to stop There we go I should get an email to my user I’ll just go on the– Well, I should’ve gotten an email here Interesting Maybe I’m sending it to this one OK Yeah, so from there, my users would get an email, assuming this guy’s a tester Maybe he’s not I’m not sure I should have one on my phone OK, I have one on my phone Is it just this one? Just pull this out? Oh, here Oh, can we do this? SPEAKER: I don’t have it JOHN LABARGE: What’s that? SPEAKER: I don’t have it JOHN LABARGE: Oh, you don’t have it? Oh, OK Will it work if I just pull this out and then put it here? It may? I’ve never really connected an Android before Anyway, on my phone if you’re really interested, I have an email from Fabric with the new version And we can also take a look at the robo test that occurred, hopefully All right, so this is the last set of robo tests that just completed All right, so they look pretty much the same as the other ones But you can see that this one just happened about 20 minutes ago So that is, in a nutshell, how to do serverless managed DevOps with Android on GCP

It’s CloudBuild plus Firebase Test Lab