Linux for Chromebooks: Secure Development (Google I/O ’19)

[MUSIC PLAYING] SUDHA BROSLAWSKY: Good afternoon, everyone I hope you are having a great time at the I/O We are here to talk to you about Linux for Chromebooks, also known as Crostini We’ll start by introducing ourselves My name is Sudha I’m a designer on Crostini for Chromebooks DYLAN REID: Hi, I’m Dylan I’m the Chrome OS Virtualization Lead TOM BUCKLEY: And I’m Tom, Product Manager for Linux on Chromebooks Now it’s the end of day two at I/O and you’ve probably already been to a bunch of different sessions that have talked about all the new frameworks that you need to be using or the platforms that you need to be building for, and everyone’s right, you absolutely should be, but we’re not really here to talk about that Instead what we want to talk about is you as developers and how you can get more peace of mind by using Linux on Chromebooks We give you that peace of mind by balancing simplicity and security SUDHA BROSLAWSKY: On that note, let’s do a quick user study How many of you are developers in the audience? Wow, that’s a full room, as expected Keep your hands raised How many of you use your computers for anything else other than development, like doing your taxes, checking email– again, 100% of you OK, one last question How many of you are worried about security? DYLAN REID: Good, that’s pretty– I mean, you all should be, so I’m glad to see many hands up anyway So I don’t know about you, but when I start a new project, I get stuck a lot I hit a lot of walls and I hit a lot of barriers and I go to look for a problem, go to look for a solution, and I turn to Google And luckily, Google’s almost always got a great answer for me Unluckily, sometimes the answer looks like this And I know I shouldn’t run this script from evilsite.com and pipe it to Sudu, but that deadline’s coming up and maybe this site looks kind of legit, so in this case, I’ll make an exception and I’ll do this And then it happens again and again, and eventually, I end up with a system that I don’t trust as much as I should, because I don’t really know what code I’ve run on it anymore I don’t have time to read all these scripts My solution to this has been to carry two laptops– one for my developer world and one for my everything else world that I want to be secure in But recently, I’ve switched to using one laptop, and Tom’s going to talk about how I do that TOM BUCKLEY: So our goal with Chrome OS has been to give you a simple and secure experience from the start, but if you tried it previously, you might have seen that it wasn’t quite ready for developers In order to be simple and secure, we couldn’t run all of the tools that developers need to get their job done But that all changed at I/O last year when we announced that we were going to start supporting Linux on Chromebooks Linux on Chromebooks lets you run all of your favorite editors, IDEs, and tools, and it’s now supported on over 50% of Chromebooks, including great devices with eighth generation Intel CPUs like the Lenovo Yoga Book C630 and the Acer Chromebook Spin 13 If you haven’t seen it, we’re going to run through a few examples First off, how do you get it? It’s really easy It’s just a couple clicks Now in the background, this is downloading a virtual machine, setting up containers, configuring it all– Dylan’s going to go more into that in a few minutes, but for you as a developer, it’s just a couple clicks to get started And this adds a terminal to your launcher Now if you open that terminal, you’ll see that you have a pretty standard Debian environment and we’ve already loaded in a bunch of the tools that developers expect, like Git and Vim, and if you need anything else, you have the APT Package Manager and you can install whatever packages you need And if you want to instead install programs via .dev files, you can do that, too This gives you access to thousands of world-class developer tools Now once you’ve installed any graphical apps, you’ll find that they all show up in your launcher, just like the rest of your Chrome OS apps, and if you open them, they show up in your Window Manager– again, just like the rest of your Chrome OS apps This is the simple experience that people have come to expect from Chrome OS and we didn’t want to change that with Linux But importantly, this is also secure You don’t have to worry about malware accessing your files, snooping on your traffic, or infecting your peripherals I’d ask you to trust us on that but this is way too important for you to take on trust alone,

so over the course of this talk, Dylan and Sudha are going to walk you through the principles behind the architecture and design of Crostini We’re then going to dissect some common developer flows to show you how these principles apply And finally, we’re going to share some tips and tricks for advanced usage for the power users out there So now I’m going to hand it over to Dylan to talk about the architecture [APPLAUSE] DYLAN REID: Good job OK So Chrome OS has always had a layered approach to security and our big layer has always been the browser and the renderer and running all untrusted code in a nice isolated renderer, and that keeps the attack surface of your core system to an absolute minimum They’re not allowed to make a lot of system calls They can’t poke at random bits of your kernel And that worked really well for web pages, web apps However, for developer tools, I need to install a lot of different programs They need a lot of different privileges They can do anything any app on Linux can do And that wasn’t acceptable for us on the core of Chrome OS, so we needed to add a layer So we added a virtualization layer, and that lives in the main Chrome OS layer, and that spins up a VM And now this VM has a much more limited interface while still exposing a full Linux kernel to the programs that run inside the VM The only way the VM can talk to Chrome OS proper is through a small API that that cross-VM program on the left up there exposes to the guest This is pretty good Now we’ve greatly reduced attack surface We were pretty happy with this We wanted to go a little further So we made sure that the guest VM was also signed by Google and somewhat trusted This lets us trust some of the actions the guest VM takes, and it’s also read-only, so users can only break things so much And that, no matter what you do, you’re going to be able to boot a VM However, with all that security solved, we’re back in a situation where you don’t have enough flexibility Your apps can’t do anything It’s a read-only thing You can’t install anything in it So we added another layer, and for this, we stole– used– LXD from Canonical That team’s been very helpful in getting this spun up with us It’s a pretty standard container runtime It’s built for running system containers And in our case, we started a system container of Debian and exposed that to the user So that cross-VM layer I was talking about, that’s kind of the most important part of the security story here It’s the last line of defense before something gets into Chrome OS So we went and we focused on this for a long time and made sure we got that as secure as possible We wrote it in a memory-safe programming language We chose Rust This eliminates buffer overflows and integer overflows and a lot of common bugs related to memory safety that are exploited by attackers We were pretty happy with that, but we again added another layer of security here in that we broke up the virtualization program into pillars and made sure that each pillar that interfaces with the guest only has access to small parts of your host Chrome OS system So your host Chrome OS system, you’ve got your bank’s web page open, you’ve got your online tax filing thing open, you’ve got all kinds of personal identifiable information everywhere We really wanted to protect that But we needed to give the guest access to things, like a random number, a display, a USB device So each of those got their own jail and they can only see the thing they need So our random number generator can generate random numbers It can’t access any files It’s in an empty file system from its perspective It doesn’t have any network access The display driver, it can access the display Again, it can’t touch the network It can’t go grab your files and upload them, even if somebody gets into it and tries to make it do things we didn’t intend it to This is all a little complicated, but we’ve added a great amount of system UI to make this easy for you to use, so when you’re just doing your job as a developer, you don’t have to worry about these pretty pictures I’ve drawn for you And Sudha will show you what we did SUDHA BROSLAWSKY: Thank you, Dylan Security is absolutely top of mind for us While crafting the Linux experience on Chromebooks, we came up with three high-level design goals The first goal was to keep your experience intuitive

Everyone here in this room has been using computers for a long time and you have established your workflows and habits So basically what we wanted to do is to match to those expectations We wanted to provide an experience that’s natural to you We want developers everywhere to be using Chromebooks and feel right at home doing it The second goal was to make your experience native We could have taken the easy path by giving you a full Linux desktop in a VM but that wasn’t good enough Our goal was to bring the Linux apps you depend on for development into your native Chrome OS experience The third goal was to make your experience simple, and I think this is very important There’s a lot of complexity that’s going on under the hood and we want to leave it there Our guiding principle is that complexity shouldn’t interfere with the user experience There’s a couple of things we are trying to balance here The security concerns that come with installing Linux apps on Chromebooks and the simplicity that comes with sticking to design patterns established by Chrome OS And our mission was to find that sweet spot TOM BUCKLEY: All right So now we’re going to talk about three common developer flows and see how they work with Crostini And the first of these is accessing files As developers, we have to do this all the time Our editors need to access files, as do our compilers, our source control, and a whole lot more But the problem is that our file systems have a lot more than just code They have our personal photos, our tax returns, maybe that novel that you’ve been working on A lot can go wrong Ransomware can hold all of that data hostage Malware can upload your files to some random server Or maybe you just get something that goes and deletes everything for the fun of it We built Crostini with those threats in mind to limit what can go wrong, and Dylan will tell you how DYLAN REID: So our goal sharing files with your VM and with your container was to make it easy for you to get the files you needed for your development tasks where you need them but not expose things you don’t want exposed to untrusted code Because ultimately, we don’t trust the code that’s running inside this VM To do this, we took a layered approach Your files all live in Chrome OS at the very bottom, and we share them out to the VM with a 9P server We named it 9S Again, we wrote it in Rust so it’s memory safe, we fuzzed it to make sure unexpected inputs don’t cause unexpected behavior, and we put it in a tight jail so it can access only the files you share with it And it takes those files and exports them to the VM The VM mounts the 9P thing that’s built into Linux, and then LXD takes that mount and exposes it into your container where your development tools are running The important thing here is that your container can only see files that you say, I want to share with my development environment Your VM can only see those same files And even the server that we wrote, running on Chrome OS, can only see those files It doesn’t get to see everything So somebody exploits this stack all the way back into Chrome OS They still don’t have access to the files you haven’t shared with the container That’s a lot of stuff to set up Setting up 9P amounts, bind-mounting things into containers We had to do this manually for a while when we were developing it It was painful So I’ll let Sudha show you how easy we made it for you SUDHA BROSLAWSKY: There are a lot of layers going on, but let’s see how simple this is in the UI Right out of the box, you have a directory called Linux Files, which is your home directory within Linux Anything in this directory is automatically shared with Linux Outside of this directory, anywhere else on the device, Linux doesn’t have access to anything until you grant permissions I’ll walk you through a couple of examples here Let’s say you’re working on a project and you see yourself needing files from this one folder called Illustrations To share this, all you have to do is access the right-click menu and click on Share with Linux And as simple as two steps, you’ve

now shared this folder with Linux If you notice, this is in Google Drive, and that’s a cool thing When you don’t want to share this anymore, you can do that by going to Settings and Unsharing Here’s another example where we made quick edits really simple for you You have a data file in your Downloads folder, and when you double-click, it automatically opens in VS code When this happens, in the background, it’s implicitly shared And the sharing lasts until you restart This is the balance of security and simplicity we wanted to bring you TOM BUCKLEY: Thank you So, for our second developer flow that we’re going to talk about, we’re going to look at running a web server Now being Chrome OS, we care a lot about people making great web apps, and we want to make sure that they can create those on a Chromebook And being able to run a web server is pretty central to being able to build any web app Unfortunately, web servers also need a pretty low level of access and that can cause some problems The code that can run a web server is also capable of snooping on your internet traffic It can know what sites you’re accessing, and in some cases, even see the contents of those pages This means that a malicious web server could potentially track everything that you’re doing Now again, we thought of this as we designed Crostini and we made sure that we prevented this kind of attack Dylan will tell you how DYLAN REID: I can be called Linux, that’s OK That’s my job All right, so starting a web server from Crostini, simple We’ve got a good demo over in the web dev sandbox already Type a command You fire up your web server, just like you would on any Linux distribution out there What’s actually happening under the hood, though, is you’re in a container and you open up a port That port’s in a network namespace inside a VM, running under our special hypervisor which puts its network stack in another namespace on the host, and then finally out to Chrome So Chrome can’t get back in, which is great for security You’ve got wonderful isolation But if I want to test this new PWA or web page I’m running in my VM, how do I get Chrome to talk to it? This was not simple So for that, we had to add some daemons along the way– actually, every layer gets a daemon for this The first one’s running in the VM and it’s sitting there waiting to check if any container that’s running happens to open a port And then it’s got to figure out which container opened that port And bundles that information up, sends it to Chrome OS Says, hey, this port in this container is listening The user might want to use that port And on the Chrome OS, we say, OK The other daemon responds Says, I will set up a route to do some forwarding I’m going to forward all of this over VSOCK, which is a protocol used to talk to local VMs on machines That’s kept under the hood So either end, it talks HTTP into the daemons, and the daemons talk VSOCK to each other So the key here is that the web server gets to talk HTTP Chrome gets to talk HTTP Everything’s normal Everything works just like you would Well, under the hood, we’ve got all these extra daemons and VSOCK forwarding going, but we’ve hidden that One other important thing, we’ve made it trusted So you can get all your PWA features You can install it to your desktop Even though it’s not technically the same machine, we know it is because we’ve got the information We set up the VM So we allow it to be trusted domain And all this complexity I think makes one of our best demos today of how complicated we made it under the hood and how simple you’re going to see it as actual use SUDHA BROSLAWSKY: I totally agree that this is very complicated under the hood, but in the UI, it’s exactly like you would expect it to be Let’s say you’re experimenting with building this cool PWA Here in terminal, you’re in your folder, PWA Starter Kit You’re running the commands to start your web server And if you see at the bottom of the screen, it’s listening at port 8080 At this point, you can launch your browser, go to local host 8080, and test your web app On the screen here, on the left, you have your web app in Chrome, and on the right, if you’re noticing it, it’s in Chrome Yes, you can test your web app on a Chromebook in Firefox,

too If you notice, we did not prompt you to give any permissions while we were in this flow This is because the host is accessing the VM and not the other way around Again, this is another way we kind of balanced the security and simplicity factor we were talking about TOM BUCKLEY: All right Finally, for our third demo, we’re going to talk about testing an Android app Now this is really exciting because just yesterday, we announced that Android Studio is officially supported on Chromebooks, and we even created an installer just for Chrome OS to make it really easy to get started with Now of course, Android Studio isn’t the only thing that you need in order to build a great Android app You also need something to test that app on– usually a phone– and while you could do that over Wi-Fi with ADB remote and all this sort of stuff, we wanted to make it easy– just the experience that you’d expect on any other device I can plug my phone in over USB and test my app that way Now, if I’m an Android developer, sure, I’ll plug my phone in to test my app, but I’m also going to plug in a lot of other devices over USB over the course of my day I may plug in a USB drive that has a lot of family photos on it I may plug in a wearable that has some health information I may even plug in my security key for work that gives me all of my access Malware can take advantage of these devices to uniquely identify you as you move between machines, to spread itself, or even to make changes to them Again, we thought of these threats when designing Crostini and made sure that we were preventing them DYLAN REID: Yeah, implementing USB was a lot of fun for us It might have been our most painful stack Same principles apply We’ve got our layers We protect the host There’s a lot of attack surface in a host USB stack It’s a very complicated kind of loosely-specced protocol Well it’s an exact spec that’s loosely implemented by a lot of people So we’ve hidden that Kept that on the host side Wrote a device that we live in cross-VM jail Again, we’ve got a USB driver It’s pretty complicated It’s got a lot of code in it and I’m sure there’s a bug or two So we made sure it was very well isolated It can’t get to your files It can’t get to the network It also can’t get to any USB device You have to explicitly say, hey, I want to give this USB device to my development environment And we’ve tried to make that as easy as possible And what actually happens under the hood, we’ve always got an emulated USB bus running, so the guest always sees, hey, I’ve got a USB bus, there’s just nothing plugged in And once you indicate that I want to give this to my VM, it says, OK, I’m going to add this device to this bus and then we show it to the guest And then the guest again in turn has to forward that into the container and the container can see it There’s two things we really focused on here One was security Again, we addressed that with the jail and we made sure that the attack surface was as minimal as possible It’s also written in Rust and it’s nice and memory-safe and it’s fuzzed But the other issue here is privacy because people somehow use lists of USB devices attached to machines to fingerprint and track users, and we wanted to make sure the untrusted code running inside the container couldn’t be another way to do that Again, this is a lot of steps We have to create a device We have to export it to a VM We have to export it to a container We have to decide which device to export or not And again, we’ll have a demo that shows how easy it is SUDHA BROSLAWSKY: OK This is the last demo Let’s say I’m on my Linux-enabled Chromebook and you’re plugging in your phone You’ll see a notification that prompts you to connect to Linux At this point, only Chrome OS has knowledge of your phone Linux doesn’t even know that your phone exists and that’s a good thing If you see here, your phone is not listed in the USB list, but when you rerun the command, once you connect on the notification, your phone shows up in the list At this point, you established access to Linux to your phone Let’s say you’re working on a project You’re developing a cool app again in Android Studio, and you’re ready to test it out You hit Run and select the phone, and boom

Just like that, you’re able to test your app on your phone At this point, you can debug and test out your app Finally, you can go to Settings and manage what Linux access to at any point of time So you can see how security is at the core of your Linux experience on Chromebooks You, the user, are in full control at all times of what Linux has access to We take advantage of a variety of UX patterns to make it simple to use and also native to Chrome OS The combination of principles of Chrome OS and Crostini make this experience pretty unique DYLAN REID: Thanks My turn All right, good We’ve got plenty of time So we’ve been talking about a lot of details and I’ve been talking a lot about layers and jails, and all that’s important and it’s a good reason for you to trust our normal flows, and when I’m using my Chromebook, I almost always stay within these common workflows that we’ve polished and made sure work However, a lot of that technical detail I was talking about is still usable, and we’ve left hooks in for you to play with it So I’m glad I’ve got time left so I can go through a few of these examples and kind of just whet your appetite for what else you can do We don’t test this stuff We don’t support this stuff We really want the standard flow to be enough for everybody But every once in a while, there might be a reason you want to do something a little more advanced, or you might just want to go have fun and play with things under the hood We’re tinkerers, right? We’re supposed to be So we’ll go through and show how some of this stuff works All this is going to be from the Chrome OS shell This has been in Chrome OS since– well, longer than I have And so Control-Alt-T gets you a shell There’s a set of debug commands you can run We’re going to focus on one command, which is the VMC command that we added to control virtual machines and containers The basic command, you can do a VMC list It’ll show you what VMs you have installed on your system The default VM is called Termina So hopefully the font’s big enough and you can see what size it is The Termina VM is the one that all the demos were done for the slides before, so it’s up and running We’ve made a shortcut to enter a container inside a VM So if you want to go into the default container, the container’s named Penguin Again, that’s where we were doing all these demos from So there’s a VMC container command that gets you into there We’ll pop out of there and then we’ll pop back into just the VM So VMC Start enters your virtual machine without entering your container So if you go back to my layers, it’s the one in the middle– the thing that LXD runs in And the reason you’d want to be in here is if you want to manipulate or change containers So I mentioned we use LXD There’s going to be a lot of LXC commands That’s the LXD control program This is well-documented online and most of it will work inside Chrome OS, just like it does on a default Ubuntu install The first one you can do is a list You can see we’ve got Penguin running We have one container It’s up and running It’s got an IP address So we’ve got our one container We might want to play with it a little bit And before we do, maybe I want to make sure I can get back to a state where I know it’s good, right? Because I’ve broken them before It’s nice to be able to just go back to where I was and play around without worrying So standard LXE command It’s called Snapshot And you give it your container name and you can give it the name of your snapshot, and now you’ve got an image saved that you can go back to if you break things There’s a copy on write, too We used btrfs in the VM so you’re not eating up a ton of disk space We can get info on our container This gives a bunch of information Again, you can go poke around with this on a Chromebook if you want to The important bit here is that we’ve got one snapshot at the bottom, the IO1 snapshot we just created You can have multiple snapshots It’s got a date on it to help you remember if you didn’t use a very creative name And then when you want to restore it back, LXE Restore These are well-named commands They did a better job with this than I did If you really want to go and play with different things,

sometimes you want more than one container So I’ve got my Penguin container and I’m going to go, say, install some different libraries in this one Maybe I want to have a container that’s got Python 2.7 and a different one that’s got Python 3 Or maybe I want a different container for writing Go than the container I have for writing Rust So we let you do that You can create as many containers as you want, disk space-limited These do cost disk space The most basic way to start off a new container is to copy an existing one There’s an LXE Copy command The example up here copies the default Penguin container over to a new container named Kingfisher You can list the containers You’ve got two By default, containers are stopped, so we have to start them Now we can list– there it’s running And you can jump in You say, I want to run Bash in Kingfisher And now I’ve got a shell in my new container and I can go off and install whatever random tool chain I didn’t want in my default container Taking that one step further, we chose Debian because it was kind of the easiest thing for us to do We don’t want to tie you down to that, though We support the Debian workflow We support some guest packages that are installed in Debian by default But some people want to use their favorite distro And there is a huge amount of distros available from the image server that Canonical runs We’ll install an Arch one here I’m not an Arch guy, I don’t really know much about Arch, but some of my coworkers talked me into doing this and playing with it So now you can see we’ve got three containers And I’ve got two Debian containers, my Penguin and my Kingfisher, and now I’ve got something called ArchTest And again, I can enter it by telling it to run Bash And if I want to install packages in this one, I’ll use PacMan instead of Act Look, it’s actually Arch, I promise That’s just a taste of what you can do from here If you go and look at the LXD and LXD documentation online, you can get some more ideas There’s even some help online about installing other ones and getting them to integrate better with the GUI if you want more than just a command line TOM BUCKLEY: All right So Dylan just showed you a bunch of the really cool tricks you can do with Crostini when you go under the hood, and if you’re interested in this kind of thing, we really recommend checking out the Crostini subreddit The folks there find features as soon as we release them, sometimes even sooner, and they’re also really welcoming to new users of Linux on Chromebooks So if you have any questions, please check it out, and a big thanks to the folks there So that’s Linux on Chromebooks As you can see, we already support a lot of web and Android developer flows And there’s a lot more to come, both in supporting other developers and in expanding what we can do with new capabilities, like multiple containers and backup and restore We’re going to keep applying these principles of simplicity and security to give you the best developer experience possible Whenever you’re ready, we hope you’ll join us Thank you [APPLAUSE] [MUSIC PLAYING]