[MUSIC PLAYING] MIKE MCDONALD: My name is Mike McDonald I’m a product manager on the Firebase team I’m here to talk today about everyone’s favorite topic How many people are here for business? Let’s start with that Your company is paying for you? Let’s imagine– you’ve been out all day in the sun It’s hot If you’re like me, maybe got a little sunburned You are now going to go to the local drugstore and pick up some aloe vera You pay for that, and you’re going to expense it, because the company’s paying for it You got sunburned And you have to deal with everyone’s favorite problem, which is expense tracking Right? You all love this process You get back from a really great trip out to I/O, and then you just have this cluster of emails, and cluster of receipts that you have to deal with You have to upload all those receipts, you have to scan them You have to determine how much it was You maybe want to know, oh god, am I spending more than everyone else on my team? Wouldn’t it be awesome if I went on a trip, and I had a receipt– you can check out my burgers and beer from a trip up to Seattle I want to just take a photo of that, have some magic occur, and then know, hey, that cost me $51.74 dollars And have that automatically tracked That’s kind of my dream It would be even better if I could allow everyone on my team to do that Maybe I have an iPhone, and my co-worker has an Android Or maybe I want to access all of those things on the web, after the fact I don’t really want an app I need this app to be available across Android, iOS, and web If I just sat down and started building an app the way I normally built an app, not only would I build all of those front ends, but I would build a bunch of backend infrastructure I would have Compute Engine VMs, or maybe I’m using App Engine, or serverless things like Cloud Run or Cloud Functions But I’m writing a bunch of back end code, to do authentication and authorization Maybe I have an ORM in there to talk to the database that I’m also managing It’s kind of a huge pain It’s a ton of infrastructure, overhead, and all I want to do is upload receipts and scan them It should be easy I’m going to use Firebase to help me build this application Firebase is Google’s mobile platform It helps you develop your applications, understand what’s going on in your application to ensure that it’s high quality, and then helps you grow your application Firebase apps are different, because they don’t require that you build all of that infrastructure It’s a bunch of managed services that let you do things like upload files directly from your phone to the cloud, or synchronize data across devices We’re going to start off I’m going to invite a couple of my friends out Frank, Jen, and Kat– I need some help building this app [APPLAUSE] FRANK VAN PUFFELEN: Thanks, Mike We’re going to build a few apps here I’m going to give the team a few moments to set up We’re going to talk through what we’re actually going to be doing Mike already said, we’re building an expense tracking app Who’s excited about that [AUDIENCE WOOS] FRANK VAN PUFFELEN: Not too bad I’m not sure about you, but when I am out having dinner– when I’m traveling– when I get the receipt, I want to take a picture straight away I grab my phone, and I take a picture I never actually do my expenses for my phone I file my expense reports from my laptop, in a browser I have the same app on my phone– typically an Android version for me– and in the browser, a web version I need to see the same receipt images there We need to store them somewhere where we can access the same files from any version of the app We’re going to use Cloud storage for that Cloud Storage is Google’s exabytes scale storage solution It’s used to back some of the biggest apps that you may have on your phone For example, Snapchat relies on cloud storage for its files That is great, because even though we’re just starting with the four of us today, this means that we know that the app will keep working even when there’s millions of users tracking their expenses We’re going to access cloud storage straight from our devices We’re going to read and write files to the cloud straight from within our application code We’re not going to spin up any servers We’re going to just use the Firebase SDK for Cloud Storage in each of our apps That means that we’ll write a file straight from the iOS app, and then later we can access that same file from within our web app because we’re using the Firebase SDK to access cloud storage We can share the file with users worldwide if we want, but since in this case we’re tracking expenses and we’re actually storing receipts, this is probably somewhat sensitive information We can also lock it down to just the users that we want to give access to these files I’ve been telling you that it’s very easy to actually write this code How about we actually write some code?
If you look carefully, it says that permission was denied We don’t have permission to write any files, which sort of makes sense But we didn’t talk about that yet So let’s switch back to slides, and see why this happens and how we’re going to fix it Remember when I said that if Mike uploads receipts, that probably only Mike should be able to see them And when I upload receipts, that only I should be able to see them That’s actually what’s happening here We store files in user specific folders that only the user can access, and we do that based on the user ID These are Firebase surface hide security rules We’ve written these, and these are automatically enforced on the server Because remember, we are accessing cloud storage directly from within our app So we can’t do security through surface hide code We have to do it through these rules And these rules tell us one very important thing In order to be allowed to read and write the receipts/userId folder, you must be signed in as an authenticated user with the same user ID That explains why nothing works yet, because we didn’t sign in yet Let’s fix that by adding Firebase authentication Firebase authentication is the second product for use, and it’s our secure serverless sign-in solution By just using the Firebase SDK in your application, you can allow your users to sign in to any of our supported providers This includes email passwords, phone number, maybe even a password word list link in email, and many social providers like Facebook, Google, Microsoft, Yahoo, Twitter, GitHub– I think we have a few more By just writing client side code, you can sign your users in with any of these providers When the user signs in, we generate a unique user ID for that user We do that the first time they sign in, when we create their accounts This ID will stay the same for the user, for as long as they’re using your application It’s the same, no matter what platform they sign in from So if today, Jen signs in on her Android app, and tomorrow she signs in on the web version of the same application, she gets the same user ID That is great, because that means we can access the same files that she wrote from her phone You do all of this without writing any surface hide code So no triple-legged OAuth validations You just write a few lines of client’s hide code, and Firebase does the rest There’s more to authentication than just code You actually also need UI You think of some of your favorite apps– you know that there’s a lot involved when you first signed into them You need to sign up, or sign in You need to enter your email address and your passwords, and wait As soon as you enter an email address, you probably want to send a verification email because if the user ever forgets their passwords, you need to send them a reset email All of these things require that you have a user interface for these actions, for these flows Let’s be honest, all of these flows have nothing to do with what your app is all about None of it is about expense tracking We’ve built a library called Firebase UI It’s built by the identity experts at Google, and includes years of best practices in building good sign-in flows This library is available for iOS, Android, and the web, and we’re going to be using that today to build our sign-in flow Let’s switch back to the codes to actually do that Can we switch back to the split screen, please? Just like before, we already included the Firebase authentication SDK We have a method here that gets called when the user clicks the sign-in button What we’re going to do is we’re going to add code to call out to Firebase UI to start a sign-in flow This is a single call to Firebase UI, but we’re passing a few parameters that tell Firebase how to actually display this The most important parameters are the providers that we want to enable So in this application, we’re going to allow the user to sign in with email and passwords, and with a Google account Now as I said before, we have many more providers, and if you want to enable those, you would just add them to this list You would also enable them in the Firebase console This is all that we need to do to kick off a sign-in flow See again how little code we have to write When the user completes the sign-in flow, it’s again going to be one of two states Either the sign-in succeeded, and in that case, we’re going to display the username Or the sign-in failed, and just like before if the sign-in failed we’ll take the error message that we get from Firebase, and we’ll display it to the user Let’s build and run these apps again, because I think this should be all that we need to do Now, if we click the sign-in button we get a pop-up that’s asking us to sign in We can select our account, or enter or email
password credentials If this is the first time that we sign into this app, we actually– for OAuth, we get a prompt whether we want to give the app access to our basic information When we sign in, we see the user’s display name With that, we should actually be ready to now upload an image Let’s see if this time, it succeeds That looks much better It’s a bit hard We don’t do anything with the image yet, so right now you have to take my word for it Let’s look at the Cloud Storage console in the bottom right, to see if any of our files actually uploaded Kat, can you go to the Cloud Storage console? You will see that we have a folder called receipts, just like in our code In there we have three sub folders Only two of them completed so far In each of these, we have a file of the receipts that they just uploaded If you look at the metadata, you can see that the time-stamp is really what just happened So our files are making it to Cloud Storage That’s a pretty good basic start for what we need for sign-in We’ve just signed into the app, we’ve taken a picture of a receipt, and we’ve uploaded it to Cloud Storage I think the next steps that Mike was talking about, is that we need to get some information from these receipts and then start storing that in a database Jen, how do you feel about actually doing that? JEN PERSON: Sure, I’ll give it a go Let’s go back to the slides We’ve stored the receipt images for our expenses in the Cloud now What’s next for our app? Right now, we just have a bunch of receipts that are sitting in Cloud Storage We don’t really show anything in our app We could ask the user to enter the details about the receipt themselves, manually, but this is Google I/O, and machine learning is all the rage So we really want a software solution to this We are going to use Google Cloud Vision to extract all the text from the receipt, find the total amount of money in there, and write that to the database We can’t use Cloud Vision right on our device, because that would give all of our users access to our API keys, to run their own Cloud Vision jobs, which we certainly do not want We could also use ML kit on iOS and Android But for this specific case, we really want to ensure that we know that the amount on the receipt is correct, so we don’t want that access to happen on the client What we really need is a trusted place where we can run code, that automatically gets called when we upload an image to Cloud Storage Then, calls Cloud Vision to get the amount for the receipt, and securely writes it to a database so that the app can read it To solve this, we’re going to use Google Cloud Functions Why use Cloud Functions, or let’s say, any server solution, or serverless solution There are several reasons where this would be an advantage For one thing, security In our case– and many other cases– there are things that you don’t want the client to be able to have access to You want to be able to process data in a place away from the client that can be secured Also, it’s really nice when you only have to write your business logic once and have it work across platforms A server or serverless environment enables us to do that And finally– and perhaps most important to your users, from their perspective– is if you’re doing something that is battery intensive, or data intensive, you really want to do that away from the client That’s going to be a better experience for them They’re not going to be too happy with an app that is draining their battery In this case, we could solve this by spinning up our own server, VM, container, but in this case we are working like a serverless microservice We need to run just a tiny bit of code in a trusted environment that automatically spins up and down as needed That’s where Cloud Functions comes in Cloud Functions are event driven, meaning that they respond to events that are happening in your app This could be a user logging in to Firebase Auth for the first time, writing a document to a database– triggering an analytics conversion event– or even triggering directly using an HTTP trigger Whatever one of these events occurs, your Cloud Function is activated and you can act on this information as necessary This could be all sorts of different things Maybe using Firebase Cloud messaging to send a notification to a user It could be uploading or changing some data in Cloud Firestore, or performing some modification
These are grouped together in what we’re calling, collections Each of these represents a collection of documents Specifically, let’s zero in on what the structure is for our own application Here we have a collection of users and you’ll see their user ID We have some fields, user cost and team cost– sort of foreshadowing what we’re going to be talking about in the future Under each user, They have their own collection of expenses which lists out all of the expenses that they have uploaded Here’s a look at some of the rules that we would use to secure the database This allows us to access the expense documents A user can only read their own expenses in this case We don’t allow users to write directly because we’re doing that from our Cloud Function Speaking of which, let’s go back to the code so we can add that Cloud Firestore code Again, you’re going to be looking at the bottom right screen to check out the Cloud Functions What we’re going to do is write the total amount to Cloud Firestore, which looks like this It uses the admin SDK Whenever you have to do any sort of server side or serverless side work, you’re going to think of the admin SDK in order to access Firebase products We still have one more step to do We are writing to Cloud Firestore, but our applications are not reading from it Let’s take a look at our client apps for a moment, and see how they’re going to set this up Our first step is going to be to attach a listener That’s going to be updated any time a new expense is added This is going to read the most recent expense and then, we’re going to want to display it in the UI so that you can see it Let’s see if this works We’re going to run our apps again, add a new receipt, and see if we can see the most recent upload We still have a couple numbers there that aren’t listing any information We really want to know about some total amounts, so I’m going to pass things over to Kat, who’s going to be able to tell you more about that KAT FANG: Thanks, Jen At this point, we have our receipt stored in Cloud Storage, and we’ve pulled out the totals and saved them as Firestore documents This means I have all of the data that I need to get useful aggregate information, and answer questions like, how much am I spending? Since I have access to all of my documents, I could do this on the client side by pulling all of my expense documents for Firestore onto my device and summing up the total there Now, that works well if I have a couple of expense documents, but as I travel more the number of receipts I have will keep growing and I could end up with hundreds or thousands of entries And it can get a little expensive to pull down this many documents It’s not efficient to pull down this much data when all I want is a single value to answer, how much am I as an individual spending? Instead, we’ll optimize for read time How can we do this? We can use another Cloud Function to help determine the total spend for each user Just like we triggered a Cloud Function every time we uploaded a receipt, we can write another function that triggers every time an expense document is written to Firestore When we get a new expense for a user, we can update their user document to add the amount from that receipt to their running total in the user cost field Let’s take a look at what this looks like in code Again, we’ll be in the lower right hand screen First, we’ll need to create a new function We’ll call this calculate user cost This should trigger whenever a document gets written to Cloud Firestore by an earlier function We specify this by putting in our function a document path pattern This function will only trigger four documents that have an expense ID and are associated with a particular user by their UID When this triggers, we will get a snapshot of the data, along with some other event context Now we can read the amount from the document, and grab that UID from the document path
Using this information, and the admin SDK, we can get a reference to the user document that we actually want to update We use a field value dot increment, and add the new expense account to the running aggregate for the user We also keep track of the last updated time which we can get Firestore to fill in by using the server time stamp function You’ll see here we’re using merge true and set This allows us to either create or update the document, depending on whether or not it already exists And like before, we return a promise so that the function continues to run until the work to update the user document is done At this point, we deploy our function, and that will write the data to Firestore Now we just need to update the clients to read this data Let’s turn our attention back to the other three screens for each of our three client platforms Next to our existing listener– getting the latest expense we uploaded– we’ll add another listener However this time, we know exactly which user document we want to listen to Instead of using a query listener, we’ll use a snapshot listener that points to that particular document Then, we will pull out the user cost field that holds our individual total and update our UI Now if we go ahead and upload a new receipt, we’ll be able to see it upload, which will then trigger the function, which parses out the total And that will in turn trigger the function, which adds it to our own user total Awesome Now I know how much I’m spending as an individual But I’m also interested in the aggregate information of how much we are spending as a team We’ll be adding this to the user document, as well, so we only have one document that we need to read to get all of the aggregate information You’ll note in our UI code, I’ve already pulled this information out and we’re updating the UI with our team cost as well The last thing we’ll need is the actual team cost data itself Let’s go back to the slides to see how we can do this At this point, we have three apps running on three platforms, each scanning receipts and summing up how much we are individually spending Since we’re often traveling together, I want to know how much we’re spending as a total, just to make sure we’re not going over budget However, we can’t read each other’s expenses or user documents, so how are we going to be able to get this information? This is another place where Cloud Functions comes in handy As Jen mentioned, functions is trusted code, which means that it is allowed to access everyone’s documents even if I, as a user, cannot This time we’ll write a function that gets triggered whenever anyone’s user cost document gets updated If it does, we’ll calculate the delta and add that amount to the team cost build, and fan that data out to all the users so everyone has access to that information Our functions pipeline looks something like this When Jen uploads a receipt, that will trigger a function to use Cloud Vision to parse out the total, and store that as a Firestore document That in turn will trigger another function, which will update her aggregate user cost in her own user dock And that in turn will trigger another function, which will update the team cost for myself, and Frank, and Mike But, wait If I update the team cost, isn’t that in the user document as well? Wouldn’t that, in turn, trigger the function which would update everyone’s team cost again, which would end up in an infinite loop? Yes, it could So we’ll need to be able to detect this case, whether it was a user cost update– like in Jen’s case– or if we have updated the team cost, in which case we want to stop the potential infinite loop Let’s go ahead and take a look at what this looks like in code We’re in the bottom right hand corner We’ll need to define a new function We’ll call this one, calculate team cost We want this to get triggered anytime a user cost gets updated, which means we’ll want to write a trigger for on write as opposed
to on create, as we did in our previous function Where on create functions give us a snapshot of the data, and on write function gives us a change object, which will tell us what the data was before and after the write operation Having the changed data allows us to determine if this is the case where Jen’s uploaded a new expense, and we have new information to fan out to all of our users Or if it’s the case where Mike’s team cost’s got updated, which means no new data was generated, and there is nothing to update We can do this by getting the old and new user cost Old total here uses the change.before fields to find out what the user cost was before the write New total here uses change.afterfields to find out what the user cost is now If these two totals are exactly the same, that means the user cost field was not updated, which is the case of what happens when Mike’s document gets updated when Jenn uploads a receipt In that case, we don’t need to update anyone’s information, and our work here is done We can return true to get out of the function Otherwise, there is an update to the user cost and we want to add this to everyone’s running team total To do this, we’ll need to loop over every user document We’ll use the admin SDK once more, using the get function to fetch all documents in the user collection This gives us a query snapshot, which we can loop over and update each document in turn Like with user cost, we’ll once again use the increment value to ensure that this amount gets added to the team cost transactionally Again, we want to make sure that the function doesn’t quit prematurely because all of these updates are happening asynchronously We’ll need to keep track of the promises returned for each one of these updates, and we’ll return promise.all, which will ensure the function waits for all updates before exiting Now, we’ve already updated the UI to pick up the team cost as soon as the data field is there Let’s try uploading a receipt one more time and see if we can get the team cost update across all three platforms You might notice that the team total is a bit smaller than expected This is because the function only picks up on new data, which means that earlier receipts are not calculated as part of this We could have another means of back filling this data However, that’s all that we have time for now, so if you have questions about this, find us in the Firebase area Mike, what do you think of our expense tracking app? MIKE MCDONALD: I just uploaded all of my receipts, so I think it’s awesome Can you go back to the slides, please? I came to you all about 38 minutes and 15 seconds ago with the problem of, I want to keep track of my expenses Frank walked us through how I can sign in, get an OAuth token, securely upload my photos into the Cloud, and then handed off to Jen, where she walked us through how we can actually respond to that We can call out to the Cloud Vision API, run our magical algorithm to find out how much money a receipt cost, and then wrote that to the database Kat has helped us through how we can aggregate that information in various interesting ways to get our team cost, as well as our individual user cost that’s then written back through Firestore, and synchronized out to all of our clients It was a really, really cool project, but we’re not done yet We want to share this all with you If you want to go to xpnz.io and start uploading your receipts, you can just share that cost, and then we can aggregate across all of I/O to see how much we have all spent It’s a nice, fun way to not feel so bad about what you’re expensing, in relation to what all of us are expensing The code will also be up on GitHub at a later date Don’t worry if you didn’t quite catch everything that we’re doing You’ll be able to build your own expense trackers in the near future Thank you all very much I hope you all enjoyed the first day of I/O. Go out there, put on sunscreen Don’t get sunburn like me, and have to get some aloe There are a couple of other talks The What’s New in Firebase talk, Wednesday, at 10:30 A deeper dive on Firestore data modeling, and then also architecting mobile web apps if you’re really interested in the web Thank you all very much Enjoy the rest of your day [MUSIC PLAYING]