Coding Challenge #147: Chrome Dinosaur Game (with Speech Commands machine learning model!)

(upbeat piano music) – Ah! Okay, hello and welcome to a coding challenge, Google Chrome dinosaur game edition Unfortunately I don’t have my train whistle I didn’t bring it with me today, so this is definitely not gonna go well Not that any of my coding challenges ever go well, but I’m gonna give this a try I’m gonna attempt to program, quickly, the Google Chrome dinosaur game I’m gonna use my own images and stuff, so it will be the unicorn game, of course I’m gonna use p5.js, the JavaScript library p5.js, and I’m going to interact with it in a unique way that you have to stick around for the end of the video I’ll just tell you now I’m gonna try to control it with my voice (laughing) Okay, so you skip to the end, maybe I’ll be doing it there Okay, so here we go, let’s get started So, what do I need to do? So, first I’m gonna make a, I’m gonna add, I’m in the p5 web editor, and I’m gonna add a file, I’m gonna call it dino.js, even though it’s not, no, let’s call it unicorn, unicorn.js And that’s gonna be my unicorn And what do I need? I also need to have, those are like cactus thingies, those are gonna be trains, so I’m gonna make a train.js, add file So, let’s start by creating the unicorn, class Unicorn So, I’m gonna use object-oriented programming because, you know, I do, this, by the way, is exactly the same as Flappy Bird, so I’m really just doing the same thing over again, but whatever This is a nice, beginner-friendly challenge, and maybe using object-oriented programming is a little bit overkill, but let’s give the unicorn an x location at 50 and a y location, which will be height, ’cause it’s sitting at the bottom of the window, but maybe I’ll say, like, height minus 50 right now, and then I’m going to create a function called show, and I’m going to draw it, I’m just gonna draw it as a rectangle right now at this.x, this.y, and I’ll make it 50, 50 So, this is the Unicorn class A Unicorn object has an x and a y, and when I draw it, it’s a rectangle at that x and y with a height of 50, and those’ll probably need to be variables eventually So, let me make a variable called unicorn I’m gonna say unicorn is a new Unicorn, and I’m gonna say here, unicorn.show And let’s run this, there we go Oh, okay So of course, I forgot what I always forget, which is that I’ve added some new JavaScript files, and I need to reference them in my HTML Unicorn, and what was the other thing called? Train All right, so that’s a unicorn, and what I want is, wait, let me play that game again How does it work? It just sits there, so every time I press the space bar, it jumps So, let’s get it to do that first, let’s get it to do that first Okay, so what does it need in order to be able to jump? It needs a velocity, a speed along the y axis, so let’s call this vy, and we’ll say that’s zero And we’ll write a function called jump, which we will give it kind of this instant force So, we’re gonna, like, push it up So, I’m gonna say this.vy equals, let’s say negative five That’s moving it up And then I’m gonna write a function called move, which says this.y += this.vy So, what we should see now is, if I say unicorn.move, and then if I add function keyPressed and say if the key is the space bar, unicorn.jump, this really is basically the same as Flappy Bird so far, then we should see that velocity kick in So, I’m always moving it based on whatever that velocity is, but the velocity is zero, the speed is zero But as soon as I say jump, it sets the speed to negative five and we should start to see it move up Okay, here we go Moving along here, I’m gonna press space, there we go Ah! Goodbye, unicorn, see you later! All right, so we need gravity, so we need some form of gravity, which probably makes sense as a global variable, but I’m just gonna, since the gravity’s really only gonna affect the unicorn, I’m gonna put it in here Let’s say it’s a 0.5 So, gravity is, in a way, a counteracting force, so no matter what it’s doing, the gravity is always adjusting the speed of the unicorn, the acceleration, basically So, what I could do now, also in move, is say this.vy += this.gravity, however, I don’t want to ever let, because see, now it’s gone I don’t wanna ever let the unicorn fall off the bottom There should be basically a ground level So, I’m gonna, to do that, let’s say, so let’s make this 50 a size

Let’s say r = 50 So, we’re going to kind of use that for all the parts where we’re drawing And then I’m also going to say, I’m gonna have a constraint So, I’m gonna say this.y = constrain Its current y value between zero and, really, where it’s sitting at the bottom, which is also height – this.r So, it’s stuck there, and now if I press, oops, press space, you can see it jumps Now, that’s a very measly little jump We need to be able to jump much more, so let’s set this to, like, 25, and let’s see what happens Okay, that’s too much Let’s just set it to 10 Maybe I need to make the gravity stronger too So, there’s gonna be a lot of fine tuning of this, but let’s just do that And let’s set this to 25 Oops Maybe let’s set the gravity to two There we go, this feels a bit more like the dinosaur game So, this is kind of sad Let’s make our unicorn look a little bit more exciting Let’s load an image So, I have on the desktop here, I have some images I have a background, which will be the background of the game I have a unicorn This’ll be the character with a little angry equal sign sitting on it And then the train will be the thing it has to jump over So, let’s, so what I’m gonna do here is I’m gonna add these files, I’m gonna do add file, and I’m going to drag them into the web editor It uploaded all of them And then, in here, I’m gonna add preload, and I’m gonna have let uImg, let tImg, and let bImg, and let’s load all of those uImg = loadImage unicorn.png, and we’ll do the same for tImg and bImg This is train and this is background Let’s change the background to, ooh, what I’d get wrong here? Background is actually a jpg So, let’s have the background now be bImg, there we go, and let’s have the unicorn, where is it, be, I’m gonna say image, uImg, there we go Ooh, it’s so tiny! And it’s all pixelated, it so sad! Let’s change this to 150 Its x value shouldn’t be, it should be here There we go, so there’s our jumping unicorn This is not a lot of screen real estate to work with, so let me actually load a background image that’s 800 pixels wide, which I think will be a little more, which I think will be a little better than 600 All right, my unicorn can jump and I’ve made the background a little wider, which means now I need obstacles for it to jump over, and those are going to be little train things So, I’m gonna make another class, call it a Train Choo-choo I’m gonna give that a constructor These will have also an x and a y, so the x will be, it’s gonna start off the edge, and the y will be actually also at the bottom So, height minus, I’ll give it a size, and this.r will be 100, and then, so now I need a show function, and that, I’m going to draw as an image (laughing) My fingers are in the wrong place The train image at x and y, this.r, this.r, and let’s also add a move function, and we’ll say this.x -=, and so, the train, so this is a weird thing It’s supposed to be, oh, this is gonna be a problem (laughing) My fixed background is gonna make this visually a problem, and people on the chat are saying paralaxed background, please All right, so maybe I’m gonna get to that This is gonna end up being a longer video if I implement that, but really, the unicorn is moving, so the unicorn is moving, but the x stays fixed The trains, well, the trains could be moving if they’re trains, that’s weird, but they’re supposed to be cactuses, cacti They’re standing still and the unicorn’s just approaching them, but we’re seeing it from the unicorn’s point of view Boy, I’ve really botched this But what I’m gonna do is just move them by, the whole scene is scrolling by three pixels, and we’ll figure out if we can

paralax the background maybe later What do I even mean by paralax? To make it appear as if the background is changing, but off in the distance, so it’s sort of moving at a different speed But for now, let’s just do this Let’s create an array called trains, and we’re going to say, in setup, well actually, you know what we’re gonna do? In draw, we’re just going to randomly, we’re going to pick a random number between zero and one, and we’ll say, like, one percent of the time, we’re going to say trains.push, we’re gonna add a new train, so it’s gonna be irregular This is different than the Flappy Bird I made, where I think the pipes, maybe it was irregular I think the pipes were at regular intervals Now there’s a random chance of adding a new train, and then all the trains that exist, we’re gonna say for all of the trains, I’ll just say t in trains, I’m gonna say t.move, t.show All right, let’s see what happens Do we have a train? Maybe I’d better increase it There we go, oh, look at that All right, they’re moving They’re kinda slow I’m gonna jump over it, oops, okay (laughing) Let’s make them move a little faster So, we’ve got some tweaking we’ve gotta do Whoa, this is a, oh, this is a nice way Oh, look, my unicorn can fly So, I need to fix the fact, I need to, like, not allow my unicorn to fly, right? It should only be able to jump if I’m a the bottom, so let’s fix that So, to do that, I need to go into Unicorn, and in jump function, I should only be allowed to jump if I’m sitting at the bottom So, in other words, if y is, oh well, if I’m at the bottom, so as long as y is equal to, is this really gonna work? I don’t like to use equal checks, but this is, by definition, sitting at the bottom, so only if I’m sitting at the bottom can I jump, so if y is equal to that spot Let’s see if that works So, jump, oops, jump Oh, this.y Come on, trains Oh, yep, I can only jump when I’m at the bottom All right, so that works, so that’s fixed I think I need to be able to jump quite a bit higher Let’s try 35 I should really use a slider or something There we go, I think this is gonna make the game more possible to win There’s too many trains, too many trains! One thing that’s bothering me is the trains are going in front of the unicorn I know that doesn’t really matter so much, it certainly doesn’t matter, but I kind of would prefer them to be behind, so I’m gonna move this here I’m also gonna make the game a little easier to play I’m gonna reduce the number of trains, so there’s gonna be a much lower probability of there being a train Let’s see if I can actually play this game somewhat successfully, even though I’m not doing the collision checking yet (laughing) No! I think if the trains moved faster, it would actually make it easier, so let’s actually double the speed of the trains All right, this looks like this game is now possible to play But I need to add collision detection So, I’ve done this before in plenty of other coding challenges There’s a pretty simple algorithm to just check if two rectangles are intersecting with each other or not I can just check their x’s and y, basically all the edges, the left, the right, the top, the bottom, against each other, but this is a nice opportunity for me to just explore a library that’s out there in the world, and this library is called p5.collide2D, and it actually has a function in it called collideRectRect, and it’ll give me a Boolean variable if I give it the x, y, width, height of one rectangle and the x, y, width, height of another rectangle So let’s actually just use this library This is also a nice excuse for me to show you that GitHub is now integrated with something called jsDelivr This is an Open Source content delivery network, meaning if a JavaScript file is hosted on GitHub, I don’t have to download that file and then add it to my project, I can just link to it So, I’m gonna go over here to GitHub There’s lots of different ways here that I can reference a file on GitHub I think the easiest way for what I’m doing right now is just to reference its path, and here’s an example path, so I’m gonna go over here to index.html, I’m gonna add another script tag I’m gonna say script src equals, I’m gonna paste that in, close script, but this is not the path that I want I want the username, which is B-M-O-R-E-N, and then I want the path to, let’s get the minified version of the library, and that would just be this right there in the root directory

So, now I’m just referencing that file Let’s see if that works This is actually not correct I’m referencing the cdn URL, the username, but I also need the repo name, so that should be p5.collide2D slash this, so this should now work, okay, and I should be able to use the collide function So, I’m gonna use this collideRectRect Let me just call this in draw to see if that works and doesn’t give us an error Great, no error And so now, I can check, when I’m going through all of the trains, if unicorn hits t, console.log game over, okay So, console.log game over, and then I’m actually gonna say no loop, so I’m just gonna actually stop the sketch, shut down the sketch completely, which isn’t a really thoughtful user experience design choice, but it’s what I’m gonna do for right now And then, I just need to add a function here called hits, and receives a train, and I can say, I can use this function, which I need this.x, this.y, and this.r, this.r, and then I need exactly the same thing, but with that particular train So, instead of this, I’m checking this rectangle with this train’s rectangle, and I’m going to put this in a variable called collide, and I’m gonna say, if, I’m gonna return that Actually, you know what, I’m just returning this Return that, so return the true or false results of this collide function from that library, and now let’s run the game We’ll wait for our first train Game over! Let’s see if I can play it now Oh, there were so many trains a while ago Oh, shoot! Let’s make the train a little bit smaller, and actually, let’s make the unicorn smaller as well Hopefully this will help (laughing) All right, let’s draw the, let’s debug this a little bit by drawing the box, like, very subtly So, let’s draw a very slight rectangle This is what we’re actually checking There we go, woo-hoo! It helps to see the full box Come on, trains Spawning a train with a mouse click would be a good idea Okay, we lost All right, so there’s obviously a lot of gameplay improvements that need to happen here One is I could be more thoughtful about, and you can see how there’s some wasted space here I should crop the image and the actual rectangle a little bit smaller I could consider using a circle, I could use a more complex shape But this is good enough for right now Let’s actually make it, you know, I could also just click the mouse in order to start a train, that might actually help things But let me comment out this, and what I wanna do, just before I go, before I finish it, and you’re gonna make a version of this with a score and all sorts of nicer things, and sounds, but I gotta finish this off, I gotta finish this video I want to control the train with my voice, so to do that, I’m gonna do this ml5.js, which has a speech commands machine learning model built into this JavaScript library So, I made a whole separate video about this particular pre-train model and how it works, so I’m just going to add it pretty quickly into this video You can go watch the other one for more details But if you’re looking for the documentation, if I click on reference and go to soundClassifier, this is what I’m looking at doing So, all I’m doing, first thing I need to do is, well, actually, one thing I need to do is I need to go to get started, and I need to get a script tag to be able to reference the library So, I wanna add the ml5 library, so I’m copying this, I’m going here to index.html, and in addition to p5.collide2D, I’m now adding ml5, and I’m going to go here, and I’m gonna, in preload, I’m gonna do let soundClassifier, I’m gonna create a variable In preload, I’m gonna say soundClassifier equals ml5.soundClassifier, and the model I’m loading is called SpeechCommands18w, because this is a model that’s been trained to recognize 18 particular words, the digits zero through nine, up, down, left, right, and a few other ones that are documented on this ml5 webpage

Let’s go look that up real quick So, if I go back to soundClassifier, up, down, left, right, go, stop, yes, no So, those are the words it’s trained to recognize, and I actually also wanna get this probably threshold, so I’m gonna add this in here, because I want it to be really sure I’m saying jump, so I’m gonna make this 95 This means it’s gotta be 95% sure the confidence score that’s coming from that particular model, and how that model was trained, and there’s a paper you can read, and I talk about that more in the other video Okay, so now that I have that, in setup, I can say soundClassifier.classify Then I can get a function I’m just gonna call it gotWord, so it’s got a word, gotCommand, let’s call it gotCommand So, this is a callback function that is going to trigger any time it gets a command It gets an error first, error first callbacks, and then the results So, just say if there’s an error, console, I’m just gonna console.error the error Otherwise, if results index zero.label So, results is an array with labels and confidence scores, and the labels are the words, and what I’m looking for is the word up If that label is up, then unicorn.jump So, this is a really quick way of me looking for a particular keyword from that library, and I’m also just gonna console.log results[0].label, just to see if I’m getting other words also And we might as well log the confidence score, just so we can see that That might be good for debugging as well So now that I’ve logged those two things, I’m gonna run this sketch Up! (laughing) Up! Up! (laughing) Shoot! But you can see, it got up 95% I’m gonna add the thing where I click the mouse and I can give myself a train And in fact, let’s comment out the random trains Also, I really should be more thoughtful about how I’m picking when the trains should come Like, maybe there’s a minimum distance between them If I’ve just added one, can I have a longer train with other train cars? I don’t know what’s going on here There’s a lot of narrative flaws in what’s going on here, but I’m just gonna add one myself Up! (laughing) Up! Shoot All right, everyone is making the suggestion in the chat that I should change it to a circle, so that’ll actually kinda be like a round, it’ll give me a little wiggle room along the edges and make the intersection happen a little less often, so let’s actually go ahead and do that So, I’m gonna go into, where the only place where I’m doing this collision detection is where? In the unicorn under hits So, I’m gonna change this to collideCircleCircle collideCircleCircle, and circle to circle assumes ellipse mode center, so I’m gonna have to think of the circle as the center point of the object, which is not a big deal. So, the way that I’m gonna do that, let’s make some separate variables Let’s say x1 = this.x + this.r * 0.5 So, this would be the middle This is the middle, and then r is the diameter of the circle, so that’s fine, and this is y And then, x2, y2 is the train Train.x, train.y, train.r, train.r And so now, I want collideCircleCircle, x1, y1, and then x2, y2 The question is, is r actually the radius or is it the diameter? Does it say? CircleDiameter, circleDiameter, so I’m good All right, so let’s try this, and then this should really be, oh, this is kind of unfortunate This should now be an ellipse Oh, you know what I can just do? I can say ellipseMode CORNER Perfect, so let’s draw this hit circles, and under Train, let’s do the same thing here and say ellipse, there So this should now, we should see, here we go That’s what it’s gonna do Okay, here we go! Oh, I have to click Oh, whoa, what happened? (dinging) All right, I passed r in twice, which is very unnecessary you just give it the diameter, not the radius,

even though I’m calling in r once And let’s see what happens, okay, here we go Jump! Wait, why is my browser really freaking out? Let’s comment out the debugging We’re just gonna go for it now, we’re gonna play this game Let me close this tab Quit Chrome We’re about to play this game! Oh, up, it’s up, I’m saying jump (laughing) Oh, up, there we go, up! Up! Up! All right, you ready everyone? (upbeat piano music) Up! (laughing) Up! Oh, maybe, I guess I can’t use the music Up! Up! Up! (laughing) Oh! All right, everyone, thus ends this coding challenge There are so many things that need to be added to this Number one, the narrative just makes no sense Why is there this unicorn standing still without any animation? The trains are coming at it The background should be sort of in paralax moving So probably it should be jumping over, I should’ve just used cacti That’s the plural of cactus, right? There’s no score, there’s no birds flying You could add a flying asterisk I’m gonna, when I upload this, I’m gonna include, when this video publishes, with the code, I will include a lot of the Coding Train characters, which are designed by Jason Heglund, illustrated and designed by Jason Heglund, so I’m gonna include a whole bunch of those if people wanna make a version of this game that we can all play online with speech commands Let me try this I’m gonna just say goodbye by trying this one more, and yes, Alka is pointing out that speech detection API in the browser might be lower latency, but let’s try this one more time Up! Up! Up! Up! Up! Up! Up! Oh, okay, goodbye everyone! Thank you very much and see you later on a future coding challenge (dinging) (upbeat techno music) (dinging)