Jenkins Selenium Meetup: Kohsuke Kawaguchi on how Jenkins + Selenium = Love

the creator of Jenkins we about myself so I’m there’s about seven years now I think and I right now watching is like this we do many things but in my context we do the martial world in the Lincoln Scott Jenkins enterprise bike rugby’s and we also have a hosted version of Jenkins where you could just get ask money to your servers in slaves it’s called a bike ride before that I used to work for Sun Microsystems I was doing eczema on Java EE stuff but that was no they went down the tubes so I started working on new things so today I’ll be talking about the Jenkins and since this is a seven new Meetup and not the Dinkins meetup I thought by about the side to introduce the Jenkins it’s the what’s typically called as continuous integration server so people use that for doing the bills and test executions including the seven email executions that we’ll be talking about today and it’s an open source software like I said it’s in eight years old and there are two things that really make stinking stand out from the rest of the pack the one is that it’s very easy to get started and once you get going you can configure everything from the UI the other thing is that it’s highly extensible meaning you could install any one of those 450 plugins at people from all around the world has developed and shared is a community which integrates sync in speech just about any tools you can body so even though this is implemented in Java you can people use this for a dotnet project PHP will be Python etcetera etcetera so thanks to all of these things and it’s kind of hard to beat the free software so we got quite a widespread adoption around we tracked around 31,000 installations and each installation is a server and so we have estimate around maybe half a million users in every corner of the world so today I’ll be talking about the Dinkins in conjunction with 17 and the reason I think you know I think this combination makes sense is multifold so the one is that the selenium test tends to be slow running so the last thing you want to do is to run them on your laptop or your at workstations and because while the selenium tests are running they pop out browsers from left to right practically it makes your computer unusable so you want to run these things on the server where you’re not impeded by a running test and then you can keep on hacking code another aspect I think is that the open especially to keep certain word in the browser’s in system you need a controlled environment so you can’t now you have to disable auto update you have got a specific versions of the browser installed inside and noren past in some cases and so on so it’s a lot more easy it’s it’s a lot easier and most able to have the server’s manage those nodes that’s dedicated for running selenium as opposed to so did I on individual double for the environment little things finally at this pace tends to be more paralyzing all day embarrassingly paralyzed of all meanings there really isn’t anything that you have to run sequentially this is particularly true if you’re thinking about running the same test with multiple different browsers which if I understand correctly is a main body prop of the selenium right so the model that we are getting to be thinking is that you could run you know lots of lots of tests in the number of servers and with different browsers running on the different machines and then you aggregate the tests the end so the companion feature in Jenkins that makes this use case really shine is called the distributed views so the way it works is the in Jenkins so if you think this is the web app I’ll show you later but that’s so primarily the you’ve interacting with the master node with who speaks HTTP so you use a browser detox to be snow doing configurations looking at the build result things like that but if you could only use one computer that only performs so well so in Dinkins what you can do is add slaves around the masters and then master would start distributing the workload into those space and because

this is implemented in Java it runs on F just about every platforms that you need to care about and today we have some people running up to 300 or so slaves connected to a single master so you know if you think about the test and build that demands 300 computers that’s a pretty heavy workload so at the scalability of this is reasonably good perhaps not at the scale of a bit but still up in a plenty scary before typical build on test purposes so I wanted to quickly show the thinking’s you I distributed to show you how easy it is and they make a demo a bit more convincing I got the second computer here which I’ll be using other slave and because I have to connect all things up I got the major network operation center going on with all these cables lying around but so if I remember correctly so I need to remember the IP so what I’m gonna do is if I come to this build executor status and then I can add a new node I’ve already added it it looks like here so all I have to do here is that I specify that this is the name of the computer because I don’t have a name in addition here I have to specify the ID and I think it’s correct see these 55 yeah so that’s it and then this is the directory where the Dinkins ofnews on this machine to store files codes or doing the build and test you need to have some local file server ok so that’s all I need to configure basically and if I scroll down a little bit and then save and then that should let me disconnect this way first and then try to reconnect and then that so what this is doing is now from here to this machine it h into the machine and it looks for if the necessary software is available so it’s checking the environment and all that stuff and then eventually it starts copying some files to run this and then now it’s under the control of the Dinkie so just by specifying a machine name and then in this case it’s logging the in serious I said you know I had the public key authentication already set up so I didn’t have to specify the password but if you wanted to do that you could have done it that way as well so basically just by specifying the connection parameters you could log in to do all sort of machines and then start using them for build slaves so that’s quickly about the distributed build support in Jenkins so I wanted to talk a little bit about the excitement so I mentioned out of the year that the key part of Jenkins is this extensibility that integrates engines with things so there are a number of plugins that integrate thinking 17 related tools and then I wanted to talk about some of those so the one is this chromedriver flagging this is a very small nice and nifty plugin as you probably already know to use chrome from webdriver you need to have a small native program called the chrome driver available on the target platform so if you only have like a two machines one you’ll be installing chrome driver is not a big deal but if you have let’s say Emma scenes I imagine that you can start to see that this little things like installing chrome drivers around you can start to get pretty tedious okay so what this plug-in does is it automates that I’m so you could just come to me um there’s a update Center in thing instead allows you to install and your plug-in so let me see if you go to manage sinkings and then there’s a manage plug-in page and then from the list of the available plugins you could just choose the one that you want to install I’ve already installed the this plugin in question but you can just in a pic of you plug-in that you want to install and then you could install it and activate it right just like that so that’s how you could install that this is how easy it is to install let’s say a chromedriver plugin and once you do that once you do that upon connecting states this plug-in you to install the right clone driver based on the texted platform and so on so it’s funny thing for you to worry about and indeed things do that when you start to have a large cluster but now the next plug-in is this is a real bigger and useful plugins that’s called the selenium fragging it really should have been called as seven grid plugin because that’s what it does it integrates with 17 great I when I initially started this plug-in I expected this to be like a

one-stop driving for everything in selenium but it turns out that the other people wanted to be other things other way so it didn’t work out that way but anyway so this is what this does is the neighbors have selenium greatly changes and for the longest time this has been with the selenium one but finally we’ve upgraded that to work with selenium 2 so the idea of this plug-in is that well as I mentioned out here that you have this distributed build support in Jenkins right so you can have a master in the middle lots of slaves so now you spend some effort setting up this you know the build Custer once in so this plugin it overlays the selenium grid on top of this thinkin’s cluster so it will automatically start the selenium grid no don’t know these states and then automatically start the grid have on the master node and then they’ll configure the grid have n grid nodes in such a way that they start talk to each other without the ammonia leak on clearing anything at all and you know things like this assisted slave banter if you think this master is the only demon that you need to be managing everything else gets lanced fine master automatically through the network so it’s a lot less overhead for the administrate that you’ve all arrived so um so this is simplicity in the deployment I think is a key benefit of this and because once this grid is running the you know it’s a plain vanilla ordinary 17 grid so you can use the service from outside thinking so if people interactively developing code in their laptop wanted to provision a one of the stage or the one of the selenium you know is from this grid they could deserve but and similarly if you need to add some extra grid nose to this grid let’s say you need some esoteric the face is six running on this portrait PC because we can’t maintain it anywhere else you could do those things as well so I think this makes it really easy to deploy geranium and another reason I like it is because this is kind of this is a showcase for what I want to see more in the Jenkins because now that you know we automated a substantial part of the headache of the setting up a large number of machines and put them under single control you could start reusing this cluster for multiple purposes being such related to the bills and test exhibitions so that’s for the reason why I keep spending more effort into this plugin now the other poor thing about running selenium with Jenkins is that is the difficult you involved around having GUI for the browser’s naturally want to see a GUI environment around it but we when you run it from Jenkins and we run on the servers we don’t really have any displace so often you have to do some extra steps to make these things work nicely for example and then in the last seven you meet up where I came I spend most of time talking about how to walk around this stuff but this time I’ll go over them quickly so if you wanted to see more details about the things you need in this space you can go look up the past slides from SlideShare so on window is right p.m. we have to in other words in the windows if it simply allows the services to interact with desktop so that’s just one check box to do it but the Microsoft in their attempt to make the windows secure they essentially eliminated this capability so nowadays the simplest and easiest way to make this work is have the windows auto log-in to some specific users and then use the startup group advanced to save a lien on unique system very easily create a virtual display environments we experience E or the exporter frame buffer there’s a Dinkins packing for experience e so that you don’t have to manually start and stop Rienzi servers thing is filled with openings for you the map is a bit strange they don’t really have any support for Bato this place but if someone needs to be interactive loading into the console the primary display but once you do that the states can connect reset and run the services so it’s different things needs to be done on different platforms I mentioned that when we run so back to the selenium grid plug-in now when we run selling a grid on top of Jenkins we mostly run it untouched aside from injecting configuration so that they know where the grid heavies and Heidi talk to each other but we also added some extra capability in the system so that we could sort of take advantage of the fact that it’s running on Jenkins so you know in using in using selenium hunting is one thing I

noticed further that you know I don’t really have any control over which node in the grid the browser gets provisioned over so I could specify the kind of rather about example I cannot specify that the is Firefox instance that I want needs to be running on Linux or it needs to be running on a particular site let’s say San Francisco side as opposed to a bangalore site or something like that so we added the capability called the dinking stuff level so this nicely takes advantage of the pack ability inside serving webdriver as well and then you could specify a label expression so that the boolean expression that specifies the requirement of the node where you run the browser and then once you do that you know the rest is just ordinary webdriver set up to be so and then while I’ve been doing this I guess the one thing that I’ve been bugging me for a long time and assist now I have the selenium people in my audience I thought I’d pitch this but you know every tutorial that I see online seems to talk about creating webdriver programmatically like this like you do the new Firefox driver right but this I think is anti father on that because it hard code is the browser in tests so it makes it very difficult for the test execution driver to stop in a different different browser so I’d rather much like to see the equivalent of selenium I’m sorry did the database connection string but like you could specify the name of the driver you know the name of the server port and username and so on into a single stream that can be flagged at the runtime not after the wolf and time I said you know we could individual users could invent things like that but then that makes it very difficult for tools like Jenkins to understand what’s going on in the test and it also makes it difficult for tool 16 can still help you know making things happen so I don’t know who is the selenium commuters in this audience but if you have their ears you know please let them know that the the leadership coming up to come up with some kind of connection URL like the database drivers have been doing for ten years does that make sense no okay so the other thing another thing about running Jenkins I miss our landing selenium testing site Jenkins is that the thinking amethyst or selenium has this capability of capturing screenshots of the page which is quite invaluable when you’re debugging the pest right so now capture those file the screenshots and then save them in the house all right but there is no way to associate that with particular tests so Jenkins have this ability to look at the test reports from web UI but when you’re doing that you know it doesn’t really know what screenshots are related to this test case so unfortunately so this is a simple thing it’s like just a few missing links in here but unfortunately this is the kind of thing that span across many different frameworks and tools and it’s a no single person could easily fix it so you know today there is no convention to associate such files to test reports and then the test reports are often generated by the test frameworks like jail unit or testing G and normal test code doesn’t have any access to those things that makes it somewhat difficult and then those test framework guys treated on how the world extensibility so they don’t know what they what to do with all things so a simple solution that I came up with is to flip this mock-up text inside the output of the past so the most of the test frameworks are capable of capturing the standard output from the test so by omitting this file name in southern encoded form the Jenkins could look at those things and notice these are the piles that needs to be associated with the pest report so they’ll be able to beat them and then pass them into the report so for a web driver or the code it looks something like that there’s a nice abstraction called output type and then the implementation of that would type that does it for systems the default file implementation about foot type does like non-persistent ordinal files by the time Jenkins is looking into test reports the screenshot of argon which is useless so we fix that I’ve been meaning to push this send a pull request for that but I got this cold the entire coil hosted on github and so you could you know once you got the driver you could take a screenshot and then you could put them into your

little brackets and then that’s all it takes to make this form and that’s on the dink inside you’d install the J unit attachment plug-in this is the one that books out the test reports and and understand the conventions and so once this plugin is installed when you’re tilling Dinkins that there are test reports that it means to look at you could also specify the test attachment to be published and then if we look at reports and then doesn’t does the right thing so let me show you how the whole thing we’d work together sorry well I think I’ll keep the font bigger even actually so in this product what I did I have a small gift project in my local computer so I’m I’m telling Jenkins to check that out and once I check to the side I got this myth so this is a maven project so I’m specifying it to be random maven and then once the build is over I’m telling Dinkins that there are a unit test report that it should be looking up and then there’s various published test attachments so to make this demo distributive I got I got a little parameter here that says 7m level and then I’m specifying the node where the selenium I’m sorry the browser should be probably done so if you do get the code so that’s the this is the test code and so here I’m doing this taking what I specified as a label and then setting it into the capability like I showed the ADI in the slides and then talking to these with having they’re running a test so the net result is once I start the build it should start the Firefox on this machine so you should stop you should see the young you should see the Yahoo pop up so let’s see what’s going on I guess here it comes so it’s running a test they’re going to the grid grid hub here which is driving the grid node on this machine which is running Firefox okay so that’s how you could so you know the Jenkins has number of gimmicks allowing you to have a lot of different machines so you can have Windows you can have math you can have Linux and if you mix them up into a single cluster so say you need to run the same test against Safari on Mac I’ve been the final focus on somewhere and then the IE on Windows you could do those things very easily with that hustling around running around and connecting computers so now the test run I can see that this report so for this I only have one test but if you do get the test reports so we can see what it has produced and here as you can see it has pretty Putman so I can see it here which is well there you go so this is the against this screenshot or Yahoo your home page that’s taking on this node so that’s the hub that’s how the whole thing can be put together in the back too so I guess you know the so while this works the thing that I want to see modeled going forward is that if this is if there’s more intelligence built into selenium underneath for the webdriver camp I notice that it’s capable of automatically taking a screenshot whenever an exception is thrown so if it doesn’t rely on need to do this gluing and instead if we that’s a kind of printing automatically when it know that it’s running indistinct in the environment then we can read the whole experience a lot better for you guys without you doing anything just by having those two projects talk to you a little bit more so one of my hope in my coming here to talk to disarranging audiences they get the hang of this arranging the wrong horse and then get some of these features into the underlying chromium code so that you don’t have to spend time doing this planning work but the only positive note

right so the the whole thing works again you can have nice you know the the distributed cluster know is running Selene capturing screenshots and all that and the setting up selenium has never been easier especially for grid because in practically this video click on deploy you can install plugins from the update Center from the UI you could install slave just from the main UI and so on and so forth you really don’t have to touch this computer at all to get going with this kind of setup and I want so we recently got another developer or inside the Jenkins community who is working on this training with so he’s looking for my ideas for the in this space one of the obvious thing is like well we’ll be super sexy if you could not only capture the screenshot but also do the video recording so that’s generally what I wanted to show so you know I don’t know if I have any more time but the fact is I should be able to take quality questions if you’re having but what’s this did this make sense or like we had already had or attract I did work yeah so I said yeah so the question was well if you using Jenkins is there any reason not to use selenium okay I see yeah so let’s see yeah there’s a slide that I have on the online I don’t know if I can show you if we work to make this network work together I got some messed up setup going so maybe doesn’t cooperate but when I was trying to show you as I have so we had this thinking since the conference last year in San Francisco and I had this flat fold for that talk I looked at the statistics after the split between things are awesome and then if you forget numbers it’s pretty clear that the old actions are we savings and then they had some guys I don’t know if they are still around so yeah so most of the people have been upgraded so sorry I couldn’t make this work but yeah so there’s the I mentioned earlier about the Jing incentivized by Travis the one of the feature that I am working on if I have it working on this before I came here today was to do this fade over with my masters so it’s coming to the I guess the commercial version of it and I didn’t ask him to ask this question right right yeah so the question was do I have any roadmap for improving the UI of Jenkins yes so the boredom that I’m using here is actually some of the newly improved border so you see that we status the breadcrumb sticks on the top and then you see the context menu for things it will help you with navigations so like if you wanted to go to a particular build you could be so and then let’s say you’re looking at the console outfit and then decided that you need to change the configurations look at this so we for the end of the last naturally this February bunch of us went to the fourth thing which is event in Belgium and then we got together and sit down and came up with a list of the improvements in the UI that we’d like to make so these are some of the things that came out of it so if you have more ideas about the things that we can do better I’d love to hear from you but otherwise these improvements are just coming in the last few devices there’s more improvements to be made obviously so stay the big yeah yeah so upgrading your version being a few months and then you should enjoy all the new improvements so the question was what’s the criteria for getting a code accepted if you have the code written that’s obviously get accepted if you write a plug-in that’s automatically gets accepted as well for the new I clean the main challenge is like everyone thinks they are UI export when they are not so like I don’t really know who is opinion that I should trust but at least I know that I not be your experts but you need to prove me that you know what you’re talking about right so that’s that’s the only challenge there yeah but yes I’m looking forward to talk to you I guess right so let’s see I don’t know how to summarize this question in quick way but so often once you start building your real test you have to have very complex choreography between different pieces like you have to compile a test and then you have to deploy the subject of the test here you have to have the test running over there you wanted to see the log files it’s produce on the server aggregate for things like wait for everything to complete and shut things down and blah blah blah so you know once you start at that point

it gets you know you need to be spending a bit more time in thinking so that you tell it to do the choreography there are a number of plugins and then some some actively being worked on some more proven that that you do that kind of inter machine set ups that that’s that’s like a that’s a that’s a worthy of a taco over on its own so I can’t be the answer ignite right then let’s see so what’s the short answer I can make there are a couple times yeah so we should talk of diastole and I can give you some pointers about some of the plugins that you might want to look into so I think is that my time is already running right so I guess that’s a that did for me so hopefully you’ll be able to use thing is I guess I forgot to show you the way Thank You Seattle all so if you haven’t been using it already please and thank you very much you