A Practical Introduction to Amazon Web Services for Testers

building the test automation framework module one a practical introduction to Amazon Web Services for testers this is the first module in part of a six module course on building the test automation framework and it starts with Amazon Web Services we’re going to use AWS and more specifically Elastic Compute cloud to build our test environment and automation system it all starts with configuring and running the virtual machines we need for our test environment in this module we’re focusing on building a control machine which will be a Windows 2008 box and a Linux machine which will be on bun 2 which will run our application under test 6 key areas to setting up AWS 1 creating the AWS account 2 configuring the security groups 3 creating the key pairs that we’ll need to connect and decrypt our passwords for creating the virtual machines or as Amazon calls them instances v connecting and configuring putty so that we can talk between our windows and our Linux machine and lastly we’ll talk a little bit about monitoring AWS usage and making sure you run your virtual servers down but before we get into this a little bit on why we use AWS 1 it’s fast to get set up you’ll see this in a minute – there’s no provisioning or messing around with hardware and operating system installs 3 you can increase the capacity or completely shut off the capacity on demand as and when you need it and fourthly you only pay you for what you use there’s no concept of paying for everything upfront and if you don’t use it you’ve still paid for it you pay for what you use a couple of prerequisites firstly you need to have a Windows machine or a Mac machine which has an RDP application so that we can RDP onto our Windows virtual machine secondly you’ll need a credit card a valid credit card when you sign up for an AWS account they will require you to enter your credit card everything we do on this course will be based around using the free tier so you shouldn’t be charged but Amazon’s still demand your credit card before that set your account up thirdly also as part of the account creation Amazon will want to confirm your account and they will do this by using your telephone number you need a valid telephone number so what are we going to cover we’ve got 11 parts to this module they range from creating an AWS account in part 2 we’ll talk about some of the fundamentals and the terminology part 3 and 4 we’ll talk about security in creating our key pairs 5 and 6 we’re into running up our Windows and UNIX instances 7 we’re looking at the connections to the Windows machine and 8 and 9 we’re looking at installing putty so that we can connect to our our Linux machine part 10 we’re just going to talk a little bit about the difference between terminate and stop when we shut down our instances and finally in part 11 we’ll look at how to check your AWS spend and monitor your account usage part 2 AWS fundamentals in this part we’re going to look at some of the terminology and concepts for AWS we’re going to cover services zones and regions the management console ec2 instances the images in the am is Amazon machine images and storage and a few basics on networks and security first up then services Amazon provide a range of different services and a service can be thought of as a type of work a particular cloud resource provides they’re all listed on this management console these services are also grouped into categories like compute storage database and application services next zones and regions in the top right corner you’ll see a list of zones these are used to reduce latency between the end users and the services that are provided Amazon provisions these services in data centers located in different regions when you start a service you’ll want to make sure that you have the correct region selected so that the service is started in your region this is also important when it comes to creating your virtual private clouds security groups key pairs they’re

all linked to a particular region select your region and stick with that region the management console when you select a service the management console changes to present a number of different panels in the ec2 service management console screen you’ll see we’ve got the nav bar at the top we’ve got the region selector again we’ve got a navigation panel on the left hand side and the content page it’s from here that you’ll start configuring manages manage the services you need ec2 then this is an abbreviation for Elastic Compute cloud essentially this is the service that delivers resizable computing capacity when you select ec2 in the management console you’re given the capability to start instances configure them and manage those virtual machines in amazon’s cloud so each see two instances from the management console you can run up and configure virtual machine instances which will run all sorts of different operating systems with different hardware platforms and a range of different default software installed each virtual machine you run up is referred to as an instance in this example we have two virtual machine instances that we’ve run up and configure ec2 instance types each instance you run up will be part of a specific type a type defines the CPU memory storage and networking capability of the instance each instance type is then given a name and more details about those instance types can be found on the Amazon Web service website as part of this course we’ll be focusing on using the T to dock micro instance types which have one virtual CPU six CPU credits per hour and one gigabyte memory with EBS storage images and a.m. eyes each time you launch an instance and you start this virtual machine you don’t want to have to install and configure the operating system and additional software from scratch to save you building the instance from scratch Amazon gives you the capability to start the machine with a predefined image already installed these images are known as a mis for the purpose of this course we’re sticking to free tier a.m. eyes and if you check this box you’ll see all of the free tier illegible a.m. eyes so for example we’ve got Amazon Linux AMI again a list of all the different ami types can be found on the Amazon website with details of all the different capabilities for each ami elastic block store and volumes a virtual machine is no good without storage Amazon provide a huge number of options here the most common is known as EBS elastic block storage EBS is again another Amazon service and this service provides storage volumes that can be attached to running instances the data stored is persistent eg the data is retained across reboots and shut nouns and each EBS volume can persist independently from the life of the instance if need be in the EBS volume screen you can see a list of your volumes along with a unique ID and if you scroll to the right you can see which instances they are attached to likewise if you view your instances you’ll see in the instance details screen a list of the devices the EBS devices that are configured and attached to this instance again full details of the types of different storage and their characteristics can be found on the Amazon AWS website network and security when you come to start creating your instances in a minute you’ll notice that each instance is created within a virtual private cloud the idea of these virtual private clouds can be found for

the each individual instance and full details about the virtual private cloud is defined under the VPC service from the management console when you view your virtual private clouds you’ll see the VPC that was created automatically when we created the instances and from within there you can see the details for that specific virtual private cloud every instance we create at this stage will be created within our virtual private cloud we then need to dictate access to machines with public IP addresses in DNS every instance you create is placed in your V PC automatically this V PC is essentially the same as the network you’d find for your physical pcs and laptops in your office just be aware that by default your instances are not given public IP addresses and host names everything is defined by default with local V PC and IP addresses for example when you look at your running hosts and you select an instance you’ll see that no public IP or public DNS information is provided you have to configure this separately security groups built into the whole AWS framework are some pretty clever security capabilities key to running our instances is the concept of security groups that are defined under the network and security entry in the navigation panel each time you start an instance you specify the security groups that you want to associate to the instance and the instance inherits the rules from that specific security group those rules for example will be firewall rules that dictate inbound connections for HTTP from a particular source machine key pairs AWS uses key pairs to encrypt and decrypt login details these are the login details that you’ll need to access both your Windows and UNIX instances the concept here is that we create a public and a private key Amazon encrypts the passwords with the public key and when it provides you with the encrypted password the only person that can decrypt it is the person holding the private key which is you and that’s it 11 concepts that you need to grasp 11 concepts that we’ll cover in more detail as we go through the practical aspects of this module so in the next part part 3 we’ll look at configuring security groups and from there we’ll start creating images and using key pairs to build up our automated test environment running on AWS part 3 configuring security groups you can configure these security groups following four steps first make sure that you have the right region selected security groups are specific to regions you cannot share them across regions second click on security groups in your ec2 area dashboard and from here you can create security groups and modify existing security groups when you create a security group you define the name the description and then dictate the inbound and outbound rules so for example when we add a rule we can say that we want inbound traffic from SSH which comes in the TCP protocol on port 22 and we can even go as far as dictating the sources from which will allow that connection if we want to we can specify the IP of the machine we’re on now or we can say anywhere which is relatively unsecure or we can define custom IP addresses in a range in here as part of this course we’ll need to set up two new security groups one for unix application under test machine and one for the windows

master machine and you’ll also notice that a default group is created automatically and this group will have a source ID that refers to the actual security group name it’s a circular reference if you like but it just means that any instance in this security group can access any other instance in the same security group so every machine within our virtual private cloud that has this security group linked to it and talked to any other instance in the same virtual private cloud part for creating a security key pair you’ll need to create security key pairs so that AWS can encrypt passwords and you can then decrypt those passwords when you create these key pairs you will be provided by AWS with a private key and you’ll need to keep this key safe so that you can use it to decrypt passwords at a later stage in this process to create your key pairs has five steps click on key pairs in the navigation panel click on the create key pairs button provide a name for your key pair and at this stage AWS will give you a download a file with the private key in automatically you just need to be aware that in your downloads folder you will find a file with a dot p.m. extension that contains your private key look after this file don’t lose it we need it later part 5 running up the windows instance we need four pieces of information to set up our windows instance the first is the ami the Amazon machine image and we’re going to use a Windows Server 2008 r2 base we need the instanced which dictates their hardware and the configuration which will be a t2 micro we need to link our security group which will be the windows master security group we’ve already created and we need to use the private key the dot p.m. file that we just downloaded in the previous module eight steps to creating this first we need to go to our ec2 dashboard and then we can click on launch instance we select the ami the easiest way to do this is to make sure we select and filter on the free tier only and then in the list we should find Microsoft Windows Server 2008 r2 base and a specific ami unique ID c5 a7b EA for select this and then select the instance type we want to make sure we stay within the free tier so we’re going to select t2 micro next we need to configure the instance we can just keep to all the defaults here but a few things to call out the network is our default virtual private cloud that Amazon created for us and the shutdown behavior is set to stop rather than terminate more about that later next stage we need to add some storage again we can select the defaults here so we end up with a root device with a 30 gigabyte Drive then optionally we can add some tags so we might have a name for this machine and you may call it Windows master for example then we associate the security groups with this instance we want to select an existing security group we will select the default security group so that anything within our virtual private cloud can connect to this instance and we want to collect select our Windows master configuration that will allow us to connect from outside using RDP and HTTP and HTTPS protocols on the various ports next we’ll review the settings and click Launch

at which point AWS asks us to select the key pair used to encrypt the password for this machine as we’ve already created our key pair you should be able to select that key pair acknowledge that you have got access to that dot PE M file that was downloaded into our downloads folder we’ll need this file to decrypt the password that AWS provides us with in a minute when we launch the instance you should see a new instance entry on our dashboard ec2 management console so at this point in time then AWS is initiating and running up this virtual machine or instance and when that’s complete you will see the instance state transition to running when we view the details for this instance a few things to call out we have a public dns value here this will need to connect to our instance from outside of our virtual private cloud we have the instance type which is a t2 Micro it has a unique instance ID we can see the virtual private cloud that it’s contained within the EBS storage device and details and the key pair that is associated with this and is used to encrypt the password so that’s our windows instance run-up and created in the next part we’ll look at running up our Linux instance and creating that within our ec2 dashboard part 6 running up the in bun to Linux instance in much the same way as we ran up the Windows instance we need four pieces of information we need the AMI the Machine details the instance type again t2 micro the security group that we created earlier and our private key file again so within the ec2 dashboard launch instance again this time we’ll want to select Ubuntu Server 1404 clicking free tier only again making sure we’ve got the unique ami identifies this image select the image select the instance type again making sure we stick to the free tier using a t2 micro instance type except all the instance configuration defaults again you’ll notice that we’re creating this instance within our own virtual private cloud again and that the shutdown behavior is set to stop or terminate and on the next step we’ll add the storage will stick to the default the games 8 gig storage device EBS storage we can create a tag on here I define this as our UNIX client and then configure our security groups this time we’re selecting from the existing security groups again we want to select the default security group that allows it to connect to anything within our virtual private cloud and select the security group which gives us HTTP and ssh protocol access from clients within our our cloud then we’re on to reviewing and launching the instance and once we’ve reviewed it and let’s start to launch it we have to select our key pair again we’re using the same key pair we’ll need that same private PM file but this time we’ll be using it when we come to connect this machine using SSH and putty click acknowledge launch the instance once the instance has been launched we’ll see it within our ec2 dashboard and once it’s run up we’ll see the instance state change from pending

to running so next in part seven we’ll look at connecting to the Windows master machine we’ve created an instance for and Rd peeing onto that Windows machine ok part seven then connecting to the Windows master machine three steps to this again from within our ec2 dashboard make sure we don’t have any filters applied and that we’re listing all of our instances and we should be able to select our Windows master machine that’s running right click and select get Windows password at this point we’ll need to load that private key file and when we click decrypt password AWS will provide us with the password and the Public DNS you’ll need to keep a note of this password because in the next step when we right click and select connect we will get the option to download the RDP file that will start our remote desktop connection if we open it with the default application remote desktop connection and click on connect we can enter our password and we now have a remote desktop connection to our Amazon Windows virtual machine or instance from here then in the next session part 8 we’ll look at installing putty and SSH on this machine so that we can add connections to our Linux machine part a installing putty for SSH access to our Linux machine so for steps here we need to open a session and RDP session on our Windows server download putty and install the putty applications and then run the installer selecting all the defaults what that’ll give us his three tools putty agent puttygen and putty that’ll allow us to at our key set up an authentication agent and then secure shell into our Linux machines so let’s have a look at that first off we start our Windows machine so we connect to this with the RDP connection downloading the remote desktop file and opening this entering in the password that we obtained and decrypted in the previous session and then on our and then on our remote desktop session we should be able to start internet explorer and we can search quickly for a download page for putty search for putty SSH and you want to go to this website the green n dot org dot uk’ site click on the download link and on here there is an installer a Windows installer which has all of the putty applications that we need we can either save it or run that directly from the website and the Installer then takes us through a few steps to install the suite of putty applications well this then gives us from the Start menu if we look in the putty folder our three applications that we’ll need to use putty agent which stores our secure key or private key putty gem which allows us to convert our private key into the correct format for the Linux machine and putty itself which makes the connection and gives us a secure shell onto the Linux machine so we’ll look at using all three of these applications in the next part of this which is connecting to the UNIX client machine part 9 connecting to the UNIX client machine so for this we’re going to use a suite of tools called putty so there’s putty agent puttygen and putty itself this suite of tools allows us connect from our Windows machines via

a what we call a secure shell onto our Linux machines now to implement this secure shell we have to use a private key that was provided by Amazon the only downside is that that key isn’t compatible with putty however putty provides a tool called puttygen which actually generates keys but also converts keys so it will convert our Amazon private key into the format that putty requires and this is step one we need to convert our dot p.m key second step then we need to run putty agent putty agent runs as a service in the background and holds our key and is used as part of the connection to our client machine are using UNIX machine third step we start putty we configure the hosts that we want to connect to our Linux machine and then we can open that connection on the client machine with a secure shell so step one then we need to confer our private key that Amazon provided us with so if you find a key that you’ve got saved on your local machine and need to copy that will copy it across to our Amazon windows instance so if we come back into our ec2 management console start up our IDP connection for our windows instance having first got our password that we decrypted in one of the previous sessions so right click on Windows Connect download the remote desktop file which allows us to connect with our DP connection so once we have that RDP connect we want to paste that dot p.m. file onto the desktop and then we can use puttygen to convert it into the correct format for putty so open up puttygen load the key that’s on the desktop and then we can save that private key in the format that putty understands you’ll be prompted to enter a pass phrase please like a password but can be a longer format and when you load the private key into putty agent it’ll ask for that it lasts for that passphrase which we’ll see in a minute so we’ll see on the desktop now we have our putty private key in the correct format so the next step then is to load this key into putty agent so that it’s stored and running in the putty agent service in the background ready for our connections so if you open putty folder again select putty agent and this runs as a service on your Windows machine and what we can do there is add the key so if you click Add key and will select the key that we’ve just created on our desktop and at this point it asks for the passphrase and we now have our private key stored in the putty agent application and if you click on View Keys you can see the key that’s been stored in putty agent so third step then we need to configure the connection details in putty itself so if you open putty and we can configure the connection details to our UNIX machine now before we can enter all the details

for this connection to our UNIX machine we need to find the IP address for our UNIX machine now because we’re working on our Windows machine within our virtual private cloud and we’re connecting to the UNIX machine within that same virtual private cloud we can use our private IP address so this will not work from outside of our virtual private cloud but it will work from our Windows machine inside the virtual private cloud so we can paste the IP address in there we can save it as a particular type of session so UNIX client is just giving it a name for the connection and save that connection the other piece of information that you you want to enter in here is under the connection data is the name of the user you’re going to log in with now the Amazon machine image we use to create our Linux machine is configured so that the roots or the admin user if you like is called Ubuntu so if we use that as the username making sure we save that again and then we should just be able to click on open and get a connection to our UNIX machine so on the first connection you’ll always have this security alert and it’s just that the the key the private key that we have loaded in putty agent hasn’t been registered with the UNIX client machine yet so to register it we just click on yes here and then we won’t have that warning on subsequent connections and there we are straight into our bun to UNIX machine our client machine and if you exit from that and come back down to the taskbar and look a putty agent you’ll see now if you right-click on here you have saved sessions and you can just click on the UNIX client and it goes straight in without you having to worry about entering any user details or passwords onto our Linux client machine and that’s it for configuring and connecting to all of our Amazon instances that have been set up within our virtual private cloud we have the ability to RDP onto our Windows machine in our virtual private cloud and from our Windows machine we can now use putty to create a secure shell connection onto our Linux machine in that same cloud in the next session we’ll look at monitoring and usage of our Amazon machines just to make sure that we don’t incur any charges as we’re running all of this within the Amazon free tea part 10 the difference between terminate and stop you’ll notice in your management console that if you right-click on one of your running instances that there are a number of options to start/stop restart your instances first one is just stop which is really the process of shutting down the machine if it has EVS storage the data on this storage is maintained when the instance is shut down you can start the instance up again when you’re ready to use it again things like the instanceid EBS storage the private DNS and IP they’re all maintained and restored across that stop and restart note that when an instance is in a stopped state you are not charged for the use of that machine however any EBS storage that is maintained you will be charged for that is assuming you’ve gone over your free tier allocation if you don’t want to be charged then you will need to delete the volume or terminate the instance once you’ve stopped your instance you then have the option to restart it again right-click instance state and click on start obviously the process of clicking stop and then starting it again is the equivalent of the reboot option in that submenu as well so the third option then is terminate now if you terminate an

instance everything is deleted only terminate an instance if you no longer lead it because you cannot restart it and you will not be able to reconnect to it again only use this option if you don’t need the data anymore you can also use this option if you want to make absolutely sure you’re not going to be charged any more any BS storage you have with it when you terminate is deleted too so there’ll be no EBS charges associated after you’ve you’ve terminated the instance either during our training course over the next six modules we want to make sure that we don’t terminate these machines we want to make sure that we only ever stop them reboot or start them we don’t want to lose any of the data that we’ve installed or set up on these machines which would happen if we use the Terminator so one final tip to make access easier to our ec2 service and our virtual private cloud service click on the edit button we can drag the ec2 icon up to the bar at the top and we can do the same for V PC and we then have fast access to our ec2 dashboard or we can get to our storage as well and our V PC service where we can set up and configure our virtual private clouds in the next session we’ll look at how to keep an eye on your AWS spend party 11 how to check your AWS spend within the AWS management console you’ll find a billing and cost management option within here you’ll see what your current balance is and what your estimated future billing charges are going to be as I say all of this course is implemented using the free tier on the Amazon Web Services but it’s advisable to keep an eye on this and monitor it to make sure that you’re not going to go over that free tier usage setting the key part you want to look at really is the top free tier services by usage and that gives you a figure for your month to date usage in relation to what the free tier limit is and at the moment I’m on 0.6 percent so well within my free tier usage allocation if you want to set up alerts to make sure you are warned before you go anywhere near this and under your preferences you can set things up so that you can configure billing alerts and billing reports there’s more on this on the Amazon website under the creating a billing alarm section of the user guide and that’s it on the final part of this 11 part series in the first module in the next module we’ll start to look at installing our application under test which is rocket chat on our UNIX machine and we’ll look at installing Jenkins on our windows machine which is going to control all of the automation and tasks that we need to automate in our test automation framework so in conclusion then we started out with the creation of a new Amazon Web service account and we went through the signup process to create that account we’ve learnt about the AWS fundamentals covering virtual private clouds elastic cloud computing the creation of instances and learned a little bit about storage with EBS from here we’ve created our first two instances we created a Windows 2008 server and a Linux Ubuntu client machine and we finished off with configuring our RDP connection from outside of our virtual private cloud and configuring our putty SSH connection from within our virtual private cloud in short we’ve started out with nothing and now we have our own cloud environment with virtual machines and storage ready for us to configure our test environment