1.2 Intro to UNIX

>> Hi, everybody Welcome to today’s lecture We are going to be talking about UNIX We’re going to introduce the operating system itself We’re going to talk a bit about its history and how we start to use this operating system That’s important because since UNIX is our model operating system that we’ll be studying for the entire term, we’re going to want to make sure that we understand how it works internally But to get to that point, we’re going to have to know how to use it So this is our introduction to the history of it and our introduction to how to start to use it So let’s look at the history first All the way back in 1965 before I was born, AT&T Bell Labs, MIT, and GE, General Electric, they started work on a joint project, and they called it Multics And Multics stood for MULTIplexed Information and Computer Services And it had a few different things going for it, which were pretty novel So first of all, it was the first operating system to have a hierarchical file system, which meant files and folders Such a thing had been discussed before, but Multics was the first stab at it as a timesharing operating system That means that it could achieve what is called apparent concurrency It could run one process for, say, 10 milliseconds and then switch to another process and run that one for 10 milliseconds By switching between processes fast enough, it appears to the end users that all of those processes are running at once That’s timesharing So these three companies– these research labs and companies get together to work on Multics together From the MIT side, their development was led by Fernando J. Corbato And there were lots of different wonderful guys and girls involved in producing this entire thing So 1969– by the time we get to here, the project fell apart Bell Labs management pulled out due to delays The Bell Labs researchers were really the core researchers involved in the effort And when they discovered that they weren’t really being met from the other companies by the same type of progress, certainly not the same fervor with what Bell Labs wanted to do, Bell Labs pulls out So they were employing some famous– well, today we would call them famous– engineers, Ken Thompson, Dennis Ritchie, and some others that are less well known, Rudd Canaday, Doug McIllroy They were all still invested in this project They’d put a lot of time into it They decide that they are going to continue working on this on their own So they call their version UNIX, which is a play on the other spelling of the term UNIX, because there is this cut down, if you’ll forgive the term, version of Multics They called theirs a UNIplexed Information and Computer Services operating system, or that’s what its name was And they were doing this on a PDP-7 So here is an example of a PDP-7 You can see that these big blue things are really just cabinets for storage But the actual computers here with their reel-to-reel recorders and levels and oscilloscopes and all kinds of crazy stuff that it took to get these things wired up and working– over here, you can see there’s a teletype or a keyboard where you could enter information on But there’s no readout that tells you what you’re doing So they were pretty primitive, impressive that they were able to do so much work on it This particular one is a PDP-7 that they are trying to restore It’s in Norway, I think By the ’70s– January 1, 1970 becomes what’s called time zero for the UNIX operating system We call that sometimes the beginning of the UNIX Epoch That particular date is used as the base, and then we count forwards from there By the time we get to the ’70s here, the early ’70s, we’ve got text processing abilities that have been added And that allows UNIX to start being used for actual work by other groups at Bell Labs So in this picture here, we see Ken Thompson sitting We’ve got Dennis Ritchie in the back And they are working on this system You can see that the computers they’re using are fairly large, lots of tapes that are being held in place And check it out here, Ken Thompson doesn’t really have a display that he’s writing at He’s just entering in characters So pretty hardcore Pretty old school They’re running now on a PDP-11/20, which was a faster computer, better than the PDP-7 So the first manual that was produced as an online man page was November 3, 1971 And the man pages are a series of, well, manual pages that describe how the various programs and utilities in UNIX work They are notorious for not being tutorials They don’t tell you why or how you would use a tool

They tell you what the tool does So they put together these things onto these man pages and started to spread the knowledge of UNIX By the time 1972 comes around, they decide that the languages they were using are no longer sufficient They were using B and some other stuff, a lot of Assembly So Dennis Ritchie takes it on himself to write a new programming language, and Dennis Ritchie writes C. So the C language that he develops is what they start to write UNIX in We see this in a lot of cases with UNIX development But the development of UNIX and the development of C go hand-in-hand There are similar utilities, similar tools, similar abbreviations that we’ll find in both cases And that informs us as to how to use each of the two, because we know the C version might tell us about the UNIX version and vice versa They present their UNIX operating system to the outside world, finally, in ’73 when they go to the ACM And they report this at the Symposium on Operating Systems Principles They couldn’t yet turn it into a product– into a commercial product And part of that was because in the ’70s, through the whole ’70s, there was some anti-trust regulation that was moving its way through the pipes And eventually, this sees the splitting of Bell So it gets split into all kinds of different companies all over the United States So Ma Bell gets split into– well, AT&T and Bell Labs and a lot of different groups have their own territorial telephone company now as the United States government breaks up Bell So because all this was happening, they couldn’t turn this thing into a product Plus, they had agreements with other companies that they were working on They were doing this as an educational thing So Linux then was developed by Linus Torvalds So as UNIX is being developed here, the kernel of it– that is, all of the different little programs that are used to derive the operating system and tell it how to access memory, tell it how to access the hardware and that contains all of the utilities that are essential to the core running of the operating system– that kernel cannot be shared There are license agreements in place However, if you copy the form of it and the effect of it and then give it away, you’re not breaking any laws And this is what Linus Torvalds does He develops his own system, calls it Linux after his own name And he does that to break out of the license agreements that are protecting UNIX, and he releases it as an open source product Everybody can change it and access it and do whatever they want So he also adopts the POSIX specification What POSIX does is it allows– if you’re following the POSIX specification, it means that your programs should be runnable in other POSIX-based systems So UNIX, Solaris, HPUX, Intel’s AIX, lots of others, they shouldn’t need many changes to run on each other, including Linux So while the servers that we use here in our course are all Linux-based, that same knowledge is applicable everywhere In fact, you’ll hear me move interchangeably between Linux and UNIX They all mean the same thing at the level we are talking about Linux is very stable It’s very robust, and it’s free And it’s a fantastic educational tool, because we can see the insides of it We can see any of the source code at any time The first version of Linux came out in 1994, so March 14, ’94 You can run Linux at home You might already be doing this It’s easy to grab Lots of different distributions are out there And distribution is where you would take, say, the kernel that Linus wrote and then bundle it with a bunch of other tools as a full featured operating system So some of the more famous ones would be Ubuntu and Debian and Red Hat And nowadays, we have things like Mint and Arch Linux, so lots of great distributions out there They can all be downloaded They’re all free Support for them is sometimes something that’s purchased So there are purchasable versions, commercial versions, of these operating systems, and that’s just fine too For our purposes at EECS at OSU, we have lots of different Linux machines that you can connect to So distributions get created with different purposes and features in mind We don’t just have a distribution that works for all general purposes We create that distribution for a certain purpose While Ubuntu was designed from the ground up to be easy to use, Arch Linux is the opposite

So it’s going to require that everything gets installed manually The Tails distribution is built to be private from the ground up So that means that there’s a lot of incognito tabs by default There’s a lot of tunneling and VPN systems that are integrated into it And lots of different features go into different distributions to separate them from each other and provide different services for the end user So how do we actually interact with UNIX? Well, modern versions of Linux all have a graphical user interface But we’re going to be looking at things a lot lower So we actually don’t touch any GUIs, any graphical user interfaces, at all in this course We’ll be using just the command line So the interface to the operating system is this command line interpreter, and it’s also called the prompt– would be another the name for it And from the prompt, we have access to all of the UNIX user level programs So we can start things up We can manage processes that are running, which is called job control We can connect the processes together, and they can communicate amongst themselves We can manage input and output to and from those programs, and then we can terminate them And we can do that from the shell So the shell is our user interface program to the operating system The shell is a piece of software just like any other piece of software The difference is that the shell has been written such that it knows how to talk to the operating system directly, which our user programs cannot do So the shell prompt itself is traditionally done as a little cursor You can move that cursor around and enter characters, and they’re all displayed on the screen And you can customize that thing So here’s a few different examples This is the one that I’m currently using 1459 is the current line that I’m entering here And here’s my user name on a particular server And that tilde represents my home directory So here is another version of it And this one has the time of day integrated into it And this one has the current directory that I’m in as part of it So you can change what the prompt of the shell looks like to match whatever you want The dollar sign at the end is the traditional symbol for bash, the particular shell bash So to create different types of shell prompts, here’s a great website, ezprompt.net The reason why doing this from a website is a good idea is because the actual strings that it takes to make your shell look differently are pretty complicated So for example, this is the line here that it takes to create a different shell look So PS1 is the variable that you’re going to put this prompt into But look how complicated it is It’s got all kinds of escape characters and special symbols that do different things And without having a giant dictionary or websites online that tell you what to do, it’s very difficult to write So I like this website just because I can enter an exact– I can drag and drop what I want, and then it tells me the string to use Once you have that string, you can put that into your .bashrc file, which is in your home directory At least if you’re using bash, this is where you would place that And that way, the next time you log in, you’ll have the shell– the prompt for your shell looking however you like it to Let’s look at some examples of running programs So if you’ve used UNIX and Linux before, this is going to be really basic But bear with me, we need to get everybody to the same level So here, I am going to run the ls program We’ll talk more about ls in a minute But ls just says, list all the files And when I hit Enter on that– type ls Enter– it then spits out the current files that are in that directory– in whatever my current working directory is Now, we don’t get much specificity out of what these particular files are Are those files? Are those directories? Are they executables or not? So having something that gives us a little more options would be good So if we add a command line switch– if we add a switch to the ls program– this one here is the -a switch– that means list all the files, including the hidden files So a hidden file is merely one that starts with a period So .bashrc now gets returned We can also see that there are some other special things that aren’t really– well, they are files, but there are special files that are used by the shell to change the working directory So the dot refers to the current directory, and the double dot refers to the parent directory We’ll look at some examples of that coming up So different UNIX shells that are out there The original one would be the Bourne shell And the Bourne shell was written by Stephen Bourne, and it was just called sh And that itself was a replacement for Ken Thompson’s original sh shell, which was based on the sh shell

from the Multics project– from the failed Multics project So the one that we use in this course is Bash, which is an upgrade to the Bourne shell I would say, probably the most other famous shell out there is the C-shell, which has a lot of C programming like syntaxes and familiarity to it But I think that Bash is the better shell These enhanced ones include, certainly, Bash, the TCSH enhanced C-shells, sometimes pronounced teach The Korn shell is another one There’s other ones out there, the Z-shell There’s tons of different shells that you can use to interact with the Linux operating system Again, we use Bash So some common UNIX commands We’ll talk about a lot of these over the course of the term But just to get you familiar with what they look like and what they’re called, you can see that, for the most part, the vowels are all gone It’s like a wave came on to the boat and washed all the vowels aside So you have to guess and learn and memorize what these things do But cd stands for change directory Pwd is print working directory We’ve already talked about ls for listing our files This is make directory, remove a directory This is move, which is also used to rename files This is copy This is remove, so rm for remove instead of delete Ln is used for linking files together It’s like creating shortcuts from one file to another And chmod or chmod is change mode, and it’s used to change the permissions involved in accessing a file For file viewing and editing, we have things like cat, which is short for concatenate The more and less pager programs, we’ll talk a lot more about these Head and tail for extracting lines from output And grep which is used to filter out lines And cut to remove columns from files Editors, you may or may not have heard about, are vi We’ve got a whole lecture coming up on vi There has traditionally been a holy war in editors in the UNIX world Vi people fight against the emacs people, and they try to convince the others that theirs is better There’s simpler ones out there that are not near as powerful but certainly can be used, pico and textedit among them Some other commands that we use are scripting languages– find for locating files, the telnet program itself, ssh for doing a secure connection, and a ton more Let’s look at some of the most important ones here for moving around So pwd and cd So pwd for print– the working directory is the where am I command So it lets you know at which hierarchy in the directory structure you are You can add this where am I functionality into the prompt itself as we showed above in some of those prompts But traditionally, it’s not part of what you see, so you have to use pwd to see where you are Cd is used to change your current working directory And it’ll allow you to move directly to an absolutely specified directory, or you can move relative to where you are So let’s look at a little example here So at the top line, I type cd -/cs344/prog1 So the tilde is a shortcut to my home directory So it says, I want to go to my home directory and then down into directory called cs344 And note, UNIX, Linux, all of these are case sensitive, so capitals are different than lower case letters That is not the case in Windows In Windows, they are treated the same But in UNIX, they’re very, very much treated as separate characters And then inside cs344, I want to go into my program 1 directory When I hit Enter on that, note that I don’t get any output Nothing says, hey, that’s succeeded or hey, that failed or anything It just gives me the next prompt So I type pwd to say, where am I now? And it tells me, I am at /nfs/stak/facult y/b/brewsteb/c344/prog1, which is where I want to be, because all the rest of this– nfs/stak/faculty/b/brewsteb– that is where my home directory is The slash at the beginning here, that’s the root of the entire directory structure And if we specify that, that means that we’re giving an absolute directory path Same thing here with tilde Tilde is an absolute directory path, because we’re saying, I want to start at my home directory and then descend into other directories So here is a relative directory movement with cd Cd .., which means go up a directory level When I do my print working directory from there, sure enough, I am now not in the program 1 directory

I’m up in it’s parent, cs344 I can cd directly to tilde, which is a shortcut from my home directory And the pwd of that shows me that I am, in fact, at my home directory Let’s look at ls here in a little more detail So what ls does– because it’s going to display all the files in the given directory And if we can specify which directory we care about– and we’ll look at some examples of that through the lecture here– it’ll accept both relative and absolute arguments in terms of displaying which directory you want In the examples on this page, I am not specifying a directory to display, which means that ls simply displays the current working directory So here when I say, ls, it returns to me cs344, cs464, and something called pidtest But what are those? We need some more options to figure out what those things are So here, I’m going to add the pla options to the ls program So one thing I want to note about this little hyphen character here, that hyphen character isn’t part of the shell It’s not part of UNIX That hyphen character is something that the ls program is looking for So as in the authors of the ls program made it such that the options that you would pass in have to be preceded by a hyphen But that’s not part of UNIX or the shell That’s just the way the ls program operates If you used a different operating system, one called BSD, a lot of the options in BSD don’t use the hyphen at all So they’re just specified as raw letters So different operating systems, different ways to do it Here on Linux, most of the utilities have a hyphen in the front of the command line switches And you can combine them together by just putting them next to each other You can also split them out So you could do -p, -l, -a, that kind of thing It depends on the utility for whether or not that works Anyway, pla, what do they do? The a, we talked about before, displays all of the hidden directories as well So we’ll see the dots and the double dots and the hidden files like bashrc The l gives us the long form That’s what’s making it be so that we get all of these details about each one of these entries, each on its own line The p is what is– actually, all the p does is it adds the little slash here at the end of the directories So it’s a way to show us that these are directories and that these other things are not So .bashrc is a file Pidtest is a file Dot, double dot and these two cs directories are all directories because of the slash In the old days, that was how you would have to tell just looking at the names of them And you can look all the way on the left These little d’s here also indicate that they’re directories, which is kind of nice Nowadays, we have color displays And if I add this extended and long option, –color=auto, it colorizes things, which makes it a little bit easier to see what they are And here, we could maybe take off the p argument such that I could tell the difference between a directory and a file just by seeing blue versus green But I like the slash there It looks like a directory when it’s got a slash So it’s a pain in the neck to type, ls -pla –color=auto every time I want a nice ls output So a program exists or a utility exists called alias What that does is it lets us set a variable or a command or a piece of text to be a longer version of it So in this case, I’m saying, I want to alias l, just l, and set that equal to all of this text, such that whenever I hit l, I get the long form of this And here is an example Here I type l, hit Return, and there’s my ls -pla with the automatic color So much, much handier to do it this way Some of these other commands– mkdir for creating directories, remove directory for deleting directories Rm is for deleting individual files and is also, confusingly, the tool that you would use to delete directories if you run rm in it’s recursive mode Mv is for moving and renaming files and directories, cp for copying Let’s look at some examples of this stuff In this example here at the beginning, we’re first going to run mkdir to create a directory called tempdir So mkdir, tempdir creates a temporary directory Now, note we don’t get any output out of that It’s just tempdir, and then we get the next prompt Next, I’m going to run a command that says, touch myfile What touch does is it either creates a file if one wasn’t there by that name, or it updates the time stamp for that file to the current time So it’s a way to reach out and touch that file to say,

I’ve modified this file or I’ve accessed this file It’s a way to manipulate which ones you’re working with, say, if you’re using other programs that rely on knowing when files got touched But in this case, if the file doesn’t exist, it’s a handy way to create a zero byte file Then I’m going to use the mv command to move this myfile that I just created into the temp directory Now, remember how we talked about how mv is also renamed So how can you tell the difference between whether it’s moving versus renaming? And in this case, it’s because tempdir, the second parameter in this line or the second argument, is an existing directory So since mv sees that, it says, I’m not supposed to rename my file into tempdir I’m supposed to move my file into the temp directory So that runs, and we get no output So we assume everything worked Then I cd into tempdir Again, no output I set the alias, because I don’t want to type all that for ls every time And then I run l When I do that, it shows that I am in a directory that contains only myfile, and that it’s 0 bytes That’s perfect That’s just what we’d expect Now, I’m going to copy myfile into a new file name called myfile_backup Then I’m going to remove the original myfile In this case, it prompts me It says, rm, remove a regular empty file, myfile And then it asks for yes or no So here I say, yes, to go ahead and delete the file Now, normally, remove just blows away the file without asking But there is a switch you can give to rm, which is the -i switch And it turns out that there is an existing alias that OSU adds on to your rm command such that whenever you type rm, it substitutes that for rm -i, just so that it can ask you to delete files every time or just so that it asks you for confirmation for deleting files It’s a really handy way to protect you so that you don’t delete things on accident So then I type ls, and it shows me that, sure enough, the only thing in here is my myfile backup file that we created Now, I’m going to move myfile_backup to myfile Since the myfile file no longer exists, I am renaming this file into that one And since that’s not a directory, this is accomplished as a rename So when I do the ls command, I, sure enough, see just that file Then I can copy myfile up a directory level to the directory above with a new name So copy myfile, make it a file called newfile at the parent directory Cd backup to the parent directory, list all the files that are in there I’ve got newfile and the tempdir itself And there’s that newfile that I just created Then I’m going to remove the temp directory And when I try to do that, it says, you can’t remove it Temp directory is not empty It’s removed or is not your tool of choice here I instead use remove directory with the recursive option, which says, delete the directory and delete all files that are in it The f switch says, don’t ask me, just do it Then I specify tempdir as the target and no output, because it then goes through and deletes all the files and directories inside tempdir and then returns So there is an example of moving around and creating directories and creating files Let’s take an aside here and talk about input and output to and from the screen So the shell itself is going to open up the terminal as the target for reading from and writing to And the way it does this is that any time a command runs, that command automatically has three files that are open in that program So when you run ls, ls comes into existence and has three files that are automatically ready to go And they are called standard in, standard out, and standard error And they’re abbreviated as standard in, standard out, and standard error, just as you can see there on the screen And they have numbers They have a file descriptor number, and we’ll talk more about file descriptors when we get to C But they have file descriptor numbers 0, 1, and 2, respectively So these files that are open, they’re a pseudo file stream thing that we’ll talk more about later But think of them as files These files that are open inside that ls command that we have run are targeting– are pointers to other files in the operating system So I can send something to standard in, but standard in itself can be redirected to point wherever we want it to By default, standard in, standard out, and standard error are pointing to the terminal So you call ls up and running And then if you told ls to get information from somewhere,

it would by default get it from the terminal And when ls output its data, it sends it to standard out, which by default points to the terminal So the point of this is that if you don’t specify otherwise, program input and output goes to and from your shell prompt to the terminal by default on every line The reason we bring this up, of course, is because we’re going to change this We’re going to start to manipulate the input and output with the next bunch of commands that we use So starting with echo– what echo does is it sends character data to standard out What echo does not do is send text to the screen That’s not what it does Echo sends character data to standard out So if I just use it by itself, I say, echo text, then echo’s command takes text as an argument, slurps into itself and then sends it to standard out Standard out points to where by default? The terminal So it’s displayed on the terminal on the next line Text There we go Then I can say, echo test text, and it’s able to take all those different arguments and combine them into one long string and output that all to the terminal as test text This next version here I’m using the -e switch with echo The reason I’m doing that is because when I use -e, it’s going to interpret the special character backslash in as a newline character So I can say, test text, newline, next line And incidentally, this backslash n is a single character when it’s represented in the file It’s displayed as a line feed carriage return, a newline thing as you can see here But in the actual file– in the actual characters that are stored, newline is a character, just like an a is, just like a q is So with the -e, I can interpret that backslash n as a newline, so my output is what I expect So echo sends all its data at a standard out And since standard out hasn’t been changed away from the terminal, then all of the data goes to the terminal So cat is a separate utility that concatenates files together, concatenates character data stored in a file, and then it sends its output to standard out And again, standard output points to the screen, typically, so that means that you take a bunch of files, concatenate them together and send it to the terminal So cat file1– and note, I didn’t give it anything else, and that tells me the contents of that file, which are, in this case, just file1 contents I can say, let me concatenate file1 and file2 together and output those to the terminal or to standard out, which points to the terminal And it tells me, sure enough, here’s the contents File1 contents concatenated with file2 contents That’s my output So since it wasn’t explicitly changed, everything goes to the screen Let’s look at redirecting now That is done with this operator This is the open alligator eats This is the greater than pointing to the right operator However you call it, this little angle operator tells standard out to instead open a different file for writing to And this is going to affect the entire line So this modifies a standard out, to point to a file instead of pointing to the terminal Now, it’s important that whenever you see this operator, you realize that it means a file is about to be the target of the operation here So here, I’ll run ls Note that ls returned nothing, so there’s nothing in the current directory or at least no non-hidden files Then I’ll run an echo -e with this as the argument, so cookie nbeefcake napple And I’m redirecting all of that to a file called foodlist So the standard out gets redirected to open a file called foodlist in this case Then when I run ls, again, there’s a new file called foodlist, which was created by my echo command I can cat out the contents of that, and I get exactly the contents I put into it, cookie, beefcake, apple So in this cat command, if there was no redirection that was done– so the terminal was open for writing inside the cat command That’s where standard out was pointing, so that’s where the data was sent And it’s pointing at the terminal, so we see it on the terminal Let’s look at the sort command So the sort command is going to take data from standard in, or it will take data from a file, and then it sends that data alphabetically sorted by line to standard out So let’s set this up I’m going to use my echo -e command here to create cookie nbeefcake napple, and store that in a file called foodlist Then I’ll cat that foodlist file out, just like we’ve seen before And then I’ll run the sort command Here, sort, and then the file I’m giving it is foodlist So the sort command opens up that file, extracts all the contents, reads them into itself,

sorts them line by line, and then outputs those lines to standard out, which points to the terminal And there we see the information there So this was me running sort, grabbing it from a file Sort can also grab the information from standard in itself So here is the same command we just saw on the previous screen where sort is opening up a file, reading it and sorting it out and sending it to standard out But in this version, what’s happening here is the sort command is simply invoked, and it wasn’t given any arguments And it has no concept about the things that are happening to the right here So what sort does– since it has no argument, since it was given no argument about a file, sort goes to read from standard in That’s where it goes It goes to this pre-built file that it has called standard in and opens it up and tries to start reading data from there Now, fortunately, we have provided redirection to standard input from a particular file So here is a new operator This is the one that points to the left And this particular operator, the less than operator, say, is indicating that it is opening up a file called foodlist and providing that as standard in to the command sort So the output is the same, but how it happened is very different Our usage of sort here– sort looks to standard in for its input And how did that input get there? Because this operator opened up a file, extracted the contents and caused that to become standard in for the command to the left Let’s take a look at what happens if we run– if we try to read from standard in and we don’t have any redirection setup So this is the redirection symbol for the standard out But I don’t have one here for standard in So the concatenate command attempts to read from standard in but has no data We didn’t redirect anything like we did on this slide where we have the less than operator We just have no input given at all So what happens is that it tries to open up what it has, which is the terminal So it opens up the terminal for getting data from Now, the terminal doesn’t have any input either, because the user provides input to the terminal So the little cursor just sits there blinking, asking you to type something in So you start entering in your lines dutifully one by one, and hitting Enter every time just advances the input on That’s just giving it newlines So what Control-D does is it sends, essentially, an end of file character And when that is read by concatenate, it says, there’s no more standard in available, and then it finishes doing the rest of its command In that case, it is take the input that I have and send it to standard out So in this case, standard out has been redirected to a file called list And when I then cat that file list out, I get the contents So kind of a complicated usage here But we’re going to use this in program 1 in order to easily capture data from standard in Let’s take a look at redirecting both standard in and standard out at the same time So at the top, again, we define our foodlist file in the same way we have before And then I am going to run the sort command where I grab the– where it is going to try to grab data from standard in And where does that standard in data come from? From the foodlist file which has been fetched by this less than operator And the output of that, of course, is sorted data In the next line, I combine the two So I run my sort command And I indicate that the input comes from a file called foodlist, and the output should go to a file called sortedlist And then when I cat sortedlist out, I get the output Now, note that when I ran sort here, there was no output to the screen at all That’s because we captured the output and redirected it to a file instead of redirecting it to the screen So this whole big, complicated, sort line has no output to the screen at all We can only see the fruits of our labors by outputting the file that was resulting from it Now, one thing that’s important about this is that the order of these two modifiers to the sort command doesn’t matter From the perspective of sort, sort has no arguments So sort thinks that it was invoked by itself with no arguments, which means it’s going to try to go to standard in, standard out, and standard error– which we haven’t talked about much, but that’s OK We’ll get to that one later It goes to those three files in order to do what it’s supposed to do So down below, we show that difference In fact, I’ve even collapsed out the space, which isn’t necessary, just to try to show you Here is the sort command being invoked

And then as a modifier, I’m saying, all of its standard outs should go here, and all of its standard ins should go here So it’s not like this is being read from left to right, like some sort of flow from left to right This is the sort command being modified in a few different ways And once those modifications have all been read in by the shell and operated on, the sort command is launched with all of these redirection modifiers in place And then it runs and does what it’s supposed to There’s a bunch more examples about this available in the articles online So shell filename expansion So certain metacharacters are going to be expanded and replaced In this case, when we’re at the shell– and I used the asterisk– this isn’t really regular expressions we’re talking about here This is what’s called filename expansion So the asterisk says, take the string that I’m entering that has an asterisk and match anything And it will actually return all of the matching file names as a result. So it expands and explodes a term into lots of terms, which are then interpreted by the command that you gave it And we’ll show an example of this in a sec The question mark, on the other hand, matches just one character whereas the asterisk can match any number of characters So again, not regular expressions This is just part of Bash, part of the shell here So top example, we’re going to ls And here we’re actually specifying the directory that we want to get a listing of So here, list me all the files in any directory whose names begin with cs344 and then any number of characters afterwards So that certainly would return things that are in cs344 as well as stuff from directories that had any more characters there on it In this next example, this is saying, return me all the files in the directory that have cs344 somewhere in it So there could be characters before There could be characters after But this is indistinct from the first one, because before we were saying it must start with capital cs344 Down below, we’re adding in a subdirectory So here’s a multi-line example In our first line, we’re creating a directory called cs344 And then in that directory, we are touching a file We’re creating a file So note the semicolon here means, I am going to have multiple commands all on one line The touch command is being given a file path, which is inside a directory It ends up creating this file in this directory that we just created Then I’m going to run it again But this time, I’ll put a 0 at the end on all of these and then run the same mkdir touch commands in sequence but this time with a 01 at the end Then I’m going to run an ls command that says, look for any subdirectory that– give me all the files in any subdirectory that begins with anything but has cs344 in the end and only one character at the end And the only one that matches that is the file called fileInCS3440 So before we talked about how we don’t– you’re not supposed to read these things from left to right Well, pipes is where we are introducing the flow from left to right What pipe does is it gives us this way to communicate from one command to another without using a temporary file to hold data as we’ve been doing up til now So here’s an echo command, echo -e, create the same data we’ve done before But this time, the pipe operator says, take the standard output coming from that echo command and redirect it to the standard in of the next command So here, the sort command gets invoked with no arguments So it doesn’t have a file that it needs to read from, so it tries to grab data from standard in And fortunately, data was provided from standard in by the echo command So here, by taking this echo command, piping it through sort, I get a nice sorted command So I can take my echo command that I’ve been using this entire time, and with a simple pipe sort at the end, I get to sort all of my data Really handy Let’s do this– a couple more compound commands here So echo -e cookie nbeefcake napple, this time I’m sending this to a file called foodlist Then I’m going to cat the foodlist file out and piping the result of that into the sort command And I get the same thing again So I can use my pipe sort little trick here on either files– well, it only operates on standard out So that standard out has to come from a file, in this case, cat foodlist, or it can come from a command like echo But in either case, the data arriving into the pipe operator is data that hits standard out from a previous command

Down here, I have cat foodlist piping into a sort command So don’t be confused by the fact that here’s an operator and here’s an operator, and where’s the break? The break is at the pipe operator That is absolutely where the break is Cat foodlist is one command, and it’s data goes to standard out Pipe operator takes that standard out, redirects it to the standard in of the sort command, which didn’t start with any input So it goes to standard in to read from, grabs the data and then sends all the output to sortedlist Then I can cat out sortedlist to see what we have We’re going to use pipes like crazy in this course The more command is a– it’s a program called a pager, which takes character data and displays it one page at a time, one screen-full at a time And while your spacing through all these different pages of really long output, you can navigate up and down with up and down But traditionally, you just hit spacebar, spacebar, spacebar to advance down through the pages And if you don’t want to have to spacebar all the way down through to the end, you can use q to exit out of the paging that you’re looking at This is what looks like a double standard out redirect operator But when they’re combined like this, this is the append operator So instead of overwrite which is what we had before, this is the append data So when there’s just one of them, that means redirect to this file And by the way, if there’s anything in that file, delete all that information and overwrite it with what I have So not just overwrite the first line and leave the rest there, but no, no, first delete all the contents of that file and replace it solely with what I’m about to send you The append operator does away with that and just adds the contents to the end of the file So a little example here Here we have a shell demo script where I am echoing a line that has multiple newlines in place to a file And then I’m going to append a lot more numbers, each separated by newlines to the same file And here is that little append operator Then I’m going to do it again with even more, again, appending them all Then I’m going to cat that rowfile out, which is a very, very long file now, pipe it through sort -nr So what does that do? Well, the n switch is going to do a numeric sort instead of an alphabetic sort, and the r is going to do it in reverse order And I’m going to take that whole thing and append it to rowfile And that gives me this same information now reversed, appended to the end of my rowfile Then I can cat the whole thing out And then I can cat it out and push it through more, because that file is really too long in the first place So let’s do an example of this Let me reach over here and grab my terminal display There we go So let’s do an example of what we’re talking about here And I’m going to shrink this a little, so I can type some of these commands in Let’s see, grab, shrink, there we go So echo -e– and we don’t have to do quite as long as the one we did down there, but let’s go -e1 and then we’ll backslash 2, backslash 3, backslash 4, backslash– I’ve got to have an n There’s my n, backslash 4, backslash n5 End that string We’ll talk more about quotes here in a coming lecture Redirect all that to rowfile So that’s all now in a file called rowfile And I can cap this thing out, so rowfile There it is Look, there is that extra 4 that we’ve got in there So if we want to change that, here I’m going to use the Up key And when I use Up, it shows me previous commands that I’ve run So I can go over here and I can find that slash 4 that isn’t supposed to be there, rerun the command And you’ll see that this time, it blows away the command that I just ran Let’s see here Echo, rowfile– I want to cap this thing out, keep pushing Up– wrong direction Cat rowfile There we go So now I can push Up, and I can execute the same thing You can see it’s been replaced So now, let’s go ahead and add some more to this So echo -e– let’s do 6 newline, 7 newline, 8 newline And this time, we’ll append to rowfile Now, I can cat rowfile So I just push Up, Up, cat rowfile And there it is You can see the 7 and the 8 have been added That’s nice And look, you can see there’s a newline at the end there, because I added one without a 9 at the end So now, what we’re going to do is we are going to cat out rowfile, and we’re going to pipe this thing through the sort command And I will do a reverse numeric sort,

and I want to append this back on to rowfile So run So no output on that because I have changed where all the output goes So that’s redirected to a file Now, I can cat this thing back out And there it is You can see that we’ve got this whole thing repeated and then reversed and appended to myfiles So that actually works pretty well If we do the same thing again, then now we have a much longer file So if I run cat rowfile, we have a very long thing And you can see, we started to double up a lot of the numbers, because we’re combining things And now, I will cat rowfile all, pipe it through more And we can see the result of having all of this be concatenated at one time, occupying multiple pages of screen texts So spacebar through to get to the next one Let’s take a look here We are going to move to another example on this shell script here I’m opening up here my cheat list on my other screen So let’s open this thing up There we go We are going to pretend that we’re in a new Bash shell So if I type bash, that does start up a new Bash shell It’s hard to tell the difference, but I am in a new Bash shell I can clear the screen to give myself a nice, clear prompt at the top I’m going to echo out some text In this case, I’m going to do the same cookie nbeefcake napple– the text that we’ve been doing before– send that into foodlist Then I will echo out– let’s do here, dog, fish, and eel And I’m going to put this thing into fishlist So I’ll run ls, and sure– got a lot of different stuff in this particular directory, including fishlist and foodlist We can cat the two out together, so cat foodlist And here we’ll put fishlist on the same one And there’s our output Definitely not sorted If I try to do something like this, I want to show you the result. There we go So what I’m doing here is I’m saying, sort, grab standard in from– and then I’m passing it a command instead of a file And this doesn’t work, because the operator here– the little less than operator– tries to open up the next thing as a file name, and cat isn’t a file name So you can’t string commands like this together This operator always wants a file name to be the next thing So let’s look at a different way to do this I’m going to cat foodlist and fishlist and push those into a file called– that should be a [INAUDIBLE] into a file called biglist Go And then I can cat this thing out And now, I’ve got all that data stored in a file, unsorted I’m going to run my sort command where I take that as input, and I provide as output, sorted biglist Run And now, I can cat out my sorted file Note, that I’m going to type this much and then hit Tab, and it will auto complete the only matching name of the file in the directory That’s a handy thing Hit Return and there is my sorted file, a b c d e So let’s reverse sort this thing So I’m going to take sort, -r, and I’ll say this time– I’ll do it the other order here So here we’ll say that my output will be– and I’ll use a space just to show you that there’s lots of different ways to do it Sort to reverse sorted biglist, and grab the input from biglist, Run And then I can cat this thing back out, cat reverse tab To see the rest of it, hit Return There is my reverse sorted list If I cat out a whole bunch of these, I can cat out all of the files that begin with list I can do that twice to give myself a whole lot of files at once, pipe all of that through sort And then pipe all of that through the more command It gives me a lot of different– well, apples, beefcakes, cookies, dog, fishes, and eels And I can space my way through all of those And note, that that was all just sent to standard out I didn’t try to put that into a file, but we could have We could take all those things and put them in a file if we wanted Well, I hope this has been an interesting introduction to you to learn about how UNIX works and how we can interact with the operating system We’ve got a lot more lectures coming up about how to do this before we start diving into programming

it– to do more operating systemsy type of things One of the paradigms that we see in UNIX a lot is that we string together small programs to do big things So that principle is illustrated by using pipe to combine the effect of small programs together to build something up as a larger whole Our first programming assignment does just that We will see you at the next lecture Thanks a lot everybody