Concatenating Commands with Pipes and Substitution

Motivation

In the last Example we have seen how we can redirect the input and output of commands from and to files. Now often enough we only produce output of one command for the purpose of feeding it to another program and we are not interested in the intermediate outputs.

In other instances we have the output of one program that we want to substitute on the command line as arguments to an other program.

With these tools we are extremely flexible and can create powerful commands.

Pipes

If we want the output of one program as the input of another we can put a | vertial bar (in this context called pipe symbol) in between them:

$ ls -l | wc -l 

So the above example would run the ls -l command that lists the files and directories in our current directory and then count the lines. So we know how many of those are there.

$ find . | grep -i doc | grep -v cache 

The find . would list all files and directories in all sub-directories form the current directory on (the dot denotes the current directory). Then the output is passed to the grep command. Grep filters out only lines that correspond to that string (actually it can do regular-expression) and only lets those through. The -i says that it should match case-insensitive. Finally the output is passed to another grep command: This time with the option -v wich inverts the match: That filters out all matching lines. In our case any file or directory name that has the word cache in it.

One specially useful tool for using with pipe is less. less is the successor of more. If the output of a command is very long and would scroll over several pages you want to page through it one page at a time.

$ find /usr/bin/ | less

The find lists all the files and directories below /usr/bin which is usually a very long list. With less you can conveniently scroll through the output. Pressing the space bar brings you to the next page. By pressing q you exit the program. Less has many more functions. See: less

Here is a list of a few commands that are useful in pipes:

$ sort     # sort lines alphabetically
$ sort -u  # unique - avoid duplicates
$ sort -n  # sort numerically - the first part of the line is interpreted as number
$ cut -b 7-15  # only display the characters 7 to 15 from each line
$ cut -f 3,5 -d ","  # display fields 3 and 5. the fields of each line are separated with , 
$ rev      # revert each line from back to start
$ grep bla # filter out lines that contain "bla"
$ wc -l    # count lines
$ uniq     # avoid duplicates (assumes already sorted with sort)
$ uniq -u  # only display unique lines. 
$ tr x u   # replaces each x with an u
$ tr A-Z a-z # converts uppercase to lowercase

Substituting the output of one command in the command line of another

Sometimes we want the output of one command not as input but as argument to an other command. This can be done by backticks ` command ` or in bash the more readable $( command )