STDIO and Pipes
Linux systems have several standard streams for input and output. When a program starts at the command line, and prints information to your terminal it is writing to standard-out (stdout) and standard-error (stderr). When you type input to it, you are (probably) writing to standard-in (stdin). Most programming languages allow you to read and write from these just as you can from files. The out and err streams appear on your screen the same way, but usually the former is used for output and information and the latter for error messages or warnings.
Generally, you interact with programs by passing them arguments. However, many can also take commands, such as filenames or data, from stdin. For example, the utility wc (word-count) can be given a filename, or can be passed data on the command line, like this:
username@godzilla:~>echo "abc" | wc -c
The '|' here is a pipe, in this case for "piping" stdout from one program (echo) into stdin on another (wc). The -c flag counts characters (note that the answer is 3, not 4. Looking at the result of
echo "abc" may help, or consider
echo "abc" | cat -E ).
The '<' and '>' characters are used to feed into stdin and out from stdout/err. For example, we can pass a filename into wc like this
username@godzilla:~>wc -l < tmp.txt
The -l flag counts lines in the file. Similarly we can redirect the output from wc into a file like this:
username@godzilla:~>echo "abc" | wc -c > wc.txt
Note that this will overwrite any existing file with that name. Also note that only output, not error, is redirected (more on this in the next secton). To append to a file, rather than overwrite it, use '>>'.
More on stdout and stderr
As noted in the wikipedia article linked above, stderr originated because error messages from typesetting became interleaved with the desired output. The '>' pipe allowed us to redirect stdout to file, but what about stderr? The following command
username@godzilla:~> ls this_file_does_not_exist.123 > tmp.txt 2>tmp.err
deliberately generates an error message and pipes it into a file tmp.err, while recording output in tmp.txt. Note the '2' which refers to stderr here. stdout came first, so is referred to with the number 1.
However, what if you want to simply capture both outputs into a single file? To do this, redirect stderr into stdout, and stdout to a file, like this:
username@godzilla:~> ls this_file_does_not_exist.123 > tmp.txt 2>&1
Note the '&1' which distinguishes stdout from a file called simply '1'.
As well as the commands above, you may come across the rather odd looking
username@godzilla:~> ls this_file_does_not_exist.123 > /dev/null
'dev/null' is a special device which means, in this context, 'nowhere'. This command therefore dumps any output into the void, while printing any error to screen. This can be useful if, for example, you want to hide errors and plan to check whether something succeeded another way.
Redirection Cheat Sheet
||||Pipe one program's output into another|
|<||Send information into stdin|
|>||Redirect information from stdout|
|>>||Redirect from stdout, but append rather than replace|
|2>||Redirect stderr to named file|
|2>&1||Redirect stderr into stdout|
|> /dev/null||Send output into the void|
|2> /dev/null||Send errors into the void|