General documentation / cheat sheets for various languages and services

Positional parameters

Var Description
$0 The path of the executed shell script (not the path to the bash interpreter).
$1 The first command line argument. ($2 is the second, etc.)
$# The number of positional arguments ($1, $2, …) available, not including the shell script itself.
"$*" All positional arguments, as one token. This should always be quoted.
"$@" All positional arguments, as multiple individually quoted tokens. It should also always be quoted, and it’s generally preferable to "$*".
$$ PID of the current process.
$! PID of the most recently executed background process.
$_ Final argument of most recently executed command
$? Exit status of most recently executed command

Operators

Description
=~ tests for a match anywhere. For example, [[ "$@" =~ '--help' ]] checks whether any of the command line arguments are --help.
-z "" or -z Evaluates to true
-z "something" Evaluates to false
${var+whatever} If $var is set to anything, evaluates to null (i.e., the empty string); otherwise evaluates to the string “whatever”
-z ${var+whatever} So, this test evaluates to true when var is unset, and “whatever” can be any non-empty string.
-z "$var" This evaluates to true if var is unset or set to the empty string. (But throws if var is unset and set -u is active.)
-n "" Evaluates to false (-n is nearly the opposite of -z, but requires its argument to be quoted.)
-n "$var" Evaluates to true if var is set to anything at all.

Internal variables

Execute the contents of this variable as a command before showing the prompt.

An array of exit statuses from the different processes in the last sequence of piped commands. $PIPESTATUS[0] holds the exit status of the first (leftmost) process, $PIPESTATUS[1] holds the exit status of the second process, etc.

String manipulation

The Advanced Bash-Scripting Guide is misleading. The string in these patterns is a variable name, while substring is the actual string value of the trigger pattern.

Syntax Explanation
Deletion
${string#substring} Delete shortest match of substring from front (implicit ^) of the $string variable.
${string##substring} Delete longest match of substring from front (implicit ^) of the $string variable.
${string%substring} Delete shortest match of substring from end (implicit $) of the $string variable.
${string%%substring} Delete longest match of substring from end (implicit $) of the $string variable.
Replacement
${string/substring/replacement} Replace first match (in $string variable) of substring with replacement.
${string//substring/replacement} Replace all matches (in $string variable) of substring with replacement.
${string/#substring/replacement} Replace substring with replacement iff the $string variable starts with substring.
${string/%substring/replacement} Replace substring with replacement iff the $string variable ends with substring.

For example, to replace all newlines in the variable IDS with commas: ${IDS//$'\n'/,}

Table of behaviors when $param is in different states, drawn from https://stackoverflow.com/a/16753536:

param set and not null param set but null param unset
${param:-word} substitute $param substitute word substitute word
${param-word} substitute $param substitute null substitute word
${param:=word} substitute $param assign word assign word
${param=word} substitute $param substitute null assign word
${param:?word} substitute $param error, exit error, exit
${param?word} substitute $param substitute null error, exit
${param:+word} substitute word substitute null substitute null
${param+word} substitute word substitute word substitute null

Array variables

All bash variables are arrays; it’s just that most of them have only one value, and the default variable-dereferencing syntax retrieves the first value. I.e., $var is equivalent to the more explicit ${var[0]} for any variable var.

The special array variables $@, $*, and $#, as well as $1, $2, …, are well known since they refer to the command’s / function’s arguments, but other arrays require a little extra syntax:

"${FILES[@]}" Reference all elements of the FILES variable.
${#FILES[@]} Return the number of elements in the FILES variable.
"${FILES[0]}" Retrieve the first entry of the FILES variable.
"${FILES[1]}" Retrieve the second entry of the FILES variable.

Array literals use round parentheses; (a b), ( a b ), and ('a' 'b') are all equivalent; (a b), ('a b'), and (a, b) are all different.

This initializes a local variable to have the value of an empty array. This is different from INPUT_FILES=, which sets the value (of the first element, implicitly) of the variable INPUT_FILES to the empty string.

This appends the string /tmp/z to the INPUT_FILES variable. The RHS of the += operator is simply an array literal of any length, which gets concatenated to the end of the current value of the INPUT_FILES (array) variable.

This appends the string /tmp/a to the first element in INPUT_FILES; probably not what you intended!

Bash does not support multidimensional arrays; e.g., GRID=((1 0) (0 1)) is a syntax error.

Arrays can be sliced much as in Python: ${FILES[@]:1} returns all but the first element; ${FILES[@]:0:2} returns just the first two elements.

And at least in Bash 4, negative indices work as counting from the end of the array: ${FILES[-1]} returns the last element in FILES.

Earlier versions of bash (and some other shells) have a special syntax to get the last command line argument: ${@: -1} (n.b.: the space before -1 is required; otherwise it’s parsed as parameter substitution’s :- default-when-null operator.)

Examples

Suppose ARR=(a b c) and argc() { printf "%d\n" $#; }:

Executed command Result
argc ${ARR[@]} 3
argc ${ARR[*]} 3
argc "${ARR[*]}" 1
argc "${ARR[@]}" 3

IO redirection

: can be used like Python pass, as a placeholder for a no-op statement where, otherwise, there would be a syntax error. For example, echo hello | | cat won’t run, but echo hello | : | cat will (though it won’t have any input, because : never produces output).

Syntax Effect
1>N or >N Write STDOUT to N
1>>N or >>N Append STDOUT to N
2>N Write STDERR to N
2>&1 Redirects STDERR to STDOUT
>>file 2>&1 Appends both STDERR and STDOUT to file
2>&1 >>file Redirects STDERR to STDOUT but writes what would have otherwise gone to STDOUT to file instead
2>>N Append STDERR to N
&> Write both STDOUT and STDERR to file (e.g., &>/dev/null to discard all output streams) (Bash 4 feature)
N>&- Close output file descriptor N (which defaults to 1, if missing)
N<&- Close input file descriptor N (which defaults to 0, if missing)
N<>filename Open file or device filename for reading and writing under the alias &N. N should be a number greater than 2.

These generally only affect the behavior of the line they appear on; to apply them to the current shell session, use exec. For example, call exec 2>>/var/log/somescript.log at the beginning of your shell script to redirect all STDERR to that file instead of the appearing in the user/caller’s TTY (and thus preempts the user’s control over STDERR).

Open file or device filename as the smallest (beginning with 10) available file descriptor, and save it to the environmental variable: NEW_FD. So after this command runs, echo $NEW_FD will return something like 10. Use the {NEW_FD} syntax whenever creating or closing the named file descriptor, but use &$NEW_FD whenever using it as a redirection target.

Send all STDERR over to syslog.

logger is a BSD (and thus, macOS) command that opens up a process that can be written to, which pipes input into syslog. E.g., echo "testing logger from shell" | logger will show up in Console.app with a timestamp and the “tag” chbrown, since that’s my username. logger -t shell-log applies the tag shell-log to everything it receives on STDIN. Each input line translate to a single syslog entry.

The three commands are effectively equivalent, since there’s no difference between overwriting and appending to the file descriptor that logger opens. The latter two commands are syntactically equivalent, but a space is required between the N> and >(...) syntax in the first command. Due to bash’s parser, exec 2>>(logger) is lexed as exec 2>> (logger), which is a syntax error.

For the remainder of the current script / scope, this will write STDOUT to the file /tmp/script.log (overwriting it if it exists).

Open the file /tmp/other.log as file descriptor 3, write the current date to it, and then close the file descriptor.

(exec 3<> /tmp/other.log syntax is equivalent to the first command.)

File descriptors 3 through 9 are all initially closed, but each can be arbitrarily opened and closed with this exec N<> syntax. Trying to read from or write to a closed file descriptor will raise the error message “N: Bad file descriptor”, where N is the closed file descriptor you tried to use.

The first step creates the file if needed, and sets the pointer to the beginning of it. Writing to it will simply overwrite whatever’s currently there, byte for byte; i.e., it doesn’t truncate the file before writing to it. Each write advances the cursor, so multiple calls to date >&3 will write each timestamp to its own line.

To append to a file descriptor opened this way, you can read through it with, e.g., cat; upon completion cat <&3 >/dev/null, all subsequent writes to file descriptor 3 will be appended to the file, because the cursor has been advanced to the end of the file.

Exit codes

In a bash session, $? refers to the exit / status code of the last-run command. Inside a bash script, $? starts out 0, and the exit code of the overall script (insofar as the caller is concerned), when it naturally finishes/exits, is set to the value of $?. You can’t manually set the exit code, e.g., ?=101, though.

Table of “Exit Codes With Special Meanings” from Appendix E of the “Advanced Bash-Scripting Guide” (with clarifications where applicable):

Value Description
0 Success!
1 Any sort of error; might indicate that the error doesn’t fall nicely into one of the subsequent categories, or the developer was too lazy (or too skeptical about standards) to pick a more descriptive error code
2 Misuse of shell builtins (missing keyword or command); permission problem; diff return code on a failed binary file comparison.
126 Command invoked cannot execute (permission problem or command is not an executable)
127 Command not found
128 Invalid argument to exit (caused by exit "tau" or exit 6.28, etc.)
128+n Fatal error signal n; kill -9 $PID (SIGKILL) causes the script running as $PID to have exit code 137; terminating a script with Ctrl-c sets the exit code to 130 (Ctrl-c is fatal error signal 2 (SIGINT), and 128 + 2 = 130)
255 Exit status out of range (caused by exit -1 or lower; calling exit 1000 is like calling exit 232, since 1000 % 256 = 232)

Exit code constants from /usr/include/sysexits.h:

Constant Value Description
EX_OK 0 successful termination
EX_USAGE 64 command line usage error
EX_DATAERR 65 data format error
EX_NOINPUT 66 cannot open input
EX_NOUSER 67 addressee unknown
EX_NOHOST 68 host name unknown
EX_UNAVAILABLE 69 service unavailable
EX_SOFTWARE 70 internal software error
EX_OSERR 71 system error (e.g., can’t fork)
EX_OSFILE 72 critical OS file missing
EX_CANTCREAT 73 can’t create (user) output file
EX_IOERR 74 input/output error
EX_TEMPFAIL 75 temp failure; user is invited to retry
EX_PROTOCOL 76 remote error in protocol
EX_NOPERM 77 permission denied
EX_CONFIG 78 configuration error

Idioms, snippets, and examples

Evaluates to default.txt if $1 is undefined, otherwise uses whatever the value of $1 is (even if it’s the empty string).

Make a directory (if needed) and navigate into it in the same breath.

Send a simple string on STDIN. Equivalent to echo "2 + 2" | bc.

List all file descriptors open in current shell.

Discard STDOUT, Redirect STDERR to STDOUT. Order matters: >/dev/null 2>&1 does not work. Depending on the behavior of some_cmd, you can replace >/dev/null with >&-, but if some_cmd writes to STDOUT without checking if it’s open, or does not catch write errors on that front, /dev/null is required.

Writes a bit of shell script to STDOUT, that, when source’d, would replicate the current value of the PATH environment variable.

Use this to debug; it prints all bash calls to /dev/stderr

Turn it off with set +x

This will safely create a new directory in $TMPDIR, and write the full path to /dev/stdout. This is the only argument structure that works equivalently on both Linux and macOS (at least, macOS 10.11). It will not take responsibility for deleting the file at any point.

This sets DIR to the absolute pathname of the directory containing the script. You should call this early, before the script has had a chance to cd to any other working directory.

When the script is executed directly because it’s on the PATH, BASH_SOURCE[0] is the full path to the script. But if it’s executed as an argument to bash, BASH_SOURCE[0] will be set to exact value of that argument. The cd does the real work of resolving and simplifying any relative or symbolic path components, but by running within a subshell, it does not affect the root-level current working directory.

In both cases, the initial working directory is the directory the script was called from. In the latter case, when BASH_SOURCE[0] is a relative directory to begin with and so the working directory is relevant. Any intermediate cd calls will probably render BASH_SOURCE[0] an invalid path, which is why it’s important to call this snippet before anything else might alter the working directory.

This method will not resolve symlinks in the path to their linked source names.

Source: Getting the source directory of a Bash script from within

This has exit code 0 if some_cmd is something you can call, e.g., a binary on PATH, an alias, or a shell function. It also prints a description of some_cmd to /dev/stdout when it succeeds, so you’ll probably want to 2>/dev/null.

type some_cmd and hash some_cmd are more flexible/fast/beneficial alternatives, but are only reliable in bash. hash some_cmd is successful if some_cmd is a command on the PATH or a function (but fails for aliases). type some_cmd covers all the bases, like command -v some_cmd, but has a type -P some_cmd variant that limits success to when some_cmd is on the PATH. (type also prints out the source code when given a function, which can be handy.)

Source: Check if a program exists from a Bash script