General documentation / cheat sheets for various languages and services

Initialization

Positional parameters

Var Description
$0 The path of the executed shell script (not the path to the bash interpreter).
$1 The first command line argument. ($2 is the second, etc.)
$# The number of positional arguments ($1, $2, …) available, not including the shell script itself.
"$*" All positional arguments, as one token. This should always be quoted.
"[email protected]" All positional arguments, as multiple individually quoted tokens. It should also always be quoted, and it’s generally preferable to "$*".
$$ PID of the current process.
$! PID of the most recently executed background process.
$_ Final argument of most recently executed command
$? Exit status of most recently executed command

Operators

  Description
=~ tests for a match anywhere. For example, [[ "[email protected]" =~ '--help' ]] checks whether any of the command line arguments are --help.
-z "" or -z Evaluates to true
-z "something" Evaluates to false
${var+whatever} If $var is set to anything, evaluates to null (i.e., the empty string); otherwise evaluates to the string “whatever”
-z ${var+whatever} So, this test evaluates to true when var is unset, and “whatever” can be any non-empty string.
-z "$var" This evaluates to true if var is unset or set to the empty string. (But throws if var is unset and set -u is active.)
-n "" Evaluates to false (-n is nearly the opposite of -z, but requires its argument to be quoted.)
-n "$var" Evaluates to true if var is set to anything at all.

Internal variables

String manipulation

The Advanced Bash-Scripting Guide is misleading. The string in these patterns is a variable name, while substring is the actual string value of the trigger pattern.

Syntax Explanation
Deletion  
${string#substring} Delete shortest match of substring from front (implicit ^) of the $string variable.
${string##substring} Delete longest match of substring from front (implicit ^) of the $string variable.
${string%substring} Delete shortest match of substring from end (implicit $) of the $string variable.
${string%%substring} Delete longest match of substring from end (implicit $) of the $string variable.
Replacement  
${string/substring/replacement} Replace first match (in $string variable) of substring with replacement.
${string//substring/replacement} Replace all matches (in $string variable) of substring with replacement.
${string/#substring/replacement} Replace substring with replacement iff the $string variable starts with substring.
${string/%substring/replacement} Replace substring with replacement iff the $string variable ends with substring.

For example, to replace all newlines in the variable IDS with commas: ${IDS//$'\n'/,}

Table of behaviors when $param is in different states, drawn from https://stackoverflow.com/a/16753536:

  param set and not null param set but null param unset
${param:-word} substitute $param substitute word substitute word
${param-word} substitute $param substitute null substitute word
${param:=word} substitute $param assign word assign word
${param=word} substitute $param substitute null assign word
${param:?word} substitute $param error, exit error, exit
${param?word} substitute $param substitute null error, exit
${param:+word} substitute word substitute null substitute null
${param+word} substitute word substitute word substitute null

Array variables

All bash variables are arrays; it’s just that most of them have only one value, and the default variable-dereferencing syntax retrieves the first value. I.e., $var is equivalent to the more explicit ${var[0]} for any variable var.

The special array variables [email protected], $*, and $#, as well as $1, $2, …, are well known since they refer to the command’s / function’s arguments, but other arrays require a little extra syntax:

"${FILES[@]}" Reference all elements of the FILES variable.
${#FILES[@]} Return the number of elements in the FILES variable.
${!FILES[@]} Return the indices (keys for associative arrays) in the FILES variable.
"${FILES[0]}" Retrieve the first entry of the FILES variable.
"${FILES[1]}" Retrieve the second entry of the FILES variable.

Array literals use round parentheses; (a b), ( a b ), and ('a' 'b') are all equivalent; (a b), ('a b'), and (a, b) are all different.

Bash does not support multidimensional arrays; e.g., GRID=((1 0) (0 1)) is a syntax error.

Arrays can be sliced much as in Python: ${FILES[@]:1} returns all but the first element; ${FILES[@]:0:2} returns just the first two elements.

And at least in Bash 4, negative indices work as counting from the end of the array: ${FILES[-1]} returns the last element in FILES.

Earlier versions of bash (and some other shells) have a special syntax to get the last command line argument: ${@: -1} (n.b.: the space before -1 is required; otherwise it’s parsed as parameter substitution’s :- default-when-null operator.)

References:

Examples

Suppose ARR=(a b c) and argc() { printf "%d\n" $#; }:

Executed command Result
argc ${ARR[@]} 3
argc ${ARR[*]} 3
argc "${ARR[*]}" 1
argc "${ARR[@]}" 3

IO redirection

: can be used like Python pass, as a placeholder for a no-op statement where, otherwise, there would be a syntax error. For example, echo hello | | cat won’t run, but echo hello | : | cat will (though it won’t have any input, because : never produces output).

Syntax Effect
1>N or >N Write STDOUT to N
1>>N or >>N Append STDOUT to N
2>N Write STDERR to N
2>&1 Redirects STDERR to STDOUT
>>file 2>&1 Appends both STDERR and STDOUT to file
2>&1 >>file Redirects STDERR to STDOUT but writes what would have otherwise gone to STDOUT to file instead
2>>N Append STDERR to N
&> Write both STDOUT and STDERR to file (e.g., &>/dev/null to discard all output streams) (Bash 4 feature)
N>&- Close output file descriptor N (which defaults to 1, if missing)
N<&- Close input file descriptor N (which defaults to 0, if missing)
N<>filename Open file or device filename for reading and writing under the alias &N. N should be a number greater than 2.

These generally only affect the behavior of the line they appear on; to apply them to the current shell session, use exec. For example, call exec 2>>/var/log/somescript.log at the beginning of your shell script to redirect all STDERR to that file instead of the appearing in the user/caller’s TTY (and thus preempts the user’s control over STDERR).

Exit codes

In a bash session, $? refers to the exit / status code of the last-run command. Inside a bash script, $? starts out 0, and the exit code of the overall script (insofar as the caller is concerned), when it naturally finishes/exits, is set to the value of $?. You can’t manually set the exit code, e.g., ?=101, though.

Table of “Exit Codes With Special Meanings” from Appendix E of the “Advanced Bash-Scripting Guide” (with clarifications where applicable):

Value Description
0 Success!
1 Any sort of error; might indicate that the error doesn’t fall nicely into one of the subsequent categories, or the developer was too lazy (or too skeptical about standards) to pick a more descriptive error code
2 Misuse of shell builtins (missing keyword or command); permission problem; diff return code on a failed binary file comparison.
126 Command invoked cannot execute (permission problem or command is not an executable)
127 Command not found
128 Invalid argument to exit (caused by exit "tau" or exit 6.28, etc.)
128+n Fatal error signal n; kill -9 $PID (SIGKILL) causes the script running as $PID to have exit code 137; terminating a script with Ctrl-c sets the exit code to 130 (Ctrl-c is fatal error signal 2 (SIGINT), and 128 + 2 = 130)
255 Exit status out of range (caused by exit -1 or lower; calling exit 1000 is like calling exit 232, since 1000 % 256 = 232)

Exit code constants from /usr/include/sysexits.h:

Constant Value Description
EX_OK 0 successful termination
EX_USAGE 64 command line usage error
EX_DATAERR 65 data format error
EX_NOINPUT 66 cannot open input
EX_NOUSER 67 addressee unknown
EX_NOHOST 68 host name unknown
EX_UNAVAILABLE 69 service unavailable
EX_SOFTWARE 70 internal software error
EX_OSERR 71 system error (e.g., can’t fork)
EX_OSFILE 72 critical OS file missing
EX_CANTCREAT 73 can’t create (user) output file
EX_IOERR 74 input/output error
EX_TEMPFAIL 75 temp failure; user is invited to retry
EX_PROTOCOL 76 remote error in protocol
EX_NOPERM 77 permission denied
EX_CONFIG 78 configuration error

“Errno” codes (from errno --list):

Constant Value Description
EPERM 1 Operation not permitted
ENOENT 2 No such file or directory
ESRCH 3 No such process
EINTR 4 Interrupted system call
EIO 5 Input/output error
ENXIO 6 Device not configured
E2BIG 7 Argument list too long
ENOEXEC 8 Exec format error
EBADF 9 Bad file descriptor
ECHILD 10 No child processes
EDEADLK 11 Resource deadlock avoided
ENOMEM 12 Cannot allocate memory
EACCES 13 Permission denied
EFAULT 14 Bad address
ENOTBLK 15 Block device required
EBUSY 16 Resource busy
EEXIST 17 File exists
EXDEV 18 Cross-device link
ENODEV 19 Operation not supported by device
ENOTDIR 20 Not a directory
EISDIR 21 Is a directory
EINVAL 22 Invalid argument
ENFILE 23 Too many open files in system
EMFILE 24 Too many open files
ENOTTY 25 Inappropriate ioctl for device
ETXTBSY 26 Text file busy
EFBIG 27 File too large
ENOSPC 28 No space left on device
ESPIPE 29 Illegal seek
EROFS 30 Read-only file system
EMLINK 31 Too many links
EPIPE 32 Broken pipe
EDOM 33 Numerical argument out of domain
ERANGE 34 Result too large
EAGAIN 35 Resource temporarily unavailable
EWOULDBLOCK 35 Resource temporarily unavailable
EINPROGRESS 36 Operation now in progress
EALREADY 37 Operation already in progress
ENOTSOCK 38 Socket operation on non-socket
EDESTADDRREQ 39 Destination address required
EMSGSIZE 40 Message too long
EPROTOTYPE 41 Protocol wrong type for socket
ENOPROTOOPT 42 Protocol not available
EPROTONOSUPPORT 43 Protocol not supported
ESOCKTNOSUPPORT 44 Socket type not supported
ENOTSUP 45 Operation not supported
EPFNOSUPPORT 46 Protocol family not supported
EAFNOSUPPORT 47 Address family not supported by protocol family
EADDRINUSE 48 Address already in use
EADDRNOTAVAIL 49 Can’t assign requested address
ENETDOWN 50 Network is down
ENETUNREACH 51 Network is unreachable
ENETRESET 52 Network dropped connection on reset
ECONNABORTED 53 Software caused connection abort
ECONNRESET 54 Connection reset by peer
ENOBUFS 55 No buffer space available
EISCONN 56 Socket is already connected
ENOTCONN 57 Socket is not connected
ESHUTDOWN 58 Can’t send after socket shutdown
ETOOMANYREFS 59 Too many references: can’t splice
ETIMEDOUT 60 Operation timed out
ECONNREFUSED 61 Connection refused
ELOOP 62 Too many levels of symbolic links
ENAMETOOLONG 63 File name too long
EHOSTDOWN 64 Host is down
EHOSTUNREACH 65 No route to host
ENOTEMPTY 66 Directory not empty
EPROCLIM 67 Too many processes
EUSERS 68 Too many users
EDQUOT 69 Disc quota exceeded
ESTALE 70 Stale NFS file handle
EREMOTE 71 Too many levels of remote in path
EBADRPC 72 RPC struct is bad
ERPCMISMATCH 73 RPC version wrong
EPROGUNAVAIL 74 RPC prog. not avail
EPROGMISMATCH 75 Program version wrong
EPROCUNAVAIL 76 Bad procedure for program
ENOLCK 77 No locks available
ENOSYS 78 Function not implemented
EFTYPE 79 Inappropriate file type or format
EAUTH 80 Authentication error
ENEEDAUTH 81 Need authenticator
EPWROFF 82 Device power is off
EDEVERR 83 Device error
EOVERFLOW 84 Value too large to be stored in data type
EBADEXEC 85 Bad executable (or shared library)
EBADARCH 86 Bad CPU type in executable
ESHLIBVERS 87 Shared library version mismatch
EBADMACHO 88 Malformed Mach-o file
ECANCELED 89 Operation canceled
EIDRM 90 Identifier removed
ENOMSG 91 No message of desired type
EILSEQ 92 Illegal byte sequence
ENOATTR 93 Attribute not found
EBADMSG 94 Bad message
EMULTIHOP 95 EMULTIHOP (Reserved)
ENODATA 96 No message available on STREAM
ENOLINK 97 ENOLINK (Reserved)
ENOSR 98 No STREAM resources
ENOSTR 99 Not a STREAM
EPROTO 100 Protocol error
ETIME 101 STREAM ioctl timeout
EOPNOTSUPP 102 Operation not supported on socket
ENOPOLICY 103 Policy not found
ENOTRECOVERABLE 104 State not recoverable
EOWNERDEAD 105 Previous owner died
EQFULL 106 Interface output queue is full
ELAST 106 Interface output queue is full

Idioms, snippets, and examples

Templates / boilerplate

Simple bash script preamble:

#!/usr/bin/env bash
set -e # exit immediately on error

usage() {
  >&2 cat <<HELP
Usage: $(basename "$0") [-h|--help]

<What the script does>
HELP
}

while [[ $# -gt 0 ]]; do
  case $1 in
    -h|--help)
      usage
      exit 0
      ;;
    -v|--verbose)
      >&2 printf 'Entering debug (verbose) mode.\n'
      set -x
      ;;
    *)
      usage
      >&2 printf '\nUnrecognized option: %s\n' "$1"
      exit 1
      ;;
  esac
  shift
done

<do something>