Initialization

• ~/.bash_profile

  • According to bash(1):

    The personal initialization file, executed for login shells.

  • It’s run once when you login, but not for each additional (sub)shell you open.

  • Evidently, macOS (Terminal) runs a full login shell for each new session you start^ref, which is why my ~/.bash_profile simply reads:

    [ -f ~/.bashrc ] && source ~/.bashrc
    

• ~/.bashrc

  • According to bash(1):

    The individual per-interactive-shell startup file.

  • It is not automatically loaded when you login, but only when you open an interactive shell.

• ~/.profile
  • Read by both sh and bash, but bash will ignore it if ~/.bash_profile exists.

Positional parameters

Operators

Internal variables

• $PROMPT_COMMAND

  Execute the contents of this variable as a command before showing the prompt.

• $PIPESTATUS

  An array of exit statuses from the different processes in the last sequence of piped commands. $PIPESTATUS[0] holds the exit status of the first (leftmost) process, $PIPESTATUS[1] holds the exit status of the second process, etc.

String manipulation

The Advanced Bash-Scripting Guide is misleading. The string in these patterns is a variable name, while substring is the actual string value of the trigger pattern.

For example, to replace all newlines in the variable IDS with commas: ${IDS//$'\n'/,}

Table of behaviors when $param is in different states, drawn from https://stackoverflow.com/a/16753536:

Array variables

All bash variables are arrays; it’s just that most of them have only one value, and the default variable-dereferencing syntax retrieves the first value. I.e., $var is equivalent to the more explicit ${var[0]} for any variable var.

The special array variables $@, $*, and $#, as well as $1, $2, …, are well known since they refer to the command’s / function’s arguments, but other arrays require a little extra syntax:

Array literals use round parentheses; (a b), ( a b ), and ('a' 'b') are all equivalent; (a b), ('a b'), and (a, b) are all different.

• INPUT_FILES=()

  This initializes a local variable to have the value of an empty array. This is different from INPUT_FILES=, which sets the value (of the first element, implicitly) of the variable INPUT_FILES to the empty string.

• INPUT_FILES+=(/tmp/z)

  This appends the string /tmp/z to the INPUT_FILES variable. The RHS of the += operator is simply an array literal of any length, which gets concatenated to the end of the current value of the INPUT_FILES (array) variable.

• INPUT_FILES+='/tmp/a'

  This appends the string /tmp/a to the first element in INPUT_FILES; probably not what you intended!

Bash does not support multidimensional arrays; e.g., GRID=((1 0) (0 1)) is a syntax error.

Arrays can be sliced much as in Python: ${FILES[@]:1} returns all but the first element; ${FILES[@]:0:2} returns just the first two elements.

And at least in Bash 4, negative indices work as counting from the end of the array: ${FILES[-1]} returns the last element in FILES.

Earlier versions of bash (and some other shells) have a special syntax to get the last command line argument: ${@: -1} (n.b.: the space before -1 is required; otherwise it’s parsed as parameter substitution’s :- default-when-null operator.)

References:

• Bash Reference Manual » 6 Bash Features » 6.7 Arrays
• Advanced Bash-Scripting Guide » Chapter 27. Arrays

Examples

Suppose ARR=(a b c) and argc() { printf "%d\n" $#; }:

IO redirection

: can be used like Python pass, as a placeholder for a no-op statement where, otherwise, there would be a syntax error. For example, echo hello | | cat won’t run, but echo hello | : | cat will (though it won’t have any input, because : never produces output).

These generally only affect the behavior of the line they appear on; to apply them to the current shell session, use exec. For example, call exec 2>>/var/log/somescript.log at the beginning of your shell script to redirect all STDERR to that file instead of the appearing in the user/caller’s TTY (and thus preempts the user’s control over STDERR).

• exec {NEW_FD}<>filename

  Open file or device filename as the smallest (beginning with 10) available file descriptor, and save it to the environmental variable: NEW_FD. So after this command runs, echo $NEW_FD will return something like 10. Use the {NEW_FD} syntax whenever creating or closing the named file descriptor, but use &$NEW_FD whenever using it as a redirection target.

• exec 2> >(logger) or exec 2>> >(logger) or exec 2>>>(logger)

  Send all STDERR over to syslog.

  logger is a BSD (and thus, macOS) command that opens up a process that can be written to, which pipes input into syslog. E.g., echo "testing logger from shell" | logger will show up in Console.app with a timestamp and the “tag” chbrown, since that’s my username. logger -t shell-log applies the tag shell-log to everything it receives on STDIN. Each input line translate to a single syslog entry.

  The three commands are effectively equivalent, since there’s no difference between overwriting and appending to the file descriptor that logger opens. The latter two commands are syntactically equivalent, but a space is required between the N> and >(...) syntax in the first command. Due to bash’s parser, exec 2>>(logger) is lexed as exec 2>> (logger), which is a syntax error.

• exec >/tmp/script.log

  For the remainder of the current script / scope, this will write STDOUT to the file /tmp/script.log (overwriting it if it exists).

• exec 3<>/tmp/other.log; date >&3; exec 3>&-

  Open the file /tmp/other.log as file descriptor 3, write the current date to it, and then close the file descriptor.

  (exec 3<> /tmp/other.log syntax is equivalent to the first command.)

  File descriptors 3 through 9 are all initially closed, but each can be arbitrarily opened and closed with this exec N<> syntax. Trying to read from or write to a closed file descriptor will raise the error message “N: Bad file descriptor”, where N is the closed file descriptor you tried to use.

  The first step creates the file if needed, and sets the pointer to the beginning of it. Writing to it will simply overwrite whatever’s currently there, byte for byte; i.e., it doesn’t truncate the file before writing to it. Each write advances the cursor, so multiple calls to date >&3 will write each timestamp to its own line.

  To append to a file descriptor opened this way, you can read through it with, e.g., cat; upon completion cat <&3 >/dev/null, all subsequent writes to file descriptor 3 will be appended to the file, because the cursor has been advanced to the end of the file.

Exit codes

In a bash session, $? refers to the exit / status code of the last-run command. Inside a bash script, $? starts out 0, and the exit code of the overall script (insofar as the caller is concerned), when it naturally finishes/exits, is set to the value of $?. You can’t manually set the exit code, e.g., ?=101, though.

Table of “Exit Codes With Special Meanings” from Appendix E of the “Advanced Bash-Scripting Guide” (with clarifications where applicable):

Exit code constants from /usr/include/sysexits.h:

“Errno” codes (from errno --list):

Idioms, snippets, and examples

• ${1-default.txt}

  Evaluates to default.txt if $1 is undefined, otherwise uses whatever the value of $1 is (even if it’s the empty string).

• mkdir -p /tmp/test && cd $_

  Make a directory (if needed) and navigate into it in the same breath.

• bc <<< "2 + 2"

  Send a simple string on STDIN. Equivalent to echo "2 + 2" | bc.

• lsof -p $$ or ls -l /dev/fd

  List all file descriptors open in current shell.

• some_cmd 2>&1 >/dev/null

  Discard STDOUT, Redirect STDERR to STDOUT. Order matters: >/dev/null 2>&1 does not work. Depending on the behavior of some_cmd, you can replace >/dev/null with >&-, but if some_cmd writes to STDOUT without checking if it’s open, or does not catch write errors on that front, /dev/null is required.

• declare -p PATH (equivalent to typeset -p):

  Writes a bit of shell script to STDOUT, that, when source‘d, would replicate the current value of the PATH environment variable.

• set -x

  Use this to debug; it prints all bash calls to /dev/stderr

  Turn it off with set +x

• set -e

  Exit script immediately if anything throws / exits with an error

• set -u

  Throw an error if we use an unset variable.

• set -o pipefail

  Set the overall result of a series of piped commands to the exit code of the last command to exit with a non-zero status.

• set -euo pipefail

  Combination of the three preceding options: exit on error, exit on unset variables, and accumulate errors in pipeline.

  ⭐ best practice!

• mktemp -d

  This will safely create a new directory in $TMPDIR, and write the full path to /dev/stdout. This is the only argument structure that works equivalently on both Linux and macOS (at least, macOS 10.11). It will not take responsibility for deleting the file at any point.

• TMPDIR=${TMPDIR-/tmp/}

  Ensure that the $TMPDIR variable exists, using a default of /tmp/ if $TMPDIR has no existing value (or is empty). (On macOS, TMPDIR is set to something like /var/folders/m8/ga778wd7rv9g0p_3cqhpti400qdfa7/T/, so I’ve applied a trailing slash to the default, for consistency.)

• DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

  This sets DIR to the absolute pathname of the directory containing the script. You should call this early, before the script has had a chance to cd to any other working directory.

  When the script is executed directly because it’s on the PATH, BASH_SOURCE[0] is the full path to the script. But if it’s executed as an argument to bash, BASH_SOURCE[0] will be set to exact value of that argument. The cd does the real work of resolving and simplifying any relative or symbolic path components, but by running within a subshell, it does not affect the root-level current working directory.

  In both cases, the initial working directory is the directory the script was called from. In the latter case, when BASH_SOURCE[0] is a relative directory to begin with and so the working directory is relevant. Any intermediate cd calls will probably render BASH_SOURCE[0] an invalid path, which is why it’s important to call this snippet before anything else might alter the working directory.

  This method will not resolve symlinks in the path to their linked source names.

  Source: Getting the source directory of a Bash script from within

• command -v some_cmd (POSIX compliant)

  This has exit code 0 if some_cmd is something you can call, e.g., a binary on PATH, an alias, or a shell function. It also prints a description of some_cmd to /dev/stdout when it succeeds, so you’ll probably want to 2>/dev/null.

  type some_cmd and hash some_cmd are more flexible/fast/beneficial alternatives, but are only reliable in bash. hash some_cmd is successful if some_cmd is a command on the PATH or a function (but fails for aliases). type some_cmd covers all the bases, like command -v some_cmd, but has a type -P some_cmd variant that limits success to when some_cmd is on the PATH. (type also prints out the source code when given a function, which can be handy.)

  Source: Check if a program exists from a Bash script

• function doit { ...; } vs. function doit () { ...; } vs. doit() { ...; }

  These are practically equivalent, but the version without the function keyword (in which case the parentheses are required) is preferred, due to increased compatibility / portability with the POSIX spec.

• ctrl+z        # suspend (to resume in the same session, call: bg)
  disown %1     # disown
  tmux          # launch another shell in screen / tmux / etc.
  pgrep sommat-in-my-process   # find PID
  reptyr 99999  # reparent process (where 99999 is the PID from above)
  

  This will transfer the current shell’s running process to another parent.^ref

  Depends on reptyr.

Templates / boilerplate

Simple bash script preamble:

#!/usr/bin/env bash
set -e # exit immediately on error

usage() {
  >&2 cat <<HELP
Usage: $(basename "$0") [-h|--help]

<What the script does>
HELP
}

while [[ $# -gt 0 ]]; do
  case $1 in
    -h|--help)
      usage
      exit 0
      ;;
    -v|--verbose)
      >&2 printf 'Entering debug (verbose) mode.\n'
      set -x
      ;;
    *)
      usage
      >&2 printf '\nUnrecognized option: %s\n' "$1"
      exit 1
      ;;
  esac
  shift
done

<do something>