Shell - examples

For years Perl was my goto for any problem, but most of the time a decent shell script was just fine. And if one is running lots of commands and looking more at exit status, perhaps a shell script will do just fine.

So there are the original single bracket [ ] tests, which were supplanted by the double bracket [[ ]] tests, and the math engine (( ))

So the evolution was:

[ $i -gt 12 ]   =>   [[ $i -gt 12 ]]   =>   (( i > 12 ))

So for testing in shell, leave the 90's behind. ksh93/bash/zsh has all this.

Test    Perl                      shell (ksh/bash/zsh)
==================================================================================
strcmp   ($var == 'hello world')  [[ $var == 'hello world' ]]
regex    ($var =~ /^\w+$/)        [[ $var =~ ^[a-zA-Z0-9_][[a-zA-Z0-9_]* ]] 
numeric  ($var eq 3.14159)        (( var == 3.14159 ))  # note the lack of $ sigil
compound (12 < $var && $var < 16) (( 12 < var && var < 16 ))
                                  [[ -f $file && $file =~ hosts ]]

add the below to a file and test out..

#!/bin/bash

#!/bin/ksh or /bin/zsh (ksh on roids)

set -u  # do not allow empty/undeclared variables - helps with typos

set -e  # bail on any failure. Can get rid of all those  && between commands.

set -x  # enable debug mode; turn off with 'set +x'.  Huh?. I just accept it.

function docleanup {
    echo 'this could be a cleanup tmp file function'
    echo and we got one arg: $1
}

true && echo 'true is always true'
false || echo 'false always non-zero exit'

host=localhost
if true
then
    echo 'true test is always true'
else
 echo 'but the && || are shorter for one liners..'
fi

# echo to do multiple things on a test
# simple for loop
for host in localhost thirud.com.foo
do
   ping -c 1 -w 3 $host 1>/dev/null 2>&1 && echo $host up ||
    echo $host down

   ping -c 1 -w 3 $host 1>/dev/null 2>&1 ||
    { echo -n $host is down; echo ' .. its really down'; }
#                          this semicolon is needed ---^ 
done

# traps - trap the exit code and do something.
# Unix has ~32 slots for doing stuff before tearing the process down..
# The zero signal had no meaning for a long time, and it was then used 
# for 'trap anything'
trap 'we are leaving the program' 0
trap 'docleanup Bob' 0

# Assignment cannot have spaces, unlike other languages.
value = 12                         # invalid
value=12                           # set value to 12
var=value           && echo $var   # simple way to set vars without whitespace
var="$value"        && echo $var   # quotes -interpret variables
var="${value}stuff" && echo $var   # delimit the variable name from other text..
var='$value'        && echo $var   # string literal - no interpretation


# the dollar in front of a variable dereferences it - thus 
# var=12 is accessed by $var
echo "cows go 'moo'" 

var=   # sets to empty

# getopt is old one, getopts is newer one..
# these are functions, reads in options and sets OPTARG, OPTIND
while getopts abc:d opt
do
    case $opt in 
    a) echo  I caught an 'a' ;;
    b) echo "I caught a 'b'" ;;
    c) echo "c had a colon, so next thing in line was set to \"$OPTARG\"" ;;
    d) echo 'just a d' ;;
    *) echo "Usage: $me [-a] [-b] [-c <string>] [-d]" ;;
    esac
done

# shift out read-in options - since OPTIND is post-incremented, subtract 1 
# and then shift that many options out. leaves $1 equal to first unprocessed
# arg from the command line.
shift $(( OPTIND - 1 ))


# (( )) and [[ ]] are ugly becuase they were added to shell later and to 
# not break older scripts that [ (symlinked to /bin/test). Most languages 
# just say 'warning: your scripts will break', but shell so widely used all
# over the OS, that would be A Very Bad Idea.  New features were instead 
# bolted-on. Frankenstein had bolts.

# number compare - the dollar sign not needed inside 
(( var == 12 )) && echo "var = $var" || echo 'var != 12'

# string compare
s=unix
[[ $s == unix ]] && echo $s

#regex  - do not wrap the compare string
[[ $s =~ nix  ]] && echo "\"$s\" is some unix variant"
[[ 192.168.0.1 =~ ^[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*$  ]] &&
    echo "we have IP"

# substitutes
# rem - to extend shell, it looks like they took the approach 'cannot alter
# any existing syntax. current syntax is golden. So we get ugly syntax:

# substitution
car=vw; car=${car/vw/Volkswagen}   # its a Volkswagen man
song=lalalalala; song=${song/la}   # one too many la's

echo man ksh to see a good explanation of the parameters

# set if unset
name={name:-bob}

# man test    to see more..

cat << HURL 
I rem this easliy becuase its like a cat barfing up a furball
so use 
   cat << HURL
   cat << HURL > file

     text to spit out, or to file

   HURL
HURL

cat <<- FURBALL

    Adding the hyphen to << will strip leading tabs.  Allows one to indent 
    heredoc and keep code looking better

FURBALL

echo 'this will exec a command, sending output to a temporary file descriptor'
echo 'yeah no more save to file'
echo 'diff -w <(date) <(sleep 2; date +%s)'

Redirect output or create new output channels (put within script)

exec 2>/dev/null    # now send STDERR to /dev/null

# open other channels for more outputs
exec 3>errors
exec 4>output
exec 5>log

echo 'use $( ) instead of `backticks` because nesting of commands works'
echo $( echo $( echo $( echo $( echo 3 ) 2 ) 1 ) ) top

cat <<- 'YAK'

    % echo $( echo $( echo $( echo $( echo 3 ) 2 ) 1 ) ) top
    3 2 1 top

    % echo ` echo ` echo ` echo ` echo 3 ` 2 ` 1 ` ` top
    -bash: 2: command not found
    echo echo 3 1 top

YAK

# parsing - the shell parses in at least 3 passes:
# original command:
# SIZE=2000000; ls -l $(find . -size +${SIZE}c -print) 1>filelist.txt  2>&1
# the semicolon separates the commands.

# scan left to right for file redirections 
# pipes are executed right to left
# the 'in between' each pipe is what is processed left to right.
#
# SIZE=2000000; ls -l $(find . -size +${SIZE}c -print) 1>filelist.txt  2>&1
#
# the amper1 means 'address of 1' - yup - thats from C.

# this would redirect STDOUT to a file, then redirect 2 to whatever 1 is 
# pointing at.  Remember these are file handles. yes - if using > files 
# will be truncated at this stage
# the command has not been run yet. it is being setup..
# which is why
# cat file | sort > file
# empties your file

# setup redirects - command not run yet
# currently, 1 and 2 are directed to the terminal. this will 'redirect' them. 
# ls -l $(find . -size +${SIZE}c -print) 1>filelist.txt 2>filelist.txt

# shell substitutions are made - command still not run yet
# ls -l $(find . -size +2000000c -print) 1>filelist.txt 2>filelist.txt

# then background commands are run - find discovered 3 files
# ls -l file1.mp3 file2mp3 file3.mp3 1>filelist.txt 2>filelist.txt

# finally the command is executed. 
# ls -l file1.mp3 file2mp3 file3.mp3 1>filelist.txt 2>filelist.txt


# Note 
# 1>file 2>&1    is often mixed up with  2>&1 1>file
# but if reading left to right, its easier to see what is wrong:
# rem the redirects and STDERR/STDOUT are pointing at terminal when the
# parsing begins.
# 
# STDOUT   terminal
# STDERR   terminal
# 2>&1     redirect 2 at whatever 1 is pointed at (STDOUT)
# 1>file   redirect 1 to 'file'

# final result
# STDOUT   file
# STDERR   terminal

# if this was a cron job, the errors are not going to the file. This 
# has given me a false sense that all is ok.
# what I meant was: 
# 1>file 2>&1 
# parse that left to right, now my file will get STDOUT+STDERR

Published on in