10.1. Loops

for loops

for arg in [list]

This is the basic looping construct. It differs significantly from its C counterpart.

for arg in [list]
do
═command(s)...
done

During each pass through the loop, arg takes on the value of each successive variable in the list.

for arg in "$var1" "$var2" "$var3" ... "$varN" # In pass 1 of the loop, arg = $var1 # In pass 2 of the loop, arg = $var2 # In pass 3 of the loop, arg = $var3 # ... # In pass N of the loop, arg = $varN # Arguments in [list] quoted to prevent possible word splitting.

The argument list may contain wild cards.

If do is on same line as for, there needs to be a semicolon after list.

for arg in [list] ; do

Example 10-1. Simple for loops

#!/bin/bash # Listing the planets. for planet in Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto do echo $planet # Each planet on a separate line. done echo; echo for planet in "Mercury Venus Earth Mars Jupiter Saturn Uranus Neptune Pluto" # All planets on same line. # Entire 'list' enclosed in quotes creates a single variable. # Why? Whitespace incorporated into the variable. do echo $planet done echo; echo "Whoops! Pluto is no longer a planet!" exit 0

Each [list] element may contain multiple parameters. This is useful when processing parameters in groups. In such cases, use the set command (see Example 14-16) to force parsing of each [list] element and assignment of each component to the positional parameters.

Example 10-2. for loop with two parameters in each [list] element

#!/bin/bash # Planets revisited. # Associate the name of each planet with its distance from the sun. for planet in "Mercury 36" "Venus 67" "Earth 93" "Mars 142" "Jupiter 483" do set -- $planet # Parses variable "planet" #+ and sets positional parameters. # The "--" prevents nasty surprises if $planet is null or #+ begins with a dash. # May need to save original positional parameters, #+ since they get overwritten. # One way of doing this is to use an array, # original_params=("$@") echo "$1 $2,000,000 miles from the sun" #-------two tabs---concatenate zeroes onto parameter $2 done # (Thanks, S.C., for additional clarification.) exit 0

A variable may supply the [list] in a for loop.

Example 10-3. Fileinfo: operating on a file list contained in a variable

#!/bin/bash # fileinfo.sh FILES="/usr/sbin/accept /usr/sbin/pwck /usr/sbin/chroot /usr/bin/fakefile /sbin/badblocks /sbin/ypbind" # List of files you are curious about. # Threw in a dummy file, /usr/bin/fakefile. echo for file in $FILES do if [ ! -e "$file" ] # Check if file exists. then echo "$file does not exist."; echo continue # On to next. fi ls -l $file | awk '{ print $9 " file size: " $5 }' # Print 2 fields. whatis `basename $file` # File info. # Note that the whatis database needs to have been set up for this to work. # To do this, as root run /usr/bin/makewhatis. echo done exit 0

If the [list] in a for loop contains wild cards (* and ?) used in filename expansion, then globbing takes place.

Example 10-4. Operating on files with a for loop

#!/bin/bash # list-glob.sh: Generating [list] in a for-loop, using "globbing" echo for file in * # ^ Bash performs filename expansion #+ on expressions that globbing recognizes. do ls -l "$file" # Lists all files in $PWD (current directory). # Recall that the wild card character "*" matches every filename, #+ however, in "globbing," it doesn't match dot-files. # If the pattern matches no file, it is expanded to itself. # To prevent this, set the nullglob option #+ (shopt -s nullglob). # Thanks, S.C. done echo; echo for file in [jx]* do rm -f $file # Removes only files beginning with "j" or "x" in $PWD. echo "Removed file \"$file\"". done echo exit 0

Omitting the in [list] part of a for loop causes the loop to operate on $@ -- the positional parameters. A particularly clever illustration of this is Example A-16. See also Example 14-17.

Example 10-5. Missing in [list] in a for loop

#!/bin/bash # Invoke this script both with and without arguments, #+ and see what happens. for a do echo -n "$a " done # The 'in list' missing, therefore the loop operates on '$@' #+ (command-line argument list, including whitespace). echo exit 0

It is possible to use command substitution to generate the [list] in a for loop. See also Example 15-54, Example 10-10 and Example 15-48.

Example 10-6. Generating the [list] in a for loop with command substitution

#!/bin/bash # for-loopcmd.sh: for-loop with [list] #+ generated by command substitution. NUMBERS="9 7 3 8 37.53" for number in `echo $NUMBERS` # for number in 9 7 3 8 37.53 do echo -n "$number " done echo exit 0

Here is a somewhat more complex example of using command substitution to create the [list].

Example 10-7. A grep replacement for binary files

#!/bin/bash # bin-grep.sh: Locates matching strings in a binary file. # A "grep" replacement for binary files. # Similar effect to "grep -a" E_BADARGS=65 E_NOFILE=66 if [ $# -ne 2 ] then echo "Usage: `basename $0` search_string filename" exit $E_BADARGS fi if [ ! -f "$2" ] then echo "File \"$2\" does not exist." exit $E_NOFILE fi IFS=$'\012' # Per suggestion of Anton Filippov. # was: IFS="\n" for word in $( strings "$2" | grep "$1" ) # The "strings" command lists strings in binary files. # Output then piped to "grep", which tests for desired string. do echo $word done # As S.C. points out, lines 23 - 30 could be replaced with the simpler # strings "$2" | grep "$1" | tr -s "$IFS" '[\n*]' # Try something like "./bin-grep.sh mem /bin/ls" #+ to exercise this script. exit 0

More of the same.

Example 10-8. Listing all users on the system

#!/bin/bash # userlist.sh PASSWORD_FILE=/etc/passwd n=1 # User number for name in $(awk 'BEGIN{FS=":"}{print $1}' < "$PASSWORD_FILE" ) # Field separator = : ^^^^^^ # Print first field ^^^^^^^^ # Get input from password file ^^^^^^^^^^^^^^^^^ do echo "USER #$n = $name" let "n += 1" done # USER #1 = root # USER #2 = bin # USER #3 = daemon # ... # USER #30 = bozo exit 0 # Exercise: # -------- # How is it that an ordinary user (or a script run by same) #+ can read /etc/passwd? # Isn't this a security hole? Why or why not?

Yet another example of the [list] resulting from command substitution.

Example 10-9. Checking all the binaries in a directory for authorship

#!/bin/bash # findstring.sh: # Find a particular string in the binaries in a specified directory. directory=/usr/bin/ fstring="Free Software Foundation" # See which files come from the FSF. for file in $( find $directory -type f -name '*' | sort ) do strings -f $file | grep "$fstring" | sed -e "s%$directory%%" # In the "sed" expression, #+ it is necessary to substitute for the normal "/" delimiter #+ because "/" happens to be one of the characters filtered out. # Failure to do so gives an error message. (Try it.) done exit $? # Exercise (easy): # --------------- # Convert this script to take command-line parameters #+ for $directory and $fstring.

A final example of [list] / command substitution, but this time the "command" is a function.

generate_list () { echo "one two three" } for word in $(generate_list) # Let "word" grab output of function. do echo "$word" done # one # two # three

The output of a for loop may be piped to a command or commands.

Example 10-10. Listing the symbolic links in a directory

#!/bin/bash # symlinks.sh: Lists symbolic links in a directory. directory=${1-`pwd`} # Defaults to current working directory, #+ if not otherwise specified. # Equivalent to code block below. # ---------------------------------------------------------- # ARGS=1 # Expect one command-line argument. # # if [ $# -ne "$ARGS" ] # If not 1 arg... # then # directory=`pwd` # current working directory # else # directory=$1 # fi # ---------------------------------------------------------- echo "symbolic links in directory \"$directory\"" for file in "$( find $directory -type l )" # -type l = symbolic links do echo "$file" done | sort # Otherwise file list is unsorted. # Strictly speaking, a loop isn't really necessary here, #+ since the output of the "find" command is expanded into a single word. # However, it's easy to understand and illustrative this way. # As Dominik 'Aeneas' Schnitzer points out, #+ failing to quote $( find $directory -type l ) #+ will choke on filenames with embedded whitespace. # Even this will only pick up the first field of each argument. exit 0 # -------------------------------------------------------- # Jean Helou proposes the following alternative: echo "symbolic links in directory \"$directory\"" # Backup of the current IFS. One can never be too cautious. OLDIFS=$IFS IFS=: for file in $(find $directory -type l -printf "%p$IFS") do # ^^^^^^^^^^^^^^^^ echo "$file" done|sort # And, James "Mike" Conley suggests modifying Helou's code thusly: OLDIFS=$IFS IFS='' # Null IFS means no word breaks for file in $( find $directory -type l ) do echo $file done | sort # This works in the "pathological" case of a directory name having #+ an embedded colon. # "This also fixes the pathological case of the directory name having #+ a colon (or space in earlier example) as well."

The stdout of a loop may be redirected to a file, as this slight modification to the previous example shows.

Example 10-11. Symbolic links in a directory, saved to a file

#!/bin/bash # symlinks.sh: Lists symbolic links in a directory. OUTFILE=symlinks.list # save file directory=${1-`pwd`} # Defaults to current working directory, #+ if not otherwise specified. echo "symbolic links in directory \"$directory\"" > "$OUTFILE" echo "---------------------------" >> "$OUTFILE" for file in "$( find $directory -type l )" # -type l = symbolic links do echo "$file" done | sort >> "$OUTFILE" # stdout of loop # ^^^^^^^^^^^^^ redirected to save file. exit 0

There is an alternative syntax to a for loop that will look very familiar to C programmers. This requires double parentheses.

Example 10-12. A C-style for loop

#!/bin/bash # Multiple ways to count up to 10. echo # Standard syntax. for a in 1 2 3 4 5 6 7 8 9 10 do echo -n "$a " done echo; echo # +==========================================+ # Using "seq" ... for a in `seq 10` do echo -n "$a " done echo; echo # +==========================================+ # Using brace expansion ... # Bash, version 3+. for a in {1..10} do echo -n "$a " done echo; echo # +==========================================+ # Now, let's do the same, using C-like syntax. LIMIT=10 for ((a=1; a <= LIMIT ; a++)) # Double parentheses, and "LIMIT" with no "$". do echo -n "$a " done # A construct borrowed from 'ksh93'. echo; echo # +=========================================================================+ # Let's use the C "comma operator" to increment two variables simultaneously. for ((a=1, b=1; a <= LIMIT ; a++, b++)) do # The comma chains together operations. echo -n "$a-$b " done echo; echo exit 0

See also Example 26-16, Example 26-17, and Example A-6.

---

Now, a for loop used in a "real-life" context.

Example 10-13. Using efax in batch mode

#!/bin/bash # Faxing (must have 'efax' package installed). EXPECTED_ARGS=2 E_BADARGS=85 MODEM_PORT="/dev/ttyS2" # May be different on your machine. # ^^^^^ PCMCIA modem card default port. if [ $# -ne $EXPECTED_ARGS ] # Check for proper number of command-line args. then echo "Usage: `basename $0` phone# text-file" exit $E_BADARGS fi if [ ! -f "$2" ] then echo "File $2 is not a text file." # File is not a regular file, or does not exist. exit $E_BADARGS fi fax make $2 # Create fax-formatted files from text files. for file in $(ls $2.0*) # Concatenate the converted files. # Uses wild card (filename "globbing") #+ in variable list. do fil="$fil $file" done efax -d "$MODEM_PORT" -t "T$1" $fil # Finally, do the work. # Trying adding -o1 if above line fails. # As S.C. points out, the for-loop can be eliminated with # efax -d /dev/ttyS2 -o1 -t "T$1" $2.0* #+ but it's not quite as instructive [grin]. exit $? # Also, efax sends diagnostic messages to stdout.

while

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is true (returns a 0 exit status). In contrast to a for loop, a while loop finds use in situations where the number of loop repetitions is not known beforehand.

while [ condition ]
do
═command(s)...
done

The bracket construct in a while loop is nothing more than our old friend, the test brackets used in an if/then test. In fact, a while loop can legally use the more versatile double-brackets construct (while [[ condition ]]).

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

while [ condition ] ; do

Note that the test brackets are not mandatory in a while loop. See, for example, the getopts construct.

Example 10-14. Simple while loop

#!/bin/bash var0=0 LIMIT=10 while [ "$var0" -lt "$LIMIT" ] # ^ ^ # Spaces, because these are "test-brackets" . . . do echo -n "$var0 " # -n suppresses newline. # ^ Space, to separate printed out numbers. var0=`expr $var0 + 1` # var0=$(($var0+1)) also works. # var0=$((var0 + 1)) also works. # let "var0 += 1" also works. done # Various other methods also work. echo exit 0

Example 10-15. Another while loop

#!/bin/bash echo # Equivalent to: while [ "$var1" != "end" ] # while test "$var1" != "end" do echo "Input variable #1 (end to exit) " read var1 # Not 'read $var1' (why?). echo "variable #1 = $var1" # Need quotes because of "#" . . . # If input is 'end', echoes it here. # Does not test for termination condition until top of loop. echo done exit 0

A while loop may have multiple conditions. Only the final condition determines when the loop terminates. This necessitates a slightly different loop syntax, however.

Example 10-16. while loop with multiple conditions

#!/bin/bash var1=unset previous=$var1 while echo "previous-variable = $previous" echo previous=$var1 [ "$var1" != end ] # Keeps track of what $var1 was previously. # Four conditions on "while", but only last one controls loop. # The *last* exit status is the one that counts. do echo "Input variable #1 (end to exit) " read var1 echo "variable #1 = $var1" done # Try to figure out how this all works. # It's a wee bit tricky. exit 0

As with a for loop, a while loop may employ C-style syntax by using the double-parentheses construct (see also Example 9-33).

Example 10-17. C-style syntax in a while loop

#!/bin/bash # wh-loopc.sh: Count to 10 in a "while" loop. LIMIT=10 a=1 while [ "$a" -le $LIMIT ] do echo -n "$a " let "a+=1" done # No surprises, so far. echo; echo # +=================================================================+ # Now, repeat with C-like syntax. ((a = 1)) # a=1 # Double parentheses permit space when setting a variable, as in C. while (( a <= LIMIT )) # Double parentheses, and no "$" preceding variables. do echo -n "$a " ((a += 1)) # let "a+=1" # Yes, indeed. # Double parentheses permit incrementing a variable with C-like syntax. done echo # C programmers can feel right at home in Bash. exit 0

Inside its test brackets, a while loop can call a function.

t=0 condition () { ((t++)) if [ $t -lt 5 ] then return 0 # true else return 1 # false fi } while condition # ^^^^^^^^^ # Function call -- four loop iterations. do echo "Still going: t = $t" done # Still going: t = 1 # Still going: t = 2 # Still going: t = 3 # Still going: t = 4

Similar to the if-test construct, a while loop can omit the test brackets. while condition do command(s) ... done

By coupling the power of the read command with a while loop, we get the handy while read construct, useful for reading and parsing files.

cat $filename | # Supply input from a file. while read line # As long as there is another line to read ... do ... done # =========== Snippet from "sd.sh" example script ========== # while read value # Read one data point at a time. do rt=$(echo "scale=$SC; $rt + $value" | bc) (( ct++ )) done am=$(echo "scale=$SC; $rt / $ct" | bc) echo $am; return $ct # This function "returns" TWO values! # Caution: This little trick will not work if $ct > 255! # To handle a larger number of data points, #+ simply comment out the "return $ct" above. } <"$datafile" # Feed in data file.

A while loop may have its stdin redirected to a file by a < at its end. A while loop may have its stdin supplied by a pipe.

until

This construct tests for a condition at the top of a loop, and keeps looping as long as that condition is false (opposite of while loop).

until [ condition-is-true ]
do
═command(s)...
done

Note that an until loop tests for the terminating condition at the top of the loop, differing from a similar construct in some programming languages.

As is the case with for loops, placing the do on the same line as the condition test requires a semicolon.

until [ condition-is-true ] ; do

Example 10-18. until loop

#!/bin/bash END_CONDITION=end until [ "$var1" = "$END_CONDITION" ] # Tests condition here, at top of loop. do echo "Input variable #1 " echo "($END_CONDITION to exit)" read var1 echo "variable #1 = $var1" echo done # ------------------------------------------- # # As with "for" and "while" loops, #+ an "until" loop permits C-like test constructs. LIMIT=10 var=0 until (( var > LIMIT )) do # ^^ ^ ^ ^^ No brackets, no $ prefixing variables. echo -n "$var " (( var++ )) done # 0 1 2 3 4 5 6 7 8 9 10 exit 0