- jot, seq
These utilities emit a sequence of integers, with a
user-selected increment.
The normal separator character between each integer is a
newline, but this can be changed with the -s
option.
bash$ seq 5
1
2
3
4
5
bash$ seq -s : 5
1:2:3:4:5
|
Both jot and seq
come in handy in a for
loop.
Example 12-42. Using seq to generate loop arguments
#!/bin/bash
# Using "seq"
echo
for a in `seq 80` # or for a in $( seq 80 )
# Same as for a in 1 2 3 4 5 ... 80 (saves much typing!).
# May also use 'jot' (if present on system).
do
echo -n "$a "
done # 1 2 3 4 5 ... 80
# Example of using the output of a command to generate
# the [list] in a "for" loop.
echo; echo
COUNT=80 # Yes, 'seq' may also take a replaceable parameter.
for a in `seq $COUNT` # or for a in $( seq $COUNT )
do
echo -n "$a "
done # 1 2 3 4 5 ... 80
echo; echo
BEGIN=75
END=80
for a in `seq $BEGIN $END`
# Giving "seq" two arguments starts the count at the first one,
#+ and continues until it reaches the second.
do
echo -n "$a "
done # 75 76 77 78 79 80
echo; echo
BEGIN=45
INTERVAL=5
END=80
for a in `seq $BEGIN $INTERVAL $END`
# Giving "seq" three arguments starts the count at the first one,
#+ uses the second for a step interval,
#+ and continues until it reaches the third.
do
echo -n "$a "
done # 45 50 55 60 65 70 75 80
echo; echo
exit 0 |
- getopt
The getopt command
parses command-line options preceded by a dash. This external command
corresponds to the getopts
Bash builtin. The getopt permits
handling long options by use of the -l
flag, and it also allows parameter reshuffling.
Example 12-43. Using getopt to parse command-line
options
#!/bin/bash
# Using getopt.
# Try the following when invoking this script:
# sh ex33a.sh -a
# sh ex33a.sh -abc
# sh ex33a.sh -a -b -c
# sh ex33a.sh -d
# sh ex33a.sh -dXYZ
# sh ex33a.sh -d XYZ
# sh ex33a.sh -abcd
# sh ex33a.sh -abcdZ
# sh ex33a.sh -z
# sh ex33a.sh a
# Explain the results of each of the above.
E_OPTERR=65
if [ "$#" -eq 0 ]
then # Script needs at least one command-line argument.
echo "Usage $0 -[options a,b,c]"
exit $E_OPTERR
fi
set -- `getopt "abcd:" "$@"`
# Sets positional parameters to command-line arguments.
# What happens if you use "$*" instead of "$@"?
while [ ! -z "$1" ]
do
case "$1" in
-a) echo "Option \"a\"";;
-b) echo "Option \"b\"";;
-c) echo "Option \"c\"";;
-d) echo "Option \"d\" $2";;
*) break;;
esac
shift
done
# It is usually better to use the 'getopts' builtin in a script,
#+ rather than 'getopt'.
# See "ex33.sh".
exit 0 |
- run-parts
The run-parts command
executes all the scripts in a target directory, sequentially
in ASCII-sorted filename order. Of course, the scripts
need to have execute permission.
The cron daemon invokes
run-parts to run the scripts in
the /etc/cron.*
directories.
- yes
In its default behavior the yes
command feeds a continuous string of the character
y followed
by a line feed to stdout. A
control-c
terminates the run. A different output string
may be specified, as in yes different
string, which would continually output
different string to
stdout. One might well ask the purpose
of this. From the command line or in a script, the output
of yes can be redirected or piped into a
program expecting user input. In effect, this becomes a sort
of poor man's version of expect.
yes | fsck /dev/hda1 runs
fsck non-interactively (careful!).
yes | rm -r dirname has same effect as
rm -rf dirname (careful!).
 | Caution advised when piping yes
to a potentially dangerous system command, such
as fsck or fdisk. It may have unintended
side-effects. |
- banner
Prints arguments as a large vertical banner to
stdout, using an ASCII character
(default '#'). This may be redirected to a printer for
hardcopy.
- printenv
Show all the environmental
variables set for a particular user.
bash$ printenv | grep HOME
HOME=/home/bozo
|
- lp
The lp and lpr
commands send file(s) to the print queue, to be printed as
hard copy.
These commands trace the origin of their names to the
line printers of another era.
bash$ lp file1.txt
or bash lp
<file1.txt
It is often useful to pipe the formatted output from
pr to lp.
bash$ pr -options file1.txt | lp
Formatting packages, such as groff and
Ghostscript may send their output
directly to lp.
bash$ groff -Tascii file.tr | lp
bash$ gs -options | lp file.ps
Related commands are lpq, for viewing
the print queue, and lprm, for removing
jobs from the print queue.
- tee
[Unix borrows an idea here from the plumbing trade.]
This is a redirection operator, but with a difference. Like the
plumber's "tee," it permits "siponing
off" to a file the output of a command
or commands within a pipe, but without affecting the result. This is
useful for printing an ongoing process to a file or paper, perhaps to
keep track of it for debugging purposes.
tee
|------> to file
|
===============|===============
command--->----|-operator-->---> result of command(s)
===============================
|
cat listfile* | sort | tee check.file | uniq > result.file |
(The file check.file contains
the concatenated sorted "listfiles",
before the duplicate lines are removed by uniq.)- mkfifo
This obscure command
creates a named pipe, a temporary
first-in-first-out buffer for
transferring data between processes.
Typically, one process writes to the FIFO, and the other
reads from it. See Example A-16.
- pathchk
This command checks the validity of a filename. If the
filename exceeds the maximum allowable length (255
characters) or one or more of the directories in
its path is not searchable, then an error message
results.
Unfortunately, pathchk does
not return a recognizable error code, and it is therefore
pretty much useless in a script. Consider instead the
file test operators.
- dd
This is the somewhat obscure and much feared "data
duplicator" command. Originally a utility
for exchanging data on magnetic tapes between Unix
minicomputers and IBM mainframes, this command still
has its uses. The dd command simply
copies a file (or stdin/stdout), but
with conversions. Possible conversions are ASCII/EBCDIC,
upper/lower case, swapping of byte pairs between input
and output, and skipping and/or truncating the head or
tail of the input file. A dd --help
lists the conversion and other options that this powerful
utility takes.
# Exercising 'dd'.
n=3
p=5
input_file=project.txt
output_file=log.txt
dd if=$input_file of=$output_file bs=1 skip=$((n-1)) count=$((p-n+1)) 2> /dev/null
# Extracts characters n to p from file $input_file.
echo -n "hello world" | dd cbs=1 conv=unblock 2> /dev/null
# Echoes "hello world" vertically.
# Thanks, S.C. |
To demonstrate just how versatile dd is,
let's use it to capture keystrokes.
Example 12-44. Capturing Keystrokes
#!/bin/bash
# Capture keystrokes without needing to press ENTER.
keypresses=4 # Number of keypresses to capture.
old_tty_setting=$(stty -g) # Save old terminal settings.
echo "Press $keypresses keys."
stty -icanon -echo # Disable canonical mode.
# Disable local echo.
keys=$(dd bs=1 count=$keypresses 2> /dev/null)
# 'dd' uses stdin, if "if" not specified.
stty "$old_tty_setting" # Restore old terminal settings.
echo "You pressed the \"$keys\" keys."
# Thanks, S.C. for showing the way.
exit 0 |
The dd command can do random access on a
data stream.
echo -n . | dd bs=1 seek=4 of=file conv=notrunc
# The "conv=notrunc" option means that the output file will not be truncated.
# Thanks, S.C. |
The dd command can copy raw data
and disk images to and from devices, such as floppies and
tape drives (Example A-6). A common use is
creating boot floppies.
dd if=kernel-image of=/dev/fd0H1440
Similarly, dd can copy the entire
contents of a floppy, even one formatted with a
"foreign" OS, to the hard drive as an
image file.
dd if=/dev/fd0 of=/home/bozo/projects/floppy.img
Other applications of dd include
initializing temporary swap files (Example 29-2)
and ramdisks (Example 29-3). It can even do a
low-level copy of an entire hard drive partition, although
this is not necessarily recommended.
People (with presumably nothing better to do with
their time) are constantly thinking of interesting
applications of dd.
Example 12-45. Securely deleting a file
#!/bin/bash
# blotout.sh: Erase all traces of a file.
# This script overwrites a target file alternately
#+ with random bytes, then zeros before finally deleting it.
# After that, even examining the raw disk sectors
#+ will not reveal the original file data.
PASSES=7 # Number of file-shredding passes.
BLOCKSIZE=1 # I/O with /dev/urandom requires unit block size,
#+ otherwise you get weird results.
E_BADARGS=70
E_NOT_FOUND=71
E_CHANGED_MIND=72
if [ -z "$1" ] # No filename specified.
then
echo "Usage: `basename $0` filename"
exit $E_BADARGS
fi
file=$1
if [ ! -e "$file" ]
then
echo "File \"$file\" not found."
exit $E_NOT_FOUND
fi
echo; echo -n "Are you absolutely sure you want to blot out \"$file\" (y/n)? "
read answer
case "$answer" in
[nN]) echo "Changed your mind, huh?"
exit $E_CHANGED_MIND
;;
*) echo "Blotting out file \"$file\".";;
esac
flength=$(ls -l "$file" | awk '{print $5}') # Field 5 is file length.
pass_count=1
echo
while [ "$pass_count" -le "$PASSES" ]
do
echo "Pass #$pass_count"
sync # Flush buffers.
dd if=/dev/urandom of=$file bs=$BLOCKSIZE count=$flength
# Fill with random bytes.
sync # Flush buffers again.
dd if=/dev/zero of=$file bs=$BLOCKSIZE count=$flength
# Fill with zeros.
sync # Flush buffers yet again.
let "pass_count += 1"
echo
done
rm -f $file # Finally, delete scrambled and shredded file.
sync # Flush buffers a final time.
echo "File \"$file\" blotted out and deleted."; echo
# This is a fairly secure, if inefficient and slow method
#+ of thoroughly "shredding" a file. The "shred" command,
#+ part of the GNU "fileutils" package, does the same thing,
#+ but more efficiently.
# The file cannot not be "undeleted" or retrieved by normal methods.
# However...
#+ this simple method will likely *not* withstand forensic analysis.
# Tom Vier's "wipe" file-deletion package does a much more thorough job
#+ of file shredding than this simple script.
# http://www.ibiblio.org/pub/Linux/utils/file/wipe-2.0.0.tar.bz2
# For an in-depth analysis on the topic of file deletion and security,
#+ see Peter Gutmann's paper,
#+ "Secure Deletion of Data From Magnetic and Solid-State Memory".
# http://www.cs.auckland.ac.nz/~pgut001/pubs/secure_del.html
exit 0 |
- od
The od, or octal
dump filter converts input (or files) to octal
(base-8) or other bases. This is useful for viewing or
processing binary data files or otherwise unreadable system
device files, such as /dev/urandom,
and as a filter for binary data. See Example 9-27 and Example 12-13.
- hexdump
Performs a hexadecimal, octal, decimal, or ASCII
dump of a binary file. This command is the rough equivalent
of od, above, but not nearly as
useful.
- objdump
Displays information about an object file or binary
executable in either hexadecimal form or as a disassembled
listing (with the -d option).
bash$ objdump -d /bin/ls
/bin/ls: file format elf32-i386
Disassembly of section .init:
080490bc <.init>:
80490bc: 55 push %ebp
80490bd: 89 e5 mov %esp,%ebp
. . .
|
- mcookie
This command generates a "magic cookie", a
128-bit (32-character) pseudorandom hexadecimal number,
normally used as an authorization "signature"
by the X server. This also available for use in a script as
a "quick 'n dirty" random number.
Of course, a script could use md5 for the same purpose.
# Generate md5 checksum on the script itself.
random001=`md5sum $0 | awk '{print $1}'`
# Uses 'awk' to strip off the filename. |
The mcookie command gives yet another way
to generate a "unique" filename.
Example 12-46. Filename generator
#!/bin/bash
# tempfile-name.sh: temp filename generator
BASE_STR=`mcookie` # 32-character magic cookie.
POS=11 # Arbitrary position in magic cookie string.
LEN=5 # Get $LEN consecutive characters.
prefix=temp # This is, after all, a "temp" file.
# For more "uniqueness," generate the filename prefix
#+ using the same method as the suffix, below.
suffix=${BASE_STR:POS:LEN}
# Extract a 5-character string, starting at position 11.
temp_filename=$prefix.$suffix
# Construct the filename.
echo "Temp filename = "$temp_filename""
# sh tempfile-name.sh
# Temp filename = temp.e19ea
exit 0 |
- units
This utility converts between different units of measure.
While normally invoked in interactive mode,
units may find use in a script.
Example 12-47. Converting meters to miles
#!/bin/bash
# unit-conversion.sh
convert_units () # Takes as arguments the units to convert.
{
cf=$(units "$1" "$2" | sed --silent -e '1p' | awk '{print $2}')
# Strip off everything except the actual conversion factor.
echo "$cf"
}
Unit1=miles
Unit2=meters
cfactor=`convert_units $Unit1 $Unit2`
quantity=3.73
result=$(echo $quantity*$cfactor | bc)
echo "There are $result $Unit2 in $quantity $Unit1."
# What happens if you pass incompatible units,
#+ such as "acres" and "miles" to the function?
exit 0 |
- m4
A hidden treasure, m4 is a
powerful macro processing filter,
virtually a complete language. Although
originally written as a pre-processor for
RatFor, m4
turned out to be useful as a stand-alone utility. In
fact, m4 combines some of the
functionality of eval,
tr, and awk, in addition to its extensive
macro expansion facilities.
The April, 2002 issue of Linux
Journal has a very nice article on
m4 and its uses.
Example 12-48. Using m4
#!/bin/bash
# m4.sh: Using the m4 macro processor
# Strings
string=abcdA01
echo "len($string)" | m4 # 7
echo "substr($string,4)" | m4 # A01
echo "regexp($string,[0-1][0-1],\&Z)" | m4 # 01Z
# Arithmetic
echo "incr(22)" | m4 # 23
echo "eval(99 / 3)" | m4 # 33
exit 0 |
- doexec
The doexec command enables passing
an arbitrary list of arguments to a binary
executable. In particular, passing
argv[0] (which corresponds to $0 in a script) lets the
executable be invoked by various names, and it can then
carry out different sets of actions, according to the name
by which it was called. What this amounts to is roundabout
way of passing options to an executable.
For example, the /usr/local/bin directory might
contain a binary called "aaa". Invoking
doexec /usr/local/bin/aaa list
would list all those files
in the current working directory beginning with an
"a", while invoking (the same executable
with) doexec /usr/local/bin/aaa delete
would delete those files.
 | The various behaviors of the executable
must be defined within the code of the executable itself,
analogous to something like the following in a shell script:
case `basename $0` in
"name1" ) do_something;;
"name2" ) do_something_else;;
"name3" ) do_yet_another_thing;;
* ) bail_out;;
esac |
|
- dialog
The dialog family of tools
provide a method of calling interactive
"dialog" boxes from a script. The more
elaborate variations of dialog --
gdialog, Xdialog,
and kdialog -- actually invoke X-Windows
widgets. See Example 34-15.
jamesthornton.com>