More Unix Utilities

1
CS465

• More Unix Utilities
• (find, diff/cmp, at/crontab, tr)

2
The find command

• The find command walks down a subtree of the file
  system looking for files which match certain
  criteria
• wildcards
• file type (e.g. file, directory, special, link)
• files size, owner, access time / modification
  time
• etc..

3
Format of the find command

• find start-dir criteria what-to-do

• start-dir defines the directory to start from.
• Each subdirectory of this directory will be
  searched
• criteria is what criteria to use to find the
  files
• Most common by filename
• what-to-do tells what to do with the files found.
  Can be any Unix command.
• Most common - print to a file
• - display to the screen (default)

4
Starting Directories

• Most often used starting directories are
• HOME users home directory
• . current directory
• / root directory (searches whole system)

5
Some find criterion options
-name pattern by filename (can use
wildcards) -perm oct by octal permission
settings -user/-group by owner or
group -size/-type by size or type -atime/-mtime
by last date accessed or modified -links by
number of links to the file ! by negated
criteria
6
find examples

• find -name projX -print
• Starting in my home directory, find any file
  whose name is projX and print their full
  pathnames
• find /etc -user root -ls
• Find all files under /etc owned by root and
  display in a long listing format
• find HOME ! -type l
• Starting in my home directory, display
  (default) all files that are NOT link-type files

7
find examples
ls -F pattern.txt pattern2.txt
regisdir/ ls regisdir pattern.txt
p.txt find . name pattern.txt ./pattern.txt
./regisdir/pattern.txt
find . type f ./pattern.txt ./pattern2.txt ./re
gisdir/p.txt ./regisdir/pattern.txt
find . type d . ./regisdir
8
find examples
ls -l -rw-rw-r-- 1 jones design 199191
May 7 2004 pattern.txt -rwxrwxr-x 1 jones
play 0 Feb 8 1352 pattern2.txt drwxrwxr
-x 2 smith design 2048 Feb 8 1342
regisdir ls -l regisdir -rwxrwxr-x 1 smith
design 120 Feb 1 1044 p.txt find .
user jones ./pattern.txt ./pattern2.txt
find . group design ./pattern.txt ./regisdir .
/regisdir/p.txt
9
find examples
ls -l -rw-rw-r-- 1 jones design 199191
May 7 2004 pattern.txt -rwxrwxr-x 1 jones
play 0 Feb 8 1352 pattern2.txt drwxrwxr
-x 2 smith design 2048 Feb 8 1342
regisdir ls -l regisdir -rwxrwxr-x 1 smith
design 120 Feb 1 1044 p.txt -rw-rw-r-- 1
smith design 120 Feb 1 1045 p2.txt
find . perm 775 ./pattern2.txt ./regisdir ./regis
dir/p.txt
find . perm 664 ./pattern.txt ./regisdir/p2.tx
t
10
Searching the Entire System

• When you search from root, redirect the error
  messages to a file
• Prevents seeing error message for every directory
  you don't have permission to view

11
Root search Example

• find / -name test -print
• find cannot read dir /lostfound Permission
  denied
• find cannot read dir /usr/lostfound Permission
  denied
• find cannot read dir /usr/dodd/progs Permission
  denied
• find cannot read dir /usr/jmsmith Permission
  denied
• /usr/kcjones/new/test
• /usr/lhlester/scripts/test
• find cannot read dir /usr/mkfox Permission
  denied
• find cannot read dir /usr/pjsmith Permission
  denied
• /usr/wsfrank/test

find / -name test -print 2gt errmsgs /usr/kcjones
/new/test /usr/lhlester/scripts/test /usr/wsfrank/
test
12
find Filename Wildcards

• You can use filename wildcards with the name
  option. But if you do, you need to ADD either
• a backslash in front of the wildcard -OR-
• single quotes around the filename containing the
  wildcard
• to prevent wildcard expansion by the shell (you
  want the find command to do the expansion of the
  wildcard, not the shell).

13
find Wildcarding Example
echo HOME /usr/home/jmsmith

• find HOME name 'p.c' print
• /usr/home/jmsmith/prog1.c
• /usr/home/jmsmith/hmwk/pass.c

14
More find examples

• find HOME -atime 30 -print
• Display all files within your directories that
  have not been accessed within the last 30 days
• NOTE 30 accessed exactly 30 days
  ago -30 accessed less than 30 days
  ago 30 accessed more than 30 days ago
• find /usr/jsmith -name '.ksh'
• - Display all Korn shell files in jsmiths
  directory structure.
• Remember, -print is the default action if none is
  specified

15
Combining find criteria

• Criteria can be combined using a and o and
  parentheses (backslashed)
• -a Logical AND
• -o Logical OR
• Example Display all files that have a .ksh
  extension and have been accessed within the last
  30 days.
• find HOME \( -name '.ksh' -a -atime \ -30 \)
  -print

16
find examples
touch soccer.doc ls -F apples
apples.c oranges oranges.c pattern.txt
pattern2.txt regisdir/ soccer.doc ls
regisdir Other pattern.txt
whole.doc find . name '.doc' o name
'.txt' ./regisdir/pattern.txt ./regisdir/whole.do
c ./pattern.txt ./pattern2.txt ./soccer.doc
17
Additional find actions
If you want to perform an action that is not in
the list of find options, you can use finds
-exec option to execute ANY Unix command
-exec command \ Notes (curly
braces) represent the files
found \ (backslash semicolon) ends the
exec command
18
-exec Examples

• Find all file ending in .c and move them to
  the cprogs subdirectory.
• find HOME -name '.c' \ -exec mv
  HOME/cprogs \
• Find all files within your directories that have
  not been accessed within the last 30 days and
  delete them
• find HOME -atime 30 exec rm \

19
find examples
ls -F pattern.txt pattern2.txt regisdir/
ls regisdir find . name 'pattern' exec cp
regisdir \ ls regisdir pattern.txt
pattern2.txt
find regisdir name '2.txt' exec rm \
ls regisdir pattern.txt
20
find Exercise
Starting in your home directory, search for
all files that have been accessed within the
last 3 days and print their full pathnames.
find HOME -atime -3 -print Expanding the
previous exercise, search for all files that have
been accessed within the last 3 days and that
have a .ksh extension and copy them to
subdirectory current, under HOME. find
HOME \( -name '.ksh' -a -atime -3 \) exec cp
HOME/current \
21
The diff command

• diff compares two files
• Format
• diff options file1 file2
• Options
• -b ignore repeating blanks
• -i ignore case
• -w ignore ALL spaces and tabs
• Output indicates which lines need be added (a),
  deleted (d) or changed (c).
• Lines in file1 are identified with a (lt) symbol
  and lines in file2 with a (gt) symbol.

22
diff command output format

• The output from diff is an ed editor script that
  would convert the first file to the second file
  if run with the first file as input.
• Additions
• firstStart a secondStart, secondStop
• gt lines from the second file to add to the
  first file
• Deletions
• firstStart, firstStop d lineCount
• lt lines from the first file to delete
• Changes
• firstStart, firstStop c secondStart, secondStop
• lt lines in the first file to be replaced
• gt lines in the second file to be used for the
  replacement

23
diff Example

• cat file1
• hello
• hello
• cat file2
• hello

diff file1 file2 2d1 lt hello
diff file2 file1 1a2 gt hello
24
Using diff output with ed

• cat file1
• hello
• hello

diff file1 file2 ed file1 cat file1 hello
diff file1 file2
25
diff examples
cat team1 cat team2 Mike Mike John Scott K
athy Kathy Martin Bob diff team1
team2 2c2 lt John --- gt Scott 4c4 lt Martin --- gt
Bob
cat team3 cat team4 Scott Mike
Scott diff team3 team4 0a1 gt Mike diff
team4 team3 1d0 lt Mike
26
Using diff with directories
Shows which files are unique to each directory

• ls sub
• sub1
• file1 prog.c test
• sub2
• file1 memo prog.c

diff sub1 sub2 Only in sub2 memo Only in sub1
test
27
The cmp command

• cmp also compares two files
• Instead of line by line, like diff, compares
  files character by character until it finds a
  character mismatch.
• Outputs the character number and line number
  where the FIRST mismatch was discovered.
• Example
• cmp filev1 filev2
• filev1 filev2 differ char 60, line 7

28
cmp command

• Format
• cmp options file1 file2
• Option
• -s work silently print nothing, but return
  exit codes, as follows
• 0 Files are identical
• 1 Files are different
• 2 Files are inaccessible

29
cmp examples
cat team1 Mike John cat team2 Stephen Scott
cmp team1 team2 team1 team2 differ char 1, line
1 cp team1 team3 cmp team1 team3
cmp s team1 team3 echo "identical" identical
30
cmp command option

• Can also create a listing of ALL character
  differences in the file, using -l option.
• Column 1 character number (byte number) within
  the file
• Column 2 ASCII code of character within first
  file
• Column 3 ASCII code of character within second
  file

31
cmp Example

• ls file1
• Hello
• Today is Monday
• ls file2
• Hello
• Today is Friday
• ls file3
• Hello

cmp file1 file2 16 115 106 17 157 162
18 156 151
cmp file1 file3 cmp EOF on f3
32
Process Scheduling

• Two Unix utilities let you schedule commands to
  run at specific times.
• crontab - lets you schedule commands that need to
  run on a periodic basis.
• at - lets you schedule a command to run only
  once.

33
The crontab file

• The crontab command uses a crontab file, which
  contains a list of commands and when to run them.
  
• Commands in your crontab file can be Unix
  utilities, shell scripts, or executable programs.
  
• If the command generates output, the output
  should be re-directed to a file or terminal
• Each user may register ONE crontab file (any
  name).
• Once the file is registered, the system runs the
  commands at the scheduled times.

34
Crontab file specs

• A crontab file contains 6 columns
• Column 1 contains minute (0-59)
• Column 2 contains hour (0-23, 24-hour clock)
• Column 3 contains day of month (1-31)
• Column 4 contains month (1-12)
• Column 5 contains weekday (1-7, where 1
  Monday)
• Column 6 contains command to run
•  
• Columns 2 to 5 may also contain a range of hours,
  days, or months, or the asterisk symbol. The
  asterisk says to match ALL hours, days, or
  months.

35
Sample crontab file
cat crontab.def 0 3 1
HOME/monthly_cleanup.ksh 45 11 1-5 echo
Lunchtime!! gt /dev/tty5

• First line will run a the script file called
  monthly_cleanup.ksh at 0300 (3 am) on the 1st of
  every month.
• Second line will display "Lunchtime!" to
  terminal 5, Monday through Friday, at 1145 (1145
  am)

36
Invoking crontab

• After you create a crontab file, you must
  register it using the crontab command, so the
  system will begin running your commands.
• Once your file is registered, you can list its
  contents using the crontab -l command.
• You can stop the system from running the commands
  by un-registering the crontab file using the
  crontab -r

37
crontab Example 1

• Remove .tmp files from home directory every
  morning at 6 am
• cat crontab.def
• 0 6 rm HOME/.tmp
• crontab crontab.def

38
crontab Example 2

• cat cronfile
• 0 3 1 HOME/monthly_cleanup.ksh
• 45 11 1-5 mail USER lt lunchmsg

crontab cronfile
crontab -l 0 3 1 HOME/monthly_cleanup.
ksh 45 11 1-5 mail USER lt lunchmsg
crontab -r
crontab -l
39
crontab Example 3

• Root can view ANY users registered crontab file

crontab l jsmith 0 3 1 HOME/removeTmps
crontab l mjones 30 11 1-5 mail USER lt
lunchmsg
40
cron Output

• cron usually mails output and errors to the cron
  user. Other methods
• Method 1 - Discard output
• 10 cronscript gt /dev/null 2gt 1
• Method 2 - Mail output to someone else
• 10 cronscript mail jsmith

41
The at utility

• If you only want to run a command once (instead
  of on a regular schedule), but at a specific
  time, you can use the at command.
• Syntax
• at time lt scriptfile
• Time field can contain
• specific date and time
• date/time relative to the current time
• month names, day names, and keywords like now,
  tomorrow, next

42
at options

• Options when running at
• -c, -k and -s tell shell to run the command under
  a specific shell (C, Korn, or Bourne)
• NOTE Not available on acadunix -- all
  automatically run under Bourne shell
• -m tells the system to send you mail when the
  command has finished running
• Options after a job has been scheduled 
• -l displays scheduled jobs
• -r job de-schedules the specifed job

43
at Examples

• Run script1 at 200 am next Saturday
• at 0200 sat lt script1
• Run script2 an hour from now
• at now 1 hour lt script2
• Display scheduled jobs
• at l
• 990259200.a Sat May 19 020000 2001
• 990231999.a Fri May 18 182639 2001

44
More at Examples

• Run script3 at 500 pm tomorrow
• at 5pm tomorrow lt script3
• Run script4 at 1000 am on October 1st
• at 1000 Oct 1 lt script4
• Cancel job scheduled for Saturday
• at l
• 990259200.a Sat May 19 020000 2003
• 990231999.a Fri May 18 182639 2003
• at -r 990259200.a
• at l
• 990231999.a Fri May 18 182639 2003

45
Re-scheduling at

• You can make a script re-schedule itself by
  making the last command in the script be another
  at command.
• Example
• cat deltmp.ksh
• ! /bin/ksh
• rm HOME/.tmp
• at now 1 day lt rmtmp.ksh
• at 6am tomorrow lt rmtmp.ksh

46
Longer Re-scheduling Script

• Check to see if a specific user is logged on, and
  pipe the output to the checkfile file (if the
  user is NOT logged on, no file will be created).
  
• Check if the checkfile file exists.
• If the file does NOT exist, the script will
  re-schedule itself to run again in 10 minutes.
• If the file does exist, the script will display a
  message on terminal 5 (assumed to be your
  terminal) and delete checkfile.

47
Script Contents

• cat check
• !/bin/ksh
• 1 userid of person to check for
• who fgrep 1 gt checkfile
• if -s checkfile
• then
• echo 1 is logged on now gt mailmsg
• mail small000 lt mailmsg
• rm checkfile mailmsg
• else
• at now 10 minutes lt HOME/check 1
• fi
• exit 0

48
Permissions for at and crontab

• The system administrator can allow or deny both
  at and crontab use for each user on the system.
• The files that control access are located in the
  directory /usr/lib/cron
• crontab is controlled by files cron.allow and
  cron.deny
• at is controlled by files at.allow and at.deny.
• If no allow files exist, then only the users
  listed in the deny files are denied access.

49
tr utility

• Translates characters in the standard input, via
  substitution or deletion of characters
• Format
• tr options string1 string2
• Meaning
• Match characters in string1 and substitute
  characters from string2.

50
tr Example 1

• Using tr for case conversion
• cat names
• Joe Smith
• Mary Jones
• tr 'A-Z' 'a-z' lt names
• joe smith
• mary jones

51
tr Example 2

• Using tr to convert spaces to newlines
• cat names
• Joe Smith
• Mary Jones
• tr ' ' '\012' lt names
• Joe
• Smith
• Mary
• Jones

52
tr d option

• The d option deletes all characters listed in
  string1
• Using tr to delete spaces and lowercase vowels
• cat names
• Joe Smith
• Mary Jones
• tr d 'aeiou ' lt names gt names.tr
• cat names.tr
• JSmth
• MryJns

53
tr s option

• The s option strips out repeated characters
  listed in string1
• Using tr to strip out blank lines
• cat names
• Joe
• Mary
• tr s '\012' lt names
• Joe
• Mary