More Filters: sed and awk

1
More Filters sed and awk

• Chapter 34

2
Using a Filter

• A filter is a Unix command that does some
  manipulation of the text of a file
• Two of the most powerful and popular Unix filters
  are the sed and awk commands

3
sed

• Here is a simple way to use the sed command to
  manipulate the contents of the apple.txt file
• miller.cs cat apple.txtcore jewel seed worm
  miller.cs cat apple.txt sed -e "s/o/WWW/"
  cWWWre jewel seed wWWWrm
• At the second shell prompt, the cat command
  displays the contents of the apple.txt file
• Contents are piped to the sed command
• The sed command changed the first occurrence of
  the letter "o" on each line to "WWW"
• The sed took as input the information it got
  through the pipe
• The sed command displayed its output to the
  screen

4
sed

• miller.cs cat apple.txt sed -e "s/e/J/gcorJ
  worm sJJd jJwJlmiller.cs
• Change all the occurrences of an e on each line
  with J.
• Note that every occurrence of e, even where there
  were more than one on a line, changed to J.
• This is because of the "g" on the end of the sed
  option value string.
• This "g" stands for global replace.
• The contents of the apple.txt file itself were
  not changed
• Only the display of its contents changed

5
awk record-based file processing

• awk views text files as records fields
• A file consists of records
• by default, each line of the file is a record
• awk operates on one record at a time
• A record consists of fields
• by default, fields are separated by spaces or
  tabs
• Data is processed according to a short script
• In the script, field number 1 is accessed with
  1, field 2 with 2, and so forth
• 0 refers to the whole record

6
Some Samples

• awk 'print 2,1' filename
• will print the second field, then the first
• All other fields are ignored
• awk 'print 1,2,sin(3/2)' filename
• will print the first and second fields, and then
  the sine of the third field divided by the second
• the second and third field had better be numbers
• awk has other built in math functions like sine

7
Some Samples

• What if you don't want to apply the program to
  each line of the file? Say, for example, that you
  only wanted to process lines that had the first
  field greater than the second. The following
  program will do that
• awk '1 gt 2 print 1,2,1-2' filename
• The part outside the curly braces is called the
  "pattern"
• the part inside is the "action"
• The comparison operators include , !, lt, gt,
  lt, gt
• If no pattern is given, then the action applies
  to all lines

8
awk

• miller.cs cat basket.txtLayer1 clothLayer2
  strawberriesLayer3 fishLayer4
  chocolateLayer5 punch cards

9
awk
miller.cs awk F 'print 1'
basket.txt Layer1Layer2Layer3Layer4Layer5

10
awk
miller.cs cat basket.txt awk 'print "HAS "
2'HAS clothHAS strawberriesHAS
fishHAS chocolateHAS punch cards
miller.cs