awk

1
awk

• Chapter 16 in 1st ed. of Textbook, chapter 12 in
  2nd ed.

2
awk options script filename

• Pattern scanning and processing language
• awk utility scans each input filename for lines
  that match any of a set of patterns specified
  in script. The script string must be enclosed in
  single quotes ( ') to protect it from the
  shell.
• For each pattern in prog there may be an
  associated action performed when a line of a
  filename matches the pattern.

3
Name Country phone number
Record 1
Mary USA
734-34-34-34 Sue W MEX
456-34-23-12 Mark Lee USA
345-67-34-53 Tomas S MEX
234-34-23-32 Yukoh Y JAP
234-34-23-32 Tim T USA
234-23-22-11
1 2 3
field
Note 0 represents the whole record (line)
4
cat phoneb MaryUSA734-34-34-34 Sue
WMEX456-34-23-12 Mark LeeUSA345-67-34-53 Tomas
SanchezMEX234-34-23-32 Yukoh
YaguchiJAP234-34-23-32 Tim TUSA234-23-22-11
awk -F 'print 1' phoneb Mary Sue
W Mark Lee Tomas Sanchez Yukoh Yaguchi Tim T
Procedures.
5
awk -F 'print 1 print 2'
phoneb Mary USA Sue W MEX Mark Lee USA Tomas
Sanchez MEX Yukoh Yaguchi JAP Tim T USA
n nth field in current record
6
awk -F 'print 1 print 2 print " "'
phoneb Mary USA Sue W MEX Mark Lee USA Tomas
Sanchez MEX Yukoh Yaguchi JAP Tim T USA
7
cat phoneb MaryUSA734-34-34-34 Sue
WMEX456-34-23-12 Mark LeeUSA345-67-34-53 Tomas
SanchezMEX234-34-23-32 Yukoh
YaguchiJAP234-34-23-32 Tim TUSA234-23-22-11
awk '/USA/' phoneb MaryUSA734-34-34-34 Mark
LeeUSA345-67-34-53 Tim TUSA234-23-22-11
pattern
8
cat phoneb MaryUSA734-34-34-34 Sue
WMEX456-34-23-12 Mark LeeUSA345-67-34-53 Tomas
SanchezMEX234-34-23-32 Yukoh
YaguchiJAP234-34-23-32 Tim TUSA234-23-22-11
awk -F '2 /MEX/ print 1, 3' phoneb Sue W
456-34-23-12 Tomas Sanchez 234-34-23-32
9
cat grades Tim1098 Sue498 Mary797
awk -F 'avg(234)/3 print 0 print
avg' grades Tim1098 9 Sue498 7 Mary797
7.66667
10
cat grades Tim1098 Sue498 Mary797
awk -F 'avg(234)/3 print 0,avg'
grades Tim1098 9 Sue498 7 Mary797
7.66667
11
NRnumber of current record 0
Entire input record
cat scr BEGIN print "Phone Book"
print NR, 0 END print "End of file" awk
-f scr phoneb Phone Book 1 MaryUSA734-34-34-34 2
Sue WMEX456-34-23-12 3 Mark LeeUSA345-67-34-5
3 4 Tomas SanchezMEX234-34-23-32 5 Yukoh
YaguchiJAP234-34-23-32 End of file
12
cat catalog PC IBM 400.00 PC HP
300.00 Macintosh 900.00 Macintosh 700.00 Sun
1,200.00 Sun 1,300.00 PC HP
700.00 1 2
13
cat sum BEGIN print "Computer
catalog" print "----------------------" to
taltotal2 print 1, "\t", 2 END print
"----------------------" print "TOTAL ",
"\t", total print "----------------------"
14
awk -F -f sum catalog Computer
catalog ------------------------------------- PC
IBM 400.00 PC HP
300.00 Macintosh 900.00 Macintosh
700.00 Sun 1200.00 Sun
1300.00 PC HP 700.00 ----------------
------------------ TOTAL
5500 -----------------------------------
15
Using the following as datafile

• cat datafile
• Henley DonDallas,TX333-3333
• Stevens RichardNew York,NY 444-4444
• Doe JohnSomewhere,SS222-2222

16
Printing fields, the comma

• awk 'BEGIN FS"" print 1 2' datafile
• Henley DonDallas,TX
• Stevens RichardNew York,NY
• Doe JohnSomewhere,SS
• awk 'BEGIN FS"" print 1, 2' datafile
• Henley Don Dallas,TX
• Stevens Richard New York,NY
• Doe John Somewhere,SS

17
Explicit or variables?

• Explicitly generates unwanted spaces around
  separator
• awk 'BEGIN FS"" print 1, "",2' datafile
  
• Henley Don Dallas,TX
• Stevens Richard New York,NY
• Doe John Somewhere,SS
• Better use the OFS
• awk 'BEGIN OFS""FS"" print 1,2'
  datafile
• Henley DonDallas,TX
• Stevens RichardNew York,NY
• Doe JohnSomewhere,SS

18
Formatted output

• awk 'BEGIN FS"" printf "s\t\ts\n",1,2'
  datafile
• Henley Don Dallas,TX
• Stevens Richard New York,NY
• Doe John Somewhere,SS

19
Dont multiply, dont clobber!

• awk 'BEGIN OFS""FS"" print 1,2'
  datafile datafile
• cat datafile
• Henley DonDallas,TX333-3333
• Stevens RichardNew York,NY 444-4444
• Doe JohnSomewhere,SS222-2222
• Henley DonDallas,TX
• Stevens RichardNew York,NY
• Doe JohnSomewhere,SS
• awk 'BEGIN OFS""FS"" print 1,2'
  datafile datafile
• cat datafile

20
With or without shell?Without shell wrappers
awk scripts run faster, take the following
equivalent examples

• cat w-sh
• !/bin/sh
• awk print 2

• cat n-sh
• !/usr/bin/awk f
• print 2

21
Patterns in specific fields

• Using shell variables with the v options
  simplifies their use inside the awk script
• !/usr/bin/bash
• echo -n "Name you are looking for? "
• read name
• nawk -F"" -v pat"name" 'toupper(1)
  toupper(pat) print 0' datafile

22
Patterns in specific fields

• Or as we had seen after the script and before the
  filename

!/usr/bin/bash echo -n "Name you are looking
for? " read name nawk -F"" 'toupper(1)
toupper(pat) print 0' pat"name" datafile