Introduction to Unix CS 21

1
Introduction to Unix CS 21

• Lecture 12

2
Lecture Overview

• A few more bash programming tricks
• The here document
• Trapping signals in bash
• cut and tr
• sed
• awk

3
The Here Document

• Redirects stdin to a specific set of text located
  inside the same file
• ltlt
• COMMAND ltlt MARKER
• Data
• MARKER

4
Example of Here
5
Why Is This Useful?

• Allows you to keep all relevant information in
  one file
• Example Database you want to search
• Dont need to clutter up your directory with
  unnecessary temporary files

6
Trapping Signals

• Catching a signal is also called trapping a
  signal
• You can tell bash programs what to do when they
  receive different signals
• Analogy When a postcard arrives, what do I do?

7
The trap Command

• Usage trap COMMAND Signals
• Example
• trap cat errorMsg 4 6
• In order to prevent you from running a program
  forever, signal number 9 cannot be trapped

8
Example Of trap
9
Two Helper Filters

• cut
• Break individual lines apart
• tr
• Change characters into different characters

10
The cut Command

• More precise control over individual lines
• Will cut out certain words from each individual
  line so they can be processed
• Usage cut FLAGS FILE

11
Flags

• -d
• Delimiter
• -f
• Field number
• Example
• cut d -f3 myFile

12
Example Of Cut
13
The tr Command

• Translate
• Change on a one to one basis characters from one
  thing to another
• Usage tr Set1 Set2
• Example tr abc ABC lt myFile

14
Example Of tr
15
Two More Powerful Tools

• sed
• Stream Editor
• awk
• Alfred Aho, Peter Weinberger, and Brian Kernighan

16
The sed Command/Language

• Filter
• Like grep, sort, or uniq, it takes input and
  performs some operation on it to filter the
  output
• Usage sed Address Command
• Address specifies where to make changes
• Command specifies what change to make
• Example
• sed 4d textFile

17
Address Specification

• Addresses could be line numbers or regular
  expressions
• No address each line
• One address only that line
• Two comma separated addresses All lines in
  between
• ! All other lines

18
Commands Available To sed

• a\
• Append text
• c\
• Replace text
• i\
• Insert text before
• d
• Delete lines
• s
• Make substitutions

19
Examples
20
More Examples
21
Substitution Example

• Same syntax as vi

22
When Would You Want To Use sed?

• sed works on streams, so it is perfect to be
  placed in the middle of a pipe in order to change
  the output from one format to another
• Example
• If a program always prints out 4 lines of junk
  for every good line, sed can be used to weed out
  the junk

23
Example
24
awk

• Answers the question
• What do I do if I want to search for a pattern
  and actually use it?
• Combination of grep and commands
• Searches for some pattern or condition and
  performs some command on it
• Complete programming language
• Looks a lot like C syntactically
• Variables are declared bash style

25
Pattern And Command

• awk in its most basic form simply executes a
  command on all lines that match (or adhere to) a
  certain pattern
• Usage awk Pattern Command FILE
• Just like sed, if there is no pattern, then every
  line will be matched

26
Example
27
Different Ways To Run Awk

• awk Pattern Command
• awk f awkFile inputFile
• Since awk itself can be a complex language, you
  can store all the commands in a file and run it
  with the f flag

28
Important awk Concepts

• Record
• Every line of an input file is a record
• The current record can be referenced with 0
• Field
• Every word in a record is called a field
• Each field is numbered and can be referred to
• 1 is the first record, 2 is the second, etc.

29
Special Predefined awk Variables

• RS
• The character that acts as a record separator
• Default is the end of a line
• FS
• The character that acts as a field separator
• Default is whitespace (space, tab, etc)
• Can be redefined with the F flag

30
Example
31
Other awk Variables

• NF number of fields in the current record
• NR Total number of records seen so far
• OFS Output field separator
• ORS Output record separator

32
BEGIN And END Blocks

• Two special patterns that can be matched
• BEGIN
• Commands are executed before any records are
  looked at
• END
• Commands are executed after all records are
  processed

33
Example
34
Awk Patterns

• /regular expression/ -gt same as egrep
• Relational expression
• gt, lt, gt, lt,
• Pattern pattern
• Pattern pattern
• Pattern1 ? Pattern2 pattern3
• If Pattern1 is True, then Pattern2, else pattern
  3
• (pattern)
• ! Pattern
• Pattern1, pattern2

35
Example Patterns
36
Awk Actions

• Enclosed in
• () Grouping
• Field reference
• -- Increment, decrement
• Exponentiation
• - ! Plus, minus, not
• / Multiplication, division, and modulus

37
Control Flow Statements

• Inside of commands, you can have control flow
• if
• while
• for

38
If Syntax

• if (condition)
• Statements
• else
• Statements

39
While Syntax

• while (Condition)
• Statements

40
For Syntax

• for (Declaration Condition Increment )
• Statements
• for ( j0 j lt 5 j)
• print hello world

41
When Would You Want To Use awk?

• Whenever you want to search for some pattern and
  perform some action
• Example I want to go through and calculate the
  average score on the Midterm

42
Example
43
Another Example

• Adding 12 points to everyones midterm score

44
Putting Them Together awk and sed
45
awk Versus bash arguments

• Always enclose everything to awk in single quotes
  so they dont get interpreted
• 1 to awk means something completely different
  than 1 to bash
• 1 in awk means first field
• 1 in bash means first command line argument

46
Next Time

• Looking at some of the string and mathematical
  awk functionality
• find
• Putting everything together
• The complete bash programming necessities
• Quiz 2 Next Tuesday