Midterm Solutions

1
Midterm Solutions
2
Stats

• Average 62.7
• Median 64.5
• Std dev 16.1
• Range 18-98
• gt80 10 A
• 70-80 19 B/A-
• 60-70 22 B/B
• 50-60 10 B-/B
• 40-50 12 C/B-
• lt40 6

3
HW1

• Average 88
• Median 64.5
• Std dev 11.2
• Range 53-100

4
Question 1

• a, c, e
• a, c, d, e

5
Question 2

• 1 file1
• 1 file2 2 gt 3 file1
• 1 file1 2 gt file2
• 1 echo file1
• 1 file1
• 1 file1 2 file2
• 1 file1 2 file2

6
Question 3

• xxxx--xx-N-.B...-N-.
  B...

7
Question 3 (cont.)

• zusage od file ...Total 11 records
• odod illegal optionusage odDone!

8
Question 4

• find ! -type d -perm -001 -mtime 10 \ \(
  -name .xml -o -name .html \)

9
Question 5

• cut -d, -f4 grep -c A
• egrep ,MSCS,(DF),(DF) \ cut -d, -f1
• grep ,\(,\),\1 cut -d,\ -f2 cut -c6-
• cut -d -f1 sort uniq \ tr a-z A-Z
• grep -v ,,,,MSIS, \ grep -c T
  B

10
Question 6

• lines01
• for (i1 iltNF i) wordsi1
• END
• for (l in lines) nl
• for (w in words) nw
• printf("There are d words and d lines.\n",
  nw, nl)

11
Question 7

• awk -F, print NF
• tr -cd ,\n wc -c

12
Question 8

• !/bin/sh
• if -lt 2 then
• exit 1
• fi
• DELIM"1"
• shift
• fileprinted0
• for file in "_at_" do
• samecol1
• nfield-1
• while read line do
• numecho line \ awk
  -F"DELIM" 'print NF'

13

• if nfield -eq -1 then
• nfieldnum
• elif num -ne nfield then
• samecol0
• break
• fi
• done lt file
• if samecol -eq 1 then
• echo file
• fileprinted1
• fi
• done
• if fileprinted -eq 1 then
• exit 0
• else
• exit 1
• fi

14
Lecture 10

• Shell (cont)
• UNIX Development Tools

15
Korn Shell / bash Features
16
Command Substitution

• Better syntax with (command)
• Allows nesting
• x(cat (generate_file_list))
• Backward compatible with notation

17
Expressions

• Expressions are built-in with the operator
• if var ""
• Gets around parsing quirks of /bin/test, allows
  checking strings against patterns
• Operations
• string pattern
• string ! pattern
• string1 lt string2
• file1 nt file2
• file1 ot file2
• file1 ef file2
• ,

18
Patterns

• Can be used to do string matching
• if foo a
• if foo abc
• Note patterns are like a subset of regular
  expressions, but different syntax

19
Additonal Parameter Expansion

• param Length of param
• parampattern Left strip min pattern
• parampattern Left strip max pattern
• parampattern Right strip min pattern
• parampattern Right strip max pattern
• param-value Default value if param not set

20
Variables

• Variables can be arrays
• foo3test
• echo foo3
• Indexed by number
• arr is length of the array
• Multiple array elements can be set at once
• set A foo a b c d
• echo foo1
• Set command can also be used for positional
  params set a b c d print 2

21
Functions

• Alternative function syntax
• function name commands
• Allows for local variables
• 0 is set to the name of the function

22
Additional Features

• Built-in arithmetic Using ((expression ))
• e.g., print (( 1 1 8 / x ))
• Tilde file expansion
• HOME
• user home directory of user
• PWD
• - OLDPWD

23
KornShell 93
24
Variable Attributes

• By default attributes hold strings of unlimited
  length
• Attributes can be set with typeset
• readonly (-r) cannot be changed
• export (-x) value will be exported to env
• upper (-u) letters will be converted to upper
  case
• lower (-l) letters will be converted to lower
  case
• ljust (-L width) left justify to given width
• rjust (-R width) right justify to given width
• zfill (-Z width) justify, fill with leading
  zeros
• integer (-I base) value stored as integer
• float (-E prec) value stored as C double
• nameref (-n) a name reference

25
Name References

• A name reference is a type of variable that
  references another variable.
• nameref is an alias for typeset -n
• Example
• user1"jeff"user2"adam"typeset n
  name"user1"print namejeff

26
New Parameter Expansion

• param/pattern/str Replace first pattern with
  str
• param//pattern/str Replace all patterns with
  str
• paramoffsetlen Substring with offset

27
Patterns Extended
Regular Expressions
Patterns

• Additional pattern types so that shell patterns
  are equally expressive as regular expressions
• Used for
• file expansion
• case statements
• parameter expansion

28
ANSI C Quoting

• '' Uses C escape sequences
• '\t' 'Hello\nthere'
• printf added that supports C like printing
• printf "You have d apples" x
• Extensions
• b ANSI escape sequences
• q Quote argument for reinput
• \E Escape character (033)
• P convert ERE to shell pattern
• H convert using HTML conventions
• T date conversions using date formats

29
Associative Arrays

• Arrays can be indexed by string
• Declared with typeset A
• Set name"foo""bar"
• Reference name"foo"
• Subscripts !name_at_

30
Software Development Tools
31
Types of Development Tools

• Compilation and building make
• Managing files RCS, SCCS, CVS
• Editors vi, emacs
• Archiving tar, cpio, pax, RPM
• Configuration autoconf
• Debugging gdb, dbx, prof, strace, purify
• Programming tools yacc, lex, lint, indent

32
Make

• make A program for building and maintaining
  computer programs
• developed at Bell Labs around 1978 by S. Feldman
  (now at IBM)
• Instructions stored in a special format file
  called a makefile.

33
Make Features

• Contains the build instructions for a project
• Automatically updates files based on a series of
  dependency rules
• Supports multiple configurations for a project
• Only re-compiles necessary files after a change
  (conditional compilation)
• Major time-saver for large projects
• Uses timestamps of the intermediate files
• Typical usage executable is updated from object
  files which are in turn compiled from source files

34
Dependency Graph
myprog
link
foo.o
bar.o
baz.o
compile
foo.c
bar.c
baz.c
baz.y
original
generated
35
Makefile Format

• Rule Syntax
• lttargetgt ltdependency listgt
• ltcommandgt
• The lttargetgt is a list of files that the command
  will generate
• The ltdependency listgt may be files and/or other
  targets, and will be used to create the target
• It must be a tab before ltcommandgt, or it wont
  work
• The first rule is the default lttargetgt for make

36
Examples of Invoking Make

• make -f makefile
• make target
• make
• looks for file makefile or Makefile in current
  directory, picks first target listed in the
  makefile

37
Make Sequence of Execution

• Make executes all commands associated with target
  in makefile if one of these conditions is
  satisfied
• file target does not exist
• file target exists but one of the source files in
  the dependency list has been modified more
  recently than target

38
Example Makefile

• Example Makefile
• CCg
• CFLAGS-g Wall -DDEBUG
• foobar foo.o bar.o
• (CC) (CFLAGS) o foobar foo.o bar.o
• foo.o foo.cpp foo.h
• (CC) (CFLAGS) c foo.cpp
• bar.o bar.cpp bar.h
• (CC) (CFLAGS) c bar.cpp
• clean
• rm foo.o bar.o foobar

39
Make Power Features

• Many built-in rules
• e.g. C compilation
• Fake targets
• Targets that are not actually files
• Can do just about anything, not just compile
• Like the clean target
• Forcing re-compiles
• touch the required files
• touch the Makefile to rebuild everything

40
Version Control

• Provide the ability to store/access and protect
  all of the versions of source code files
• Provides the following benefits
• If program has multiple versions, it keeps track
  only of differences between multiple versions.
• Multi-user support. Allows only one person at
  the time to do the editing.
• Provides a way to look at the history of program
  development.

41
Version Control Systems

• SCCS UNIX Source Code Control System
• Rochkind, Bell Labs, 1972.
• RCS Revision Control System
• Tichy, Purdue, 1980s.
• CVS Concurrent Versions System
• Grune, 1986, Berliner, 1989.

42
RCS Basic Operations

• Set up a directory for RCS
• mkdir RCS
• Check in a new file into the repository
• ci filename
• Check out a file from the repository for reading
• co filename
• Check out a file from the repository for writing
• co l filename
• Acquires lock
• Compare local copy of file to version in
  repository
• rcsdiff rltIDgt filename

43
RCS Keywords

• Keywords in source files are expanded to contain
  RCS info at checkout
• keyword ? keyword value
• Use ident to extract RCS keyword info
• Author Username of person checked in the
  revision
• Date Date and time of check-in
• Id A title that includes the RCS filename,
  revision number, date, author, state, and (if
  locked) the person who locked the file
• Revision The revision number assigned

44
SCCS Equivalents

• Function RCS SCCS
• Setup mkdir RCS mkdir SCCS
• Check in new foo.c ci foo.c sccs create foo.c
• Check in update to foo.c ci foo.c sccs delta
  foo.c
• Get read-only foo.c co foo.c sccs get foo.c
• Get writeable foo.c co -l foo.c sccs edit foo.c
• Version history of foo.c rlog foo.c sccs print
  foo.c
• Compare foo.c to v1.1 rcsdiff sccs diffs r1.1
  foo.c -r1.1 foo.c

45
CVS Major Features

• No exclusive locks like RCS
• No waiting around for other developers
• No hurrying to make changes while others wait
• Avoid the lost update problem
• Client/Server model
• Distributed software development
• Front-end tool for RCS with more functions

46
CVS Repositories

• All revisions of a file in the project are in the
  repository (using RCS)
• Work is done on the checkout (working copy)
• Top-level directories are modules checkout
  operates on modules
• Different ways to connect

47
CVSROOT

• Environment Variable
• Location of Repository
• Can take different forms
• Local file system /usr/local/cvsroot
• Remote Shell user_at_server/usr/local/cvsroot
• Client/Server pserveruser_at_server/usr/local/cvs
  root

48
Getting Started

• cvs basic-options ltcommandgt cmd-options
  files
• Basic options
• -d ltcvsrootgt Specifies CVSROOT
• -H Help on command
• -n Dry run
• Commands
• import, checkout
• update, commit
• add, remove
• status, diff, log
• tag...

49
Setting up CVS

• Importing source
• Generates a new module
• cd into source directory
• cvs dltcvsrootgt import ltnew-modulegt
  ltvendor-branchgt ltrelease-taggt
• cvs dltcvsrootgt checkout ltmodule-namegt

50
Managing files

• Add files add (cvs add ltfilenamegt)
• Remove files remove (cvs remove ltfilenamegt)
• Get latest version from repository update
• If out of sync, merges changes. Conflict
  resolution is manual.
• Put changed version into repository commit
• Fails if repository has newer version (need
  update first)
• View extra info status, diff, log
• Can handle binary files (no merging or diffs)
• Specify a symbolic tag for files in the
  repository tag

51
tar Tape ARchiver

• tar general purpose archive utility (not just
  for tapes)
• Usage tar options files
• Originally designed for maintaining an archive of
  files on a magnetic tape.
• Now often used for packaging files for
  distribution
• If any files are subdirectories, tar acts on the
  entire subtree.

52
tar archiving files options

• c creates a tar-format file
• f filename specify filename for tar-format
  file,
• Default is /dev/rmt0.
• If - is used for filename, standard input or
  standard output is used as appropriate
• v verbose output
• x allows to extract named files

53
tar archiving files (continued)

• t generates table of contents
• r unconditionally appends the listed files
  to the archive files
• u appends only files that are more recent
  than those already archived
• L follow symbolic links
• m do not restore file modification times
• l print error messages about links it cannot
  find

54
cpio copying files

• cpio copy file archives in from or out of tape
  or disk or to another location on the local
  machine
• Similar to tar
• Examples
• Extract cpio -idtu patterns
• Create cpio -ov
• Pass-thru cpio -pl directory

55
cpio (continued)

• cpio -i dtum patterns
• Copy in (extract) files whose names match
  selected patterns.
• If no pattern is used, all files are extracted
• During extraction, older files are not extracted
  (unless -u option is used)
• Directories are not created unless d is used
• Modification times not preserved with -m
• Print the table of contents -t

56
cpio (continued)

• cpio -ov
• Copy out a list of files whose names are given on
  the standard input. -v lists files processed.
• cpio -p options directory
• Copy files to another directory on the same
  system. Destination pathnames are relative to the
  named directory
• Example To copy a directory tree
• find . -depth -print cpio -pdumv /mydir

57
pax replacement for cpio and tar

• Portable Archive eXchange format
• Part of POSIX
• Reads/writes cpio and tar formats
• Union of cpio and tar functionality
• Files can come from standard input or command
  line
• Sensible defaults
• pax wf archive .c
• pax r lt archive

58
Distributing Software

• Pieces typically distributed
• Binaries
• Required runtime libraries
• Data files
• Man pages
• Documentation
• Header files
• Typically packaged in an archive
• e.g., perl-solaris.tgz or perl-5.8.5-9.i386.rpm

59
Packaging Source autoconf

• Produces shell scripts that automatically
  configure software to adapt to UNIX-like systems.
• Generates configuration script (configure)
• The configure script checks for
• programs
• libraries
• header files
• typedefs
• structures
• compiler characteristics
• library functions
• system services
• and generates makefiles

60
Installing Software From Tarballs

• tar xzf ltgzipped-tar-filegt
• cd ltdist-dirgt
• ./configure
• make
• make install

61
Debuggers

• Advantages over the old fashioned way
• you can step through code as it runs
• you dont have to modify your code
• you can examine the entire state of the program
• call stack, variable values, scope, etc.
• you can modify values in the running program
• you can view the state of a crash using core files

62
Debuggers

• The GDB or DBX debuggers let you examine the
  internal workings of your code while the program
  runs.
• Debuggers allow you to set breakpoints to stop
  the program's execution at a particular point of
  interest and examine variables.
• To work with a debugger, you first have to
  recompile the program with the proper debugging
  options.
• Use the -g command line parameter to cc, gcc, or
  CC
• Example cc -g -c foo.c

63
Using the Debugger

• Two ways to use a debugger
• Run the debugger on your program, executing the
  program from within the debugger and see what
  happens
• Post-mortem mode program has crashed and core
  dumped
• You often won't be able to find out exactly what
  happened, but you usually get a stack trace.
• A stack trace shows the chain of function calls
  where the program exited ungracefully
• Does not always pinpoint what caused the problem.

64
GDB, the GNU Debugger

• Text-based, invoked with
• gdb ltprogramfilegt ltcorefilegtltpidgt
• Argument descriptions
• ltprogramfile gt executable program file
• ltcorefilegt core dump of program
• ltpidgt process id of already running program
• Example
• gdb ./hello
• Compile ltprogramfilegt with g for debug info

65
Basic GDB Commands

• General Commands
• file ltfilegt selects ltfilegt as the program to
  debug
• run ltargsgt runs selected program with
  arguments ltargsgt
• attach ltpidgt attach gdb to a running process
  ltpidgt
• kill kills the process being debugged
• quit quits the gdb program
• help lttopicgt accesses the internal help
  documentation
• Stepping and Continuing
• continue continue execution (after a stop)
• step step one line, entering called functions
• next step one line, without entering
  functions
• finish finish the function and print the return
  value

66
GDB Breakpoints

• Useful breakpoint commands
• break ltwheregt sets breakpoints. ltwheregt can
  be a number of things, including a
  hex address, a function name, a
  line number, or a relative line offset
• rwatch ltexprgt sets a watchpoint, which will
  break when ltexprgt is written to or read
• info breakpoints prints out a listing of all
  breakpoints
• clear ltwheregt clears a breakpoint at ltwheregt
• delete ltnumsgt deletes breakpoints by number

67
Playing with Data in GDB

• Commands for looking around
• list ltwheregt prints out source code at ltwheregt
• search ltregexpgt searches source code for ltregexpgt
• backtrace ltngt prints a backtrace ltngt levels
  deep
• info ltwhatgt prints out info on ltwhatgt
  (like local variables or function args)
• print ltexprgt prints out the evaluation of
  ltexprgt
• Commands for altering data and control path
• set ltnamegt ltexprgt sets variables or arguments
• return ltexprgt returns ltexprgt from current
  function
• jump ltwheregt jumps execution to ltwheregt

68
Tracing System Calls

• Most operating systems contain a utility to
  monitor system calls
• Linux strace, Solaris truss, SGI par

27mS 1 close(0) OK
27mS 1 open("try.in",
O_RDONLY, 017777627464) 29mS 1
END-open() 0 29mS 1
read(0, "1\n2\n/bin/date\n3\n/bin/sleep 2",
2048) 31 29mS 1 read(0,
0x7fff26ef, 2017) 0 29mS 1
getpagesize() 16384 29mS 1
brk(0x1001c000) OK 29mS 1
time() 1003207028 29mS 1
fork() 31mS 1
END-fork() 1880277 41mS 1
(1864078) was sent signal SIGCLD 31mS 2
waitsys(P_ALL, 0, 0x7fff2590,
WTRAPPEDWEXITED, 0) 42mS 2
END-waitsys(P_ALL, 0, signoSIGCLD, errno0,
codeCLD_EXITED, pid1880277, status0,
WTRAPPEDWEXITED, 0) 0 42mS 2
time() 1003207028