Programming with the Shell.

1

• Programming with the Shell.

2
Shell Types in Unix

• Bourne Shell.
• Bourne Again Shell (bash).
• C Shell (c-shell).
• Korn Shell (k-shell).
• TC Shell (tcsh)

3
The Shell.

• A program that interprets users requests to run
  programs.
• Provides environment for user programs.
• A command line interpreter that
• Reads user inputs.
• Executes commands.

4
Shell Features.

• Interactive and background processing.
• Input/output redirection
• Pipes.
• Wild card matching.
• Programmable.
• Shell Variables.
• Programming language constructs.
• Shell Scripts.

5
Korn Shell Features

• Features appropriated from the C shell include
• ? Job control
• Aliases
• Functions
• Command history
• The Korn shell's major new features include
• Command-line editing,

6
Korn Shell Features

• Integrated programming features
• Debugging primitives
• Regular expressions
• Advanced I/O features
• New options and variables
• Increased speed
• Security features

7
What is a shell ?
Figure 1.1 The shell is a layer around the UNIX
operating system                                
                                                  
                                                  
                
8
What is a Shell Script

• A shell script is a text file containing a
  combination of following
• Unix commands (such as ls, cat).
• Shell commands (such as variable assignment) and
  programming constructs (such as if statement and
  loops)
• Since a shell script is a program, it is normally
  assigned execute permission using the chmod
  command.

9
What is a Shell Script

• To write effective shell scripts, you need to
  have working knowledge of basic commands and
  utilities in Unix. These include
• regular expressions
• grep, cut, awk and sed utilities
• This course does not permit time to cover the
  above utilities. Where relevant, we might
  demonstrate the use of these utilities.

10
Executing a shell script

• There are many ways of executing a shell script
• By passing the shell script name as an argument
  to the shell. For example
• sh script1.sh
• If the shell script is assigned execute
  permission, it can be executed using its name.
  For example
• ./script1.sh

11
Executing a shell script

• There are many ways of executing a shell script
• If the shell script has execute permission and is
  stored in a directory listed in PATH, it can be
  executed using its name. For example
• script1.sh
• -- If the shell script has execute permission and
  is stored in a directory listed in PATH, it can
  be executed using the dot command. For example
• . script1.sh

12
Filenames and Wildcards

• Wildcard Matches
• ? Any single character
• Any string of
  characters
• set Any character in set
• !set Any character not in set
• A set is a list of characters
• Expression Matches
• abc a, b, or c
• ., Period, comma, or semicolon
• -_ Dash and underscore
• a-c a, b, or c
• a-z All lowercase letters
• !0-9 All non-digits

13
Filenames and Wildcards

• Expression Matches
• a-zA-Z All lower- and uppercase letters
• a-zA-Z0-9_- All letters, all digits,
  underscore, and dash

14
Special Characters or Metacharacters Quoting

• Table 1.6 Special Characters
• Character Meaning
• Home directory
• Comment
• Variable expression
• Background job
• String wildcard
• ( Start subshell
• ) End subshell
• \ Quote next character
• Pipe

15
Special Characters or Metacharacters Quoting

• Character Meaning
• Start character-set wildcard
• End character-set wildcard
• Start code block
• End code block
• Shell command separator
• Strong quote
• Weak quote
• lt Input redirect
• gt Output redirect
• / Pathname directory separator
• ? Single-character wildcard

16
Command grouping ( )

• Executes the command group in a sub-shell
• Ex-
• pwd
• /root
• (cd xyz pwd )
• /root/xyz
• pwd
• /root
• A change of the directory in the child can't
  reflect in the parent process.

17
Command grouping

• It uses current shell only
• Ex-
• pwd
• /root
• cd xyz pwd
• / root/xyz
• pwd
• /root/xyz
• Directory change is permanent

18
Here-documents

• The ltlt label redirector essentially forces the
  input to a command to be the shell's standard
  input, which is read until there is a line that
  contains only label
• The input in between is called a
  here-document.
• Ex-
• 1) cat gtgt xyz ltlt EOF
• IBM UNIX TRAINING
• EOF

19
Here-documents

• Ex-
• 2) tr 'a-z' 'A- Z' ltlt EOF
• hi hello
• EOF
• 3) cat gt file1 ltlt EOF
• djhdksjhfksfkdsf
• EOF
• cat file1
• It is used to take away user interaction
  provide automation.

20
Tees joints

• Tee uses standard input and standard output
• It can be placed anywhere in a pipeline
• It breaks up the input inito two components, one
  component is saved in a file and the other is
  connected to the standard output
• It is used to store the intermediate output of a
  pipeline
• Ex-
• ls -l tee abc
• who tee list wc -l

21
Command-line Editing

• Enabling Command-line Editing

set -o emacs or set -o vi

22
Simple Control Mode Commands

• Basic vi Control Mode Commands
• ESC to Enter control mode
• Command Description
• h Move left one character
• l Move right one character
• w Move right one word
• b Move left one word
• e Move to end of current word
• 0 Move to beginning of line
• Move to first non-blank character in line
• Move to end of line

23
Entering and Changing Text

• Commands for Entering vi Input Mode
• Command Description
• i Text inserted before current character
  (insert)
• a Text inserted after current character (append)
• I Text inserted at beginning of line
• A Text inserted at end of line
• R Text overwrites existing text

24
Deletion Commands

• Command Description
• dh Delete one character backwards
• dl Delete one character forwards
• db Delete one word backwards
• dw Delete one word forwards
• dB Delete one non-blank word backwards
• dW Delete one non-blank word forwards
• d Delete to end of line
• d0 Delete to beginning of line

25
Moving Around in the History File

• Command Description
• k or - Move backward one line
• j or Move forward one line
• G Move to line given by repeat count
• ?string Search backward for string
• /string Search forward for string
• n Repeat search in same direction as
  previous
• N Repeat search in opposite direction of
  previous

26
Shell Variables.

• Positional Parameters.
• Special Parameters.
• Named variables

27
Positional Parameters.

• Acquire values from the position of arguments in
  command line.
• 1, 2, 3,..9
• sh file1 10 20 30

1
2
3
28
Special Parameters.

• Shell assigns the value for this parameter.
• Number of Command Line Arguments.
• 0 Command Name.
• Displays all the command line arguments.
• ? Exit Status.
• ! Process number of the last background
  command
• _at_ Same as , except when enclosed in double
  quotes.
• PID number.

29
Named Variables.

• User-defined variable that can be assigned a
  value.
• Used extensively in shell-scripts.
• Used for reading data, storing and displaying it.

30
Accepting Data.

• read.
• Accepts input from the user.
• Syntax read variable_name.
• Example read sname

Variable Name
31
Display Data.

• echo
• Used to display a message or any data as required
  by the user.
• echo Message, Variable
• Example
• echo IBM.
• echo sname

Variable Name
32
test command.

• Used extensively for evaluating shell script
  conditions.
• It evaluates the condition on its right and
  returns a true or false exit status.
• The return value is used by the construct for
  further execution.
• In place of writing test explicitly, the user
  could also use .

33
test command (Contd).

• Operators used with test for evaluating numeral
  data are
• -eq ? Equal To
• -lt ? Less than
• -gt ? Greater than
• -ge ? Greater than or equal to
• -le ? Less than or equal to
• -ne ? not equal to

34
test command (Contd).

• Operators used with test for evaluating string
  data are

str1 str2 ? True if both equals str1 ! str2 ?
True if not equals -n str1 ?True if str1 is not
a null string -z str1 ? True if str1 is a null
string
35
test command (Contd).

• Operators used with test for evaluating file data
  are

-f file1 ? True if file1 exists and is a regular
file. -d file1 ? True if file1 exists and is
directory. -s file1 ?True if file1 exists and
has size greater than 0 -r file1 ? True if file1
exists and is readable. -w file1 ? True if file1
exists and is writable. -x file1 ? True if file1
exists and is executable.
36
Logical Operators.

• Logical Operators used with test are

! ? Negates the expression. -a ? Binary and
operator. -o ? Binary or operator.
37
expr command.

• Used for evaluating shell expressions.
• Used for arithmetic and string operations.
• Example expr 7 3
• would give an output 10.
• When used with variables, back quotes need to be
  used.

Operator has to be preceded and followed by a
space.
38
Conditional Execution.

• The second command is executed only when first is
  successful.
• command1 command2
• The second command is executed only when the
  first is unsuccessful.
• command1 command2

39
Program Constructs

• if
• for
• while
• until
• case

40
if statement.

• Syntax.

41
for statement.

• Syntax.

42
while statement.

• Syntax.

43
until statement.

• Syntax.

44
case statement.

• Syntax.

The symbols are used as option terminators.
45
Useful Shell Scripting commands.

• break
• To come out of a loop.
• continue
• To jump to the start of loop.
• exit
• To prematurely terminate a program.
• To interpret the rest of line as comments.

46
export command.

• export
• To make a variable a part of environment and also
  be accessible to the child shell.
• export variable_name

47
Customizing User Environment

• gtTo customize the environment various built-in
  shell variables are available.
• gtTo change the values of variables permanently ,
  define it in .profile file.
• The .profile File
• the Korn shell reads and runs this file whenever
  you log in to your system
• Various environment variables can be defined in
  this file
• Alias can be defined in .profile file

48
Aliases

• Alias is a synonym for a command or command
  string
• Syntax
• alias neworiginal
• Ex-
• alias searchgrep
• alias cdnewcd /xyz/x1/x2
• gtQuotes are necessary if the string being aliased
  consists of more than one word
• gtit is possible to alias an alias, aliases are
  recursive
• Ex-
• alias ccdnew

49
Aliases

• type alias without any arguments, to get a list
  of all the aliases you have defined as well as
  several that are built-in.
• The command unalias name removes any alias
  definition for its argument

50
set command.

• set command
• Used for display all the environment variables.
• Shows the current values of system variables.
• Also allows conversion of arguments into
  positional parameters.
• Syntax set

51
Shell Variables

• Shell variables can specify everything from your
  prompt string to how often the shell checks for
  new mail
• built-in variables have names in all capital
  letters
• The syntax for defining variables is
• varnamevalue
• if the value is more than one word, it must be
  surrounded by quotes
• To delete a variable type the command
• unset varname

52
Print Command

• To check value of a variable print built-in
  command can be used
• Print command is strongly recommended over echo
  because its options are the same on all UNIX
  systems, whereas echo's options differ between
  BSD-derived and System V-derived UNIX versions.
• Ex- print x

53
System Variables or Built-in Variables

• PATH
• Search path referred by Unix for any command.
• echo PATH
• HOME
• Indicates the home directory for the user.
• echo HOME
• HISTFILE
• - Name of history file, on which the editing
  modes operate.

54
System Variables (Contd).

• FCEDIT
• Pathname of editor to use with the fc command.
• PS1
• Used for displaying changing the primary
  prompt.
• echo PS1
• PS2
• Used for changing the secondary prompt.
• MAIL
• Name of file to check for incoming mail (i.e.,
  your mail file)

55
System Variables (Contd).

• SHELL
• Pathname of the shell you are running
• PWD
• Current directory
• ENV
• Name of file to run as environment file when
  shell is invoked
• HISTSIZE
• Number of lines kept in history file

56
System Variables (Contd)

• MAILCHECK
• How often, in seconds, to check for new mail
  (default 600 seconds, or 10 minutes)
• EDITOR
• Pathname of your favorite text editor the suffix
  (emacs or vi) determines which editing mode to use

57
Environment Variables

• Environment Variables are known to all kinds of
  subprocesses
• Any variable can become an environment variable.
  First it must be defined as usual then it must
  be exported with the command
• export varnames
• To find out environment variables and their
  values ,type
• export

58
The Environment File

• Although environment variables will always be
  known to subprocesses,
• the shell must define which other variables,
  options, aliases, etc., are to communicated to
  subprocesses.
• The way to do this is to put all such
  definitions in a special file called the
  environment file instead of your .profile.
• 1. Decide which definitions in your .profile you
  want to propagate to subprocesses. Remove them
  from .profile and put them in a file you will
  designate as your environment file.

59
The Environment File

• 2. Put a line in your .profile that tells the
  shell where your environment file is
• ENVenvfilename
• 3 . For the changes to take effect, type either
  . .profile or login. In either case, your
  environment file will be run when the shell
  encounters the ENV statement.

60
Functions

• A function is sort of a script-within a-script
• Functions improve the shell's programmability
  significantly
• To define a function, you can use either one of
  two forms
• function functname
• shell commands
• or
• functname ()
• shell commands

61
Functions

• to delete a function definition issue command
• unset -f functname.
• To find out what functions are defined in your
  login session
• functions

62
String Operators

• string operators let you do the following
• Ensure that variables exist (i.e., are defined
  and have non-null values)
• Set default values for variables
• Catch errors that result from variables not being
  set
• Remove portions of variables' values that match
  patterns

63
Syntax of String Operators

• Operator Substitution
• varname-word If varname exists and isn't
  null, return its value otherwise return word.
• Purpose Returning a default value if the
  variable is undefined.
• Example count-0 evaluates to 0 if count is
  undefined.

64
Syntax of String Operators

• varnameword If varname exists and isn't
  null, return its value otherwise set it
  to word and then return its value
• Purpose Setting a variable to a default value
  if it is undefined.
• Example count0 sets count to 0 if it is
  undefined.

65
Syntax of String Operators

• varname?message If varname exists and isn't
  null, return its value otherwise print
  varname followed by message, and abort
  the current command or script.
• Purpose Catching errors that result from
  variables being undefined.
• Example
• count?" undefined!" prints "count
  undefined!" and exits if count is undefined.

66

• varnameword If varname exists and isn't
  null, return word otherwise return null.
• Purpose Testing for the existence of a
  variable.
• Example count1 returns 1 (which
  could mean "true") if count is defined.

67
select

• select allows you to generate simple menus easily
• Syntax-
• select name in list
• do
• statements that can use name...
• done
• what select does
• Generates a menu of each item in list, formatted
  with numbers for each choice
• Prompts the user for a number
• Stores the selected choice in the variable name
  and the selected number in the built-in variable
  REPLY
• Executes the statements in the body
• Repeats the process forever

68
shift

• Shift command is used to shift position of
  positional parameter
• supply a numeric argument to shift, it will shift
  the arguments that many times over
• for example, shift 3 has the effect
• 1 4 2 5 ...

69
Integer Variables and Arithmetic

• The shell interprets words surrounded by (( and
  )) as arithmetic expressions. Variables in
  arithmetic expressions do not need to be preceded
  by dollar signs
• Korn shell arithmetic expressions are equivalent
  to their counterparts in the C language
• Table shows the arithmetic operators that are
  supported. There is no need to backslash-escape
  them, because they are within the ((...))
  syntax.
• The assignment forms of these operators are also
  permitted.
• For example, ((x 2)) adds 2 to x and stores
  the result back in x.

70
Arithmetic Operators

• Operator Meaning
• Plus
• - Minus
• Times
• / Division (with truncation)
• Remainder
• ltlt Bit-shift left
• gtgt Bit-shift right
• Bitwise and
• Bitwise or
• Bitwise not
• Bitwise exclusive or

71
Relational Operators

• Operator Meaning
• lt Less than
• gt Greater than
• lt Less than or equal
• gt Greater than or equal
• Equal
• ! Not equal
• Logical and
• Logical or
• Value 1 is for true and 0 for false
• Ex- ((3 gt 2)) has the value 1
• (( (3 gt 2) (4 lt 1) )) also has the value
  1

72
Arithmetic Variables and Assignment

• The ((...)) construct can also be used to define
  integer variables and assign values to them. The
  statement
• (( intvarexpression ))
• The shell provides a better equivalent the
  built-in command let.
• let intvarexpression
• there must not be any space on either side of the
  equal sign ().

73
Sample Integer Expression Assignments

• Assignment Value
• let x x
• 14 5
• '1 4 5
• '(23) 5 25
• '2 3 5 17
• '17 / 3 5
• '17 3 2
• '1ltlt4 16
• '48gtgt3 6
• '17 3 1
• '17 3 19
• '17 3 18

74
I/O Redirectors
75
I/O Redirector

• Redirector Function
• lt- Close the standard input (ex catlt- , wc
  lt- )
• gt- Close the standard output (ex cal gt- )
• 2gt1 says, "send standard error (file descriptor
  2) to the same place as standard output (file
  descriptor 1)
• ex- cat abc 2gt1 (where, abc is
  not-existing )
• In other examples the error message is not
  displayed on standard output
• ex- cat abc 2gt xyz , cat abc 2gt1

76

• The redirector ltgt is mainly meant for use with
  device files (in the /dev directory), i.e., files
  that correspond to hardware devices such as
  terminals and communication lines. Low-level
  systems programmers can use it to test device
  drivers

77
print

• print escape sequences
• print accepts a number of options, as well as
  several escape sequences that start with a
  backslash
• Sequence Character printed
• \a ALERT
• \c Omit final NEWLINE
• \n NEWLINE
• \t TAB
• \\ Single backslash
• Ex- 1) print \\a hi hello it gives
  beep
• 2) print \\c hi hello

78
read

• read command, allows you to read values into
  shell variables
• Options to read
• Option Function
• -p prompt it prompts for input
• -n nchars it restrict no. of characters to a
  variablepter 1.
• EX-
• read -p "enter value for x " x
• read -n 5 x

79
Handling Signals

• Signals are sent to processes by UNIX kernel in
  response to certain events.
• Most signals cause the process receiving them to
  terminate abruptly. However, if you have set a
  trap'' for the signal, you can use them to
  recover from the emergency.
• Shell recognizes a number of signals. However,
  not all of them can be trapped.

80
Signals

• A signal is a message that one process sends to
  another when some abnormal event takes place or
  when it wants the other process to do something
• signal is another way of processes to communicate
  with each other.
• To get list of all the signals on your system
• kill -l

81

• Some signals are
• Value Signal Description
• -------- -------- ------------------------------
  -----------------
• 0 EXIT Exit from the shell.
• 1 HUP A signal used by the shell,
  indicating that
• the standard output has hung up sending
  this signal logs you out.
• 2 INT Sent by the Control-C (intr)
  keystroke sends an
• interrupt to the current
  program.
• 3 QUIT Sent by the Control-\ keystroke
  causes the current program to abort,
  leaving behind a core dump (for use in
  program debugging).

82

• Value Signal Description
• ----------------- -----------------------------
  -----------------
• 9 KILL Cannot be trapped or
  ignored forces the receiving program to
  die.
• 15 TERM Terminates the receiving
  program. This signal should be used in
  preference to Signal 9, as the receiving program
  can catch it and carry out clean up tasks
  such as closing open files Signal 9 forces
  the process to terminate immediately.

83
Signals

• CTRL-Z sends TSTP ( for "terminal stop").
• To customize the control keys used to send
  signals , use options of the stty command
• For example, to set INT key to CTRL-X , use
• stty intr X
• When you log out by pressing ltCtrlgtD or typing
  exit, you are sending a signal to the shell.
• A signal notifies the shell that a special event
  has occurred, and the shell should take action.
  Several different types of signal are available
  to the system, but the one you send by logging
  out is signal 0, called EXIT.

84
Trap

• To alter the effects of certain events that
  generate signals, which generally tends to halt a
  process.
• You can use the trap command to catch any or all
  of the trappable signals. Its format is as
  follows
• trap command signals_list
• In other words, command is executed whenever one
  of the listed signals (which are specified using
  the signal number shown earlier) is received.

85
Trap

• For example, if you have a shell script that uses
  a temporary file called scratchpad, the file will
  be left behind whenever the script is
  interrupted, unless you add the following near
  the top of the script
• trap "rm scratchpad" 0 1 2 3 15
• trap 'print \'You hit control-C!\' ' INT
• trap 'print \'You hit control-C!\' ' INT
  TERM

86
kill

• kill is used to send a signal to any process you
  created-not just the currently running job
• kill takes as argument the process ID, job
  number, or command name of the process to which
  you want to send the signal.
• Ex-
• kill 1
• kill -QUIT 1
• kill -KILL 1
• kill -QUIT 2389

87
Ignoring Signals

• HUP (hangup), the signal the shell sends to all
  the background processes when you log out.
• nohup is used to ignore HUP

88
Subshells

• whenever a shell script is executed , it invoke
  another copy of the shell that is a subprocess of
  the main, or parent, shell process
• Subshell Inheritance
• Characteristics subshells get, or inherit, from
  their parents are
• The current directory
• Environment variables
• Standard input, output, and error plus any other
  open file descriptors
• Any characteristics defined in the environment
  file
• Signals that are ignored
• The first three of these are inherited by all
  subprocesses, while the last is unique to
  subshells
• surround some shell code with parentheses
  (instead of curly brackets), and that code will
  run in a subshell

89
Summary

• Types of Shells
• Shell Variables
• Program Constructs
• System Variables.

90
Thank You!