Scripting on Linux

1
Scripting on Linux

• Origins of scripting languages
• Learning scripting languages
• A simple example of a shell script
• A script with a loop and debugging
• Processing a table of data by selecting rows and
  columns
• Doing arithmetic internally within the shell
• Perl, Python, Ruby etc.
• Scheduling scripted jobs using cron

2
Origins of scripting languages

• Hardwired programs and embedded OS allowed little
  system control. Job control language JCL allowed
  IBM mainframe operators to "batch" a sequence of
  programs into a routine job, e.g. deducting
  payments made from accounts outstanding.
• Benefits
• greater accuracy of job control
• less typing
• ability to restart job from initial data
  when system goes down part way through.
• Mainframe features reinvented on smaller/cheaper
  machines.

3
Command languages developed

• TSO/Clist (IBM on-line), PCL (Prime Primos), DCL
  (DEC VAX/VMS), Aegis (Apollo DomainOS). MS-DOS
  Batch File (.BAT) etc.
• Microsoft froze .BAT language early 1990s. Early
  versions of MS Basic packaged with MS-DOS, later
  versions sold separately.
• Unix opened development and transfer of ideas
  between shells.
• Bourne Shell /bin/sh 'C' Shell
  /bin/csh
• Korn Shell /bin/ksh Ash Shell
  /bin/ash
• GNU Bash Shell /bin/sh and /bin/bash
• Unix shell scripts have usual 3rdG features,
  including loop constructs, variables, arrays,
  branches and functions.

4
Learning scripting languages

• Many features shared between shell languages and
  'C', Perl,
• Python etc.
• The 80/20 rule 80 of the benefit derives from
  knowledge of the 20 most useful subset.
• Programmers learn by
• Reading tutorials, books, reference
  material
• Studying code examples
• Running modified code examples
• Many small experiments
• Knowing how to program using a similar
  language type

5
Learning scripting languages 2

• Those requiring an in-depth understanding of
  these languages will need to read the books and
  on-line tutorials and carry out a comprehensive
  series of programming exercises.
• In other cases a usable subset of knowledge can
  be obtained by reading the source code of
  existing programs and executing these, and by
  conducting small experiments supplemented with
  tactical use of the reference information
  provided with languages. Programming knowledge
  can grow on an as-needed basis.

6
A simple shell script example
7
Return values and tests

• The Unix shell convention is for a program which
  exits successfully to end with a return of 0
  (which the shell considers true), and for an
  error exit to result in a return of 1 or more
  (which the shell considers false).
• This is the opposite way round to how this is
  done within 'C'. The number N returned by a
  program (e.g. using the bash exit N statement) is
  different from the standard output of the same
  program. The shell command
• echo ?
• outputs the return code of the last foreground
  command.

8
A script with a loop and debugging
9
menu script part 2
10
A table with rows and columns

• Here are some records from the may.logins input
  file
• reboot Tue May 11
  1314
• shutdown Tue May 11 1315
• usr11361 console Tue May 11 0904 -
  1022 (0118)?
• usr11187 console Mon May 10 1853 -
  2030 (0136)?
• usr11187 console Mon May 10 1850 -
  1853 (0002)?
• usr11187 console Mon May 10 1838 -
  1850 (0012)?
• usr11513 console Mon May 10 1515 -
  1627 (0111)?
• usr11451 console Mon May 10 1211
  still logged in
• usr11456 console Mon May 10 1053 -
  1514 (0421)?
• usr11138 console Mon May 10 0903 -
  1040 (0136)?
• usr12069 console Sat May 8 1105 -
  0901 (12155)?
• usr12069 console Sat May 8 1100 -
  1104 (0004)?
• This gives us time logged, one record per login
  session.

11
Login analysis script 2

• This combines a number of features of previous
  examples, using awk, grep and sed filters to
  access specific rows then columns, and to exclude
  unwanted data from the analysis. The input data
  is a set of login records. This application was
  used to analyse average usage of 20 workstations
  during particular months.

12
Login analysis script 3
13
Doing arithmetic internally within the shell

• Smaller or earlier Unix shells don't have builtin
  arithmetic operators, so they farm this job out
  to external programs such as expr as we saw
  above. This can be done using the Bash shell let
  builtin, as in the following example script

14
Advanced scripting Perl, Python, Ruby etc.

• Shell scripts provide glue logic together with
  other utilities, pipelines and redirection.
• Advantage Very rapid development of
  systems/network administration and automated
  operations.
• Disadvantages Slow, not for very large programs,
  non-portable. Having to load and execute external
  programs very many times ?
• Adding 200 user login accounts to a system
• Shell 1 hour to run (1992), helped us
  learn how to do it
• Perl ('93) 2 minutes to run, we already
  knew how to do it
• 'C' ('94) 6 seconds but 10 times as long to
  write the program.

15
Advanced scripting Perl, Python, Ruby 2

• Fully portable languages Perl, Python and Ruby
  used to handle simple scripting-type applications
  or more complex requirements, e.g.
  object-oriented webapp development. These
  languages build upon features in shell languages,
  using similar syntax for many purposes, e.g.
  handling regular expressions.
• These languages trade machine for programmer
  efficiency, compare against 'C', 'C' .
• Can combine benefits by using 'C' modules in
  scripting interpreter or split application design
  after profiling to find where cycles used.

16
Advanced scripting Perl, Python, Ruby 3

• The login
• analysis
• program
• described
• above was
• rewritten in Perl.

17
Advanced scripting Perl, Python, Ruby 4

• This program runs much faster than the shell
  script, because everything is done inside the
  same process. There are many syntactic
  similarities, but Perl borrows array and loop
  notation from 'C' and some other notation from
  grep, awk and sed. As in Bash, is used to
  introduce single (scalar) variables and _at_ is used
  for arrays. _ is used for a default scalar
  variable, so that the substitution operation e.g
  s/\)// which strips a closing round bracket
  from a string, doesn't state the string it
  modifies.

18
Scheduling cron jobs on Linux 1

• If scripts are the key to systems automation, how
  do we cause these to be run at set times of the
  day, or on set days of the week or month ? The
  program we use for this is called cron.
• Cron is a background process which runs other
  programs at set times. It is controlled using a
  file called crontab. Crontab contains entries to
  run all scripts placed in directories
  /etc/cron.hourly
• /etc/cron.daily, /etc/cron.weekly etc.

19
Scheduling cron jobs on Linux 2

• /etc/crontab system-wide crontab
• Unlike any other crontab you don't have to run
  the crontab'
• command to install the new version when you
  edit this file
• and files in /etc/cron.d. These files also have
  username fields,
• that none of the other crontabs do.
• SHELL/bin/sh
• PATH/usr/local/sbin/usr/local/bin/sbin/bin/us
  r/sbin/usr/bin
• m h dom mon dow user command
• 17 root cd / run-parts
  --report /etc/cron.hourly
• 25 6 root test -x /usr/sbin/anacron
  ( cd / run-parts --report /etc/cron.daily
  )?
• 47 6 7 root test -x /usr/sbin/anacron
  ( cd / run-parts --report /etc/cron.weekly
  )?
• 52 6 1 root test -x /usr/sbin/anacron
  ( cd / run-parts --report /etc/cron.monthly
  )?
• local modifications
• update spam black and whitelist DNS zones every
  10 minutes
• 01,11,21,31,41,51 root /usr/local/bin/upda
  te_dnsfiles