Job%20control

1
2.0.0.3.3 Introduction to the Shell Session 3

• Job control
• Start, stop and monitor running processes
• Monitoring disk usage
• Command line tools
• grep, cut, sort, uniq, paste, fold, tr, head,
  tail
• Command line Perl
• When command line tools dont quite do it

2
Seeing Running Processes top

• See a constantly updated list of whats running
  with
• Sorted by CPU use
• Also shows available memory and processors
• Processes can be controlled based on the PID
  (process ID)

Overall CPU use
Overall memory use
Process memory use
Process CPU use
top
Im not running all that! Many processes are run
by the OS (as user root) in the background.
These processes generally have very low PIDs.
3
Seeing Running Processes ps

• See a complete list of whats running with
• By default only shows what you are running (no
  root or other users processes)
• Can select which processes to show eg.
• U by user
• p by PID
• r only running processes
• See full list of everything running with ps auxw

ps
4
Seeing Running Jobs jobs

• See currently running jobs with
• These are processes that the current user is
  running in this terminal (does not show other
  users processes, or jobs started from other
  terminal windows)
• Useful for job control.

jobs
5
Putting Processes in the Background , bg and fg

• What if youre running a long job (eg. BLAST),
  but you want to use your prompt at the same time?
• Put the job in the background by starting it with
  ltcommandgt
• Or, once the job is running, stop it with Cntl-Z
  (interrupts but does not kill), then use
• To put the job back in the foreground (and lose
  your prompt), use
• To specify which job to fg or bg, use the PID or
  ltjob idgt

bg
fg
Job ID
6
Ending Jobs Cntl-C, kill

• To end (without saving anything) a job currently
  running in the terminal, use Cntl-C
• To kill a job that is running in the background
  or from another terminal, use
• If it really wont die, try kill -9 but only as
  a last resort!

kill
Killing applications If you are running
applications (eg. word processors, web browsers)
from the command line, do NOT kill them unless
you cannot exit the application normally this
can corrupt files or cause application errors.
7
Sharing the CPU nice

• Jobs are assigned priority (the lower the number,
  the higher the priority) which determines sharing
  of CPU
• Set the priority of your job lower when running
  intensive jobs so that others can use the CPU too
  with
• Or, for running jobs, with

nice
renice
Niced
Priority
8
Memory and Swap

• Nice does not control memory use
• When running memory-intensive jobs, avoid
  requiring swap (virtual memory actually disk
  space) as this will slow all jobs on that computer

9
Checking Disk Location and Usage df

• To confirm that the disk you are on has enough
  space for you to create large files
• -h human-readable

df
10
Determining Disk Space Usage du

• To check total space taken by a directory
  structure, use
• Find out what is taking up all your disk space!
• -h human-readable
• -c produce total
• --max-depth only print totals for this deep
  in directory tree

du
Saving space Cut down on disk usage by using du
and find to find large, old files, and use gzip
to compress them.
11
Working with Text

• Simple text processing is a large part of
  bioinformatics
• Converting between program formats
• Data processing
• Command line tools are a fast and easy way to do
  a lot of text processing
• Usually faster than Perl
• Dont require any coding!

12
Searching for Text grep

• Search for text in a file with
• Prints matching lines
• Can do regular expression matching (slightly
  different from Perl)
• Useful options
• -i case-insensitive
• -w text must appear as a word (surrounded by
  white space)
• -v find non-matching lines
• -c count matching lines
• -B ltgt print lines before matching lines
• -A ltgt print lines after matching lines
• -f ltfilegt find lines matching patterns listed in
  ltfilegt

grep
13
Sorting Text sort

• Sort a file using
• Useful options
• -f case-insensitive
• -g sort numerically (i.e. 11lt100)
• -r reverse sort
• -k ltgt sort on column
• -u only print one copy of unique lines

sort
14
Working with Columns cut and paste

• Get particular columns from a file with
• Have to specify which field (column) to get
• Useful options
• -f ltgt fields to output
• -d field delimiter
• Put columns back together with

cut
paste
15
Finding Unique or Repeated Lines uniq

• Determine unique lines with
• Collapses repeated lines
• Useful options
• -i case insensitive
• -c count copies of each line
• -d print only duplicated lines
• -u print only unique lines
• -f ltgt do not compare the first fields

uniq
Pipe it! The UNIX text tools are particularly
powerful in combination use pipes to do a
series of processing commands on your text.
16
Counting wc

• Count lines, words, and characters with
• Good for checking if files have correct number of
  records
• Useful in combination with other commands

wc
bytes (characters)
lines
words
How many SAGE tags match more than one gene
17
Replacing Characters tr
Capitalize

• Replace characters with
• Like Perls tr///
• Useful options
• -d delete
• -s replace repeats with single occurrence
• -c use complement (replace everything EXCEPT
  what is specified)

tr
Format comma-separated list
Input to tr Instead of reading from a file, tr
reads from standard input. If you want it to read
from a file, redirect that file to standard input
with lt, or use cat and pipe the output to tr.
18
Formatting Text fold

• To wrap long lines in a file, use
• Good for formatting before printing text files
• Useful options
• -w width to wrap to
• -s break at spaces

fold
19
Getting Parts of Files head and tail

• Print the first 10 lines with
• Print the last 10 lines with
• Specify how many lines to get with head ltgt
• eg. head -100 -gt gets first 100 lines
• Good for making smaller test files

head
tail
Make a test file
Divide a file into three
20
Comparing files diff

• Find differences between lines in files with
• Can also compare directories
• Useful options
• -b ignore differences in white space
• -i ignore changes in case

diff
Confirm file copying
21
More advance processing sed and awk

• Simple programming languages
• Designed for command-line use
• Good for more advanced text processing
• But I prefer.

22
Command Line Perl

• Perl can be run on a script, or directly on text
  input at the command line (a very short Perl
  script that does not have to be saved to disk)
• Input file with cat or lt
• Options to use command line
• -e find code on command line not in file
• -n run command line script on each line of input
• Check out other Perl command line options with
  man perlrun

What? Quotes? When running Perl on the
command line, use single quotes () so that the
shell does not try to interpret the Perl
commands. Within these quotes, you can then use
double quotes () for eg. print statements.
23
2.0.0.3.3 Introduction to the Shell Session 3

• Now you know.
• How to control your processes
• How to keep up on system usage
• How to play with text on the command line
• How to run Perl from the command line

24
2.0.0.3.3 Further Readings

• Job control
• Linux Cookbook Ch. 4.3
• Learning the UNIX Operating System Ch. 6
• Command line goodies
• Linux Cookbook Ch.13.1, 15.1-15.4
• Command line Perl
• Linux Cookbook Ch. 13.5, 15.5