Introduction to Unix

1
Introduction to Unix
2
Outline

• Overview of the Unix Operating System (slides
  3-5)?
• Shell, CL, Basic Ops, Help, Special Characters
  (slides 6-16)?
• Files, Directories, Access Rights (slides
  17-18)?
• Defining Your Environment (slides 19-25)?
• Text Editors (slide 26)?
• Input/Output Redirection - Pipelines Filters
  (slides 27-39)?
• Processes Multitasking (slides 40-41)?
• Scripting (slides 42-45)?
• Compressing Archiving (slides 46-47)?
• Getting Help from ACCRE (slide 48) OReilly
  books (slide 49)?
• screen, a nice Unix tool (slide 50)?

Introduction to Unix
2
3
What is Unix?

• Unix is an operating system (OS) like Windows
• UNIX was originally created in the late 1960's by
  Ken Thompson and Dennis Ritchie of ATT Bell Labs
• It was created out of frustration with MULTICS
  (UNICS was a pun on MULTICS)?
• It was designed to be a programmers OS
• It turned out to be a portable, multitasking,
  multiuser OS

Introduction to Unix
3
4
Unix or UNIX?

• UNIX is trademarked by the Open Group
• To be called UNIX, an operating system must be
  certified as meeting the specification (currently
  UNIX 03) by the Open Group
• Four UNIX's Solaris (Sun), AIX (IBM), HP-UX
  (Hewlett-Packard), MacOS X Leopard (Apple)?
• Unix refers to any of the Unix-like operating
  systems such as Linux, OpenFreeNetBSD, or
  pre-Leopard MacOS X

Introduction to Unix
4
5
All Unix's aren't the same

• Each Unix usually develops its own niche or
  specialization
• AIX / HP-UX heavy duty business server
• Solaris e-mail / web server
• MacOS X - User friendly desktop
• Linux High performance OS and free, which makes
  it ideal for HPC clusters
• OpenBSD ultra-secure server

Introduction to Unix
5
6
Unix Shell / Command Line

• The shell is an interpreter used to communicate
  with the OS interactively on the command line
  (CL)?
• The cluster has two flavors tcsh and bash.
  Online manuals and FAQ
• http//www.gnu.org/software/bash/
• http//www.tcsh.org/Home
• For historical reasons, tcsh was the default
  shell for all users on VAMPIRE (the ACCRE
  cluster) now it's bash
• If you wish to change your default shell, use the
  chsh command on vmpschedCL promptgt chsh -s
  /bin/bash

6
Introduction to Unix
7
CL Operations and Files

• There are Unix commands (same function in any
  shell) and shell built-in commands
• Commands entered at the CL prompt have options
  and arguments
• Files and directories are the primary abstraction
  in Unix (similar to files and folders in
  Windows)?
• Directories are files with information on their
  contents
• Some useful Unix commands for interacting with
  files and directories

Introduction to Unix
7
8
Basic Commands

• pwd - print working directory
• ls - list contents of directories
• mkdir - make (create) new directories
• cd - change the current directory
• cp - copy files or directories
• mv - move files or directories
• rm - remove (delete) files or directories
• cat - concatenate file contents
• more/less - scroll file contents
• file - show file type
• simple bash examples with CL editing

Introduction to Unix
8
9
Command Help

• Unix traditionally includes instructions and help
  files (manual pages) on most commands and API's
  and their options. To access a manpage
• CL promptgt man foo
• For csh/tcsh and sh/bash built-in commands
• CL promptgt man builtin
• Some commands have command line help (usage hints
  or --help)?
• Other sources of help /usr/share/doc, Google

Introduction to Unix
9
10
Basic CL Editing

• lttabgt complete filenames and commands on CL
• up/down arrows scroll CL history
• left/right scrolls back and forth on CL
• history shell command lists CL history (also
  tcsh)?

Introduction to Unix
10
11
Basic CL Editing

• allows more complex CL editing, e. g.,
  select last event containing specific word,
  replace word, then execute command
• CL promptgt !?olds/old/new/
• This repeats the last event containing old, but
  substitutes new for old

Introduction to Unix
11
12
Basic CL Editing
Introduction to Unix
12
13
Files, Directories, Special Characters

• Files to care about
• hidden (filenames begin with a dot, list with
  ls -a),
• e. g., user initialization, .login, .bashrc,
  .cshrc,
• global initialization, e. g., /etc/bashrc
• devices
• symbolic links
• / top root directory
• A few special Unix shortcuts for file/directory
  names/paths
• expands word to your home directory path
• username home directory of any username
• . current directory
• .. parent directory

Introduction to Unix
13
14
Files, Directories, Special Characters

• N. b., best to avoid using these characters in
  filenames

Introduction to Unix
14
15
Files, Directories, Special Characters

• E. g., globbing (expanding wildcard to match
  pattern)
• CL promptgt ls -1
• file1
• file2
• file3
• CL promptgt ls ?ile1
• file1
• CL promptgt ls file1-2
• file1
• CL promptgt ls filea-z0-9
• file2

Introduction to Unix
15
16
Multiple commands and

• You can use the operator to append many
  commands on one line
• CL promptgt date uptime
• Tue May 8 114126 CDT 2007
• 1141 up 8 days, 2226, 1 user, load averages
  0.81 0.69 0.67

Introduction to Unix
16
17
File Permissions

• Files/directories have an owner and a group
• The owner can grant read/write/execute
  permissions to three groups (the user, the user's
  group, and all others on the system)?
• promptgt ls -l /home/username
• total 16
• drwxr-xr-x 2 username group 8192 May 1 1559
  certs/
• drwxr-xr-x 3 username group 8192 Sep 15 1728
  classes/

Introduction to Unix
17
18
File Permissions

• These permissions are modified by the chmod
  command, e. g.
• CL promptgt chmod grx file
• permits other users in the owners group to read
  and execute file

Introduction to Unix
18
19
Startup Files, Variables, Aliases

• Two types, environment and shell, can be
  (re)initialized at any time.
• If defined in .login, environment variables are
  set upon login
• If defined in shell run command files (e. g.,
  .bashrc, .cshrc, .tcshrc), shell variables set
  upon each instance of shell
• Frequently used variables should be set in these
  files

Introduction to Unix
19
20
Startup Files, Variables, Aliases

• A few common variables common to both shells

Introduction to Unix
20
21
Startup Files, Variables, Aliases

• Important environment variables
• PATH - points to executables
• LD_LIBRARY_PATH points to libraries
• Reference a variable with or
• echo and printenv commands
• CL promptgt echo PATH
• CL promptgt echo PATH
• CL promptgt printenv PATH

Introduction to Unix
21
22
Setting Variables

• To assign a value to a variable (shells have
  different syntax)
• Example of adding to your PATH
• rehash lets shell know PATH was just updated

Introduction to Unix
22
23
Setting Variables

• Example of setting a new variable to the output
  value of a command
• CL promptgt current_date_timedate
• CL promptgt echo current_date_time
• Mon Apr 23 141535 CDT 2007
• Another example (note pwd vs. PWD)
• CL promptgt echo "I am in pwd on HOST
• I am in /home/username on vmps08
• CL promptgt echo "I am in PWD on HOST
• I am in /home/username on vmps08

Introduction to Unix
23
24
Setting Command Aliases

• Shorten frequently used or lengthy commands
• tsch syntax
• alias rm 'rm -i
• bash syntax
• alias rm'rm -i
• other useful aliases
• alias cp'cp -i'
• alias mv'mv -i'
• alias ls'ls --color-tty -F'
• alias ll'ls -laF'

Introduction to Unix
24
25
Setting Command Aliases

• Return to default
• backslash temporarily returns to default
• \rm junk
• unalias returns default for rest of session
• unalias rm
• rm junk

Introduction to Unix
25
26
Text Editors on Cluster

• Fairly basic
• Nano (http//www.nano-editor.org/)?
• More advanced
• Vi / Vim (http//www.vim.org/)?
• Emacs (http//www.gnu.org/software/emacs/)?
• Create/edit a new file example.txt

Introduction to Unix
26
27
Input/Output Redirection

• Most programs have three I/O streams
• stdin standard input
• stdout standard output
• stderr standard error.
• They all default to the console ("console" means
  the keyboard for the input and the screen for the
  output)?

Introduction to Unix
27
28
Input/Output Redirection

• To redirect stdout of a program to a file bash
  myprogram 1gt output.log
• tcsh myprogram gt output.log
• To redirect stderr of a program to a file
  bash myprogram 2gt error.log
• To redirect both stdout and stderr to same file
  (order matters) bash myprogram gt
  combined.log 2gt1
• To redirect both stdout and stderr separately
• myprogram gtoutput.log) 2gt error.log

Introduction to Unix
28
29
Input/Output Redirection

Introduction to Unix
29
30
Input/Output Redirection

• Example of gtgt operator to append information to
  a filepromptgt date gt foopromptgt cat fooWed
  Aug 31 172752 CDT 2005promptgt date gtgt
  foopromptgt cat fooWed Aug 31 172752 CDT
  2005Wed Aug 31 172756 CDT 2005

Introduction to Unix
30
31
Input/Output Redirection

• Use of single back quotes to substitute in the
  value of another command
• Compare to slides 14 and 22
• promptgt ls find . -name filea-z0-9
• file2
• promptgt echo find . -name filea-z0-9
• file2

Introduction to Unix
31
32
Input/Output Redirection

• script to redirect CL stdin/out to record session
  to a file (however, formatting not so nice)?
• To redirect output to nowhere use the null
  device, /dev/null
• To redirect stdout to null device (tcsh/bash)
• myprogram gt/dev/null
• To redirect stdout and stderr to null
  (bash) myprogram gt/dev/null 2gt1

Introduction to Unix
32
33
I/O Redirection - Pipes/Filters

• Pipelines are a set of processes chained by their
  standard streams, so that the stdout of each
  process feeds directly as the stdin of the next.
• Pipelines are defined using the character.
• E. g., use a pipe and tee to direct output of
  echo to both stdout and to a file
• promptgt echo "Hello World" tee output.txt

Introduction to Unix
33
34
I/O Redirection - Pipes/Filters

• Commands can be used as filters which take the
  output of a program and modify it. E. g., use a
  pipe to count words from the output of echo
• promptgt echo "Hello World" wc -w2
• Very useful filters include
• grep - Pattern matching
• sed - Search and Replace
• cut - Print specific columns
• sort - Sort alphabetically / numerically
• uniq - Remove duplicate lines from a file

Introduction to Unix
34
35
I/O Redirection - Pipes/Filters

• grep examplepromptgt cat example.txtHello
  WorldGoodbye Worldpromptgt cat example.txt
  grep HelloHello Worldpromptgt cat example.txt
  grep -v HelloGoodbye World

Introduction to Unix
35
36
I/O Redirection - Pipes/Filters

• sed examplepromptgt cat example.txtHello
  WorldGoodbye Worldpromptgt cat example.txt
  sed s/Hello/Goodbye/gGoodbye WorldGoodbye
  World

Introduction to Unix
36
37
I/O Redirection - Pipes/Filters

• cut examplepromptgt cat example.txt1,Hello,Worl
  d2,Goodbye,Worldpromptgt cat example.txt cut
  -d "," -f 2-Hello,WorldGoodbye,World

Introduction to Unix
37
38
I/O Redirection - Pipes/Filters

• sort examplepromptgt cat example.txt2
  Goodbye1 Hello2 Goodbyepromptgt cat
  example.txt sort -n1 Hello2 Goodbye2 Goodbye

Introduction to Unix
38
39
I/O Redirection - Pipes/Filters

• uniq examplepromptgt cat example.txt2
  Goodbye1 Hello2 Goodbyepromptgt cat
  example.txt sort -n uniq1 Hello2 Goodbye

Introduction to Unix
39
40
Processes and Multitasking

• To run a program in the background, use the
  character (or Z followed by bg) promptgt
  myprogram 1 7895
• myprogram is now running in the background as
  process id (PID) 7895
• Whenever your process finishes, it will print
  Done to the console.

Introduction to Unix
40
41
Processes and Multitasking

• To check on the status of your jobs running on
  the system, use the ps command promptgt ps
  -a PID TTY TIME CMD 8095 pts/3
  000000 ps
• You can get an expanded list by typingps agux,
  or by using the top command
• Use uptime to check the load average (how hard
  system is working) on slowly responding machines

Introduction to Unix
41
42
Simple Shell Scripting

• String Unix commands into a shell
  script/bin/csh N. b., not a comment on line
  1
• To executetsch promptgt source myscript01.csh
• bash promptgt . ./myscript01.sh
• Or run as executable
• promptgt chmod ux myscript01.csh
• promptgt myscript01.csh
• promptgt ./myscript01.csh (if not in path)?

Introduction to Unix
42
43
Simple Shell Scripting

• Simple bash programming myscript02.sh
• !/bin/bash
• define variable
• list"Mercury Venus Earth Mars Jupiter Saturn
  Uranus Neptune
• initiate counter
• i1
• for loop
• for planet in list
• do
• echo "planet i is planet" Print to STDOUT
• iexpr i 1 Increment
  counter
• done
• Execute it
• promptgt chmod ux myscript02.sh ./myscript02.sh

Introduction to Unix
43
44
More on Variables

• Can also define conditional variables

Introduction to Unix
44
45
More on Variables

• Example bash uses of conditional variables
• promptgt echo VAR ERROR
• promptgt echo ERROR?An error was found
• bash ERROR An error was found
• promptgt ERRORTRUE
• promptgt echo ERRORAn error was found
• An error was found
• promptgt echo ERROR-An error was found
• TRUE
• promptgt echo ERRORAn error was found
• TRUE

Introduction to Unix
45
46
Compressing and Archiving

• There are several ways you can compress files to
  reduce disk usage or transfer time.
• The Windows way is using the zip and unzip
  commandspromptgt ls -alh testfile1
  testfile2-rw-r--r-- root root 1.0M
  testfile1-rw-r--r-- root root 1.0M
  testfile2promptgt zip testfile.zip testfile1
  testfile2promptgt ls -alh testfile.zip-rw-r--r--
  root root 2.4K testfile.zip

Introduction to Unix
46
47
Compressing and Archiving

• Traditional UNIX archiving tools, tar and gzip.
• tar takes a number of files/directories and
  combines them into a single file
• gzip takes combined archive and compresses it
• tar -c file1 file2 ... gzip -9 gt archive.tgz
• Or simply
• tar -zc archive.tgz file1 file2
• To extract files from a tar archivetar xfzp
  archive.tgz

Introduction to Unix
47
48
Getting Help from ACCRE

• ACCRE website FAQ and Getting Started pages
• www.accre.vanderbilt.edu/support
• ACCRE helpdesk
• www.accre.vanderbilt.edu/support/contact/submit
  _RT.php
• accre-forum mailing list
• Office hours at ACCRE M-F 4-5 PM

Introduction to Unix
48
49
Give UNIX a try!

• There are several free flavors of UNIX available
• Fedora http//www.fedora.redhat.com
• Debian http//www.debian.org
• FreeBSD http//www.freebsd.org
• Ubuntu http//www.ubuntu.com
• OReilly has excellent desktop reference
  materials (I. e., books)
• http//www.oreilly.com/
• (E. g., Linux in a Nutshell, Class Shell
  Scripting, )?
• OReilly pocket guides also very useful quick
  references (Linux OS, shells, editors, )?

\
Introduction to Unix
49
50
Screen

• Another useful unix tool with remote session
• Tutorial
• Log onto cluster noting the gateway (i. e.,
  vmpsxx)
• promptgt screen
• promptgt nano junk.txt
• While in editor, close window
• Must log back onto node you were on, i. e.,
  vmpsxx.accre.vanderbilt.edu
• Find SCREEN process ID (PID)
• promptgt ps augx grep ltyour user namegt
• promptgt screen -R ltPIDgt
• Your process is Restored where you left off

Introduction to Unix
50