UNIX: Background and the Traditional Process and Kernel Abstractions

1
UNIXBackground and the Traditional Process and
Kernel Abstractions

• Fred Kuhns
• fredk_at_cse.wustl.edu
• Applied Research Laboratory,
• Department of Computer Science and Engineering,
• Washington University in St. Louis

2
A Little History First UNIX

• Initial design Ken Thompson, Dennis Ritchie and
  others at ATT's Bell Telephone Laboratories
  (BTL) in 1969.
• ATT made the source available to Universities
  for research and educational use.
• 1973 UNIX Version 4 is a C rewrite of previous
  version.
• The C language originally developed for use on
  the UNIX system by Dennis Ritchie
• C evolved from 'B', developed by Thompson.
• ATT sold UNIX to Novel Novel passed the UNIX
  trademark to X/OPEN and sold source code to Santa
  Cruz Operation (SCO).
• Plan 9 is ATT's successor to UNIX
• ATT unable to market UNIX as a product, source
  code made available to Universities for use in
  Research and Education.
• Influential variant Berkeley Software
  Distributions (BSD) distributed by the Computer
  Systems Research Group, University of California
  at Berkeley
• Berkeley obtained UNIX from ATT in December of
  1974
• UNIX ported to many different architectures
• Microsoft and SCO collaborated to port UNIX to
  the Intel 8086 XENIX
• ATT purchased 20 of Sun result resulting in
  joint effort to develop SVR4
• In 1982 ATT broken up and able to market UNIX.
  Released System III in 1982 and System V the
  following year.
• System V UNIX introduced virtual memory
  (different from BSD, called regions), IPC (shared
  memory, semaphores, message queues), remote file
  sharing, shared libraries and STREAMS.

3
BSD UNIX

• 2 BSD text editor vi
• 3 BSD demand-paged virtual memory
• 4.0BSD performance improvements
• 4.1BSD job control, autoconfiguration
• 4.2/4.3BSD reliable signals, fast filesystem,
  improved networking (TCP/IP reference
  implementation), sophisticated IPC primitives
• 4.4 BSD stackable and extensible vnode
  interface, network filesystem, log-structured
  filesystem, other filesystems, POSIX support, and
  other enhancements.

4
Commercialization

• Interactive Systems first commercial (1977)
• Microsoft and SCO release XENIX (1980)
• 1982 Bill Joy left Berkeley and founded Sun
  Microsystems.
• SunOS originally based on BSD 4.2
• SunOS 5 (Solaris 2.X) was a collaborative effort
  based on System V, release 4 (SVR4).
• AIX from IBM
• HP/UX from Hewlett Packard Corporation
• ULTRIX from Digitial Equipment Corporation,
  followed by DEC OSF/1. DEC purchased by Compaq

5
CMU and MACH

• As UNIX grew, the kernel became large and complex
  - originally small and elegant.
• Mid-1980 CMU researches began development of a
  new OS
• microkernel providing a small set of essential
  services.
• Support UNIX API
• Support uniprocessor and multiprocessors
• supported distributed environments
• collection of servers provide necessary
  functionality
• New source so not encumbered by ATT licenses
• OSF1 and NextStep based on MACH

6
Standards

• Problem Incompatible programming APIs and core
  service implementations across the different UNIX
  variants
• Solution Standard set of interfaces. Several
  standards exist that define the functions, syntax
  and semantics
• IEEE POSIX (Portable Operating System Interface
  for computing environments or portable OS for
  UNIX) specifications.
• System V Interface Definition (SVID) from ATT
• X/Open Portability Guide (XPG) from the X/Open
  Consortium

7
Standards

• IEEE POSIX specifications.
• 1003.1 (1990) system interfaces and headers
• 1003.1b (1993) realtime extensions
• 1003.1c (1995) threads extensions (pthreads)
• 1003.1 (1996) includes 1003.1 (1990), 1003.1b
  (1993), 1003.1c (1995), 1003.1i (1995)
• 1003.1g protocol independent interfaces
• 1003.2 (1992) Shell and utilities
• 1003.2a (1992) Interactive shell and utilities
• SVID SVID4 is for SVR4
• defines the System V programming interface. Must
  be conformant to call an OS SVR5. ATT published
  the System V Verification Suite (SVVS)

8
Standards

• XPG - formed to develop the Common Applications
  Environment (CAE).
• XPG3 - Superset of POSIX.1 1998 and utilities
  from SVID3
• XPG4 - Superset of POSIX.1 1990, POSIX.2 (1992),
  POSIX.2 (1992), and extension from XPG3
• XPG4v2 (SUS or single UNIX specification or spc
  1170 or UNIX 95) - superset of XPG4 with widely
  used BSD interfaces
• XNS4 (X/Open Network Services) - sockets and XTI
  extensions
• SUSV2 (UNIX98)- superset of SUS, POSIX.1b (1993),
  POSIX.1c (1996) and ISO/IEC 9899 (C standard).
  Includes CDE specification
• XNS5 - superset of LP64 - clean derivative of XNS4

9
Traditional Kernel Architecture
execution environment
application
trap
libraries
user
System call interface
kernel
File subsystem
IPC
System Services
Process control subsystem
Buffer cache
scheduler
memory
hardware
10
Basic Concepts and Terminology

• Two privilege levels, or modes user and system
• System (Interrupt) versus process context
• Kernel space protected
• operates with system level privileges
• requires special instruction sequence to change
  from user to kernel mode
• reentrant
• Each process has own "protected" address space
• Per process state (u_area) in kernel space
• All process share same kernel space
• Per process kernel stack (since kernel is
  re-entrant)

11
Mode, Space and Context
Privileged
user
kernel
mode
context
process
Application (user code)
System calls Exceptions
kernel
X not allowed
Interrupts System tasks
system space
UNIX uses only two privilege levels
12
Overview

• Process
• compete for system resources
• blocks if can not acquire a resource
• Kernel Trusted software component, interfaces
  with hardware, implements system abstractions and
  interfaces
• manages resource - allocate resource to processes
  "optimally"
• implements policy for resource allocation
  (sharing)
• provide high-level abstract interface to
  resources
• centralized management simplifies synchronization
  and error recovery

13
The Process

• Fundamental abstraction
• Executes a sequence of instructions
• Competes for resources
• Address space - memory locations accessible to
  process
• virtual address space memory, disk, swap or
  remote devices
• System provides features of a virtual machine
• shared resources (I/O, CPU etc)
• dedicated registers and memory
• Most created by fork or vfork
• well defined hierarchy one parent and zero or
  more child processes. The init process is at the
  root of this tree
• different programs may be run during the life of
  a process by calling exec
• processes typically terminate by calling exit

14
Process states
fork
USER RUNNING
INITIAL IDLE
system call, interrupt
return
fork
swtch()
exit
KERNEL RUNNING
ZOMBIE
RUNNABLE
swtch()
wait
sleep()
continue
stop
wakeup
stop
stop
STOPPED
ASLEEP
continue
wakeup
STOPPED ASLEEP
15
Process Context

• Address Space
• text, data, stack, shared memory ...
• Control information (u area, proc)
• u area, proc structure, maps
• kernel stack
• Address translation maps
• Credentials
• user and group ids
• Environment variables
• variablevalue
• typically stored at bottom of stack
• Hardware context
• program counter
• stack pointer
• processor status word
• memory management registers
• FPU registers
• Machine registers saved in u area's process
  control block (PCB) during context switch to
  another process.

16
Process Address Space (one approach)
0xffffffff
Kernel address space
0x7fffffff
17
Big Picture Another look
kernel memory
18
Process Control Information

• U area.
• Part of user space (above stack).
• typically mapped to a fixed address.
• contains info needed while running.
• Can be swapped
• Proc
• contains info needed when not running.
• Not swapped out.
• traditionally fixed size table

19
U area and Proc structures

• U Area
• PCB - HW context
• pointer to proc
• real/effective ids
• args, return values or errors to current syscall.
• Signal info
• file descriptor table
• controlling terminal vnode

• Proc structure
• process ID and group
• u area pointer
• process state
• queue pointers - scheduler, sleep, etc.
• Priority
• memory management info
• flags

20
User Credentials

• Every user is assigned a unique user id (uid) and
  group id (gid).
• Superuser (root) has uid 0 and gid 0
• every process both a real and effective pair of
  Ids.
• effective id gt file creation and access,
• real id gt real owner of process. Used when
  sending signals.
• Senders real or effective id must equal receivers
  real id

21
The Kernel

• program that runs directly on the hardware
• loaded at boot time and initializes system,
• creates some initial system processes.
• remains in memory and manages the system
• Resource manager/mediator
• Time share (time-slice) the CPU,
• coordinate access to peripherals,
• manage virtual memory.
• Synchronization primitives.
• Well defined entry points
• syscalls, exceptions or interrupts.
• Performs privileged operations.

22
Entry into Kernel

• Synchronous - kernel performs work on behalf of
  the process
• System call interface (UNIX API) central
  component of the UNIX API
• Hardware exceptions - unusual action of process
• Asynchronous - kernel performs tasks that are
  possibly unrelated to current process.
• Hardware interrupts - devices assert hardware
  interrupt mechanism to notify kernel of events
  which require attention (I/O completion, status
  change, real-time clock etc)
• System processes - scheduled by OS
• swapper and pagedaemon.

23
Trap or Interrupt Processing

• Hardware switches to kernel mode, using the
  per/process kernel stack.
• HW saves PC, Status word and possibly other state
  on kernel stack.
• Assembly routine saves any other information
  necessary and dispatches event.
• On completion a return from interrupt (RFI)
  instruction is performed.

24
Interrupt handling

• Asynchronous event such as disk I/O, network
  packet reception or clock tick.
• Must be serviced in system context.
• Must not block.
• This does take CPU time away from the currently
  running process
• Multiple interrupt priority levels (ipl)
  traditionally 0-7.
• User processes and kernel operate at the lowest
  ipl level.
• Higher level Interrupts can preempt lower
  priority ones.
• The current ipl level may be set while executing
  critical code.

25
Exception handling

• Synchronous to the currently running process for
  example divide by zero or invalid memory access.
• Must run the the current processes context.
• An Interrupt can occur during the processing of
  an exception or trap.

26
Software Interrupts

• Interrupts typically have the highest priority in
  a system.
• Software interrupts are assigned priorities above
  that of user processes but below that of
  interrupts.
• Software interrupts are typically implemented in
  software.
• Examples are callout queue processing and network
  packet processing.

27
System Call Interface

• Implemented as an assembly language stub.
• Trap into kernel dispatch routine which may save
  additional state and invoke the high-level system
  call.
• User privileges are verified and any data is
  copied into kernel (copyin()).
• On return, copy out any user data (copyout()),
  check for an Asynchronous System Trap, or AST
  (signal, preemption, etc).

28
Synchronization

• Kernel is re-entrant.
• Only one process may be active within the kernel
  at any given time (others are blocked).
• Nonpreemptive.
• Blocking operations
• Masking interrupts

29
Blocking Operations

• When resource is unavailable (possibly locked),
  process sets flag and calls sleep()
• sleep places process on a blocked queue, sets
  state to asleep and calls swtch()
• when resource released wakeup() is called
• all processes sleeping on this resource are
  woken state set to runnable and placed on the
  runnable queue(s).
• When running, process must verify resource is
  available.
• Why?

30
Process Scheduling

• Preemptive round-robin scheduling
• fixed time quantum
• priority is adjusted by nice value and usage
  factor.
• the more cpu time, the lower the priority gt
  prevents starvation
• Processes sleeping in the kernel are assigned a
  kernel priority which is higher than user
  priorities.
• priority boost may depend on the reason for sleep
• may give boost to interactive, I/O bound
  processes
• Kernel 0-49, user 50-127.

31
Signals Event notification

• Asynchronous events and exceptions
• Signal generation using the kill() system call
• Default operation or user specific handlers
• sets a bit in the pending signals mask in the
  proc structure
• All signals are handled before return to normal
  processing
• System calls can be restarted

32
Process creation

• Fork
• create a new process
• copy of parents virtual memory
• parent return value is childs PID
• child return value is 0
• exec
• overwrite with new program

33
Process Termination

• exit() is called
• closes open files
• releases other resources
• saves resource usage statistics and exit status
  in proc structure
• sends SIGCHLD signal to parent
• wakes parent
• any children are inherited by the init processes
• calls swtch()
• Process is in zombie state
• parent collects, via wait(), exit status and usage