CSE522 Advanced Operating Systems Midterm Review

1
CSE522Advanced Operating SystemsMidterm Review

• Fred Kuhns
• (fredk_at_cse.wustl.edu, http//www.arl.wustl.edu/fr
  edk)
• Applied Research Laboratory
• Department of Computer Science and Engineering
• Washington University in St. Louis

2
Terminology for WFQ

• Generalized Processor Sharing (GPS)
• provides server an infinitesimally small time
  slice of length proportional to server size.
• Not practical
• Density of a sporadic job Ji
• ei maximum execution time
• ri release time
• di deadline, density ei/(di-ri)
• active in feasible interval (ri, di
• Instantaneous utilization of a sporadic task
• ui maxj(eij/pij)
• independent, preemptable sporadic jobs are
  schedulable by EDF if total density lt 1

3
Fairness and Starvation

• Fairness
• Consider a system of n servers.
• let wi(t1,t2) equal the processor time used by
  server Si (i.e. allocated processor time) in
  interval I (t1, t2 for t2 gt t1.
• Normalized service wi(t1,t2)/ui fair if the
  normalized service of all servers differ by no
  more than a fairness threshold FRgt0.
• Goal is to provide fair access to processor while
  permitting jobs to use an idle processor
• Non-preemptive version used for network packet
  scheduling
• Jobs are assigned a finish number that represents
  the round in which its deadline occurs (think of
  this as virtual time).

4
Weighted fair queuing preemptive

• FN system finish number, Ub backlogged
  utilization
• Scheduling
• A server is eligible when it has budget and an
  assigned finish number
• Priority assigned to eligible servers based on
  finish number Server with smallest finish time
  has highest priority

• Consumption
• Consume only when running
• Initialization
• I1 Idle system, set FN0, Ub0, t-1 0 ei,fni
  0 for all i.
• I2 first job arrives to an Idle system for some
  server FQk set t-1 t, Ub uk, budget ek e
  and fnk e/uk
• Updating current finish number and replenishment
• R1 job arrives at an idle server FQi, then 1
  increment system FN FN (t-t-1)/Ub 2 set t-1
  t and increment Ub ui 3 update FQi ei
  e fni FN e/ui and place in ready queue
• R2 FQi completes a job, remove it from the
  queue if still backlogged ei e fni
  e/ui else (idle) FN (t - t-1)/Ub, set t-1
  t decrement Ub - ui

5
Protection

• Two parts to protection
• authentication
• authorization
• If an intermediate entity can gain access to
  information then Cryptography is used to protect
  sensitive data
• Protection mechanisms within OS detect and
  contain errors or generally any policy violations
• Confinement
• Access rights allocation
• Common considerations
• Accounting and audit trails
• Granularity of protection (fine or course)
• Ability to vary access rights over time
• Well known Principles
• Principle of least privilege
• Need to know principle
• Resource access at the instruction (hardware) or
  system (abstract) level.

6
Protection Model

• Computer System is a collection of passive and
  active entities.
• Passive entities correspond to resources and
  other secure entities. Known as objects or
  protection objects.
• Active entities processes and threads acting on
  the behalf of users.
• Access right object, operation(s) ordered pair
  ltobject,opsgt, for example ltfileX,read,writegt
• A process can also be considered an object and
  thus have access rights associated with it
• A particular set of access rights held by a
  process at any point in time is known as its
  Protection Domain
• Subject a process executing in a specific
  protection domain.
• Domains access rights may vary over time
• Specify objects that may be accessed (authorized
  objects)
• Specify operations that may be performed on the
  authorized objects
• Process-Domain associations may be static or
  dynamic
• Defining domains
• per user domain per process domain per
  procedure domain
• Protection system composed of a set of objects
  and set of subjects, and a set of rules
  specifying the protection policy.
• Protection state defines the accessibility of
  objects by subjects
• System guarantees protection state is checked for
  each access of an object O by a subject S.
• Internal protection state can be changed only
  according to a set of rules that implement the
  external security policy.

7
Protection Access matrix

• Protection state represented by an access matrix
  A.
• row for each subject column for each object
• An array element AS,O represents a set of
  access rights. Subject S requests operation x on
  object O.
• The protection system authenticates S
• generates the 3-tuple ltS,x,Ogt.
• Monitor for object O checks entry AS,O.
• if x ? AS,Othen access is permitted otherwise
  it is denied.
• If domain switching is supported then
• switch from domain Si to Sk IFF switch ? Ai,k
• Example Rules
• Copy right, denoted by an , three variations
• Copy copy right R from Ai,k to Am,k
  (subject i to subject m, object k)
• Transfer a right is copied from Ai,k to
  accessm,k then removed from Ai,k
• Limited copy only the right R (not R) is copied
  from Ai,k to Am,k
• Owner right If Ai,k includes the owner right
  then a process executing in domain Si may add or
  remove any right in any entry in column j.
• Control right (applies only to domain objects)
  if Ai,k includes a control right then a process
  executing in Si can modify row k.
• Access list All nonempty entries in a column
  form a list associated with the corresponding
  object
• For each object there is a list of ltdomain,
  rightsgt
• Capabilities list All nonempty entries for a row
  with the corresponding domain

8
Protection
9
Sandboxing

• Review Mikes slides on OKE and Singularity.
• Issues
• safety, liveness, performance
• Techniques
• Safe Execution Environment/Virtual machines VMM,
  Java, KoffeOS, packet filters
• Language based (type safety) Singularity, OKE
  (Cyclone), mobile code (STP), SPIN
• Proofs proof carrying code (PCC)
• Software Fault Isolation (SFI) VINO
• Hardware Fault Isolation (HFI) kernel plugins,
  Denali, XEN, Exokernel, Palladium, NOOKS.
• Focus on two approaches
• kernel extension to support simple interpreted
  environment (packet filtering) with protocols
  implemented in user space
• sandbox for in-kernel protocol implementations
  using a type safe language and run-time support.
  In the sprit of OKE and mobile code (with
  concepts from OKE

10
Memory
11
Memory

• Primary Memory Design Requirements
• Minimize access time hardware and software
  requirement
• Maximize available memory using physical and
  virtual memory techniques
• Cost-effective limited to a small percentage
  total
• Memory Manager Functions
• Allocate memory to processes
• Map process address space to allocated memory
• Minimize access times while limiting memory
  requirements
• Process address space
• Compiler produces relocatable object modules
• Linker combines modules into an absolute module
  (loadable module).
• addresses are relative, typically starting at a
  small offset from 0.
• Loader loads program into memory and adjusts
  addresses to produce an executable module.

12
Memory Management

• Central Component of any operating system
• Memory Partitioning schemes Fixed, Dynamic,
  Paging, Segmentation, Combination
• Placement algorithms, fragmentation, compaction
• Addressing and HW support (paging and
  segmentation)
• Relocation, Protection and Sharing
• Hierarchical layering to optimize performance and
  cost
• registers, cache, primary (main) memory,
  secondary (backing store, local disk) memory
• file servers (networked storage)
• Policies target expected memory requirements of
  processes
• consider short, medium and long term resource
  requirements
• long term admission of new processes (overall
  system requirements)
• medium term memory allocation (per process
  requirements)
• short term processor scheduling (immediate
  needs)
• Common goal optimize number of runnable process
  resident in memory and minimize overhead

13
Memory

• Programs tend to cluster memory references for
  both data and instructions. Further, this
  clustering changes slowly with time.
• Hardware and software exploit principle of
  locality.
• Temporal locality if location is referenced
  once, then it is likely to be referenced again in
  the near future.
• Spatial locality if a memory location is
  referenced then other nearby locations will be
  referenced.
• Stride-k (data) reference patterns
• visit every kth element of a contiguous vector.
• stride-1 reference patterns are very common.
• Caching

Client Host
Web Server
Disk (files)
CPU
DRAM (Primary)
page.html
page.html
4
page.html
page.html
cache
image.jpg
image.jpg
2
3
1
14
Memory management and Paging

• HW Requirements
• Protection Prevent process from changing own
  memory maps
• Residency CPU distinguishes between resident and
  non-resident pages
• Loading Load pages and restart interrupted
  program instructions
• Dirty Determine if pages have been modified
• Memory Management Unit
• Translates Virtual Addresses
• Page tables
• One for kernel addresses
• One or more for user space processes
• Page Table Entry (PTE) one per virtual page
• 32 bits - page frame, protection, valid,
  modified, referenced
• Translation Lookaside Buffer Cache of page
  translations
• Associative cache of address translations
• Hardware managed versus software managed TLB
• Entries may contain a tag identifying
  context/domain.
• Virtual address
• virtual page number offset
• Translation failure MMU raises an exception -
  page fault

MMU
Virtual address
physical address
TLB
Page tables
15
Cache

• Placement policy where new block (i.e. cache
  line) is placed
• Replacement algorithm
• determines which block to replace (LRU)
• Block size
• data unit exchanged between cache and main memory
• Associativity
• Direct mapped one-to-one mapping between cache
  lines and memory locations.
• Fully associative any line in memory can be
  cached in any cache line
• N-way set associative A line in memory can be
  stored in any of N-lines associated with the
  mapped set.
• Write policy
• hit write-through versus write-back
• miss write-allocate versus no-write-allocate

16
Virtual Memory Costs

• Space Translation tables and other data used by
  VM system reduce memory available to programs
• Time Address translation time is added to the
  cost (execution time) of each instruction.
• Asynchronous Page fault handling may result in
  page I/O operations increasing latency and
  possibly affecting unrelated processes.
• Overhead Memory management operations have been
  measured to consume up to 10 of the CPU time on
  a busy system.
• Efficiency Allocating memory in pages may result
  in internal fragmentation

17
Memory Allocation
physical page
Page-level allocator
Kernel memory Allocator
Paging system
Network buffers
Data structures
temp storage
process
Buffer cache
18
Virtual Memory

• Transparent address translation
• Support virtual memory abstraction
• Address space along with process register context
  reflects the current state
• Typically demand paged implementations
• Text and initialized data read in from executable
  file
• generally, memory objects with explicit initial
  values reside on a persistent storage device
• Page level access
• Uninitialized data are typically zero-filled
  pages
• Shared libraries from library file generally
  shared among executing processes
• Text is shared (Read-Only) while data
  (read/write) is part of programs state
• Swap Area Pages are copied to the swap device to
  free up space for running programs.
• read-only segments can reference the original
  executable file
• modifiable data, when not physically resident in
  RAM, must be stored on a device so as not to
  loose vital process state.
• Swapping plus paging for two-tiered scheme
• swapping generally refers to moving the entire
  process out of RAM while paging moves individual
  pages
• Requires a swap map to locate swapped out pages
• MMU set dirty bit for page if it has been
  modified
• maintain copy of original page on swap device
  then if in-memory page is not modified we can
  save an update operation.
• Text pages need not be backed by the swap device
  (i.e. the original file is sufficient)

19
VM

• Replacement algorithm Deciding when to reclaim a
  page, defined in terms of criteria used for
  selecting pages to reclaim
• Reference string pages referenced over time
• Fault rate page faults for some length of a
  reference string (i.e. over a period of time)
• Algorithms evaluated based on effectiveness on
  collected (real) reference strings
• Implementations usually require sample reference
  strings
• Working Set Model
• Assumes a slowing changing locality of reference
• if working set is in memory then few page faults
• A simple model is a least recently used (LRU)
  policy
• if a page has been accessed recently then
  assume it will againelse assume it will not be
  needed
• free pages not accessed recently
• Implement using an approximate set
• number of pages held versus fault rate.
• Set high and low water marks
• Most kernels implement a scheme whereby pages are
  periodically freed and placed on a free pool.
• Prepaging working set resident before scheduling
  process

20
VM Support

• Hardware
• Protection Prevent process from changing own
  memory maps
• Residency CPU distinguishes between resident and
  non-resident pages
• Loading Load pages and restart interrupted
  program instructions
• Dirty Determine if pages have been modified

21
Kernel Memory management

• Requirements
• Minimize Waste (max utilization)
• Fast since extensively used (efficient)
• Simple API similar to malloc and free.
• Properly aligned allocations for example 8 byte
  alignment
• Support cyclical and bursty usage patterns
• Interacts with paging system able to borrow
  pages from paging system if running low
• Example implementations
• Resource Map Allocator
• Simple Power-of-Two Free Lists
• The McKusick-Karels Allocator
• The Buddy System
• SVR4 Lazy Buddy Allocator
• Mach-OSF/1 Zone Allocator
• Solaris Slab Allocator

22
Virtual Machines

• Read and know the first VM lecture materials
  (04/07, virtual_machines.ppt)
• understand material in cited reference papers
  the architecture of virtual machines and
  Virtual machine monitors current technology and
  future trends
• Thoroughly read, understand and be able to
  critically asses the paper Container-based
  operating sytem virtualization a scalable,
  high-performance alternative to hypervisors