Chapter 5: CPU Scheduling

1
Chapter 5 CPU Scheduling
2
Chapter 5 CPU Scheduling

• Basic Concepts
• Scheduling Criteria
• Scheduling Algorithms
• Multiple-Processor Scheduling
• Real-Time Scheduling
• Algorithm Evaluation
• Operating Systems Examples
• Thread Scheduling
• Java Thread Scheduling

3
Process Scheduling Queues
Job Pool
Job Scheduler (Long-Term Scheduler)
Job Queue
CPU
Done
CPU Scheduler (Short-Term Scheduler)
Ready Queue
I/O Wait Queue (wait after I/O is done, to get
back into ready queue)
Device Queue (wait for I/O to complete)
Partially executed swapped out process
Medium-Term Scheduler
4
Basic Concepts

• Maximum CPU utilization obtained with
  multiprogramming
• Multiprogramming is accomplished through
• A pool of jobs
• CPU scheduling
• Many CPU scheduling algorithms depend on the
  concept of a CPU-I/O Burst Cycle occurring
  within processes
• CPUI/O Burst Cycle Process execution consists
  of a cycle of alternating CPU execution and I/O
  wait.
• CPU burst distribution most CPU bursts are
  short
• This makes scheduling allthe more important (and
  useful)

5
Alternating Sequence of CPU And I/O Bursts

• Example of CPU - I/O Burst Cycle
• Process execution consists of a cycle of
  alternating CPU execution and I/O wait.
• Take advantage of this alternating sequence to
  schedule processes in the CPU

6
Why Schedule the CPU?

• A typical process runs as alternating CPU and IO
  bursts
• Consider two processes, P1 and P2
• Each runs by executing for 1 sec. and then
  waiting 1 sec. 60 times
• If run P1 to completion and then P2, it takes 4
  minutes total
• 2 minutes total CPU time 2 minutes wasted time
• 50 CPU utilization
• If interleave them, it takes 2 minutes and 1
  second
• 2 minutes for P1, 2 minutes for P2 no wasted
  time
• 100 CPU utilization

P1
P2
Consecutive
P1
P2
Interleaved
7
CPU Scheduler

• Selects from among the processes in memory that
  are ready to execute, and allocates the CPU to
  one of them.
• CPU scheduling decisions take place when a
  process
• 1. Switches from running to waiting state (e.g.,
  for I/O).
• 2. Switches from running to ready state.
• Higher priority event needs attention
• Higher priority process needs to run
• Interrupted by timer
• 3. Switches from waiting to ready.
• I/O completed process wakes up
• 4. Terminates.
• Scheduling under 1 and 4 is nonpreemptive.
• Process keeps CPU until goes into wait or is
  terminated
• All other scheduling is preemptive.
• CPU can be taken away from currently running
  process (actually there are cases where 1 can
  also be preemptive)

8
Dispatcher

• Dispatcher module gives control of the CPU to the
  process selected by the short-term scheduler
  this involves
• Switching context
• Interrupt switches to system mode
• Save state of old process
• Load state of new process
• Switching back to user mode
• Restarting execution of the user program at the
  point where the program had been stopped
  previously
• Dispatch latency time it takes for the
  dispatcher to stop one process and start another
  running (to do all of the above).
• Must be very fast
• May be combined with short term scheduler
• Because short term scheduler is also code that
  must run

9
Scheduling Criteria

• CPU utilization amount of time is CPU busy
  compared with time it is waiting
• Throughput of processes that complete their
  execution per unit of time
• Turnaround time amount of time to execute a
  particular process (time it takes a job to run
  from start to finish)
• Waiting time total amount of time (not just
  initial time) that a process has been waiting in
  the ready queue neither processing nor doing I/O
• Response time amount of time it takes from when
  a request is submitted until the first response
  is produced, i.e., until response begins, but is
  not necessarily complete
• Fairness whether all processes eventually have
  a chance to run in the CPU, or some processes are
  passed over enough that they starve

10
Optimization Criteria

• Maximize CPU utilization
• Maximize throughput
• Minimize turnaround time
• Minimize waiting time
• Minimize response time
• Guarantee fairness
• Evaluate CPU scheduling algorithms using these
  criteria
• Study principles by beginning with 1 CPU burst
  per process all request service at roughly the
  same time
• First look at simplest First Come First Served

11
First-Come, First-Served (FCFS) Scheduling

• Process Burst Time
• P1 24
• P2 3
• P3 3
• Suppose that the processes arrive in the order
  P1 , P2 , P3
• The Gantt Chart for the scheduling and execution
  is
• Waiting time for P1 0 P2 24 P3 27
• Average waiting time (0 24 27)/3 17
• Average turnaround time (24 27 30)/3 27
  (Note all arrived at the same time, but in a
  particular order)

12
FCFS Scheduling (Cont.)

• Suppose that the processes arrive in a different
  order
• P2 , P3 , P1 (i.e.,
  3,3,24).
• The Gantt chart for the schedule is
• Waiting time for P1 6 P2 0 P3 3
• Average waiting time (6 0 3)/3 3
• Average turnaround time (3 6 30)/3 13
• Much better than previous case but can't
  control order
• NOTE poor CPU utilization if poor mix of IO/CPU
  bound jobs
• Particularly troublesome for multiuser systems
• Seems like might be advantageous try to get
  shortest jobs to run first (control ordering of
  jobs other than FCFS)

13
Shortest-Job-First (SJF) Scheduling

• Associate with each process the length of its
  next CPU burst.
• Using those length values, look at the jobs in
  the job queue and select the process that has the
  shortest upcoming CPU burst as process that will
  be allowed to run next
• Use FCFS to break ties
• Two versions of SJF
• Nonpreemptive once CPU given to a process, it
  cannot be preempted until it completes its CPU
  burst.
• Preemptive if a new process arrives with a CPU
  burst length less than remaining time of the
  currently executing process, preempt. This
  scheme is known as the Shortest-Remaining-Time-Fi
  rst (SRTF).
• SJF is optimal it is BEST in minimizing average
  waiting time for a given set of processes.
• BUT it is not best for response time

14
Example of Non-Preemptive SJF

• Note this is the first slide to include
  the concept of arrival time

• Process Arrival Time Burst Time
• P1
  0.0 7
• P2
  2.0 4
• P3
  4.0 1
• P4
  5.0 4
• SJF (non-preemptive)
• Average waiting time (0 6 3 7)/4 4

15
Example of Preemptive SJF (SRTF)

• Process Arrival Time Burst Time
• P1
  0.0 7
• P2
  2.0 4
• P3
  4.0 1
• P4
  5.0 4
• SJF (preemptive)
• Average waiting time (9 1 0 2)/4 3

16
Determining Length of Next CPU Burst

• PROBLEM how do we know length of next CPU burst
  ?
• ANSWER we cant !!!
• Can only estimate the length.
• Can be done by using the length of previous CPU
  bursts, using exponential averaging.
• t n actual length of nth CPU burst (the
  most recent burst)
• ?n1 predicted value for the next CPU burst
• ? weight of past history, where 0 lt ? lt 1
  (can adjust)
• Definition
• ?n1 ? tn(1 - ?) ? ?n

17
Prediction of the Length of the Next CPU Burst

• Example where ? .5 and ?0 10

18
Examples of Exponential Averaging

• If ? 0
• ?n1 ?n
• Recent history does not count.
• If ? 1
• ?n1 tn
• Only the actual last CPU burst counts.
• If the formula is expanded, the result is
• ?n1 ? tn (1 - ?) ? tn-1 (1 - ? )j ? tn-j
  (1 - ? )n1 ?0
• Since both ? and (1 - ?) are less than or equal
  to 1, each successive term has less weight than
  its predecessor.
• NOTE these calculations are very expensive
  (i.e., S L O W)
• So not very practical because scheduler must be
  very fast

19
Priority Scheduling

• Associate a priority number (integer) with each
  process
• The CPU is allocated to the process with the
  highest priority (often smallest integer ?
  highest priority).
• FCFS is priority scheduling where all jobs have
  the same priority
• SJF is priority scheduling where priority is the
  predicted length of the next CPU burst
• Often priority based on job characteristics,
  purpose, etc.
• Quite common often used in combination with
  other algorithms
• Can be set internally (by OS) or externally (by
  user)
• Can be preemptive or nonpreemptive
• Problem Starvation low priority processes may
  never get a chance to execute.
• Solution Aging as time progresses, increase
  the priority of the process.

20
Round Robin (RR)

• Each process gets a small unit of CPU time (time
  quantum), usually 10-100 milliseconds. After
  this time has elapsed, the process is preempted
  and added to the end of the ready queue.
• If there are n processes in the ready queue and
  the time quantum is q, then each process gets
  1/nth of the CPU time in chunks of at most q time
  units at once.
• Guaranteed no process waits more than (n-1)q time
  units.
• Length of time slice important in performance
• q too large ? FCFS
• q too small ? waste much CPU time in context
  switches
• q must be large with respect to context switch
  time
• To minimize turnaround time, q should be slightly
  longer than average CPU burst length
• But may not provide best response time
  tradeoffs
• May want to set different time quanta for
  different types / classes of jobs

21
Process Scheduling Queues
Job Pool
Job Scheduler (Long-Term Scheduler)
Job Queue
CPU
Done
CPU Scheduler (Short-Term Scheduler)
Ready Queue
I/O Wait Queue (wait after I/O is done, to get
back into ready queue)
Device Queue (wait for I/O to complete)
Time Slice End (dont wait here, just go back to
ready queue)
Partially executed swapped out process
Medium-Term Scheduler
22
Example of RR with Time Quantum 20

• Process Burst Time
• P1 53
• P2 17
• P3 68
• P4 24
• The Gantt chart is
• Typically, higher average turnaround than SJF,
  but better response time (each process gets
  initial response sooner)
• NOTE wait time for RR includes ALL the time a
  process waits in the ready queue, not just the
  time the process waits before it first runs
• Sometimes mix with other algorithms, such as
  priority

Current contentsof ready queue
23
Time Quantum and Context Switch Time
Note if q is small, will spend / waste more
time in context switches. And context
switches today also mean reloading cache lines,
TLBs, page tables, etc
24
Turnaround Time Varies With The Time Quantum
Note changing the value of q does not have a
deterministic effect on average turnaround
time Sometimes ATT increases, and sometimes it
decreases Depends on the mix of jobs and their
characteristics
25
Multilevel Queues

• Ready queue is divided into separate queues
  processes are divided into several classes, for
  example foreground (interactive) background
  (batch)
• Each queue has its own scheduling algorithm,
  e.g., foreground RR background FCFS
• Scheduling must be done between the queues.
• Fixed priority scheduling (i.e., serve all from
  higher priority queues before lower priority
  queues)
• Possibility of starvation.
• Time slices among queues each queue gets a
  certain amount of CPU time which it can schedule
  amongst its processes i.e., 80 to foreground in
  RR, 20 to background in FCFS
• Can be many levels of queues

26
Multilevel Queue Scheduling
27
Multilevel Feedback Queues

• A process can move between the various queues
• Aging can be implemented this way.
• Multilevel-feedback-queue scheduler defined by
  the following parameters
• Number of queues
• Scheduling algorithms for each queue
• Rules to decide when to upgrade a process
• Rules to decide when to demote a process
• Rules to decide in which queue a process is
  assigned to start

28
Example of Multilevel Feedback Queue

• Three queues
• Q0 time quantum 8 milliseconds
• Enter in FCFS order, but preempt and move if
  doesnt complete within q
• Q1 time quantum 16 milliseconds
• Enter in FCFS order, but preempt and move if
  doesnt complete within q
• Q2 FCFS runs until goes into I/O wait
• Scheduling
• A new job enters queue Q0 which is served FCFS.
  When it gains CPU, job receives 8 milliseconds.
• If it does not finish in 8 milliseconds, job is
  moved to queue Q1.
• At Q1 job is again served FCFS and receives 16
  additional milliseconds.
• If it still does not complete, it is preempted
  and moved to queue Q2.
• Stays in Q2 until it finishes its CPU burst, then
  goes back to Q0

29
Multilevel Feedback Queues
30
Multiple-Processor Scheduling

• CPU scheduling more complex when multiple CPUs
• Loosely coupled don't share memory
• Tightly coupled share memory
• Partly coupled have more affinity for some
  memory than other
• Homogeneous processors within a multiprocessor.
• Single ready queue for entire system
• Any processor can be used to run any job on
  system
• Potential impacts to caches (unless single,
  shared cache then probably too small)
• Load sharing
• Separate ready queue for each processor
• Can starve one processor (i.e., waste processor
  resources)
• Processor affinity
• Processes have an affinity for a particular
  processor, can be hard or soft
• Asymmetric multiprocessing master/slave
  relationship
• Only one processor accesses the system data
  structures, alleviating the need for data sharing
  but can become bottleneck.

31
NUMA and CPU Scheduling
32
Multicore Processors

• Recent trend to place multiple processor cores on
  same physical chip
• Faster and consume less power
• Multiple cores on same chip add another dimension
  of complexity
• Because often share parts of cache lines or
  on-chip memory
• Multiple threads per core also growing
• Takes advantage of memory stall to make progress
  on another thread while memory retrieve happens
• Some processors also support simultaneous
  multi-threading e.g., Power 5, Power 6, etc.

33
Multithreaded Multicore System
34
Real-Time Scheduling

• Hard real-time systems required to complete a
  critical task within a guaranteed amount of
  time.
• Soft real-time systems require that critical
  processes receive priority over less fortunate
  ones.
• Must consider time required to dispatch new
  process, if must respond to critical event in
  real time
• Time to process interrupt
• Dispatch latency
• Find proper process to dispatch
• Switch context to new process
• New process runs to handle the event
• Response to event completed

35
Dispatch Latency
36
Algorithm Evaluation

• How to determine which scheduling algorithm to
  use
• Or whether the implementation of an algorithm can
  be improved?
• Deterministic modelling / simulation
• Define (write) the model(s)
• Take a particular predetermined workload and
  determine the performance of each algorithm for
  that workload.
• Random distributions
• Standard workloads / benchmarks
• Trace measurements from actual runs
• Queuing models
• Complex mathematical analysis
• Implement a prototype
• Then use inputs / workloads on prototype
• Choice influenced by other cost required for
  evaluation or implementation

37
Evaluation of CPU Schedulers by Simulation
38
Process Scheduling Queues
Job Pool
Job Scheduler (Long-Term Scheduler)
Job Queue
Last ---gt Middle-gt First
---gt Middle-gt ABOVE IS ORDER OF ACCESS
TO READY QUEUE AT TIME SLICE END
CPU
Done
CPU Scheduler (Short-Term Scheduler)
Ready Queue
I/O Wait Queue (wait after I/O is done, to get
back into ready queue)
Device Queue (wait for I/O to complete)
Time Slice End (dont wait, just pass through)
Wakeup Wait Queue (wait after wake up, to get
back into ready queue)
General Wait Queue (sleep wait for interrupt)
Medium-Term Scheduler
Partially executed swapped out process
OTHER
39
Solaris Scheduling
40
Windows XP Priorities
41
Linux Scheduling

• Two algorithms time-sharing and real-time
• Time-sharing
• Prioritized credit-based process with most
  credits is scheduled next
• Credit subtracted when timer interrupt occurs
• When credit 0, another process chosen
• When all processes have credit 0, recrediting
  occurs
• Based on factors including priority and history
• Real-time
• Soft real-time
• Posix.1b compliant two classes
• FCFS and RR
• Highest priority process always runs first

42
Linux Priorities and Time-slice length
43
List of Tasks Indexed According to Priorities
44
Thread Scheduling

• Local Scheduling
• The threads library (often at the user level)
  decides which thread to put onto an available
  process
• Global Scheduling
• The kernel decides which kernel thread to run
  next
• We will discuss threads in future sessions

45
Java Thread Scheduling

• Each JVM is essentially one process
• So sub-processes within it are really threads
• And the scheduling described here is really
  thread scheduling
• JVM Uses a Preemptive, Priority-Based Scheduling
  Algorithm
• FIFO Queue is used if there are multiple threads
  with the same priority
• i.e., if threads are same priority, then becomes
  FCFS within that group of threads

46
Java Thread Scheduling (cont)

• JVM Schedules a Thread to Run When
• The currently running thread exits the runnable
  state
• Goes into a wait, exits, time-slice end, etc.
• --- OR ---
• A higher priority thread enters the runnable
  state
• Then the higher priority thread preempts the
  current thread
• Note the Java Specification does not specify
  whether threads are time-sliced or not
• It is up to the implementation of each particular
  JVM
• Also, runtime characteristics are very dependent
  on processor speed and memory availability

47
Time-Slicing

• Since the JVM doesnt ensure time-slicing, the
  yield() method may be used to force a thread to
  give up the CPU (cause it to be pre-empted)
• while (true)
• // perform CPU-intensive task
• . . .
• Thread.yield()
• This yields control to another thread of equal
  priority

48
Thread Priorities

• Priority Comment
• Thread.MIN_PRIORITY Minimum Thread Priority
• Thread.MAX_PRIORITY Maximum Thread Priority
• Thread.NORM_PRIORITY Default Thread Priority
• Priorities May Be Set Using setPriority() method
• setPriority(Thread.NORM_PRIORITY 2)

49
End of Chapter 5