Vassal: Loadable Scheduler Support for Multi-Policy Scheduling

1
Vassal Loadable Scheduler Support for
Multi-Policy Scheduling

• George M. Candea, Oracle Corporation
• Michael B. Jones, Microsoft Research

2
The Problem

• OS multiplexes CPU among tasks
• OS not always aware of scheduling requirements
• No algorithm is good enough for all task mixes
• Compromise Hardcode set of scheduling policies
  into the operating system
• Desirable Dynamically extensible set of policies

3
Overview of Vassal

• Tasks can use custom scheduling policies
• Custom schedulers
• are special Windows NT drivers
• coexist with the Windows NT scheduler
• have negligible impact on global performance
• In current prototype, one external scheduler
  loaded at once

4
Outline

• Motivation and Overview
• Windows NT Scheduling
• Vassal Design and Implementation
• Sample Scheduler
• Results
• Conclusions

5
Windows NT Scheduling

• Schedulable unit thread
• Priority-based thread scheduling
• Two policies, in distinct priority ranges
• Variable (dynamic priority round-robin)
• Real-Time (fixed priority round-robin)

6
NT Scheduling Precedence

• 1. Interrupts
• 2. Deferred Procedure Calls (DPCs)
• 3. Threads

• Not all time gets scheduled based on priorities
• Scheduling predictability is limited

7
NT Scheduling Events

• Scheduling decisions triggered by
• End of thread quantum
• Priority or affinity changes
• Transition to Wait state
• Wakeups

8
Windows NT Timers

• Hardware Abstraction Layer (HAL) provides kernel
  with a periodic timer
• Resolution selectable from 1 to 15 ms (default
  10 or 15 ms)
• Not all HALs implement all values
• MP HAL provides 1, 2, 4, 8, 16 ms
• Some HALs just implement 10 ms

9
Outline

• Motivation and Overview
• Windows NT Scheduling
• Vassal Design and Implementation
• Sample Scheduler
• Results
• Conclusions

10
Separate Policy from Mechanism

• NT scheduler thread dispatcher with scheduling
  policies interspersed
• Vassal separate scheduling and dispatching
  modules
• Schedulers policy modules that decide which
  threads to run
• Dispatcher runs threads selected by schedulers

11
Details of Present Prototype

• Standard NT policies remain in kernel
• Schedulers are in a hierarchy
• Give loaded scheduler first choice
• Ask native scheduler if loaded scheduler makes no
  choice
• Could easily support deeper hierarchy
• By default, threads use NT policies

12
Vassal Entities

• Schedulers
• Register decision making routines with dispatcher
• Dispatcher
• Invokes decision routines when scheduling events
  occur
• Threads
• Communicate with schedulers to request services

13
Vassal Architecture
Hardware Abstraction Layer (HAL)
Drivers
14
Interface Modifications

• Extend driver interface for schedulers
• RegisterScheduler
• SetSchedulerEvent
• Extend syscall interface for threads
• MessageToScheduler

15
Registering a Scheduler

• RegisterScheduler (scheduler identifier,
• decision making routine,
• message dispatcher routine)

• Invoked by driver at initialization time
• Dispatcher checks for conflicts and updates
  scheduler hierarchy
• Dispatcher queries scheduler by invoking the
  decision making routine

16
Communicating with a Scheduler

• MessageToScheduler (scheduler identifier,
• message buffer,
• message length)

• Thread sends message to specific scheduler
• Corresponding schedulers message dispatcher
  extracts message from buffer and responds

17
Precisely Timed Events

• SetSchedulerEvent (scheduler identifier,
• absolute time value)

• Scheduler requests control of CPU at certain
  absolute time
• Dispatcher invokes schedulers decision routine
  at specified time

18
Vassal Interfaces
Application Thread
User space
MessageToScheduler
External Scheduler
Thread Dispatcher
RegisterScheduler
Kernel
SetSchedulerEvent
Hardware Abstraction Layer (HAL)
Drivers
19
Outline

• Motivation and Overview
• Windows NT Scheduling
• Vassal Design and Implementation
• Sample Scheduler
• Results
• Conclusions

20
Sample Real-Time Scheduler

• Allows threads to get scheduled at
  application-specified time instances
• Demonstrates potential for more interesting
  time-based policies

21
Using The Real-Time Scheduler

• Tell system to use the real-time scheduler
• status MessageToScheduler (RT_scheduler,
  JOIN)
• if status ! SUCCESS
• error (Could not join R/T scheduling class.)
• We want one iteration every 1ms
• while TRUE do
• status MessageToScheduler (RT_scheduler,
• SET, wakeup_time)
• wakeup_time wakeup_time 1 msec

22
Execution of Sample Code
T
23
Outline

• Motivation and Overview
• Windows NT Scheduling
• Vassal Design and Implementation
• Sample Scheduler
• Results
• Conclusions

24
Windows NT Kernel Changes

• Added 188 lines of C code
• Added 61 assembly instructions
• Replaced 6 assembly instructions

25
Context Switch Times
Context switch times on original and modified
systems (µs, P-133)

• No significant difference when external
  schedulers not loaded
• 8 overhead on untuned prototype when using
  loaded schedulers

26
Writing a Scheduler

• Proof-of-concept real-time scheduler
• 116 lines of C code
• No assembly language
• Only need to code the policy

27
Periodic Wakeup Times
Wakeup times using multimedia timers on vanilla
system and sample scheduler on Vassal (µs,
P-133). Desired value is 1000.

• No early wakeups when using our scheduler
• Predictability significantly improved
• Believe late samples due to unscheduled activities

28
Outline

• Motivation and Overview
• Windows NT Scheduling
• Vassal Design and Implementation
• Sample Scheduler
• Results
• Conclusions

29
Vassal Take-Home

• Demonstrates viability and effectiveness of
  loadable schedulers
• Frees OS from anticipating all possible
  application scheduling requirements
• Encourages scheduling research by making it easy
  to develop and test new policies
• Insignificant performance impact

30
Limitations and Future Work

• Timing precision limited by HAL
• Predictability limited by interrupts and DPC
  activity
• Only one loaded scheduler supported
• External schedulers not fully MP aware

31
Related Work

• Solaris scheduler class drivers
• Must map scheduling decisions onto global thread
  priority space
• Extensible OS work
• Spin, Exokernel, Vino
• Hierarchical schedulers
• Utah CPU inheritance scheduling
• UIUC Windows NT soft real-time scheduler

32
For More Information...

• http//pdos.lcs.mit.edu/candea/research.html
• http//research.microsoft.com/mbj/