Ashish Gupta

1

• Ashish Gupta
• Bin Lin
• Peter Dinda
• Department of Computer Science
• Northwestern University

2
Overview

• Our Goal
• Introduce resource borrowing and its motivation
• Current model and further possibilities
• What we want to study ?
• Methodology
• Actually try it out !
• Design of a system which emulates resource
  borrowing and captures feedback
• Design of our application
• What it does ?
• Extensive System Monitoring
• Resource Exercisers we have
• For technical details refer to paper and tech
  report
• System Design the client server system,
  creating and analyzing testcases
• The controlled study
• What the user was asked to do, user details etc.,
  set of testcases ( ramp and step)
• Results
• Aggregated
• Context
• Person

3
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled User Study
• Its Results
• Conclusions

4
New Models of Distributed Computing
5
Observation and ideas

• Idea Why not borrow the unused resources ?

6
Effect of performance slowdown
7
Effect of performance slowdown
8
The question we want to answer
What level of resource borrowing leads to user
discomfort for significant fraction of users ?

• Depends on Other Factors as well

9
CPU
RAM
Disk
10
Context
Person
Hardware
Others ?
11
Power Users
Typical Users
Beginners
12
(No Transcript)
13
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions

14
How can we find this out ?

• Actually test it with user feedback

A system for emulating resource borrowing and
measuring user discomfort using direct feedback
http//comfort.cs.northwestern.edu
15
How can we find this out ?

• Measure existing contention

1
16
How can we find this out ?

• Emulate Resource Borrowing

2
CPU Usage vs. Time
Final Contention
Borrowed Contention
Real Contention
17
How can we find this out ?

• Allow User Feedback

3
F11
18
How can we find this out ?

• Find MAX Level of tolerable contention

4
CPU Usage vs. Time
Final Contention
Borrowed Contention
19
How is it useful ?
Comfort f (Contention)

• To find a quantitative relationship between
  resource borrowing and user comfort

20
How is it useful ?
Enhances the degree of workstation sharing
21
Interactivity Aware Scheduling
More intelligent scheduling of interactive tasks
possible via user feedback and our results
22
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions

23
Can do large scale user studies and maintain
testcase control over wide-area
Server
Results Transfer
Testcase Transfer
Client
Resource borrowing using testcase profile
24
The client design
25
Testcases

• Each testcase stores details of resource
  borrowing
• Testcase consists of exercise functions for each
  resource
• Exercise function is a vector of contention
  values
• Example
• CPU contention 0,0.5,1.0,1.5,2.0 at 1 Hz

26
Exercise Functions
Has all sort of strange exercise
functions Power-law, exp, pareto Plot our real
testcases
Exercise Function 1
Contention
Exercise Function 2
Exercise Function 4
Exercise Function 3
Time
27
Flowchart of Testcase Execution
No Testcase running
28
Resource Exercisers

• CPU Exerciser
• Contention describes the expected extra number of
  threads in ready queue
• Fractional resource borrowing using stochastic
  scheduling methods
• Validated to contention level of 10
• Disk Exerciser
• Random seek and read/write in a large file (twice
  the memory)
• Validated to contention level of 7
• Memory Exerciser
• Borrows a fraction of physical memory from 0 to
  1

29
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions

30
The control study description

• 33 undergrad and grad students
• 1.5 hrs each
• Each user was assigned 4 popular tasks to do
• MS Word
• MS Powerpoint
• MS Explorer ? searching and saving information
• Quake III

31
Machine Configuration
32
Testcases for the Control Study
Three types of testcases step, ramp and blank

• Ramp allows us to know the point of discomfort
  for each resource
• Step and Ramps comparing the manner of
  borrowing resources
• Blank To test the background level of discomfort

33
Frog in the Pot Hypothesis
Step/RampDynamics


34
Testcases for the Control Study

• Testcases different for each application
• Some apps more intensive than others
• Example Word and Quake
• Requires Calibration

35
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions

36
Resource borrowing vs User Discomfort

• CDFs for discomfort contention level
• Not all contentions cause discomfort exhausted
  region

37
Aggregated Results
Exhausted Region
Discomfort Region

• C0.05 0.35 ( aggregated over all applications)

38
Resource borrowing vs User Discomfort

• CDFs for discomfort contention level
• Not all contentions cause discomfort exhausted
  region
• CDFs allow us to read age of people discomforted
  for a given contention
• Metric c0.05 At what contention do we
  discomfort only 5 of the people ?

39
Results overview
Aggregated Results
Dependence on Resource
Dependence on Context
Dependence on Users
Dependence on Step/Ramp
40
Aggregated Results
41
Aggregated Results
Exhausted Region
Discomfort Region

• C0.05 0.35 ( aggregated over all applications)

42
Aggregated Results
Exhausted Region
Discomfort Region

• Can borrow a lot, discomforting only a small
  fraction\
• C0.050.33

43
Aggregated Results
Exhausted Region
Discomfort Region

• Can borrow a lot, discomforting only a small
  fraction
• C0.05 1.11

44
Dependence on Resource
45
An overview of factors
Dependenceon Resource
User sensitivity by task and resource (Low,
Medium, High)
46
Dependenceon Resource
CPU
47
Dependenceon Resource
Memory
48
Dependenceon Resource
Disk
49
Dependence on Context
50
Comparison of CPU discomfort
Dependenceon Context
Quake
Powerpoint
IE
Word
51
Dependence On Context - CPU
Dependenceon Context
Word
Powerpoint
IE
Quake
52
Dependence on Context - Memory
Dependenceon Context
Word
Powerpoint
IE
Quake
53
Dependence on Context - Disk
Dependenceon Context
Powerpoint
Word
IE
Quake
54
Fraction of users discomforted in Disk
Dependenceon Context
IE
Quake
Powerpoint
Word
55
Internet Wide-Area Study

• No controls here, hope to get more general and
  complete data
• Users can download a client from
  http//comfort.cs.northwestern.edu
• Clean installation/un-installation and privacy
  options

Join us !
56
Outline

• Motivation behind Resource Borrowing
• Measuring user comfort
• System Design for Measuring User Comfort
• Controlled Study
• Its Results
• Conclusions

57
Conclusion
Comfort f(contention)
Disk and memory can be borrowed more
aggressively, CPU less so
A throttle for resource borrowing ? very useful
CDFs can be used to set the throttle
58
Conclusion
Comfort f(contention)
Knowing the user context helps
Using Direct user feedback can be useful
59
Current and Future Work
User feedback for interactive VM Scheduling
60
Acknowledgements
http//comfort.cs.northwestern.edu
Special Thanks to Don Norman Andrew Ortony Ben
Watson Jack Tumblin Leslie Lamport and All our
beta testers Study participants
61
Conclusion
Comfort f(contention)
Disk and Memory surprisingly easy to borrow, CPU
borrowing also liberal
Set of empirical CDFs provided
Depends of other factors Resource, Context and
Users
62
Dependence on User
63
Dependence on User
Dependenceon User
Beginner
Typical
Power
64
Step/Ramp Dynamics
65
Frog in the Pot Hypothesis
Step/RampDynamics


66
An example with PPT
Step/RampDynamics
0.98
67
The placebo effect

• Users express discomfort even when no testcase is
  running
• Can never guarantee that the user will never
  express feedback

68
Memory Exerciser

• Contention indicates fraction of total memory
  allocated in addition to occupied memory
• Pages are touched frequently to keep them in
  physical memory

Occupied Mem.
0.5
1024 MB
0MB
Memory Contention
Clipping Point
69
(No Transcript)
70
Related Work

• User Frustration in HCI
• Computer Response to User Frustration K99
• Sensing and measurement of frustration with
  computers R01
• Impact of latency of user utility
• Behavioral aspects of text editors EN81
• Psychological Upper and lower limits of system
  response time and users preference on skill
  level K97

71
Virtual Machine Scheduling

• Virtuoso Distributed Computing with Virtual
  Machines
• User pays for the resources he uses
• Need to know minimal resources needed to keep an
  interactive VM user happy

72
Questions and Answers

• Other types of irritation
• Bursty natures of Desktop Computing, bursts
  indicate sudden action-response tasks
• Large number of factors for irritation
• Since there is usually a power law for process
  run times, why do we need this model ? (CPU is
  idle most of the time)
• Why cant we just run the bg app at a lower
  priority ? It may benefit much more than just
  running at a fixed contention.
• Memory exerciser What does contention mean ? How
  do you find out free physical memory ?
• How do you avoid thrashing ? Why do you avoid
  thrashing ?
• How does the CPU exerciser work ?
• Why do you disable buffering in the Disk
  exerciser ?
• The aggregated graphs are not useful. Memory
  usage may be much more dynamic then in your tests
  due to user multitasking etc. So this may not
  represent reality ?
• Our study is a start in this direction and we
  realize that there are limitations to the data we
  have collected. But we expect to collect and
  analyze more results from the Internet Wide area
  study and generalize our results to more real
  environments. There are no controls in the
  internet wide area study. The control study was
  meant to specifically answer the questions we
  have posed, like the affect of other factors in
  user discomfort and resource borrowing.

73

• Lower priority ?
• Disk and memory ?
• There is a limit (max) possible amount of cycles
  stolen. Our goal is to sestimate that.
  Seperately, there are many possible sched mech
  that would get closer to that max. Lower priority
  is an example. Linger longer is an example.
  Screen saver is an example.
• Quake example
• Dont argue for/against a particular scheduling
  scheme
• It is a step in the right direction (for
  results)

74
Server
Results Transfer
Testcase Transfer
Client
Resource borrowing using testcase profile
75
The client-server system

• Data Exchange
• Hotsync
• Results transfer
• Testcase update
• Testcase Tools

76
(No Transcript)
77
(No Transcript)
78
An irritating Interface
79
Bursty Nature of Interactive Computation
Interactive Computation
80
(No Transcript)
81
(No Transcript)
82
Understanding Testcases
CPU
MEM
ramp 1 ramp 2 ramp 3 step 1 step 2 ..
ramp 1 ramp 2 ramp 3 step 1 step 2 ..
Ramp
Step
83
Combination of Resources

• CPU Usage
• Memory Usage
• Hard Disk activity
• Network Traffic

Comfort Zone surface
84
(No Transcript)
85
(No Transcript)