David P. Anderson

1
Designing Middleware for Volunteer Computing

• David P. Anderson
• Space Sciences Laboratory
• University of California Berkeley
• davea_at_ssl.berkeley.edu

2
Why volunteer computing?
home PCs
your computers
academic
business

• 2006 1 billion PCs, 55 privately owned
• If 100M people participate
• 100 PetaFLOPs, 1 Exabyte (1018) storage
• Consumer products drive technology
• GPUs (NVIDIA, Sony Cell)

3
Volunteer computing history
95 96 97 98 99 00 01 02 03 04
05 06
GIMPS, distributed.net
SETI_at_home, folding_at_home
commercial projects
climateprediction.net
BOINC
Einstein_at_home Rosetta_at_home Predictor_at_home LHC_at_home
BURP PrimeGrid ...
4
Scientific computing paradigms
Control
Bang/buck
least
most
Supercomputers
Cluster computing
Grid computing
Volunteer computing
least
most
5
BOINC
Climate
SETI
physics
biomedical
projects
volunteers
Joe
Jens
Alice
6
Participation in gt1 project

• Better short-term resource utilization
• communicate/compute in parallel
• match applications to resources
• Better long-term resource utilization
• project A works while project B thinks

work
work
project computing needs
think
think
time
7
Server performance
How many clients can a project support?
8
Task server architecture
Work creator
Feeder
Shared mem
MySQL
Transitioner
Scheduler
clients
Validator
Assimilator
File deleter
DB purger
9
Server load (CPU)
10
Server load (disk I/O)
11
Server limits

• Single server (2X Xeon, 100 Mbps disk)
• 8.8 million tasks/day
• 4.4 PetaFLOPS (if 12 hrs on 1 GFLOPS CPU)
• CPU is bottleneck (2.5 disk utilization)
• 8.2 Mbps network (if 10K request/reply)
• Multiple servers (1 MySQL, 2 for others)
• 23.6 million tasks/day
• MySQL CPU is bottleneck
• 21.9 Mbps network

12
Credit
13
Credit display
14
Credit system goals

• Retain participants
• fair between users, across projects
• understandable
• cheat-resistant
• Maximize utility to projects
• hardware upgrades
• assignment of projects to computers

15
Credit system

• Computation credit
• benchmark-based
• application benchmarks
• application operation counting
• cheat-resistance redundancy
• Other resources
• network, disk storage, RAM
• Other behaviors
• recruitment
• other participation

16
(No Transcript)
17
Benchmarks not whole story
18
Limits of Volunteer Computing

• How much processing/disk/RAM is out there?
• Combinations of resources
• Data from 330,000 SETI_at_home participants

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Goals of BOINC

• gt 100 projects, some churn
• Handle big data better
• BitTorrent integration
• Use GPUs and other resources
• DAGs
• Participation
• 10-100 million
• multiple projects per participant