Benchmarks

1
Benchmarks

• By
• Michael Stonebraker
• Adjunct Professor, MIT

2
Purpose

• B3MS beat Oracle/BEA/MQ-Series
• On a semi-persuasive app
• By 1 order of magnitude

Can use as a metric for our implementations, but
this is second order effect.
3
Take Away

• DM will lead a 3 month effort
• Define a benchmark
• Run it on Oracle and B3MS
• Publish a paper
• We need 3-4 volunteers (workers) to help
• Jennifer?

4
From 100K Feet, Exploit

• Scalable trigger system
• But Oracle is in the process of fixing this
• And its not clear we have any better ideas..

5
From 100K Feet Exploit

• Extreme dynamism
• I dont see this in a real app

6
From 100K Feet Exploit

• Insert rate
• Has been fixed by Oracle for retrievals
• Fixable through UDFs, etc.

7
From 100K Feet Exploit

• Pushing processing into the sensor network
• Worth at least an order of magnitude
• But not a server benchmark (I.e. Oracle can say
  they do the same thing)

8
From 100K Feet Exploit

• Time series
• Worth an order of magnitude on Wall Street
  (Informix UDFs)

9
From 100K Feet Exploit

• Time series
• Put app server, rules system and DBMS together
  eliminate crossings
• Probably worth an order of magnitude (?)

10
From 100K Feet Exploit

• Time series
• Put app server, rules system and DBMS together
  eliminate crossings
• Non ACID (QoS, drops, etc.)
• Worth an order of magnitude

11
Ball Fields to Play on..

• Medical monitoring
• Not very hard
• Doesnt really exploit one of our unfair
  advantages

12
Ball Fields to Play on..

• Joe DiLiberti
• Exploits our unfair advantages
• Maybe we shouldnt be so timid.

13
Ball Fields to Play on..

• Traffic monitoring
• Could be ok
• Candidate for more thought

14
Ball Fields to Play on..

• Fraud detection (cell phone, Visa, calling card,
  money transfer)
• Could be ok
• Candidate for more thought

15
Ball Fields to Play on..

• Yahoo IM