Lies, Damn Lies, and Statistics

1
Lies, Damn Lies, and Statistics

• Holger Bast
• (based on an observation together with Ingmar
  Weber)

2
Famous Quotes
There are three kinds of lies lies, damn lies,
and statistics Benjamin Disraeli, 1804
1881 (reported by Mark Twain)
Never believe any statistics you havent forged
yourself Winston Churchill, 1874 1965
3
A Typical Figure
Y-axis some cost measure, e.g computing time
Each point represents an averageover a number of
iterations
4
Theirs
3.5
Ours
3
X-axis some parameter,e.g. cache size
4
A monotone transformation

• Assume we changed our cost measure from c to 2c
  (a monotone function)

This is from authentic data!
5
Dont say you didnt know

• It can happen that
• EX lt EY
• E2X gt E2Y
• and not only for 2x but for any monotonically
  increasing function
• It cannot happen though (it seems) that
• EX sX lt EY sY
• E2X s2X gt E2Y s2Y

6

• Thank you!

7
(No Transcript)
8
A Slight Change

• Assume we changed our measure of badness from b
  to 2b (a monotone function)

4.2
20
4
3.8
3.6
15
3.4
3.2
10
3
2.8
9
A Typical Figure
Y-axissome measure of badness
Each point represents an averageover a number of
iterations
4.2
4
3.8
Theirs
3.6
3.4
Ours
3.2
3
X-axis some parameter,e.g. problem size
2.8