The Discipline of Business Experimentation

1
The Discipline of Business Experimentation
2
Innovations Dont Always Pay Off. Sometimes They
Fail Spectacularly.
3
Big Data Has Its LimitsIt cant always predict
the future.
4
Innovations Need Rigorous Testing
A more reliable way to evaluate new initiatives.
5
A RIGOROUS EXPERIMENT

• Relies on proven scientific and statistical
  methods.
• Has a sample size that will yield valid results.
• Tests one independent variable against a
  dependent variable while holding all other
  variables constant.
• Incorporates careful observation and analysis.

6
Five Keys to Good Experiments
7
1Does the Experiment Have a Clear Purpose?

• Start with a Strong Hypothesis
• Avoid if the Hypothesis results are weak

8
2Have Stakeholders Agreed to Abide by the
Results?

• Weigh All the Findings
• Walk Away if Theyre Negative

9
3Is the Experiment Doable?

• Potential Roadblocks
• Too Much Complexity
• Costly Sample Size
• Operational Disruptions 

10
4How Can We Ensure Reliable Results?

• Randomized Field Trials
• Blind Tests
• Big Data

11
5 Have We Gotten the Most Value out of the
Experiment?

• Invest in areas where the ROI will be highest.
• Determine which components have a positive return.

12
It All Comes Down to Rigor
13
Experiment Checklist

• Purpose
• What specific management action are we
  considering?
• What do we hope to learn?
• Buy-In
• What specific changes will we make on the basis
  of the results?
• How will we ensure that the results arent
  ignored?

14
Experiment Checklist

• Feasibility
• Do we have a testable prediction?
• What sample size do we need?
• Can we avoid disrupting operations at the test
  locations?
• Reliability
• What measures will we take to counteract bias?
• Would others conducting the same test obtain
  similar results?

15
Experiment Checklist

• Value
• Can we do a targeted rollout focusing on
  areaswhere the payback is highest?
• Have we implemented only the components with the
  highest returns?
• Do we understand which variables are causing
  which effects?

16
Thank You
You can contact us at
00966 56 100 4748 info_at_dtechsystems.co www.dtechsy
stems.co