Management Science

1
Management Science

• Term 1, 2003/2004
• Class 7Benchmarking Using Data Envelopment
  Analysis
• Moritz Fleischmann
• Rotterdam School of Management / FBK

2
Agenda Class 7

• Data Envelopment Analysis (DEA)
• Performance Measurement
• Measuring Operational Efficiency
• Developing a DEA Model
• Results of a DEA Analysis
• Application Modells

3
Objectives

• Introduce performance measurement
• Highlight difficulties in measuring performance
• Introduce concept of DEA
• Develop understanding for key elements of DEA
• Highlight application of DEA for generating
  insight into performance
• Explain interpretation of DEA results

4
Data Envelopment Analysis

• Quantitative technique for measuring therelative
  efficiency (performance)of relatively
  homogeneous organizational units.
• such as bank branches, schools, hospitals,
  rehabilitation centers, retail stores, local
  authority departments, courts of justice
• Quite controversial

5
PERFORMANCE MEASUREMENTISRISKY,
CONTROVERSIAL,OR EVEN EMOTIONALBUSINESS!!
DEA is no Panacea
6
Benchmarking Example

• Drinking Water Industry
• public utility sector
• 17 companies in the Netherlands
• voluntary benchmark since 1997
• regulation, law since 2001, with obligatory
  benchmarking on
• 1. Quality
• 2. Customer service
• 3. Finance tariffs costs
• 4. Environment

7
Cost ComparisonDrinking Water Companies
8
Cost ComparisonDrinking Water Companies
9
Cost ComparisonDrinking Water Companies
10
Cost ComparisonDrinking Water Companies
11
Common Complaints

• You are measuring the wrong things!
• You are weighing the performance measures
  incorrectly!(I find this aspect more important
  than you do.)

12
LETS INTRODUCE SOME SYSTEMATIC THINKING!
13
Single Input and Single Output
Slide 13
14
Basic Metric
OUTPUT
EFFICIENCY
INPUT
15
Many Other Examples

• Production targets
• Conversion (marketing)
• sales transactions / store traffic
• Return on investment
• Impact factors of scientific journals
• quoted papers / published papers
• GPA rankings of students
• UEFA soccer ranking

16
6 5 4 3 2 1 0
SALES
B
A
0 1 2 3 4 5
6 7 8 9
EMPLOYEE
17
Two Inputs and Single Output
18
FLOORAREA
6 5 4 3 2 1 0
G
E
A
B
H
I
F
D
C
0 1 2 3 4 5
6 7 8 9
EMPLOYEE
19
General Setting

• Multiple Inputs
• Multiple Outputs
• Different Resources
• Different Activities

gt How to compare ?
20
Example

• FT 2003 International MBA Ranking
• 1. Wharton
• 2. Harvard
• 28. !

21
Generalized Metric
EFFICIENCY
WEIGHTED SUM OF OUTPUTS
WEIGHTED SUM OF INPUTS
22
Weight doctor 5 Weight nurses 1 Weight
outpatient 1 Weight inpatient 3
Two Inputsand Two Outputs
23
HOW TO DETERMINE SUCH A COMMON SET OF WEIGHTS?
24
Data Envelopment Analysis

• Key idea (Charnes, Cooper Rhodes, 1978)
• Measure efficiency as the ratio between weighted
  sum of outputs and weighted sum of inputs
• However, do not use a fixed set of weights, but
  choose the weights dependent on the unit to be
  assessed
• For assessing a unit, choose the weights that are
  most favorable for this unit and compare its
  efficiency with that of other units for these
  weights

25
Data Envelopment Analysis

• Assessing the DEA efficiency of a given unit
  comes
• down to solving an optimization problem
• Decision variables weights for inputs and
  outputs
• Objective maximize efficiency of the given unit
• Constraintsefficiency of all units (for these
  weights) ? 1

26
Data Envelopment Analysis

• Formally (for assessing Hospital 1 in the
  example)
• find weights i1, i2, o1, o2
• that
• maximize (efficiency Hosp.1)
• such that
• (efficiency Hosp.1)
• (efficiency Hosp.2)
• (repeat for all other hosp.)

27
DEA Linear Programming Model

• This model can be transformed into a linear
• optimization model by
• rewriting the constraints weighted output /
  weighted input ? 1as weighted output ? weighted
  input
• normalizing the weighted input of the unit to be
  assessed to an arbitrary number weighted input
  unit 1 100 (say)

28
DEA Linear Programming Model

• Conclusion
• Determining the DEA efficiency of a given unit
  comes down to solving a LP model
• Determining the DEA efficiencies for all of a
  given group of n units requires successively
  solving n LP models
• These models differ only in their objective
  function and in the normalization constraint. All
  other constraints remain the same.

29
Results for Hospital A

• Efficiency Hospital A 1.00
• Weights
• i1 0.00 i2 0.66
• o1 0.31 o2 0.76
• Other efficiencies for these weights
• Hospital B 0.98
• Hospital C 0.87
• Hospital D 1.00
• Hospital E 0.76
• ...

30
Results for Hospital C

• Efficiency Hospital C 0.88
• Weights
• i1 0.00 i2 0.63
• o1 0.36 o2 0.56
• NB different from those in the case of Hospital
  A
• Other efficiencies for these weights
• Hospital A 0.92
• Hospital B 1.00
• Hospital D 1.00
• Hospital E 0.72
• ...

31
Results for Hospital C

• Why is Hospital Cs efficiency not equal to 1.00?
• Compare with Hospitals B and D B C D
• Doctors 19 ltlt 25 lt 27
• Nurses 131 ltlt 160 lt 168
• Outpatients 150 lt 160 ltlt 180
• Inpatients 50 lt 55 ltlt 72
• Whatever weights we choose, Hospital C is always
  dominated by either Hospital B or D(or both)

32
Two Inputs and Two OutputsDEA Ranking
33
Analyzing DEA Output

• Two possible types of outcomes
• The efficiency of a unit equals 1.0
• The efficiency of a unit is lt 1.0

What do they mean/suggest?
34
Analyzing DEA Output

• Flexibility in the choice of weights
• is a weakness (when efficiency 1.0)
• Why?

35
Analyzing DEA Output

• Flexibility in the choice of weights
• is a strength (when efficiency lt 1.0)
• Why?

36
Analyzing DEA Output
Broadly speaking, Hospital E should have been
able to support its activitylevels with only 76
of its resources
37
Additional DEA Output

• For an inefficient unit, at least one other unit
  will be efficient with the target units set of
  weights!
• The set of units that are efficient for these
  weights forms the so-called peer group for the
  inefficient unit and serves as a role model

38
Additional DEA Output

• In addition, the sensitivity report of the LP
  solution provides target input and output levels
  for an inefficient unit
• To this end, one can construct a virtual
  composite unit as a weighted sum of all
  efficient units

39
Including Environmental Factors

• Relevant environmental factors (non-discretionary
• variables) can be incorporated as additional
  inputs
• or outputs. For example
• of parents with university degree for assessing
  the performance of high schoolsgt model as
  additional input (resource)
• Level of competition for assessing the
  performance of retail storesgt model as
  additional output (activity)

40
DEA Applications

• Retail stores
• Bank branches
• Local authorities (police departments, lighting
  facilities, public utility companies)
• Gas stations
• Schools, universities
• Hospitals
• Airports

41
DEA Analysis Extensions

• The basic DEA model can be extended in manifold
• ways, tailored to specific applications
• Limiting the admissible set of weights through
  additional constraintsin particular, one may
  prohibit setting weights to 0(see following
  example)
• Taking into account scale effects by modifying
  the objective function

42
FLOOR AREA
IS UNIT J EFFICIENT?
6 5 4 3 2 1 0
G
E
A
B
H
I
F
Efficient Frontier
D
C
J
0 1 2 3 4 5
6 7 8 9
EMPLOYEE
43
Summary DEA Method

• Specify relevant input and output factors
• For each unit, check whether it is possible to
  assign weights to the factors such that this unit
  becomes the most efficient among all units
• To answer this question, solve an LP model for
  each unit
• For inefficient units, determine peer group and
  target input / output levels

44
REACTIONS?

• USEFUL TECHNIQUE?
• DO YOU BELIEVE IN IT?
• HOW WOULD YOU APPLY IT IN PRACTICE?
• WHICH ADDITIONAL APPLICATIONS CAN YOU THINK OF?

45
Data Envelopment Analysis
46
Data Envelopment Analysis
47
CONCLUSIONS

• A useful, practical, insightful, refreshing
  method
• Not a panacea!!
• Remember that you only measure relative
  performance
• Try to explain the results!!
• Perform audits to find explanations!
• Try different models (inputs, outputs)
• Dont be afraid to combine it with other methods
• Be careful! A tool in the hand of a fool

48
Modells DEA Analysis
49
INPUT/OUTPUT MODEL
Store area
Backroom area
Regular time
Traffic
Over- time
STORE
Transactions
Footware
Sports goods
Apparel
50
Modells DEA Analysis
WHAT QUESTIONS CAN BE ASKED AND ANSWERED?
51
(No Transcript)
52
WORST-PERFORMING STORES
No. Name DEA Effec. 79 WALDORF 66.48 87 GRBGR
ENB 73.14 71 DPTFRDMAL 74.12 24 NESHAMINY 76.
81 25 CHERY_HILL 77.07 53 NEWARK 78.74 92 RK
VL 80.49 26 EXTON 80.70 64 RFL_ROSE 81.58
93 MRKT 81.65
53
BEST STORES??
54
(No Transcript)
55
(No Transcript)
56
(No Transcript)
57
Top performing
Good/Fair
58
HERMANS ACQUISITION
No.
DEA Eff.
Name
Region
71
DEPTFORD MAL
PHIL
74.12
72
SECAUCAS
NJ
85.19
73
UNION PLAZA
NJ
86.15
74
WILLOWBROOK
NJ
92.92
75
DOUGLASTON
NYC
100.00
76
LEHIGH VALLEY
PHIL
90.50
78
BETHESDA
WASH
100.00
79
WALDORF
WASH
66.48
81
RESTON
WASH
95.50
82
SPRINGFIELD
WASH
86.78
83
LIVINGSTON
NJ
87.29
59
CANDIDATES FOR CLOSURE?
60
CANDIDATES FOR CLOSURE??
61
CANDIDATES FOR CLOSURE??
62
Candidates for closure
63
(No Transcript)
64
(No Transcript)
65
(No Transcript)
66
(No Transcript)
67
(No Transcript)
68
(No Transcript)
69
Observations

• Physical size (sq-ft), average sales and traffic
  are not strongly correlated with efficiency.
• Transactions, sales volume and sales/sq-ft are
  more strongly correlated with efficiency.

70
Conclusions

• Modells DEA Analysis
• identifies best stores
• raises conversion issue for inefficient stores
• suggests that inefficient stores use too much
  space
• footware and sporting goods often problematic
• compares stores across different locations
• evaluates new acquisition
• suggests candidates for closure

71
Key Insights

• DEA provides a tool for benchmarking the
  performance of an organizational unit against
  best practice within a given reference group
• DEA evaluates the relative efficiency within the
  group, not the absolute efficiency
• Careful interpretation is key to a successfulDEA
  analysis
• DEA is based on Linear Programming
• Performance measurement is controversial business

72
Reading

• Review reading for this class
• Winston and Albright, Section 4.8 (pp 162-168)
• If you want to know more
• Various applications Interfaces
  http//www.interfaces.smeal.psu.edu, for
  example
• Volume 29, nr. 3, May-June 1999 (DEA in Banking)
• Volume 24, nr. 1, Jan-Feb 1994 (School Bus
  Services)
• L.C.Hendry and R.W.Eglese (eds.) Data
  Envelopment Analysis, in Tutorial Papers in
  Operational Research, Operational Research
  Society, 1990.
• Charnes, A., Cooper, W.W., and Rhodes, E. (1978),
  Measuring the efficiency of decision making
  units, European Journal of Operational Research
  2, 429-444.

73
Web Resources

• Introductions / Tutorialshttp//www.deazone.com
  (excellent site!)http//www.wiso.uni-dortmund.de/
  lsfg/or/scheel/doordea.htmhttp//www.emp.pdx.edu/
  dea/homedea.html
• Free DEA Softwarehttp//www.wiso.uni-dortmund.de
  /lsfg/or/scheel/ems/
• Commercial DEA Softwarehttp//www.banxia.com/fam
  ain.html

74
Outlook on Next Class

• Tomorrow (Thu 12/11) DA Workshop
• Jan. 12/13/14, 2004
• Topic Project Management
• Reading material will be posted in early January