Apresentao do PowerPoint

1
DECISION CONFERENCING WITHIN A MULTIMETHODOLOGICAL
FRAMEWORK DEVELOPING A PERFORMANCE APPRAISAL
MODEL FOR THE PORTUGUESE ARMY OFFICERS
João P.C.F. Thomaz, LtCol Portuguese Military
Academy Research Centre (CINAMIL), Lisbon
Technical University of Lisbon Centre of
Management Studies (CEG-IST) fernandesthomaz_at_leiri
anet.pt Carlos A. Bana e Costa Technical
University of Lisbon Centre of Management
Studies (CEG-IST) London School of Economics
Department of Operational Research carlosbana_at_netc
abo.pt
2

• João Pedro da Cruz Fernandes Thomaz

• Bachelor in Military Sciences Artillery
  (Portuguese MA)
• Post-Graduation in Safety Engineering (WQI) /
  Flight Safety (Portuguese AF)
• Master in Management International Trade
  (Lusíada Univ.)
• Doing a PhD in Industrial Engineering and
  Management (Decision Sciences) (Engineering
  Institute IST support of the London School of
  Economics)
• LtCol Artillery w/General Staff Course
• Professor of Decision Analysis and Software
  Engineering
• Researcher at the Centre of Management Studies of
  IST (CEG-IST) and at Portuguese Military Academy
  Research Centre (CINAMIL)

• Contacts
• fernandesthomaz_at_leirianet.pt http//thomaz.pt.vu
  /

3

• The Problem Actors
• The Methodology
• A Competency Based Model
• Obtaining the weights for the competencies (a new
  MACBETH weighting process)
• Interpretation of the results (Performance
  indicators)

4
Decision Problem
Definition of a Competency-based Performance
Appraisal model for the Portuguese Army Officers
What for?

• Evaluate the performance of the military officers
  in a clear and simple manner
• Support the Armys Personnel Strategic
  Management
• Identify potential development opportunities.

5
Actors Involved

• Decision Maker
• General Chief of Staff (Armys Commander) (CEME)
• Work Group
• Representatives of
• - Army General Staff
• - GNR General Staff
• - Army Personnel Command
• - Institute of Higher Military Studies (IAEM)
• one Sociologist
• one Psychologist.

6
METHODOLOGY Multimethodology
Decision Conferencing
Actors
Problem
Facilitators
(Multiple Criteria) Decision Analysis
Group Process Consultation
7
Multiple Criteria Decision Analysis
Can be defined as a package of activities grouped
into three main phases of analysis

• Structuring

1. Problem structuring the definition of the
problem
2. Model structuring definition of the basic
elements of the problem
3. Impact assessment qualitative analysis of
options.
2. Evaluation
1. Construction of a quantitative additive value
model
2. Options evaluation compare options and
analyse intrinsic attractiveness or
unattractiveness
3. Sensitivity and robustness analysis.
3. Elaboration of Recommendations
8
Workshop 1 Model Structuring Competency Clusters
Technical Competencies
Personal Competencies
Institutional Competencies
Interpersonal Competencies
9
Workshop 1 Model Structuring How do we measure
each competency?
Determine a way to measure each competency
To
Decide upon the next step
Non intervention
Consensus
dialogue
Direct Statements Definitions (DSD)
Create statements to measure on the job
competencies
Group Process Consultation Model
10
Workshop 1 Model Structuring Direct Statement
Definitions
Interpersonal Competencies
11
Workshop 2 Evaluating Response Scale

• Response Scale
• 1 0 1
• Theory of Determinants
• Prospect Theory
• (Tversky Kahneman)

Very Important vs Important Very Important 2X
more in NO answers
TVERSKY, A., KAHNEMAN, D. (1981), The framing of
decisions and the psychology of choice, Science,
211, 453-458.
12
Workshop 2 Evaluating Response Scale
13
Workshop 3 Determining Weights A new high level
group
Special Group for the Weighting Decision
Conferencing

• 4 Generals
• 1 Lieut-General ??? (Armys Personnel Commander)
• 3 Major-Generals ?? (representatives of the
  Infantry,
• Artillery Cavalry branches)
• 2 Colonels (one from Army General Staff Personnel
  Division and other
• from Personnel Command Evaluation area)
• 1 Lieut-Colonel (from Military Academy
  Operational Research area)
• 1 Major (Adjunct of Armys Personnel Commander).

14
Workshop 3 Determining Weights A new MACBETH
weighting process

• After explaining the objective of the work
  session and all the process to achieve this
  Competency-based model, all members of the group
  received for each competency a document with a
• Formal definition of the competency
• Set of Direct Statements Definitions for the
  competency
• Fictitious maximum reference level (Level )
  that characterizes an individual with an
  excellent performance in the competency
• Fictitious neutral reference level (Level o) that
  characterizes an individual with the minimum
  acceptable performance in the competency.
• Referential
• A Lieutenant-Colonel of any branch in peace time
  situation

15
Workshop 3 Determining Weights A new MACBETH
weighting process
Teamwork refers to the capacity to work in a
team and actively contribute to solve a problem.
Formal definition
Direct statements definitions (DSD)

• Acknowledges individual differences in others,
  values these differences and adjusts them in
  order to reach identified goals
• Puts the team goals before his own
• Seeks to establish winwin solutions.

Fictitious reference levels
Level (excellent) Individual that revels the
capacity to aggregate different characteristics
of team elements to obtain a better solution,
creating consensus and a good intra-grupal
climate.
What is the global difference of attractiveness
in change one Officer in the level o on all
competencies to level in this competency ?
?
Level o (minimum acceptable) Individual that
revels the capacity to aggregate different
characteristics of team elements to obtain a
better solution.
16
Workshop 3 Determining Weights A new MACBETH
weighting process
What is the global difference of attractiveness
in change one Officer in the level o on all
competencies to level in this competency ?
Based on this question the group answered
qualitatively, using the MACBETH procedure. Note
that there was some hesitation in the judgements,
e.g. 4-5 (strongvery strong) or 3-4
(moderatestrong), that are supported by the
MACBETH software.
17
Workshop 3 Determining Weights A new MACBETH
weighting process
Then using the Build (Macbeth) scale button
the software proposed a new ordination of the
competencies and automatically included a P
(for positive difference of attractiveness) in
the pairs with a preference relation between them
(obtained by transitivity).
?
Now the group had to make a top-down validation
of the pre-ordination by groups of different
semantic categories, e.g. the 5 (very strong)
versus 4-5 (strong very strong)
competencies, and so on.
18
Workshop 3 Determining Weights A new MACBETH
weighting process
The validation is done comparing the
competencies of one semantic category with the
others in the next semantic category, e.g.
competencies with a 4-5, Courage, Determ,
Initiative, Leader, DecMaking with the others
with a 4, Loyalty and ProbSolv.
Blue 5 4-5 Red 4-5 4 Green
4 3-4 Brown 3-4 3
Now the group had to validate the competencies
within each group of equal semantic category,
e.g. at category 3, Discip, Teamwork,
ProfExpert, PlanOrg .
19
Workshop 3 Determining Weights A new MACBETH
weighting process
The validation now (in the diagonal of the
matrix) is done comparing the competencies within
each semantic category (3, 3-4, 4 and
4-5).
So, for category 3 - moderate the group agreed
that Discip Teamwork and ProfExpert PlanOrg
were indifferent ( I ) and that ProfExpert
PlanOrg had a positive difference of
attractiveness ( P ) over Discip.
Blue 3 Red 3-4 Green 4 Brown
4-5
20
Workshop 3 Determining Weights A new MACBETH
weighting process
Then clicking on Build (Macbeth) scale the
software reorganized the matrix and proposed a
weight scale for the competencies.
The group discussed the values obtained by the
software and agreed on it with a small refinement
of the weights.
21
Workshop 3 Determining Weights Final Weights
for the competencies
The group was satisfied with the dynamics of the
process and the results obtained in a short time
(approx. 3 hours).
22
Workshop 3 Determining Weights Weights for the
clusters of competencies
23
Workshop 4 Determining Weights Additive
Aggregation Model
With the value functions and the weights, the
additive model could now be used to associate a
global evaluation to each Officer.
where V(o) Global value of fictitious Officer o,
vj(o) Partial value of fictitious Officer o on
the j-th FPV kj Weight (scale constant) for the
j-th FPV
24
Workshop 5 Evaluating Fictitious Reference
Levels of Performance
Very Important (V.IMP) vs Important (IMP) Very
Important 2 x more in NO answers
Maximum Level YES in all V.IMP and IMP
Direct Statement Definitions (DSD) Superior
Level YES in all V.IMP and ONE hesitation
in IMP DSDs per competency Neutral Level()
YES in all V.IMP and ONE NO in IMP
DSDs per competency Neutral Level () ONE
hesitation in V.IMP and ONE NO in IMP
DSDs per competency Inferior Level NO in
all V.IMP and YES in IMP DSDs per
competency Minimum Level NO in all V.IMP
and IMP DSDs.
25
Workshop 5 Evaluating Qualitative Performance
Indicators
Superior YES in all V.IMP and ONE
hesitation in IMP Neutral () YES in all
V.IMP and ONE NO in IMP Neutral () ONE
hesitation in V.IMP and ONE NO in
IMP Inferior NO in all V.IMP and YES
in IMP
Superior 194.5 Neutral ()
109.1 Neutral (-) 9.1 Inferior
219.6
26
CONCLUSIONS
Multimethodology framework is an efficient
alternative when tackling with complex problems,
helping facilitators adjust to the decision
makers needs during the whole process
Decision Conferencing, enriched with Group
Process Consultation, proved to be a methodology
that improves the group productivity, controlling
the group dynamics and saving time Multiple
Criteria Methodology was enriched with
contributions borrowed from other decision
group support methodologies, under a single
paradigm the learning paradigm that
guaranteed the theoretical and practical cohesion
and consistency of the process The Special
Group for the weighting decision conferencing
session was satisfied with the dynamics of the
process and the results obtained in a short time.
The decision aid process becomes tailor made,
and helps to
GENERATE A SHARED UNDERSTANDING OF ISSUES
DEVELOP A COMMON SENSE OF PURPOSE
AGREE ABOUT THE WAY FORWARD
27
Thanks for your attention.
28
DECISION CONFERENCING WITHIN A MULTIMETHODOLOGICAL
FRAMEWORK DEVELOPING A PERFORMANCE APPRAISAL
MODEL FOR THE PORTUGUESE ARMY OFFICERS
João P.C.F. Thomaz, LtCol Portuguese Military
Academy Research Centre (CINAMIL), Lisbon
Technical University of Lisbon Centre of
Management Studies (CEG-IST) fernandesthomaz_at_leiri
anet.pt http//thomaz.pt.vu/ Carlos A. Bana e
Costa Technical University of Lisbon Centre of
Management Studies (CEG-IST) London School of
Economics Department of Operational
Research carlosbana_at_netcabo.pt