Total quality management based on self-assessment

1
Total quality managementbased on self-assessment
The experience with in Flanders

• Andre Vyt

PARTNERS IN QUALITY MANAGEMENT
2
PROZA exists in different versions

• Higher education
• Secondary education
• Vocational training for handicapped persons
• Adult education training centers
• Health Care Social care centers

3
The goal of is

• to diagnose strengths and weaknesses of all
  relevant aspects in the organization
• inspired by an all-encompassing and recognized
  framework (EFQM Excellence model)
• by the process-owners (lecturers, researchers,
  administrators, technicians ...) in quality teams
• together with all parties involved supervisors,
  colleagues, and clients (students, qualified
  professionals)
• to set up well-supported specific actions for
  improvement

4
Inspired by EFQM Excellence
Basic conditions
Core processes
Results
5
The 9 areas of attention
Policy and strategy
Resources and co-operation
Leadership
Core processes
Student satisfaction
Human resources policy
Personnel satisfaction
Appreciation by society
Performance of the organization
6
The content of PROZA

• It contains more than 60 focal points over the 9
  areas of attention
• In a focal point 25 specific items are to be
  checked/answered
• The 25 items are grouped into 5 phases, according
  to the level of quality management

7
Areas and points of attention
8
Focal points in core processes

• Admissions coaching
• Communication with students
• Curriculum design
• Design of course units
• Implementation of the curriculum
• Teaching and tutoring
• Student coaching

9
Focal points in core processes

• Practice-based learning
• Internationalisation
• Testing and assessment
• Graduation and alumni-oriented activities
• Postgraduate education
• Public service provision
• Research
• Development and practice of arts

10
The 5 phases of development

1. Quality is variableQuality is linked to the
   individual
2. Initiation of process-based thinking/actingStarti
   ng to work systematically
3. (High level of) ProfessionalismQuality is
   guaranteed
4. Systematic innovationContinuous improvement
5. Aiming for excellenceRecognition of expertise
   and high level of quality

11
The PDCA-cycle of QM (Deming)
quality
12
The higher the level attained,the better the

• Formulation and management of processesMeasuremen
  ts and information gathering
• Systematisation and reliability
  Openness and transparency
• Attunement and integration
  Innovation and improvement
• Involvement of different persons
  Client and external orientation

13
1 Quality linked to the individual

• Quality is the result of individual effort
• Short-term thinking
• Problems are resolved when they arise
• Limited exchange of information
• Lack of follow-up commitments
• A minimum of quality is present

14
2 Initiating process-based thinking/acting

• Formulation of needs and priorities
• Overall commitments enforceable to a limited
  extent
• Increasing attention to information management
• Real causes of most problems are known
• Performance is evaluated regularly
• Improvement is clearly noticeable

15
3 (High level of) Professionalism

• Planned approach
• Availability of reliable data
• Binding by written commitments
• Continuity of service is guaranteed
• Level of quality achieved is supported by
  assessment data
• Information flow is effective

16
4 Systematic innovation

• Systematic analysis and assessment of operations
• Use of quality indicators as a policy instrument
• Structural problem detection handling
• Innovative thinking
• Emergence of synergy effects

17
5 (Aiming for) excellence

• Frames of reference, indicators and working
  methods internationally recognized as best
  practice
• Broad base of support for moving beyond current
  levels of quality
• All quality indicators are moving in a positive
  direction
• Large number of personnel are internationally
  recognized experts

18
E.g., the focal point course units has items
such as

1. Do lecturers know the function of their course
   unit in the curriculum?
2. Do colleagues work together in a team to design
   course components?
3. Is the design of nearly all course units geared
   to the frame of reference of the department?
4. Are goals of improvement defined on a yearly
   basis for all course units?
5. Is the design of specialized course units
   entrusted to experts?

19
Testing and assessment

• Do all students receive a copy of the exam
  regulations in time?
• Is there an attempt to safeguard objective
  assessment by involving more assessors?
• Are assessment methods for each course component
  clearly geared to the objectives?
• Are assessment methods systematically adjusted
  following evaluation?
• Is the quality of the assessment methods
  regularly tested by external experts?

20
Student admission coaching
21
Answering the questions

• Answering categories yes / no / not applicable /
  impossible to answer
• Lack of measurement data provides relevant
  information on the need to professionalize the
  management of information
• Concrete questions provide immediate goals of
  improvement

22
The method used in PROZA

• The checklists are filled out individually by the
  team members
• They gather to agree in consensus on the scores
  of the items and deduce the level of quality
  management
• but also to identify immediately specific goals
  of improvement
• that are selected according priority and written
  down in a self-evaluation report

23
Score (in consensus)

• Actual level phase within which 3 items are
  positive
• Potential or growth level phase within which 1
  item is positive
• each time on condition that in previous phases
  at least 3 items are positive

24
Chart of scores within an area
25
Chart of scores over the 9 areas
26
The PROZAself-assessment process

• Preparation
• Diagnosis
• Improvement

27
Preparation

• Policy decision to use PROZA and formation of a
  steering group
• Getting people on board by engagements and good
  communication
• Initial screening with quickscan to determine
  which areas (and foci) to address
• Formation of self-assessment groups
• Agreement on working method and prior
  clarification of terminology and gradation words

28
Screening with the quickscan

• 200 statements that probe the overall level of
  quality (20 per area, 40 for core processes)
• To be answered with
• Adequate (A)
• Must be improved / not adequate (NA)
• Important (I)
• Not so important (NSI)
• Preferably by the steering group or key persons

29
The Quickscan
30
The Quickscan
31
Carrying out the diagnosis

• Relevant information on the foci can be made/kept
  available
• Gaining a view on hot issues by an inventory of
  individual scores before the meeting
• Value of individual comments on the questions
• Short-circuiting endless discussions
• Right/wrong is less essential than identifying
  improvement
• Eventual agreement on skipping items on which
  only one member answered negatively

32
From diagnosis to improvement

• Diagnosis and proposals for improvement are
  formulated on a report form
• Assignment of a priority value
• Definitive selection of ultimate priorities by
  steering group or management
• For each selected goal, an action plan must be
  drawn up on a predefined form by an improvement
  team

33
Work sheets in PROZA
34
Action plan

• Concrete goal
• Responsible person
• Actions that have to be taken
• Persons involved
• Time plan
• Expected results / deliveries
• Budget

35
Client-orientation of PROZA

• Evaluation criteria and items were tested on
  content and applicability in different HE
  institutions.
• Assessment questions have grown out of practical
  experience and are based on a consensus between
  TQ managers
• The checklists are accompanied by a users guide
  and a list of concepts
• The checklists are delivered electronically, so
  that institutes can adjust words to match the
  structure of the organisation

36
Some pros of PROZA

• Exhaustive and flexible separate areas and focal
  points can be analysed for different units in the
  organisation
• It is easy to deduce questionnaires
• More than 1000 items result in very specific
  goals of quality improvement
• Based on individual analysis and on consultation
  with others in teams

37
Links between self-evaluations and external
quality assurance
Themes for audits
Self-evaluation
Quality management policy Human resources Means
and finances Learning goals Programme content and
organization Outside school practice
training International policy and
activities Training methods and tools Assessment
and examination Counseling and guidance Study
load and effectiveness Research, arts and
societal services Graduates
38
Implementation requirements

• Improvement as the final goal diagnosis and
  scoring is only the starting point.
• Explicit policy and engagement on all levels of
  the management.
• Agreement on aspects of confidentiality and
  openness of reports
• Explicit rules about implementation of selected
  goals, follow-up and feedback

39
Avoiding pitfalls

• If not properly introduced and coached, it may be
  perceived and labeled as a mass of paperwork
  which demands more energy than that it should
  help to improve quality.
• If not complemented by a leading and steering
  management, self-evaluation may bog down in the
  diagnostic process.
• If the instrument is put forward as to be used
  primarily on teaching processes , it may result
  in adverse effects of shooting the piano-player
  instead of improving the piano.

40
Guaranteeing the achievementof goals

• Diagnoses by self-evaluation teams can exist
  alongside the management instead of in the heart
  of the management, leading to quality management
  on paper instead of effective improvement in
  reality
• Avoid misuse by management to solve management
  problems, or to evaluate employees.
• Diagnoses must not only result in clear
  objectives of improvement but must also be made
  concrete in realistic project planning according
  to SMART rules.

41
Criteria for project goals

• Relevant
• Understandable
• Measurable
• Behaviorable
• Attainable

• Specific
• Measurable
• Agreed/acceptable
• Relevant/realistic
• Time-related

42
For further information
www.prose.be
www.prose.be
Info_at_prose.be