Using the Delphi Technique to Identify Leadership Issues in Career

1
Using the Delphi Technique to Identify Leadership
Issues in Career Technical Education
Implications for Administrators

• Edgar I. Farmer
• Professor of Education and
• Department Head
• Robert W. Clark
• Associate Professor of Education
• Steven M. Welch
• Research Assistant
• Workforce Education Development
• Penn State University
• 2009 ACTE Convention, Nashville, TN
• November 20, 2009

2
Introduction

• In Pennsylvania, Career and Technical
  Education (CTE) administrators are faced with a
  variety of issues and challenges that command a
  significant portion of their time and resources.

3
Our question to you !

• In your opinion, what are three major leadership
  issues facing CTE administrators in technical
  education programs?

4
Sequence of Presentation

• Purpose of Study
• Conceptual Framework
• Significance of Study
• Research Questions
• Review of Literature
• Research Method
• Procedures/Data collection
• Target Audience
• Findings
• Data Analysis
• Limitations
• Conclusions
• Implications

5
Something to Consider
If your actions inspire others to dream more,
learn more, do more and become more, you are a
leader. John Quincy Adams 6th President of
the United States 1825-1829
6
Purpose of Study

• The purpose of this study is to
• 1) identify the major leadership issues in
  Pennsylvania secondary programs using a Delphi
  approach and
• 2) discuss implications that could be helpful to
  all administrators of Career and Technical
  Education programs.

7
DEFINITION OF DELPHI STUDY
Delphi Approach - may be described as a means to
gather views from a panel, though privately
given, of subject-matter experts (SMEs) to
determine the extent of consensus regarding a
specific issue or research problem. The
intention of the Delphi is to ensure that changes
in your opinions or estimates reflect rational
judgment, not the influence of certain opinion
leaders. The Delphi approach prevents strong
personalities from forcing judgments in certain
directions as frequently occurs in meetings of
SME panels (Farmer, 1998).

8
Conceptual Framework

• The conceptual framework for this study is based
  upon the principles in the Delphi technique,
  developed by the RAND Corporation, to gather
  expert opinions from a group of selected experts
  (Farmer, 1998 Rowe Wright, 1999 Watba
  Farmer, 2006).
• Four key principles that should be considered in
  using the Delphi technique are anonymity,
  iteration, controlled feedback, and the
  statistical aggregation of group response (Rowe
  Wright, 1999).

9
Significance of Study

• Transparency of the leadership issues
  identified in Pennsylvania makes this study most
  important to CTE administrators in other states
  such as Delaware, Maryland, New York, New Jersey,
  Ohio and West Virginia.
• In many cases, CTE administrators throughout the
  United States are faced with similar issues and
  challenges.
• By identifying leadership issues facing CTE
  administrators today, solutions can be targeted
  to establish a more productive and dynamic
  workforce tomorrow.

10
Research Questions

• Several research questions were used to guide
  this study
• What major leadership categories or themes were
  developed from the list of priorities and/or
  issues identified by CTE administrators?
• What degree of importance do Pennsylvania CTE
  administrators place on leadership priorities
  identified in the study?
• How important was the consensus building process
  in determining agreement among the experts on the
  leadership priorities identified in the study?

11
Review of Literature

• Rowe, G. Wright, G. (1999). The Delphi
  technique as a forecasting tool Issues and
  analysis. International Journal of Forecasting,
  15 pp. 353-375.
• Stitt-Gohdes, W. Crews, T. (2004). The Delphi
  technique A research strategy for career and
  technical education. Journal of Career and
  Technical Education, 20(2).
• Watba, U. Farmer, E. (2006). Challenges
  confronting community college deans Using a
  Delphi approach. Community College Journal of
  Research and Practice, 30(3) pp. 243-251.

12
Research Method

• The Delphi Technique
• Procedures
• Three-round Delphi Approach
• Round 1 August 2009
• Generated and identified relevant issues
• Reworded relevant statements if necessary
• Listed relevant issues identified
• Prepared questionnaire for Round 2

13
Round 1 Questionnaire
Directions As an expert in the field of Career
and Technical Education you are asked to
generate three (3) responses to the following
question What are the major leadership issues
in technical education programs in
Pennsylvania? PLEASE LIST AT LEAST THREE (3)
SPECIFIC LEADERSHIP ISSUES OR PROBLEMS BELOW NO
PARTICULAR ORDER IS REQUIRED 1. 2. 3.
14
Research Method (Contd.)

• Procedures
• Delphi Study
• Round 2 September 2009
• - Developed categories from issue list
• Rated issues listed, using Likerts Scale
• Conducted statistical analysis on issues and
  categories
• Calculated inter-quartile range, median, and mean
  on each issue
• Prepared the list and categories for Round 3

15
Round 2 Questionnaire
Dear CTE Administrator This questionnaire
represents Round 2 of a three-round Delphi
technique, which lists specific issues or
problems that CTE administrators have identified
as major leadership issues. Your comments, along
with those of other participants, were reviewed
and categorized from your written responses
generated in Round 1. The purpose of completing
Round 2 is to rate each item in terms of its
importance by placing an X in the appropriate
space using the assigned 5-point Likert scale
,with 1 having the lowest importance and 5 having
the highest importance.
16
Research Method (Contd.)

• Procedure
• Three-round Delphi Study
• Re-rated the issues
• Reviewed the categories

17
Round 3 Questionnaire
Dear CTE Administrator This questionnaire
represents the third and final round of our
Delphi study. Round 3 requires you to review your
Round 2 responses and contrast them using group
consensus data. Your Round 2 responses have been
computed to include the groups mean, median,
inter-quartile range (middle 50 of all
responses) . The ratings you initially made for
Round 2 are identified in bold as Your Original
Responses. You may keep your original response
or change any response by putting an X in the new
response grid. An explanation is requested only
when the rating falls outside of the
inter-quartile range. You may write your
explanation in the space provided under each
question.


18
Procedure

• Career and Technical Education Administrators
• Selection of Experts 60

38
Made a written commitment
21
26
27
Completed Round 1
Completed Round 1 2
Completed All 3 Rounds
19

Subject Matter Experts Throughout Pennsylvania
N 27
1
20
Findings

• Major leadership categories/themes developed from
  the list of issues identified by CTE
  administrators
• Curriculum and Instruction
• Professional Development
• Image and Perception of CTE
• Recruitment and Retention
• Budget and Funding
• Legislation and Policies
• Leadership

21
Findings
Curriculum and Instruction
1. Manage and integrate the new levels of
accountability (Academics, PSSA, etc). 2.
Integrate CTE with rigorous academic
instruction. 3. Provide effective ways to help
struggling high school students increase
their academic and technical competence. 4.
Include the academic and technical skills
Pennsylvania employers demand. 5. Develop
career pathways and programs of study that
allow CTE students to transition to postsecondary
ed.
22
Findings
Professional Development

• Provide professional development for teachers to
• improve the academic and technical
  performance of
• IEP (Individual Education Plan) students.
• 2. Provide staff development so faculty acquire
  and utilize
• classroom and laboratory instructional
  practices that
• engage and motivate students.
• Build a professional development system that
  prepares
• educators to be effective administrators in
  CTE.
• 4. Help new teachers adapt to a very different
  career,
• including the requirement to pass challenging
  PRAXIS
• examination.

23
Data Analysis

• Calculations of statistical values for issue
  items
• Inter-quartile range
• Median
• Mean
• Calculations of statistical values for
  categories
• Group mean
• Group Median
• Group Inter-quartile range

24
Data Analysis

• Issue Items final ratings (top Seven)
• Issue Rating Mean
• Provide staff development
• so faculty acquire and
• engage students 1
  4.71
• Provide ways to help students
• increase their competence 2 4.57
• People in leadership who can
• inspire others to support mission 3 4.50

25
Data Analysis

• Issue Items final ratings (top Seven)
• Issue Rating Mean
• Use of data-driven decision
• making for CTE administrators 4 4.35
• Integrate CTE with rigorous
• academic instruction 5 4.31
• Include the academic and
• technical skills Pennsylvania
• employers demand 6 4.29
• Manage and integrate new
• levels of accountability 7 4.29

26
Conclusions

• What are the findings Saying?
• CTE administrators are most concerned about
  faculty and staff development.
• CTE administrators are concerned about providing
  effective ways to help students increase their
  academic and technical competence.
• Priority is given to the need for more CTE
  leaders .
• The use of data-driven decision making should be
  a core management skill for CTE administrators.

27
Limitation of Study Pennsylvania
28
Implications

• Although the current study focuses primarily on
  leadership issues of CTE administrators in the
  Commonwealth of Pennsylvania, it has national
  implications in terms of similarities in which
  CTE programs are managed and delivered.

29
Further Implications

• Pennsylvania Career and Technology Centers (CTC)
  will benefit from a consensus of current
  leadership issues and potential problems facing
  them in the future.
• CTE administrators may benefit by being better
  prepared to serve their faculty, staff and
  students as well as the industry needs of their
  local community.

30
A WORK IN PROGRESS
Further research is needed to identify national
leadership issues faced by CTE administrators Fu
rther research is needed to address the
identified leadership issues and develop possible
solutions Further research is needed to find
the best practices from other states regarding
possible solutions to the identified leadership
issues in Pennsylvania.
31
Q A Session Thank You
Using the Delphi Technique to Identify Leadership
Issues in Career Technical Education
Implications for Administrators
For Additional Information About this Study
Contact Steven M. Welch Research
Assistant Workforce Education Development Penn
State University smw348_at_psu.edu