Title: McCoy College of Business Administration (MBA Program)
1McCoy College of Business Administration (MBA
Program)
- Why Assessment?
- We give grades dont we?
- What is an Assessment Program?
- What can we do?
- How do we do it?
- Special thanks to Dr. Doug Eder and Dr. Kathryn
Martell for much of the content in this
presentation.
2The First and Only Goal
- To teach for long-term retention of information
and application to new situations - --after Halpern Hakel
3- Deep (expert) learning is a curriculum function.
- It is not a course function.
Universities are systems
4Direct vs. Indirect Assessment
- Direct assessment acquires evidence about student
learning and the learning environment Exams,
projects, logs, portfolios, observations.... - Indirect assessment acquires evidence about how
students feel about learning and their learning
environment Surveys, questionnaires, interviews,
focus groups, reflective essays....
5Whatever Assessment Is...
- ...Its Multiple Measures Over Time.
6Why Assessment Wont Go Away
- Limits on money
- Shifts in public view of Higher Education
- State laws and education standards
- Accountability to accrediting agencies to
maintain operating credentials -
- Result We must demonstrate quality of outcome
and return on investment. - Through assessment educators meet
responsibilities to themselves, to students, and
to the public.
7What is Program-Level assessment as specified
by AACSB?
- The college or school must specify program-level
learning goals for each separate degree program.
Generally, such goals are anticipated for each
degree, not for separate majors or concentrations
within a degree. For example, the McCoy College
offers a Bachelor of Business Administration
(BBA) degree with defined majors in accounting,
CIS, economics, finance, management, and
marketing. A set of program-level learning goals
for the BBA degree must be provided and assessed
goals for each major will not be required for
AACSB program review accreditation. The MBA
program has program-level learning goals
reflecting graduate-level learning outcomes.
8Strongest LinksThe Scholarship of Teaching
- Clear Goals
- Adequate Preparation
- Appropriate Methods
- Significant Results
- Effective Presentation
- Reflective Critique
9Assessment vs. Evaluation
- Assessment focuses on the student and the
learning environment. - Evaluation focuses on the professor and the
teaching performance.
10We Give Grades, Dont We?
- Arent grades
- (by themselves)
- enough?
NO!
11When used alone, grades are no longer regarded as
adequate indicators of student learning.
- Professor autonomy Grades in one course or
section may be recorded using a different
standard than grades in another. - General accountability Much of the current
public annoyance with higher education comes from
a lack of skill and content mastery by students
who have received good grades.
12Grades may reflect many things besides student
mastery of course objectives
- Verbal ability
- Participation
- Cooperation
- Extra credit
- Attendance
- Effort
- Criterion Performance vs. Value Added
13Good Assessment Asks Good Questions
- Do we have a curriculum or a set of common
courses? Which of these do our students
experience? - What do we want our students to be or to have
when they have completed the program? What are
our curricular goals? - What do our students do to demonstrate
accomplishment of curricular goals (i.e. course
objectives)? How much and what kinds of writing?
What kinds of math? Oral Presentations? Analysis
and Reports? Are these relevant to our curricular
goals? - How do we demonstrate to ourselves that students
are achieving or have achieved curricular goals?
14Graduate Common Core Courses
- 1. MGT 5314 Organizational Behavior and Theory
- 2. ECO 5316 Managerial Economics
- 3. MKT 5321 Marketing Management
- 4. QMST 5334 Advanced Statistical Methods for
Business - 5. ACC 5361 Financial and Managerial Reporting
and Analysis - 6. FIN 5387 Managerial Finance
- 7. CIS 5354 Decision Support Models in
Management - 8. MGT. 5313 Administrative Policy
15GRADUATE LEARNING GOALS (MBA)Upon completion of
the Master of Business Administration (MBA), a
student will be able to
- 1. Integrate knowledge of fundamental business
disciplines to effectively manage domestic and
global organizations in a dynamic environment. - 2. Integrate appropriate information technologies
for managing business data for decision making,
enhancing productivity, and communicating with
others. - 3. Evaluate analytical skills and critical
thinking processes used in business decision
making. - 4. Evaluate the issues associated with ethical
leadership and conducting business in an ethical,
legal, and socially responsible manner. - 5. Demonstrate the ability to communicate
effectively, both orally and in writing, in new
and unfamiliar circumstances. - 6. Apply the skills needed to effectively lead
and contribute to dynamic workgroups.
16An Assessment Question How Do You Know...
- ...that students walk out your door looking like
you want them to? What behaviors have they
exhibited or products have they produced? What
are the indicators for your goals?
17Goals and Objectivesfor Students
- Goals express what we want our students to be.
- Objectives describe what we want our students to
do. - .....
- Objectives are indicators of goals.
18Graduate Faculty Teaching Common Core Courses
Should
- Talk to each other (if multiple faculty teach the
course) . . . - Agree on specific objectives for the course . . .
- Make sure course objectives are relevant to
Program-Level goals . .
19To be assessment friendly, objectives should
- Focus on students
- Make the learning goals visible (serve as
indicators) - Describe behaviors or products (doing, making)
that can be captured by assignments.
20Blooms Taxonomy
- LEVEL SOME COGNITIVE BEHAVIORS
- Evaluation Appraisal of an Analysis
- or Synthesis
- Synthesis Assembly of Application
- Analysis Disassembly of Application
- Application Use of Understanding
- Understanding Management of Knowledge
- Knowledge Memorization of facts, language,
concepts, principles, theories
21The Next Step . . .
- Graduate Faculty Should Agree on How Best to
Measure Student Performance on each Objective
Contributing to Program-Level Goals.
22Some Assessment Ways and Means
- Assessment days or centers
- Case studies
- Classroom assessments
- Completion and retention studies
- Content analyses
- Debates
- Direct observations
- Focus groups
- Graduate success
- Internships and service learning
- Interviews (including videotapes)
- Exams for certification and licensure
- Matrices
- Performances
- Portfolios of several kinds
- Projects (Primary Trait Analysis)
- Questionnaires and surveys (Direct and telephone
employer, alumni, and student attitude and
satisfaction) - Reflective essays
- Study and activity logs
- Tests
- (Locally-developed and standardized)
- Transcript analyses
23Another Step . . .
- Depending on the method (ways and means) used to
assess a course objective, develop a rubric or a
method of measurement. - Evaluate student performance on the specific
objective on three levels Exceeds Expectation
Meets Expectation Does Not Meet Expectation. - Report your findings.
- Discuss the result(s), determine any actions to
take (if necessary).
24Example Rubric for MGT. 4335
25Responsibilities of Core Course Coordinators
- 1. completing the Course Alignment Matrix which
specifies which program-level learning goals are
covered in a specific core course, - 2. developing course learning objectives that
lead to fulfillment of program-level learning
goals, - 3. collecting data from faculty teaching course
sections to support or measure student learning
of program-level learning goals (including a
representative sample of measurement
devices/techniques/assignments to be maintained
in the office of the core course coordinator), - 4. reporting collective results on program-level
learning goals to the McCoy College Assessment
Committee, and - 5. discussing and acting on specific
recommendations made by the college assessment
and curriculum committees.
26(No Transcript)
27(No Transcript)
28What documentation must be retained by the Core
Course Coordinators?
- Core course coordinators should retain copies of
instruments used for direct assessment of student
learning, such as assignments, written products,
test questions, rubrics, scoring grids, etc. The
course coordinators also must retain sample
student products relating to the instruments
used. Aggregate results of direct assessments
for various program-level learning goals gathered
and/or submitted to the assessment committee
should be retained by the course coordinator by
semester or year for review. Minutes of core
course faculty meetings should be retained with
special emphasis given to items concerning
improvements in instruments used to assess
student learning and acknowledgement and review
of data.
29Must all students be assessed?
- For the purpose of meeting AACSB-International
standards relating to Assurance of Learning (16,
18 and 20), sampling may be utilized as long as
it is representative. Faculty teaching college
core courses for various programs, as scholars in
their respective fields and under the leadership
of the Core Course Coordinator, have the
responsibility for determining appropriate
representative sample sizes. Graduate core
course coordinators may use census data from
graduate core courses for assessment purposes.
30Should college core course faculty and
coordinators gather data and submit results every
year?
- Program-level learning goals for undergraduate
(BBA) and graduate (MBA) programs are reviewed on
a scheduled rotational basis. At least two
program goals are reviewed each year, and college
core courses covering that particular program
goal are required to submit the results of
course-embedded assessment to the assessment
committee during the scheduled review year.
Since the McCoy College views assessment as a
continuous process, core course coordinators and
core course faculty are encouraged, but not
required, to gather data each semester for the
reporting of assessment results.
31Example Undergraduate Goal Rotation
- Program-Level Goal Rotation Sequence
- Academic Year 2005-2006
- Goal 1 Conceptualize a complex issue into a
coherent, persuasive written or oral statement. - Goal 5 Understand the importance of group
dynamics in achieving organizational goals and
use the skills needed for effective teamwork. - Academic Year 2006-2007
- Goal 2 Use critical thinking skills to evaluate
information, solve problems, and make sound
decisions. - Goal 3 Use information technology skills in
decision-making. - Academic Year 2007-2008
- Goal 4 Apply general concepts of ethical
behavior in dealing with stakeholders. - Goal 6 Understand the importance of culture and
diversity.
32We have never been asked...
- ...to assess everything all at once.
33Who sets performance standards? How should Core
Course Coordinators report student performance on
various college-level learning goals?
- For each program-level learning goal assessed in
a college core course, the faculty will determine
their minimum expectation or standard. For
reporting data on any specific program-level
learning goal, core course coordinators should
aggregate data and report the results by three
levels Exceeds Expectations Meets Expectations
or Does Not Meet Expectations. In order to
preserve the integrity of the assessment process,
core course coordinators should release aggregate
data by course (all sections) and not individual
results by the professor teaching the core
course. Program Assessment in the McCoy College
is not the evaluation of faculty performance in
the classroom it concerns the aggregate level of
student learning of various program-level goals.
34Example Report on Program Goal
35An Important Lesson from the Farm
- A pig doesnt get any fatter merely by weighing
it.
36How many students must meet or exceed the
performance standard on a college-level program
goal?
- There is no prescribed percentage of students
that must meet the standard articulated by
faculty. AACSB and other constituent groups will
look at whether and how these data are used for
continuous improvement purposes. Thus, a poor
showing of student mastery of a given
program-level learning goal would only be a
concern if the curriculum was not subsequently
modified to improve student skills/performance of
that learning goal. A second emphasis in the
Assurance of Learning Standards is that
program-level learning goals are used to
communicate competencies of graduates to
students, employers and other constituents.
Thus, faculty have the responsibility to develop
and modify curricula to insure that program-level
goals are achieved by most graduates and not
just a portion.
37What Assessment Asks of You!
- One day of initial development time to determine
course objectives and agreeing on a grading
rubric for specific course objectives. - Two to three hours per semester (depending on the
goal rotation) to extract student performance on
course objectives leading to the fulfillment of
program-level goals. - Writing a short report (1 page) concerning
student performance. - Maintaining examples of student work, grading
rubrics or other examples demonstrating student
achievement of course objectives. - Talking to other faculty about continuously
improving classroom processes and products to
improve student learning of program level goals
over time.
38Closing Thought
- The enemy of the good is the perfect.