Developing and Assessing Students

1
Developing and Assessing Students Research
Skills in the CurriculumA seminar for the
Faculty of Business and Economics,Monash
University
2 Australian Learning and Teaching Council
Dr John Willison Centre for Learning and
Professional Development University of Adelaide
2
Overview

• Analyse collaboratively, and draw own conclusions
  about, the Research Skill Development (RSD)
  framework.
• Examine examples from Business and disciplines
  that have used wikis
• Apply the RSD to a context of relevance

3
Disciplines Involved in Trialling RSD

• Business (Melbourne and Monash)
• Dentistry/ Oral Health, Electronic Engineering,
  Introduction to Academic Learning for
  International Students, Medical Science, Nursing,
  Petroleum Engineering (Uni of Adelaide)
• Psychology (Macquarie)
• Introduction to Tertiary Learning (University of
  South Australia)
• Impending Chemical Engineering and Computing
  Science
• In train VU (Engineering), QUT, USQ (Multiple)

4
Why Research Skill Development (RSD) in the
curriculum?

• Undergraduate research experiences are claimed to
  increase student motivation, satisfaction,
  understanding of content, progression to, and
  completion of HDR, and to reduce attrition and
  plagiarism.
• But
• Without modeling the practice of at a
  higher level of thinking, students continue to
  practice investigative .. at the introductory
  level, even though their exposure to
  content is considerably richer with each
  course they take.
• (Chaplin, 2003 p.238)

5
Research Skill Development in Curriculum Design
and Assessment

• In 12 Disciplines so far, changes that lectures
  have made are small
• Lecturers take existing assessment, and rework
  the marking criteria according to the RSD
• Use analogous criteria eg Oral Health used
  Business
• Some missing facets e.g. evaluate, are
  incorporated
• Diagnostic Assessments have been changed or
  added
• Otherwise, the nature of assessments is unchanged
• The focus of the course is either renewed or
  clarified
• Explicit nature of RSD changes curriculum

6
Research Skill Development Framework
Students A. Embark, determining need for
knowledge B. Find/generate C. Critically evaluate
D. Organise information E. Synthesise, analyse,
apply F. Communicate
7
(No Transcript)
8
Assessments Based on the RSD framework In
Business Courses (see pages 58-61 of RSD
Handbook)
9
(No Transcript)
10
RSD Framework

• Provides the Big Picture and relates this to the
  assessment details for course coordinators,
  lecturers, tutors, and especially students
• Informs assessment-first curriculum redesign
• Same facets for multiple assessments, various
  levels
• Explicit Transparent assessment criteria
• Coherent Incremental skill development
• Revisited (potentially) Cyclic Conceptual
  Framework

11
References

• Bloom, B., Engelhardt, M.D., Furst, E.J., Hill,
  W.H., Krathwohl, D.R (1956). Taxonomy of
  Educational Objectives, NewYork David McKay
  Company.
• Australian and New Zealand Institute of
  Information Literacy (2004). Australian and New
  Zealand Information Literacy Framework
  Principles, standards and practice (2nd Edition).
  Retrieved from http//www.caul.edu.au/info-literac
  y/InfoLiteracyFramework.pdf on 15/4/08.
• Cohen, J. (1988). Statistical power analysis for
  the behavioral sciences (2nd edition). Hillsdale,
  NJ Erlbaum.
• Chaplin, S.B. (2003). Guided development of
  independent inquiry in an anatomy/physiology
  laboratory. Advanced Physiology Education 27
  230-240, 2003
• Willison, J.W. ORegan, K. (2007). Commonly
  known, commonly not known, totally unknown A
  framework for students becoming researchers.
  Higher Education Research and Development 26 (4).

Web Site www.adelaide.edu.au/clpd/rsd Lot
s of downloadable RSD examples are available here.
12
Acknowledgement Support for this activity has
been provided by The Australian Learning and
Teaching Council, an initiative of the Australian
Government Department of Education, Science and
Training. The views expressed in this activity
do not necessarily reflect the views of
Australian Learning and Teaching
Council. Application of RSD slides were
provided by Dr Mario Ricci and Dr Eleanor Peirce,
University of Adelaide For more Information
or to discuss ideas further, contact
John john.willison_at_adelaide.edu.au 08 8303
3219
13
r 0.57
14
Correlations between RSD Assessments 2005-2007
15
Correlations between RSD Assessments 2005-2007
16
Correlations between RSD and pre-RSD Assessments
2003-2007
17
Facets associated with research processes

• In researching, students
• embark on an inquiry and so determine a need for
  knowledge/understanding
• find/generate needed information/data using
  appropriate methodology
• critically evaluate information/data and the
  process to find/generate
• organise information collected/generated
• synthesise and analyse and apply new knowledge
• communicate knowledge and the processes used to
  generate it, with an awareness of ethical, social
  and cultural issues.
• (Willison ORegan, 2007 based on Bundy,
  2004).

18
Levels of Student Autonomy

• Level I
• Students research at the level of a closed
  inquiry and require a high degree of
  structure/guidance

Facets of research inquiry A. Students embark on
inquiry and so determine a need for knowledge/
understanding B. Students
find/generate needed information
C. Students critically evaluate
information /data and the process to
find/generate D. Student organise information
collected/generated E. Students synthesise and
analyse and apply new knowledge F. Students
communicate knowledge and understanding and the
process used to generate them.

• Appropriately identifies key ideas from 1
  source. When using 2 or more sources, identifies
  some peripheral or duplicated ideas as key
• Significance of the paper is stated, but not
  based on leads from, or gaps in, the literature
• http//www.adelaide.edu.au/clpd/rsd/cells/a/a1.htm
  l

19
Levels of Student Autonomy
Level II Students
research at the level of a closed inquiry
and requires structure
Facets of research inquiry A. Students embark on
inquiry and so determine a need for knowledge/
understanding B. Students
find/generate needed information
C. Students critically evaluate
information /data and the process to
find/generate D. Student organise information
collected/generated E. Students synthesise and
analyse and apply new knowledge F. Students
communicate knowledge and understanding and the
process used to generate them.

• Identifies key ideas across several sources
• Significance of the paper is stated explicitly
  and is based on leads from, or gaps in, a limited
  number of references
• http//www.adelaide.edu.au/clpd/rsd/cells/a/a2.htm
  l

20
Levels of Student Autonomy
Level
III Students research independently at the
level of a closed inquiry
Facets of research inquiry A. Students embark on
inquiry and so determine a need for knowledge/
understanding B. Students
find/generate needed information
C. Students critically evaluate
information /data and the process to
find/generate D. Student organise information
collected/generated E. Students synthesise and
analyse and apply new knowledge F. Students
communicate knowledge and understanding and the
process used to generate them

• Significance of the paper is clearly stated and
  based on leads from, or gaps in, a substantial
  number of sources
• http//www.adelaide.edu.au/clpd/rsd/cells/a/a3.htm
  l

21
Levels of Student Autonomy
Facets of research inquiry A. Students embark on
inquiry and so determine a need for knowledge/
understanding B. Students
find/generate needed information
C. Students critically evaluate
information /data and the process to
find/generate D. Student organise information
collected/generated E. Students synthesise and
analyse and apply new knowledge F. Students
communicate knowledge and understanding and the
process used to generate them
Level IV Students research at the level of an
open enquiry within structured guidelines

• Aims/hypothesis clear, focussed and innovative
• http//www.adelaide.edu.au/clpd/rsd/cells/a/a4.htm
  l

22
Levels of Student Autonomy
Facets of research inquiry A. Students embark on
inquiry and so determine a need for knowledge/
understanding B. Students
find/generate needed information
C. Students critically evaluate
information /data and the process to
find/generate D. Student organise information
collected/generated E. Students synthesise and
analyse and apply new knowledge F. Students
communicate knowledge and understanding and the
process used to generate them
Level V Students research at the level of an
open inquiry within self-determined guidelines
http//www.adelaide.edu.au/clpd/rsd/cells/a/a5.htm
l
23
Locating the Development and Assessment of
Students Research Skills in the CurriculumA
workshop for the Faculty of Business and
Economics,Monash University
2 Australian Learning and Teaching Council
Dr John Willison Centre for Learning and
Professional Development University of Adelaide
24
Workshop Objectives

• Develop collaboratively a plan of action for
  Research Skill Development in a specific course
• Or
• Develop specific marking criteria for a
  particular assessment task
• Or
• Another RSD-based possibility that has come to
  mind