Does Technology Improve Student Learning?

1
Does Technology Improve Student Learning?

• Michelle Barton, Palomar College
• Darla Cooper, Oxnard College
• Kelvin Leeds, Santa Ana College
• Susan Longacre, City College of San Francisco
• John Whitmer, _at_ONE Project, Evergreen Valley
  College

2
Outline

1. Overview (John)
2. Researchers (Darla Michelle)
3. The Effect of Tablet PCs on Learning and
   Collaboration (Kelvin)
4. Developing Critical Thinking in the Nursing
   Student Using Simulated Online Clinical Scenarios
   (Sue)
5. Q A

3
Overview (1)

• _at_ONE Project
• Vision evaluate effectiveness of technology in
  CCC classrooms
• Goals of Scholar Program
• To create and share original research on the
  impact of instructional technology on student
  learning in California Community College
  classrooms
• To support the improvement of faculty members
  research abilities, technology skills, and
  pedagogical approaches

4
Overview (2)

• Faculty-designed research project (competitive
  application), using Classroom Action Research
  methodology
• Ongoing program 6 Scholars 04-05, 10 Scholars
  05-06, 10 Scholars 06-07
• Research support added to 05-06 cohort through
  Center for Student Success
• Support team Pedagogical Support, Research
  Support, Instructional Tech Support, Logistics
• Program Elements Summer Retreat, Winter Retreat,
  Spring Graduation (06-07 only), Virtual Group
  Meetings, individual support

5
2005-06 _at_ONE Carnegie Scholars (1)

• Gregory Beyrer - Distance Education and Media
  Services, Cosumnes River College"Online Student
  Success" Making a Difference
• Carolyn Brown - Graphic Design, Foothill
  CollegeStudent engagement and outcome in online
  courses what can we learn from face-to-face
  learning communities?
• Cheryl Carter - Counseling, Diablo Valley
  CollegeNon-Traditional Student Participation in
  Synchronous Online Workshops
• Bobby Hutchison - Business, Behavioral and Social
  Sciences, Modesto Junior CollegeThe Impact of
  Online Human Sexuality Education on Percpetions
  of Risk, Worry and Knowledge
• Kelvin Leeds - Mathematics, Santa Ana
  CollegeTechnology Fad or Fixture

6
2005-06 _at_ONE Carnegie Scholars (2)

1. Patrick Len - Physical Sciences, Cuesta
   CollegeAnalyzing Individual and Cooperative
   Electronic Response Systems to Improve Student
   Learning and Attitudes in Astronomy
2. Susan Longacre, - Registered Nursing, City
   College of San FranciscoUtilizing Online
   Simulated Clinical Scenarios to Develop Critical
   Thinking in the Nursing Student
3. Scott Lukas - Anthropology/Sociology, Lake Tahoe
   CollegeAn Assessment of Reflexive Student
   Learning in Online Social Science Courses
4. Frances Rice-Farrand - Nursing, Los Angeles City
   CollegeEducation for Future
5. Denise Stiglich - Math and Science, Antelope
   Valley CollegeInteractive PowerPoint and Online
   Homework Programs in the Classroom

7
The Researchers

• The Odd Couple

8
Overview

• What Did We Do?
• How Did We Do It?
• Why Did We Do It?
• Would We Do It Again and
• What Would We Do Differently?
• What Did the Scholars Study and How Did They Do
  It?

9
What Did We Do?

• Worked with _at_ONE staff
• Retreats
• Worked with the scholars

10
Worked with _at_ONE Staff

• Helped select the scholars
• Met regularly to review program
• Met regularly with scholars to get updates on
  their progress
• Discussed drafts of monographs
• Helped plan for next year

11
Retreats

• Helped plan the retreats
• Prepared and gave presentations at the retreats
• Met individually with each scholar to review
• Research design and methodology (1st
  retreat)
• Data analysis and interpretation (2nd
  retreat)

12
Worked with the Scholars

• Helped refine and solidify research question,
  design and methodology
• Reviewed and helped develop survey instruments,
  databases, and definition files
• Helped with data analysis and interpretation
• Reviewed monographs and provided feedback to
  scholars
• Helped with data presentation in the monograph
• On call for help

13
How Did We Do It?

• TEAMWORK!!
• Communication (Lots of it!)
• Humor
• Patience

14
Why Did We Do It?

• To do research again
• To learn about technology and its impact on
  learning in different disciplines
• To do research that matters and directly impacts
  students

15
Would We Do It Again and What Would We Do
Differently?

• YES! YES! YES!
• Lessons learned
• Focus is key
• Data storage and organization
• Data presentation
• Review instruments
• Submission of written methodology

16
What Did the Scholars Study andHow Did They Do
It?

• Impact of collaborative use of clickers on
  student learning and attitudes in astronomy
  (Mixed design, pre-test post-test two groups)
• Comparison of student perceptions and experiences
  regarding participation in face-to-face and
  online discussion groups (Questionnaire and
  interview)
• Impact of an online preparation course on student
  performance in online courses (Analysis of
  historical data and questionnaire)

17
What Did the Scholars Study andHow Did They Do
It?

• Participation and satisfaction of non-traditional
  students in synchronous online student services
  workshops (Questionnaire)
• Student perceptions of learning using Tablet PCs
  (Questionnaire and interview)
• Effect of online clinical scenarios on the
  development of critical thinking skills (Mixed
  design, pre-test post-test with experimental and
  comparison groups)

18
The Effect of Tablet PCs on Learning and
Collaboration A Study of Student Perceptions

• Kelvin Leeds
• Santa Ana College
• April 11, 2006

19
Background

• Technology is changing the classroom setting
• Keyboards are not conducive for Math and Science
  Type classes
• Tablet PCs may be more suitable for Math and
  Science classes

20
Technology Uses

• Tablet PC
• Use Tablet PC as virtual paper to take notes
• Synchroneyes
• Software program that enables the instuctor to
  view student computer screens
• Blackboard
• Online portal used to post course materials
• Whiteboard
• Software program that allows students to form
  virtual groups

21
Research Questions

• Do students think the use of Tablet PCs in the
  classroom enhances their learning when compared
  to traditional methods?
• Do students think the use of Tablet PCs during
  class increases collaboration between them and
  their classmates?
• Are students hesitant to share their classroom
  work since others in class may know whose work is
  whose?

22
Methodology

• MESA Pre-Calculus Course at Santa Ana College,
  Fall 2005 Semester, 23 students
• Developed two questionnaires
• Background Questionnaire
• Student Perception Questionnaire
• Follow up interviews

23
Background Questionnaire

• 11 questions designed to investigate
  students knowledge/experience with computers, 22
  students
• Overall computer use
• Course work and computer use
• Computer access

24
Background Questionnaire

• Sample Question
• In general, how often do you use a computer
  during scheduled class time at this college?
• a.) I never use computers during class time
• b.) Less than half of the time
• c.) Half of the time
• d.) More than half of the time

25
Background Questionnaire

• Key Findings for Background Questionnaire
• Overall Computer Use
• Virtually all students use computers outside of
  class
• Half of students use computers for more advanced
  tasks beyond word processing/email/Internet
• Course Work and Computer Use
• Most students use computers outside of class to
  complete course related materials
• Half of students use computers in class to
  complete course related materials
• Computer Access
• Most students have access to a computer at home
  or work
• Half of students use computers on campus, the
  other half have access to a more convenient
  computer

26
Student Perception Questionnaire

• 18 Questions designed to investigate students
  perceptions about using Tablet PCs during class
• Instructor's Notes
• Students Notes
• Homework
• General Comments
• Two Administrations, 14 students
• Middle and end of the term

27
Student Perception Questionnaire

• Sample Question
• When the instructor uses Synchroneyes to
  present my notes on the screen in front of the
  class
• a.) I feel very comfortable
• b.) I feel somewhat comfortable
• c.) I feel somewhat uncomfortable
• d.) I feel very uncomfortable
• e.) It does not bother me one way or the other
• f.) Not Applicable

28
Student Perception Questionnaire

• Key Findings for Instructors Notes
• All students felt it was helpful to have
  instructors notes available on the Internet
• Most students access these notes at least half of
  the time

29
Student Perception Questionnaire

• Key Findings for Students Notes
• More than half of students felt using the Tablet
  PC to take notes was more helpful than using
  traditional pencil/paper
• Most students feel comfortable when their notes
  are presented in front of the class using
  Synchroneyes
• Most students think using Synchroneyes to present
  other students notes is helpful

30
Student Perception Questionnaire

• Key Findings for Homework
• Most students think using Whiteboard to review
  homework is more helpful than traditional
  paper/pencil
• Most students think using Whiteboard to review
  homework creates more collaboration than
  traditional paper/pencil

31
Student Perception Questionnaire

• Key Findings for General Comments
• Most students think that using computers in
  non-computer related courses during regular class
  time is more helpful than not using computers in
  class
• More than half of students think that using
  computers during class time improves their
  performance during class

32
Follow up Interviews

• Students feel it is helpful to use Synchroneyes
  since they see more than one way to solve a
  problem
• Some students do not take notes because this
  allows them to focus on what the instructor says,
  knowing they will have access to the instructors
  notes later

33
Overall Results

• More than half of students felt that using
  computers in class improved their performance
  during class
• More than half of students felt that they
  collaborate more using Whiteboard and the Tablet
  PC rather than traditional paper/pencil
• Most students felt comfortable when their work
  was presented to class using Synchroneyes with
  the Tablet PC

34
Future Research

• Do students think it is worth the time and effort
  that it takes to learn to use the Tablet PC?
• Do students think that using the Tablet PCs in
  combination with blackboard helps to keep course
  information more organized?
• Students have access and knowledge about
  computers, thus the background survey is not
  necessary

35
Implications

• Although it takes time both to learn and use in
  the classroom, students think that using
  computers during class improves their classroom
  performance
• Having access to a copy of the instructors notes
  online is helpful to students
• Tablets PCs can be used to increase collaboration
  amongst students in other classes

36
Developing Critical Thinking in the Nursing
Student Using Simulated Online Clinical Scenarios

• Sue Longacre, RN, MS
• Registered Nursing Department
• City College of San Francisco

37
Problem Statement
The new nurses can detect there is a problem
they may be able to determine that it is urgent,
but then all they do is call the doctor. They
cannot diagnose what is wrong, even in instances
where the problem is as common as an oncoming
stroke or heart attack. Betty and Gordon Moore
Foundation Focus Group Study, 2005

38
Critical Thinking Definition

Critical thinking is a purposeful,
results-oriented process that focuses on decision
making, problem solving, and clinical judgment.
Brunt (2005)
39
Critical Thinking Competencies

• Facciones (1990) Critical Thinking Competencies
• Interpretation
• Analysis
• Evaluation
• Inference
• Explanation
• Self-Regulation

.
40
Course Content

• Maternal/Newborn Nursing
• Simulated online clinical scenarios
• Multi-media and text
• Asynchronous learning, available via WebCT
• Students completed written responses via WebCT
• Assignments returned with faculty comments via
  WebCT

41
Research Hypothesis

Students who complete the online clinical
scenarios will demonstrate a larger increase in
their post-test critical thinking skills over
their pre-test critical thinking skills than
those students who do not complete the online
clinical scenarios.
42
Study Design
Research Group Pre-Test (first day of course) Treatment (4 times during course) Post-Test (last day of course)
Experimental (n19) X O X
Comparison (n19) X X
X Critical Thinking Assessment Instrument O
Simulated Online Clinical Scenarios
Formal survey administered on the last day of
class to experimental students to gather feedback
on students perceptions of the online
assignments.
43
Demographics by Research Group

• Gender predominantly female
• Ethnicity experimental group Asian/Filipino
  (68), comparison group Caucasian (68)
• Median age 32
• Direct patient care experience 5 experimental
  and 4 comparison students

44
Clinical Scenario AssignmentExample
45
Patient Assessment

• The patient is a 25 year old Gravida 2 Para 1 _at_
  28 weeks gestation.
• Her first pregnancy and delivery were normal.
• She takes prenatal vitamins.
• Todays fundal height measurement is 28
  centimeters.

Sample Online Clinical Scenario slide using
picture and text.
46
Pregnancy Weight Gain
The patients BMI before pregnancy was 21. Her
prepregnant weight was 59K (130). Her weight
today is 68K (150)
Sample Online Clinical Scenario slide using a
graph and text.
47
Patients Question
The patient is complaining of frequent leg
cramps of her calf area. The patient asks the
nurse, What can I do to stop this muscle
cramping of my legs?
Sample Online Clinical Scenario slide with
patients question.
48
Assignment
You are the nurse caring for this patient today
in the clinic. Based on the information provided

• How would you answer the patients question?
• What would be pertinent nursing diagnoses,
  starting with the highest priority and why?
  (This may include risk for...)
• Describe appropriate nursing actions and/or
  interventions for the patient. Describe your
  rationale for these actions and/or interventions.

.
49
Critical Thinking Assessment Instrument

• DEVELOPMENT
• Standardized Critical Thinking Instruments not
  adequate for nursing
• Instrument developed to measure critical thinking
  competencies in a nursing environment
• Reviewed by _at_ONE staff and nursing faculty
• Piloted on previous students

50
Critical Thinking Assessment Instrument

• SCORING
• Scored by two faculty blind to purpose of
  research project
• Reviewers agreed with majority of scores
• When scores differed, a 3rd faculty assigned
  final score, always agreed with one of previous
  scores

51
Results
Mean Scores for Critical Thinking Pre and
Post-Tests by Research Group

• Both groups mean individual scores show an
  increase in each of the critical thinking
  competencies

• The majority of students in each group improved
  their scores

• Two students in each group demonstrated a
  decrease in their post-test scores over their
  pre-test scores

52
Survey

• The clinical scenario assignments improved my
  ability to analyze multiple assessments in a
  complicated patient.
• 78 strongly agreed
• 22 agreed
• The clinical scenario assignments better prepared
  me to base my nursing care on sound nursing
  theory.
• 67 strongly agreed
• 33 agreed

53
Survey

• Students spent considerable time completing
  assignments, 33 reported spending 5 hours or
  more on each assignment
• Faculty feedback was important in learning
• The most helpful thing is the patients
  question.
• Pictures made it real life.
• It provided me a chance to practice critical
  thinking.

54
Conclusion

• While the performance of both groups improved
  significantly over time, the experimental group
  shows a significantly larger improvement in mean
  critical thinking test scores from pre-test to
  post-test.

55
Implications

• Construct validity of critical thinking
  assessment instrument needs to be established
• Instrument specific to maternal/newborn nursing,
  could easily be adapted to other areas of nursing
• Cost-effective alternative to supplement
  students clinical internship hours
• Students expressed satisfaction with computer
  based clinical scenarios
• Students thought assignments should be part of
  course curriculum

56
Future Research

• Further research may determine what component of
  these clinical scenario assignments improved the
  students critical thinking skills. Was it
• the technology tool,
• the faculty feedback,
• the patients questions,
• or possibly all of these factors combined?

57
Questions?
Contact Information
Michelle Barton MBarton_at_palomar.edu [phone] ext. 2360
Darla Cooper dcooper_at_vcccd.net [phone] ext 7697
Kelvin Leeds leeds_kelvin_at_sac.edu [phone]
Sue Longacre slongacr_at_ccsf.edu [phone]
John Whitmer jwhitmer_at_cccone.org [phone]