SURVEY

1
SURVEY EVALUATION
PHILLIP R. MUSICH, Ph.D. Dept. of Biochemistry
Molecular BiologyJ.H. Quillen College Of
MedicineEast Tennessee State UniversityJohnson
City, TN
2
End-of-course surveys

• The policy at most schools, administered by
  Academic Deans office.
• Advantages
• Provide an overall view of a course, its faculty
  and their teaching.
• Useful in planning future course topics and
  organization.
• Disadvantages
• Faculty appearing early in course not well
  remembered by semesters end.
• End-of-semester exhaustion may bias feedback.
• Feedback too late for current course
  modifications.

3
Why do surveys during a course?

• Identify administrative problems that might
  reoccur during the same semester
• Course organization and event scheduling
• Pre-exam review sessions
• Exam formats
• Provide anonymous communication for students.
• Promote open discussion of perceived problems
• Identify topic- or faculty-specific difficulties.
• Handout organization, availability,
• Presentation style, student interactions,

4
Why do surveys during a course?

• Resolve problems as they occur, reduce student
  frustration.
• Provides a conduit for student venting
  concerning perceived problems
• ? decrease student stress about course, exams
• Also, can be very informative and allow quick
  solutions
• As another conduit, employ a student feedback
  team
• OUTPUT STAYS WITHIN THE BMB DEPARTMENT

5
Survey software Ultimate Survey
v7.0.4http//www.prezzatech.com/
Can be installed on users or companys
server Cost is per server installation Unlimited
surveys of unlimited population (creators and
respondents)
The ETSU link (Export numeric data to Excel for
analysis)
6
Keep it simple
keep it short

• Need 15 surveys 11 clinical conference 4
  blocks
• Enhance response rate in composing the survey.
• Keep the surveys short
• Keep it simple ? easy to complete
• Structure survey for numeric data PLUS commentary
  by students.
• Depend on end-of-course survey for greater
  detailed review.

7
Clinical Case Conference survey

• This conference provided new information on this
  clinical topic/disease.
• This conference enhanced my understanding of the
  biochemical basis of this disease.
• I would rate Dr. Clinician's presentation of this
  conference and his ability to answer questions
  as
• I was adequately prepared for this conference by
  Dr. BSci's previously presented material (in the
  basic science lectures).
• Overall, I would rate this clinical case
  conference as

SA strongly agree, A agree, U undecided, D
disagree, SD strongly Disagree Ex
excellent, G good, Ave average, BAve below
average, P poor
8
Clinical Case Conference survey

• I was adequately prepared for this conference by
  Dr. BSci's previously presented material (in the
  basic science lectures).
• Overall, I would rate this clinical case
  conference as

SA strongly agree, A agree, U undecided, D
disagree, SD strongly disagree Ex
excellent, G good, Ave average, BAve below
average, P poor
9
Chemotherapy Conference (7) survey

• Please provide constructive comments below on
  this conference, its presentation by Dr.
  Clinician, and your preparation for it by the
  BMB facultys (Dr. BScis) lectures. (12 reply
  comments received)
• Dr. Clinician should present every year, he is a
  great speaker and explains everything ...
• So many drug names, so little time.
• I believe I benefited most by the fact that this
  was presented by an MD. I think physicians should
  teach every class in our curriculum...
• I enjoyed the opportunity to coordinate our
  in-class learning with clinical application.
  Anytime that we can do this is greatly
  appreciated.

10
Response Rates - Conferences
11
Faculty teaching survey

• The biochemistry topics presented by Dr. BSci
  were clearly organized and presented.
• The in-class Powerpoints and/or smart-board
  presentations by Dr. BSci provided me with the
  necessary information to learn the topic.
• Dr. BSci enhanced my understanding of the items
  presented.
• In class, Dr. BSci was receptive to and clearly
  answered student questions.
• Dr. BSci was available outside of class time to
  answer questions and/or discuss the biochemistry
  material.
• Overall, as an instructor I would rate Dr. BSci
  as
• Please provide constructive comments below on Dr.
  BSci's material/presentations

12
Faculty teaching Dr. BSci1 comments

• Please provide constructive comments below on Dr.
  BSci1's material/presentations (29 reply
  comments)
• There is obviously a language barrier , but his
  notes and study questions compensated very well.
• test questions were more challenging than what
  I expected. I still thought he did a good job
  though.
• very knowledgeable and did a good job of
  preparing only needs to work on his speaking
  skills
• Excellent, basic presentation of material --
  appreciative.
• Dr. BSci1's study questions helped organize his
  material for studying purposes.
• Good job. Some text to go with the powerpoint
  slides and elaborate on them would be nice.

13
Faculty teaching survey Dr. BSci2 comments

• Please provide constructive comments
• handouts were well-organized, practice quizzes
  helped ...
• One of the best instructors that Quillen has. His
  lectures are interactive and engaging. His
  enthusiasm for the subject, respect for the
  students, friendliness and sense of humor are all
  qualities that professors should strive to
  emulate.
• Dr. BSci2 moved a bit quickly through topics and
  it was difficult to tell what information was
  going to be important for the tests.
• He obviously enjoys the material, which
  encourages us.
• very receptive to any and all questions and was
  very patient I appreciate his approach
• My only complaint is that some of the exam
  questions written by Dr. BSci2 were misleading
  and somewhat ambiguous.

14
Exam survey

• I thought that the exam was fair in its coverage
  of the material presented.
• I had enough time to complete the exam.
• Overall, the questions and possible answers were
  clearly written.
• The honor-code style administration of this exam
  was acceptable.
• Independent of how I did on the exam, I thought
  it was of appropriate difficulty to discriminate
  between different knowledge levels in the class.
• Please provide constructive comments about the
  format and content of the exam.

15
Exam survey comments

• Please provide constructive comments about the
  format and content of exam I. (25 reply comments)
• I definitely think that pre-medical students
  should be required to take Biochemistry before
  their admission.
• Very fair and straightforward. It tested what you
  knew not whether or not you're a good test taker.
• Exam I was fair. I thought it was difficult
  enough without being too picky.
• A lot of trivia on the test. Not a whole lot of
  real understanding was tested.
• The exam questions need to be more clearly
  written. Many questions seemed ambiguous
• Questions should not come from case conferences.
  They should be for enjoyment and learning only.

16
Exam survey comments

• Please provide constructive comments
• The honor-code style administration of this exam
  was acceptable.
• 41 SA, 52 A, 4 U, 2 D, 1 SD
• 93 (SA or A) vs. 4 U vs. 3 (D or SD)
• I really like the honor code method always
  disliked how some students get an edge by
  repeatedly badgering the teacher with questions
  ... This distracts everyone else and takes away
  somewhat from the even playing field...
• The honor-code administration is pleasant, but
  sometimes it is hard to decipher the test
  questions so I would like to a have a professor
  to answer questions.
• There should have been a proctor present.
• There were no teachers available to answer
  questions during the exam.

17
Response Rates Lectures, Exams
18
Outcomes based on feedback

• Early feedback can improve recurring items during
  course
• Improve questions/statements in the survey
  instrument
• Modify later clinical case presentations
• Useful feedback for faculty that appear again
  later in semester
• Modify pre-exam review sessions
• Enhanced communication between faculty and
  students.
• Points for discussion already raised in survey
  feedback
• Especially with the Feedback Team

19
Outcomes based on feedback

• Enhanced ongoing communication between course
  director and course faculty.
• Discuss his/her feedback shortly after
  presentations, exam
• Discuss feedback on his/her associated clinical
  conferences
• Compare to general /- feedback of other faculty,
  cases
• Mentor new basic science and clinical faculty for
  teaching 1st year medical students

20
General conclusions

• Very useful for course director and course
  faculty.
• Valued by students (relatively high response rate
  throughout semester).
• Enhanced ongoing discussions with students and
  team effort for all associated with the Medical
  Biochemistry course.