High Tech Questions, Low Tech Answers

1
High Tech Questions, Low Tech Answers

• Maggie Austen
• Department of Chemistry
• TEL York, April 26, 2006

2
High Tech Questions, Low Tech Answers

• Abstract
• Following (loosely) the lead of Harvard physicist
  Eric Mazur, freshman chemistry lectures are
  interspersed with multiple choice concept
  questions allowing students to check their
  understanding.
• Our feedback mechanism is easy to implement and
  almost as informative. (Instructions follow.)
  Suggestions will be made for further
  improvements.
• More open ended concept questions are posted on
  WebCT before each quiz to guide students in their
  studying. Student response has been very positive
  to both initiatives. (Not discussed in session.)

3
Making a low-tech classroom response system

• Start with construction paper in three colours.
• I used 12 9 paper in red, yellow and green.
• Single colour packs of 24 sheets cost less than
  2 each
• Before the first class, cut paper to rectangles
  in an appropriate size for your classroom needs.
• I used 6 3, for a 200 seat lecture hall.
• It is not easy to read the names at the back of
  the room, but larger would become quite
  cumbersome to keep on the desk.
• Assemble other materials to take to class
• Doutang-type fasteners (2 per student)
• Several rolls of transparent tape
• Several (2- or 3-) hole punchers
• Markers (black / blue), thickness depending on
  size of classroom.

4
Assembling the clickers 1

• Have students tape together three different
  colours of paper, such that they can be easily
  folded along the tape line, but will remain
  sturdy all term.
• Students use markers to write their name (what
  they wish to be called) on each colour.
• Students then continue with assembly

NAME
tape
NAME
tape
NAME
5
Assembling the clickers 2

• Fold over the sheet so that the top and bottom
  edges align. (Do not crease.)
• Punch (2) holes in the edges, which will be used
  both for fastening, and for storing the clicker
  in a binder with the students notes. (3-hole
  punch will give the correct spacing).

6
Assembling the clickers 3

• Fasten with duotangs, to give a triangular prism.
• This will be a bit pinched at the fastened edge.
• Ask a question
• Clearly assign one colour per answer option.
• Students answer by facing the appropriate colour
  towards you.
• This may be built into your lecture notes, as
  follows

NAME
7
First thoughts

• Students learn best when they ____ about the
  topic.
• ? read
• ? hear
• ? talk

How can we help them do this? When there are 500
in a class?
8
The class

• CHEM 1001, Section P, Winter Term
• CHEM 1000 is not a prerequisite should be.
• About 100 students
• Tuesday tutorial / quiz 6 7 pm
• Tuesday lectures 7 10 pm
• After the final exam I posted an anonymous survey
  about the teaching tools I used. The last
  question was open ended for comments.

9
Franks Comments

• Dear Maggie, I think the concept questions have
  succeeded in stimulating students to actively
  digest and critically review the course
  materials. And in most cases, the level of
  challenge in these questions is just enough to
  effectively create an ample environment for
  students to engage in academic debate at the
  first year level.
• Also, the unique idea that multiple choice
  questions are to be answered in class using
  colour-coded cards (instead of electronic
  clickers) makes interactions between the lecturer
  and the audience more personal and visually
  appealing.
• It has succeeded not only in making learning a
  more spontaneous process by seizing students'
  attention, but also in pinpointing and clarifying
  the focal and doubtful points.
• These innovations, along with your conscientious
  input, have worked together to greatly enhance
  the accessibility and applicability of some
  potentially dry concepts. Thank you for making
  the overall learning experience a very pleasant
  one and rekindling my spark for chemistry.
  Sincerely yours, Frank

10
Monikas Comments

• 1. Tricolor flashing cards great idea. It helps
  to focus the students during the lecture. May I
  suggest 2 things that may improve this system?
• One thing is stressing in the first lecture how
  difficult this course is and how it requires to
  study before each lecture just as before each
  lab.
• This may help to ease in the flashing cards the
  first reaction to them is are we in
  kindergarten or what?.
• Second thing - cutting down on chatter during the
  lecture. It is very counterproductive for the
  ones who try to follow the flashing cards
  are very helpful to curb some of that but they
  dont do the job entirely.

11
First impressions of the coloured cards

• For each statement please indicate whether you
  ? agree ? are undecided ? disagree
• This is silly.
• This could be fun.
• This is a unique idea.
• This could be helpful in my classes.

12
Students impressions (as reported on the
end-of-term survey)

• In my lecture notes I often included little
  multiple choice questions, to be answered using
  the coloured cards we made in the first class.
• What did you think of this idea? Choose all that
  apply. (26 respondents)
• a. I thought it was a silly idea. 2
• b. I thought it was a fun idea. 14
• c. I thought it was a unique idea. 17
• d. I thought it was a helpful idea. 18
• I never thought about it at all. 0
• (1 chose a only, 1 chose a/b/c/d)

13
In my lecture notes I often included little
multiple choice questions, to be answered using
the coloured cards we made in the first class.

• How useful, for paying attention and
  understanding during class, did you find them to
  be?
• How useful, for preparing to write quizzes and
  exams, did you find them to be?
• a. Very useful 11 5
• b. Somewhat useful 14 11
• c. Not very useful 1 8
• d. Very useless 0 2
• e. I never looked at them 0 0

• What did the students find the in-class questions
  more useful for?
• paying attention in class ? test preparation
• ? equally useful for both ?

14
A student repeating the course said

• In regards to Maggie's actual teaching style, I
  would pretty much stick to how things are
  currently because they were a significant
  improvement. ...
• However, for the multiple choice questions, I may
  include slides which explain how one got those
  answers, because sometimes on going back and
  reviewing, it was difficult, if not impossible,
  to work through them again to get the same answer
  or to apply it to similar answers.
• Would you include answers in your posted notes?
• Yes / No / ?Sometimes? Discuss with your
  neighbour!

15
Other uses for the coloured cards

• Polling about anything where you just want a
  general impression, not precision.
• Gauging the students level of comfort with the
  material
• Should I move on / go faster, or do we need to
  spend more time on this?
• Hence the traffic light colour scheme Go /
  / Stop
• This could be on a continuous basis during class,
  but it is very hard (for both instructor
  students) to attend to.
• Better to build specific check-in points into
  your lecture?

Slow
16
Date Tue, 10 Jan 2006 234404 -0500 To
austenm_at_yorku.ca Subject Chemistry 1001 Hi
Maggie Im a student in your chemistry 1001
class on Tuesday evenings. I just wanted to
welcome you to York. I cant emphasize how nice
it is to have a professor that is up beat and
actually seems to care about the students
level of absorption especially in a course like
chemistry. It is a nice change to say the
least. I hope you have a good experience at
York Sincerely (Name removed)
A few hours after our first class of the term
17
Same student (self-identified) on (anonymous)
end-of-term survey

• Ok. First of all Maggie I thought your approach
  to this class was excellent. I've never had such
  an enthusiastic professor in a chemistry course.
  I really thought you made the class more
  enjoyable with your enthusiasm.
• The coloured cards idea was very smart, the only
  thing which was bad was that many people didn't
  bring them.
• Your office hours were extremely helpful when I
  came.
• hhmmmm...
• ok toooo many concept questions, anything more
  than 10 is a lot.

These are explained in the next section.
18
Was the number of questions appropriate?

• a. There were too many questions.
• b. There were about the right number of
  questions.
• c. There were not enough questions.
• d. Don't know / don't care.
• a b c d
• In-Lecture MC 0 17 8 1
• Concept review 7 16 2 1
• Next year, what should I have more of?
• ? concept review questions ? multiple choice
  questions ? both ?

19
More MC questions needed

• Clearly many students would like to see more of
  these questions, and none felt that there were
  too many.
• Having more questions would probably also help
  with encouraging students to bring their cards to
  class, and to take them out and use them.
• I would encourage you to have the students take
  out their cards right at the beginning of class,
  perhaps always starting with a quick question.
• The mental barrier to getting them out in the
  middle of the class seems to be significantly
  higher.

20
More recommendations from survey responses of the
1st day e-mailer

• One more thing to mention was that the people who
  answer questions are the minority generally, so
  when someone yells out an answer and most of the
  other people don't know the answer, it's not a
  good indication to follow what people shout out
  in the middle of class.
• Sometimes a small minority would know and a
  majority would not know, but when they yell the
  answer you'd assume everyone knew, this is not
  your fault and Im not blaming you Im just
  trying to bring subtle things to your attention.
  Sometimes we'd carry on and I'd be lost.

More reason to use the response system more often
and respond to it!
21
Titration of strong base with strong acid
Examples of in-class questions
What is the pH at the equivalence point? 10 / 7 /
4
22
Solution Buffering and Weak-Strong Titrations
Titration of a weak acid with a strong base
Where is the buffer region? Near pH of 5 / 9 / 12
23
Which of the following mixtures are buffers?
These ones should be straight forward for
attentive students

• Equimolar amounts of
• KH2PO4 and H3PO4
• NaClO4 and HClO4
• C5H5N (pyridine) and C5H5NHCl
• Na2CO3 and NaHCO3
• NaOH and HCl

Discuss with a neighbour, then well vote
Yes Dont know No
24
Which of the following mixtures are buffers?
These ones are more challenging

• 1 mol KH2PO4 and 0.5 mol HCl
• 1 mol KH2PO4 and 0.5 mol NaOH
• 1 mol NH3 and 0.5 mol NaOH
• 1 mol NH3 and 0.5 mol HCl
• 1 mol NH3 and 1 mol HCl

We need a strategy
Yes Dont know No
25
Determining which solutions are buffers.
Here is the strategy, then we tried the last set
again

• Identify all species present as strong/weak (or
  neutral) acid/base (or amphiprotic).
• Assume all strong acids and bases (H and OH-)
  react to completion with any weak bases or acids
  present.
• Check for significant amounts of both a weak acid
  and its conjugate base. (Use the buffer game.)
• A common approach is to add half as much strong
  acid to a weak base, or half as much strong base
  to a weak acid.

26
What kind of questions?

• In-class multiple choice can highlight important
  points, test key skills or help clarify confusing
  details. See acid-base examples above.
• I used open-ended concept (p)review questions to
  help guide and focus the students reading and to
  encourage reflection and application / synthesis
  of ideas.
• Identify the key points or learning goals for the
  chapter / section. Transform these into
  questions.
• These were posted on webCT before each quiz. I
  would have preferred to post them before the
  class started each topic, but I was behind all
  term in my preparation.
• Preview versus reviewnext time!

27
For each quiz, I posted to webCT a set of
concept questions for you to answer.

• If and when you did look at these, how useful
  (for preparing to write quizzes and exams)
  did you find them to be? MC
• a. Very useful 9 5
• b. Somewhat useful 14 11
• c. Not very useful 2 8
• d. Very useless 0 2
• e. I never looked at them 1 0

For comparison we saw this data earlier

• Which did the students find more useful
• in preparing for their quizzes and final exam?
• concept review questions ? multiple choice
  questions
• ? they were equal ?

28
Review Questions

• For each quiz, I posted to webCT a set of concept
  questions for you to answer while reading the
  textbook.
• What did you normally do with these questions?

29
Review questions as study aids

• When you used these questions to study, which of
  the following did you normally do? (Choose all
  that apply)Which of the following did you once
  or sometimes do?

30
PReview Questions

• In the very first lecture I wrote out a set of
  questions for you to answer while reading the
  textbook. They were later posted on WebCT.
• What did you do with these questions?
• In hindsight, what do you wish you had done?
• Read the book and tried to answer the relevant
  questions

31
For each quiz, I posted to webCT a set of concept
questions for you to answer while reading the
textbook.

• If the later review questions had been available
  from the beginning of term, or at least a week in
  advance of the lectures,
• what do you think you should have done with them?
• what do you think you would have done with them?
• Read the book and tried to answer the relevant
  questions

Which way do the answers shift?
Did 1st
32
Would you post the answers to review questions?
Yes/Maybe/No

• According to Daniel
• I would have preferred to have solutions
  available for the concept/review questions. There
  were some questions that I had trouble with. Not
  having the accurate solution creates an insecure
  atmosphere for the student and is
  counter-productive.
• Essentially, questions are raised and when I
  couldn't get a definitive answer immediately it
  left me questioning my understanding of the
  subject.
• Although I could have made an appointment to
  discuss this with you, I didn't find that it was
  a time-effective approach.
• For example, the use of the study guide with full
  solutions to textbook questions proved to be a
  tremendous help because when I had difficulty on
  any question (assigned or unassigned), I found
  the answer immediately and was able to master the
  concepts that I had trouble with.
• now compare his friend Monika

33
Would you post the answers to review questions?
Yes/Maybe/No

• 2. I would like to thank you for bringing
  enthusiasms and element of fun to this course. I
  would also like to thank you for being so present
  and helpful in our electronic discussions.
• 3. Maggies review questions are one of the most
  useful studying tools. I wish we could have the
  rough layout of them at the beginning of each
  section and a final, more detailed version at the
  beginning of the pre-quiz time.
• Having to answer them personally, comparing the
  answers with fellow students and you pitching in
  the discussion as well was one of the best ways
  of mastering the knowledge in a deep and lasting
  way.
• It was also a breeze to review these notes before
  the final exam, yet at the same time it was the
  most in-depth review of the whole material.

34
Who took the survey?
35
Who took the survey?
36
Who took the survey?
Similar results for other help sessions available
37
Grade Expectations

• At the beginning of term, what grade were you
  hoping to achieve in this course?
• At the end of term, what grade do you expect to
  achieve in this course?

Note this survey was taken after the final exam.
Compare to these students results in CHEM 1000.
38
Overall impressions

• Overall, how useful would you say this course was
  (relevance of content, etc.)?
• a. Very useful 14
• b. Somewhat useful 11
• c. Not very useful 1
• d. Pretty useless 0

• Overall, how challenging would you say this
  course was?
• a. Very challenging 17
• b. Somewhat challenging 7
• c. About average 2
• d. Not very challenging 0
• e. Pretty easy 0

39
Overall impressions

• Overall, how enjoyable would you say this course
  was?
• a. Very enjoyable 5
• b. Somewhat enjoyable 11
• c. Not very enjoyable 9
• d. Not at all enjoyable 1

• Overall, how would you rate my (Maggie's)
  effectiveness in terms of teaching style?
• a. Very effective 19
• b. Somewhat effective 6
• c. Not very effective 1
• d. Very ineffective 0

40
Various students comments on end-of-term survey

• Most students thought that the multiple choice
  idea was good, as it made the class interactive.
• I thought the interactive and personal approach
  that you used was excellent in providing a
  comfortable teacher-student relationship. You
  were very approachable which facilitated
  discussions.
• You are a great teacher, you know your material
  very well and more importantly, you know how to
  deliver it. I love your teaching style and
  presentation skills. I wish you all the best.
  Thank You.
• If it wasn't for your innovative way of teaching
  I don't think I would have passed my quizzes.
  Your lecture notes were extremely helpful,
  perfect supplement to the text (which at times
  was confusing) until I re-read it. I enjoyed your
  enthusiasm, you made taking chem fun The
  examples you added to your lecture notes were
  great as well as the problem sets (very helpful).

41
More unsolicited e-mail, from a different student

• Date Fri, 24 Feb 2006 133633 -0500 To
  austenm_at_yorku.ca
• Subject chem 1000 feedback.
• Good afternoon Professor,
• I just want to compliment on your superior
  teaching skills, care for students, joyful
  personality, and stringent method of open-based
  inquiry for students.
• Certainly, as a student who once geared going to
  Mcmaster (and was accepted), I can see why
  Mcmaster's education is top class bar none. I
  have never met a professor _at_ York so committed to
  his/her work like yourself. Kudos!!!!!
• Many thanks.
• (Name removed)
• P.S. any reasons for coming to York?
• Any plans to teach more courses, etc?
• Thanks again.
• Enjoy the sunshine )

End of reading week.
I used to teach at McMaster, before my mat leave.
42
Engaged students loved it!(and got the highest
marks in the class)
Final thoughts

• Others were less happy with the approach I took.
• Combined with non-traditional demographics and a
  bad schedule, this led to low attendance on most
  nights.
• Most (?) appreciated the fact that I expressed my
  interest in their learning process. What did the
  other 75 think?
• How to get more students engaged?
• Will it work with a class of 500, rather than
  100?
• What are the implications for doing this in the
  largest section of a multi-section course (versus
  the smallest)?
• I emphasized ideas, but the exam emphasized
  numerical problem solving. Marks suffered. What
  to do about this?