The Use of Webbased Technology in an Introductory Environmental Engineering Course

1
The Use of Web-based Technology in an
Introductory Environmental Engineering Course

• Susan J. Masten, Kuan-Chung Chen,
• Michael Najera, Jaime Graulau-Santiago,
• Subhash L. Kari, and Kyung-Hyuk Lee
• Department of Civil and Environmental
  Engineering.
• Michigan State University
• East Lansing, MI 48824

2
Abstract

• This paper will focus on the use of web-based
  technology in the course Introduction to
  Environmental Engineering (CE 280). An extensive
  website, web-based quizzes and homework
  assignments, and tutorials have been developed
  and are available on the course website
  (www.egr.msu.edu/classes/ce280). In 1999,
  web-based quizzes and tutorials were introduced.
  In 2001, homework assignments were computerized
  using the CAPA (Computer Assisted Personalized
  Approach) software developed at MSU. A small
  percentage of the class completed CAPA
  assignments in the final hours that the homework
  was due. This percentage increased when the due
  date was changed from Friday to Sunday. There was
  only a very weak correlation between the homework
  score and in-class midterm exams. As such, the
  homework assignments do not appear to assess
  individual learning or mastery of the material.
  Traditional homework is a better assessment tool
  (of examination performance) than are the
  CAPA-based assignments. When CAPA was used, there
  was a shift of the final grades away from the
  traditional bell-shaped curve, although this
  shift could not be correlated with increased
  performance on in-class examinations. The shift
  is likely to be due to the increased weighting of
  CAPA homework (compared to traditional homework)
  and the additional points gained in completing
  the take-home examinations.

3
Introduction to Environmental Engineering (CE
280)

• CE 280 is part of the Civil Engineering and
  Engineering Arts Curricula. It covers
• surface- and ground- water hydrology
• water quality
• municipal water and waste water treatment
• noise pollution
• air pollution

4
Introduction to Environmental Engineering (CE 280)

• It is offered twice per year. Enrollment is
  typically about 120-150 students per semester.
• 30 Civil Engineering
• 10 Environmental Engineering
• 30 Engineering Arts
• 15 Other engineering Disciplines
• 15 Non-engineering

5
Course objectives

• Understanding of mass balance equations applied
  to
• surface water hydrology
• chemical contamination in lakes and rivers
• indoor air pollution
• Understanding of how mathematical expressions are
  used to model complex phenomena
• Hortons infiltration equation
• Streeter-Phelps DO Sag Curve Equation
• Understanding of tradeoffs in engineering design
• water treatment
• pollution from automobiles
• Understanding of the relationship between
  environmental engineering and other disciplines

6
Course website (www.egr.msu.edu/classes/ce280)

• The website serves as a repository of course
  information and material. Using the website, at
  any time of the day or night students have access
  to course description, syllabus, reading
  assignments, study guides, past examinations,
  homework assignments, and practice problems,
  along with general information such as
  announcements, location and the time of office
  hours and help sessions and the grading criteria.
  The location of this material on the website,
  also allowed students to access the desired
  information from their homes, or their residence
  halls, avoiding the need to visit the Engineering
  library or travel long distances. In addition, a
  virtual library was created where links to
  relevant information and resources are given.
• The website was created using HTML and the
  scripting language, JavaScript. The website can
  be viewed using any standard browser, such as
  Netscape3.0 or Internet Explorer4.0. Frames were
  used to make the website user friendly and to
  provide a uniform look to the website. This
  allows the students to return to the index page
  without having to experience the frustration of
  getting lost in the website

7
Course website (www.egr.msu.edu/classes/ce280)
8
Course website
9
Interactive Tools

• It is our hypothesis that due to the rapid
  changes in communication and computing, current
  students are much more demanding of instant
  feedback. As such, we have attempted to develop
  interactive tutorials to assist student learning.
  These tools were developed using the programming
  languages JavaScript and Java. We developed
  interactive multiple-choice questions to aid in
  the students conceptual understanding of the
  material.. If a students chose the wrong answer,
  he/she was immediately prompted about the
  possible flaw in his/her approach. Java applets
  were used to develop interactive calculators such
  as the Dissolved Oxygen Sag Curve plotter. The
  purpose of these calculators was to give the
  students the resources needed to master certain
  important concepts and the effect of various
  parameters on the nature of the solution
  (presented as a curve). We believe that such
  calculators, which are easy to use, greatly
  reduce the problems associated with the students
  having to do such study in isolation. In
  addition, some problems were solved and students
  were guided through the various steps with
  different levels of help available at each level.
  If the students did not follow the steps they
  could click on the Help link and hints were
  provided.

10
Interactive Tools
11
Interactive Tools Assessment

• Based upon student surveys, we found that in
  all four semesters students did not use tutorials
  unless required to do so. This may have been
  because they did not know they were there, forgot
  they were there or simply it was easier to ask
  someone else than to go to the tutorial. In 2000,
  fewer students used the tutorials than in the
  previous year. This may have been the result of
  omitting the web-based quizzes in 2000.
  Additionally, CAPA had not been implemented,
  resulting in less web usage in general. Although
  base groups and active learning exercises were
  used, tests were traditional in nature. In 2002,
  even when the tutorials were included on a
  take-home exam assignment, only 51 used the
  tutorials.
• The tutorials were found to be useful when
  used. 46 percent of the 1999 class said that
  tutorials aided their learning of course
  material. Since 59 of the same class said they
  used the tutorials, this means that the vast
  majority (78) of those using the tutorials
  found them helpful to their learning process.
• In order to facilitate the use of the
  tutorials, we have begun to place links in the
  homework assignments to direct them to the
  tutorials when they have help.

12
CAPA Computer Assisted Personalized Approach

• CAPA was developed by Physics Department at MSU.
  It is supported by
• MSU and National Science Foundation and Sloan
  Foundation grants
• One of our major goals in implementing the use
  of CAPA was to engage
• students in the learning process. CAPA was used
  for both homework
• assignments and examinations.

13
CAPA Homework assignments

• Assignments were individualized and given on a
  weekly basis. The data given in problems can be
  randomized so that each student had a unique
  problem. Similarly, the choices of correct
  answers for multiple choice problems were
  randomized. Multiple attempts can be allowed (set
  at 2-99). Problem types included multiple
  choice, problems, fill-in blank, matching, essay,
  identification of items on a figure,
  quantitative. Link or tags can be provided for
  additional help to students.

14
CAPA Assessment

• One of the advantages of using CAPA is that it
  allows the instructor to monitor student
  performance. For example, during the semester,
  the instructor can access multiple statistical
  information - including assessment of class and
  individual performance on particular problems or
  the entire set. We found this very useful since
  during the week as students were completing their
  homework we could see which problems students
  were having trouble with - allowing me to review
  specific material in lecture. CAPA also allows
  the instructor to track student submissions and
  course records.

15
CAPA Asynchronous Learning

• CAPA allows the instructor to set up a
  discussion page. Students can post questions or
  comments anonymously. The instructor has
  monitoring privileges so that she/he can see
  who is posting even if the student wishes to
  remain anonymous to the other students. She/he
  can remove messages if desired.
• Students can respond to each others questions.
  In effect, the discussion page provides students
  with a mechanism to form a virtual base group,
  providing the intellectual and emotional support
  that a physical base group would do., but
  avoiding the issues of meeting in the same
  location.

Screen dump discussion page
16
CAPA Examinations

• CAPA was also used to administer examinations.
  Using CAPA, individualized exams could be
  prepared for each student. Exams contained both
  multiple choice, 1 of n or n of m and
  quantitative problems. They were scored by a
  computer.
• After completing the in-class examination, each
  student was given a take-home examination upon
  completion of in-class examination. Completion of
  take-home examination was used to reinforce
  concepts and allow students to earn additional
  points (since no partial credit was given on
  quantitative problems).
• The total exam score was calculated as
• the inclass exam score
• 30 X (Difference between the in
  class and take-home exam). Completion of the
  take home exam could only add points a lower
  score on the take-home exam would not result in a
  reduction in the total exam score.

17
CAPA Examinations
18
CAPA Assessment

• By surveying the students, we found that the
  vast majority of students felt that CAPA aided
  their learning of course material. Only 16 of
  the class disagreed with this statement. Similar
  results were observed in 2001 and 2002.
• We also found that attendance at office hours
  increased greatly with 56 of the students
  stating that they visited office hours in 2002
  and 44 in 2001 - compared to 17 in 2000). The
  question of attending office hours was not asked
  in 1999.
• Additionally a greater percentage of the class
  felt that material was graded fairly (2002 84
  2001 71, 2000 53, 199966).

19
CAPA Student Frustrations

• Despite the many positive aspects of CAPA, we
  found several problems associated it with it.
  Much more so than with conventional homework, we
  found that students expect to earn 100 on
  homework assignments. They will continue trying
  problems and get frustrated if they cannot get
  the correct answer. Often they will spend long
  periods of time trying to to get a problem
  correct and then get very anxious as they use up
  the allotted tries.
• Example of typical student e-mail message
  (edited to simplify message).

Ok, Either CAPA is coded wrong, or there is
something else wrong with this problem. I have
tried it two more times, and I only have one try
left. After putting Do(mix)3.512 and the
BOD(mix)12.483 into the Tc equation I got 1.95
days, and that is wrong.... of course. I can't
get the rest of the problems without getting this
one... So I don't know what else to do. I hope
you can help me.. I can not see what I am doing
wrong, and I am scared to try anything else
because I only have one try left. I have tried
doing this problem sooo many different ways.. I
don't even know what is left. Hopefully you can
help me. (emphasis added by author).
20
CAPA Student Frustrations

• Since homework can be completed 24 hours a day -
  students often worked on CAPA late at night. In
  2001, homework was due on Friday nights.
  However, students suggested that it would have
  been better to have had the weekend to complete
  the assignments. So in 2002, homework due dates
  were changed to Sunday nights at 1130 pm. As a
  result, many students completed their homework
  late on Sunday nights at a time when neither the
  instructor nor TAs were available for assistance.
  Although students were encouraged to work on
  homework before Sunday night, many did not, as
  shown in the figure below. For homework
  assignment 10, 75 (out of 134) students were
  on-line in the last 5.5 hours completing their
  assignment. This compared to 27 students (out of
  141) in 2001 when homework was due on a Friday
  night. Unfortunately, this leads to a sense of
  student frustration when they could not get the
  assistance they desired.

21
CAPA Student Frustrations
Number of students on-line completing the
required assignment in the last 5.5 hours before
it was due. In 2001, the homework was due on a
Friday night. In 2002, it was due on a Sunday
night. In both cases it was due at 1130 pm.
Conceptual problems
Quantitative problems
Conceptual problems
Quantitative problems
22
CAPA Student Frustrations

• Students are used to getting partial credit for
  problems on exams - and even though they had the
  opportunity to earn additional exam points by
  completing the take-home - they were still
  frustrated when they made some progress on a
  problem but did not get any points for the
  in-class portion of the exam.
• Finally, although not documented there seems to
  be a greater sense of isolation and alienation
  resulting with the use of instructional
  technology. Despite the time commitment for
  developing CAPA students seem to perceive that it
  is done to save time, and because of a lack of
  commitment to undergraduate teaching.

Comment in response to announcement that final
exam would be graded by hand and partial credit
would be given for work on quantitative problems
I am positive having partial credit on the
final will help me out tons, since I seem to know
what I am doing on most of the questions, I just
make stupid mistakes. Comment in response to
lack of partial credit for quantitative problems
on mid-semester examinations I just feel that if
a TA or yourself could have seen the amount of
work, which most of it was correct, I had for
each problem my efforts would have increased my
grade.
23
CAPA Assessment
Ultimately the question we need to ask is Does
CAPA increase learning? I have attempted to
answer that by looking at the correlation between
the in-class exam grades and homework scores. In
1997, before the implementation of any web-based
technology, we see a reasonable correlation
between the in-class exam grades (reported as a
percentage of the total possible score attainable
for the three exams) and the homework
grades. However, in 2001, with CAPA we see that
the homework grades are clustered in the 80-100
range with many in the 90 range. This led us to
the conclusion that homework grades not well
correlated with exam grades and that CAPA
homework cannot be used as an assessment of
student learning.
24
CAPA Assessment
25
CAPA Assessment of learning

• By surveying the students, we found that the
  vast majority (67) of students felt that CAPA
  aided their learning of course material. Only
  16 of the class disagreed with this statement.
  Similar results were observed in both 2001 and
  2002.
• At the 2000 FIE meeting in Kansas City, MO,
  Kashy et al. reported that when CAPA was used ,
  there was a clear shift away from the traditional
  bell-shaped curve. In the next figures, the
  final grade data obtained for CE280, Spring 1997,
  2000, 2001 and 2002 are given. In 1997, CE 280
  was taught without the use of any web-based
  technology. In 2000, only the website was used.
  In both 2001 and 2002, CAPA was used. While in
  2001 and 2002 there was somewhat of a shift
  skewing the grades towards the higher values, we
  did not see the skewing reported by Kashy et al.
  Additionally, when one considers only the raw
  in-class examination scores, one does not either
  a shift in the mean score for each exam or the
  distribution of grades for the in-class exams
  that would be consistent with increased learning.
  In fact, the increase in the number of grades in
  the 40-59

26
CAPA Assessment of learning
range on Exams 2,3 and the final in 2002,
suggests just the opposite. The shift is likely
to be due to the increased weighting of CAPA
homework (compared to traditional homework) and
the additional points gained in completing the
take-home examinations.
27
CAPA Assessment of learning
28
CAPA Assessment of learning
Note partial credit was given on quantitative
problems in 1997 and 2000 and on the final exam
in 2002.
29
Conclusions

• Students did not take advantage of all material
  available, although those that did found the
  tutorials useful to their learning of material.
• CAPA can be used to track student performance
  during the semester and therefore modify lectures
  accordingly.
• CAPA can be used to assist students to learn
  material and to provide virtual base groups via
  the discussion board.
• The implementation of CAPA appears to have
  increased student engagement as measured by the
  percentage of students attending office hours.

30
Conclusions

• Some students became frustrated completing their
  CAPA assignments.
• Students did not like not getting partial credit
  on quantitative problems.
• Performance on CAPA homework cannot be used to
  assess examination performance.
• In-class examination scores did not support the
  hypothesis that the use of CAPA increased student
  learning, although the majority of students
  self-reported that it did.

31
Acknowledgements

• Funding
• GE Fund
• Department of Education Graduate Assistantships
  in Areas of National Need
• Michigan State University
• Colleagues at MSU
• Prof. James Fairweather, College of Education
• Prof. J.D. Fisher, Dept. of Electrical
  Engineering
• Prof. G. VanDusen, former Assistant Dean of
  Undergraduate Education, College of Engineering
• Prof. Thomas Wolff, Associate Dean, MSU