Can We Teach Design of Experiments On-Line?

1
Can We Teach Design of Experiments On-Line?

• Dr. Dan Rand Winona State University
• 46th Annual Fall Technical Conference - ASQ ASA
• E-mail drand_at_winona.edu

2
Can We Teach Design of Experiments On-Line Agenda

• DoE as a prime tool of Six Sigma
• Instructional design impact on E-learning
• Voice of customer requirements for on-line
  education
• Requirements focus on a DoE course
• Web Search for on-line DoE courses
• Assessing on-line courses
• Examples/ assessments from my course
• An Experiment Design Our Field of Dreams
• Links to resources

3
DoE as a prime tool of Six Sigma

• Design of Experiments is the most effective and
  most complex quantitative tool of Six Sigma Black
  Belts
• Most Six Sigma training is light on DoE
• Black Belts form the latest group that needs
  supplemental DoE training
• They add to the audience that cant sit for a
  traditional classroom course

4
Instructional Design to Meet Learning Needs -
Blooms Taxonomy1
Analysis

• Cognitive skills

Synthesis
Evaluation
Application
Comprehension
Knowledge
5
Instructional Designin Distance Education2

• Contextualize instruction
• Present lesson content
• Present vocabulary
• Provide examples
• Use cueing programs
• Advise learner
• Activate learner processing
• (more)

6
More Instructional DesignAspects from IDDE

• Assess learning
• Provide feedback
• Provide practice
• Test learning
• Sequence instruction
• Logical order
• Prerequisite order
• By content organization

7
General Instructional Design Concepts for
E-Learning

• Modular structure
• Learning objectives for each module
• Active learning
• Assessment / feedback

8
Instructional Design to Meet Organizational ROI

• 4 desired outcomes / assessments of skills
  development training

Learn it Test at end of course
Retain it Apply it
Apply it report on application
Cause benefits ROI
9
What customers want from on-line education

• LINK Learning Network InitiativeCreating
  Effective Collaboration Between Industry and
  University Center Rochester (MN)
• An End-to-End Framework for High Tech
  Workforce Training
• From Technology Industry Working Group
• Drew Flaada, IBM
• Warren Kemplin, Rochester Community Technical
  College
• Brian Nass, XIOtech (Seagate)
• Dr. Daniel Rand, Winona State University

10
End-to-End Learning Management System
1
2
3
4
5
6
A
A
A
A
A
A
A
A assessment of skill / knowledge gap
7
Phases 1 Navigating to relevant offerings
(not pertinent today) 2 - Skill / knowledge gap
assessment selection of optimal offering 3 -
Connecting to the offering (registration,
payment, getting course materials) 4 -
Absorbing the offering (take the class) 5 -
Completing the offering and post-offering
assessment of remaining gaps 6 - Application -
Effectively using the newly acquired
skill/knowledge on the job
11
From Phase 2 of End-to-End LMS Pre-assessment
Selection

• Problem Statement (Voice of Customer)
• I dont know which offering is the best fit for
  me.
• Requirements (Voice of Customer)
• I need a pre-assessment in order to further
  filter out offerings that arent at the
  appropriate level
• I need to be able to filter the possible
  offerings by
• Terms of payment, match from skills assessment,
  cost
• Reputation of provider (quality), certification
  level I require
• Availability, language, delivery method
• Any tools associated with this phase must be
  accessible anytime, from anywhere
• My personal information needs to be protected
  (ongoing on-line education concern)
• Pre-assessment, pre-test for pre-requisites
• Identify goals from employee and employer point
  of view, with live or e-counselor

12
Phase 3 Connecting to the offering

• Problem Statement
• I dont know how to get started and its
  difficult to find out. The whole process is a
  hassle.
• Requirements
• Automate the provisioning of all facilities I
  need (textbooks, databases, labs, server access,
  web site access) Bundle as a student kit
• Test and ensure that my workstation or browser
  will work with the delivery methods used before I
  proceed (example what plug ins are required?)

13
Phase 4 Absorbing the offering (taking the class)

• Problem Statements
• I may not complete the course unless I have
  specific interim goals to hit and/or peers to
  keep me going.
• My day is interrupt-driven.
• If I miss a traditional class, I cant get
  caught up. I need an alternative.
• While in the midst of an offering, I dont know
  if Im on target or not.

14
Phase 4 Absorbing the offering (taking the
class)

• Requirements
• I can communicate with the instructor and peer
  students either real time (chat), via email, or
  via a forum
• I have access to supplemental resources online
• I can take the course from my desk or anywhere
• There is an online assessment tool available to
  help me gauge my progress
• The materials become customized to my specific
  needs, based upon my online assessments (which
  point out remaining gaps)
• I can pause, resume, and replay any of the
  offering
• The offering includes real hands-on projects that
  support my learning
• Estimate total effort needed for a course
• I need (small) modular content

15
Phase 5 Final assessment closure

• Problem Statements
• I have no idea how I did vs. the expected
  outcome or standard.
• I cant drive to a classroom to take the final.
• Im out of town the day of the final.

16
Phase 5 Final assessment closure

• Requirements
• Results of the final assessment need to be fed to
  me, my employer, the instructor
• The final assessment vehicle needs to be
  available online and on my schedule
• I need to be able to pause and resume the
  assessment vehicle
• Assuming successful completion of the assessment,
  any completion credentials (grade, certificate,
  et al) need to be automatically issued
• I need frequent checkpoint assessments given to
  employer (payer), like a completed bar.
• I need a means to re-take course free, or even
  remedial material.

17
Phase 6 Application

• Problem Statements
• I dont know where to go for ongoing help.
• Im unsure how this topic/knowledge should be
  applied in the real world.
• I forgot a great deal in the X months/years
  since the class ended.
• I dont know how to extend this knowledge to
  more advanced levels.

18
Phase 6 Application

• Requirements
• Give me access to all related materials (online)
• Give me access to peer learners and the
  instructor (online)
• Point out options for follow-on instruction
• Provide me with application examples
• Provide me assessment vehicles and
  recommendations for remediation
• Keep material available to student after
  completion of course.
• Form graduate communities / networks to gain
  knowledge of content improvements, growth.

19
Additional General Concerns

• Motivating employee to completion
• Synthesis of training into job performance
• How to direct students to where their success
  rate is the highest
• Need a disconnected mode carry course content
  away on CD-ROM

20
Summarized needs, applied to an on-line DoE
Course

• Content and procedure
• Motivation to finish
• Dont be totally dependent on high speed Internet
• Animation, bells, and whistles may de-motivate
  students if they slow down processing
• Interact with others to discuss and assimilate
  methods
• Feedback from an expert
• Link to textbook

21
Summarized needs, applied to an on-line DoE
Course

• Application
• Practice actual experiments apply it
• Try software to design and analyze
• Work with others in team experiment (bring
  together for catapults or simulation)
• Apply it to students problem- as project, w/
  expert consultant
• Provide a support group of graduates

22
Pragmatic (Market) Limitations

• Even most motivated students need checkpoints
• Market is relatively small for advanced content
  like DoE animation and customization are
  expensive

23
On-line Design of Experiments courses Web Search

• http//www.margaret.net/doe/
• Online Course in Design of Experiments (DOE)
  interactive tutorials, case studies and email
  support. access for 4 weeks, extend 2 weeks,
  expect 30 hours.
• http//www.thequalitygroup.net/
• The Quality Group has a DOE overview module in
  their on-line Six Sigma Green Belt training
• Features animated with Quick Time, pre-tests,
  interactive quizzes, choice of video, captions,
  text, or still pictures

24
More Web Sources

• Eastern Michigan U sponsors ASQ course in DOE
  (1165)
• Stanford - open source learning management system
  (LMS) no DOE
• MIT (Stellar) Open Courseware - no DOE
• University of Phoenix 3 doctoral programs added
  to 1 existing. 125,364 students 5/31/02
• average age38
• no DOE ?- MBA in Technology Management

25
Assessing On-line Courses

• http//taste.merlot.org/
• Multimedia Educational Resource for Learning and
  Online Teaching (MERLOT)
• Non-profit on-line community
• Peer review, user comments on 3 evaluation
  standards
• Quality of Content
• Effective Teaching-Learning Tool
• Ease of Use

26
Industrial Design of Experiments course - online
at Winona State - assessments

• Pre-test on course material.
• After student enrollment and course introduction,
  do survey about the enrollment and start-up
  process.
• After each homework problem with supplied answer,
  ask if the solution is understood, if they could
  do the problem independently, if they want to ask
  for help.
• 1st learning assessment- after review of basic
  statistics and hypothesis testing.
• In conjunction with 1st learning assessment, do
  survey about how its going

27
DoE at Winona State Further assessments

• Further (5) learning assessments, after each
  logical section of course material, including
  assessment of projects (catapult-by-team or
  individual and self-defined)
• Another how its going survey after two-thirds
  of course content
• Final, comprehensive learning assessment
• Final how it went survey, including questions
  of what the student wants for follow-up

28
A pre-test question on BlackBoard
29
(No Transcript)
30
Surface Mount Technology (SMT) experiment -
problem solving in a manufacturing environment

• 2 types of defects, probably related
• - Solder balls
• - Solder-on-gold
• Statistician invited in for a quick fix
  experiment
• High volume memory card product

31
Homework assignments and answers on Blackboard