Title: Teaching Statistical Concepts with Activities, Data, and Technology
1Teaching Statistical Concepts with Activities,
Data, and Technology
- Beth L. Chance and Allan J. Rossman
- Dept of Statistics, Cal Poly San Luis Obispo
2Goals
- Acquaint you with recent recommendations and
ideas for teaching introductory statistics - Including some very modern approaches
- On top of some issues we consider essential
- Provide specific examples and activities that you
might plug into your courses - Point you toward online and print resources that
might be helpful
3Schedule
- Introductions
- Opening Activity
- Activity Sessions
- Data Collection
- Data Analysis
- ltlt lunchgtgt
- Randomness
- Statistical Inference
- Resources and Assessment
- QA, Wrap-Up
4Requests
- Participate in activities
- 23 of them!
- Well skip/highlight some
- Play role of student
- Good student, not disruptive student!
- Feel free to interject comments, questions
5GAISE
- Emphasize statistical literacy and develop
statistical thinking - Use real data
- Stress conceptual understanding rather than mere
knowledge of procedures - Foster active learning in the classroom
- Use technology for developing conceptual
understanding and analyzing data - Use assessments to improve and evaluate student
learning - www.amstat.org/education/gaise
6Opening Activity
- Naughty or nice? (Nature, 2007)
- Videos http//www.yale.edu/infantlab/socialevalua
tion/Helper-Hinderer.html - Flip 16 coins, one for each infant, to decide
which toy you want to play with (headshelper) - Coin Tossing Applet http//www.rossmanchance.com/
applets
73S Strategy
- Statistic
- Simulate
- Could have been distribution of data for each
repetition (under null model) - What if distribution of statistics across
repetitions (under null model) - Strength of evidence
- Reject vs. plausible
8Summary
- Use real data/scientific studies
- Emphasize the process of statistical
investigation - Stress conceptual understanding
- Idea of p-value on day 1/in one day!
- Foster active learning
- You are a dot on the board
- Use technology
- Could this have happened by chance alone?
- What if only 10 infants had picked the helper?
9Data Collection Activities Activity 2 Sampling
Words
- Circle 10 representative words in the passage
- Record the number of letters in each word
- Calculate the mean number of letters in your
sample - Dotplot of results
10Sampling Words
- The population mean of all 268 words is 4.295
letters - How many sample means were too high?
- Why do you think so many sample means are too
high?
11Sampling Words
- Tactile simulation
- Ask students to use computer or random number
table to take simple random samples - Determine the sample mean in each sample
- Compare the distributions
12Sampling Words
- Java applet
- www.rossmanchance.com/applets/
- Select Sampling words applet
- Select individual sample of 5 words
- Repeat
- Select 98 more samples of size 5
- Explore the effect of sample size
- Explore the effect of population size
13Morals Selecting a Sample
- Random Sampling eliminates human selection bias
so the sample will be fair and unbiased/representa
tive of the population. - While increasing the sample size improves
precision, this does not decrease bias.
14Activity 3 Night Lights and Near-Sightedness
- Quinn, Shin, Maguire, and Stone (1999)
- 479 children
- Did your child use a night light (or room light
or neither) before age 2? - Eyesight Hyperopia (far-sighted), emmetropia
(normal) or myopia (near-sighted)?
15Night Lights and Near-Sightedness
Darkness Night light Room light
Near-sighted 18 78 41
Normal refraction 114 115 22
Far-sighted 40 39 12
16Night Lights and Near-Sightedness
17Morals Confounding
- Students can tell you that association is not the
same as causation! - Need practice clearly describing how confounding
variable - Is linked to both explanatory and response
variables - Provides an alternative explanation for observed
association
18Activity 4 Have a Nice Trip
- Can instruction in a recovery strategy improve an
older persons ability to recover from a loss of
balance? - 12 subjects have agreed to participate in the
study - Assign 6 people to use the lowering strategy and
6 people to use the elevating strategy - What does random assignment gain you?
19Have a Nice Trip
- Randomizing subjects applet
- How do the two groups compare?
20Morals
- Goal of random assignment is to be willing to
consider the treatment groups equivalent prior to
the imposition of the treatment(s). - This allows us to eliminate all potential
confounding variables as a plausible explanation
for any significant differences in the response
variable after the treatments are imposed.
21Activity 5 Cursive Writing
- Does using cursive writing cause students to
score better on the SAT essay?
22Morals Scope of Conclusions
The Statistical Sleuth, Ramsey and Schafer
Allocation of units to groups Allocation of units to groups
By random assignment No random assignment
Selection of units Random sampling A random sample is selected from one population units are then randomly assigned to different treatment groups Random samples are selected from existing distinct populations Inferences to populations can be drawn
Selection of units Not random sampling A groups of study units is found units are then randomly assigned to treatment groups Collections of available units from distinct groups are examined
Cause and effect conclusions can be drawn
23Activity 6 Memorizing Letters
- You will be asked to memorize as many letters as
you can in 20 seconds, in order, from a sequence
of 30 letters - Variables?
- Type of study?
- Comparison?
- Random assignment?
- Blindness?
- Random sampling?
- More to come
24Morals Data Collection
- Quick, simple experimental data collection
- Highlighting critical aspects of effective study
design - Can return to the data several times in the
course
25Data Analysis ActivitiesActivity 7 Matching
Variables to Graphs
- Which dotplot belongs to which variable?
- Justify your answer
26Morals Graph-sense
- Learn to justify opinions
- Consistency, completeness
- Appreciate variability
- Be able to find and explain patterns in the data
27Activity 8 Rower Weights
- 2008 Mens Olympic Rowing Team
28Rower Weights
Mean Median Full Data Set 197.96 205.00 Wi
thout Coxswain 201.17 207.00 Without Coxswain
or 209.65 209.00 lightweight rowers With
heaviest at 249 210.65 209.00 With heaviest at
429 219.70 209.00 Resistance....
29Morals Rower Weights
- Think about the context
- Data are numbers with a context -Moore
- Know what your numerical summary is measuring
- Investigate causes for unusual observations
- Anticipate shape
30Activity 9 Cancer Pamphlets
- Researchers in Philadelphia investigated whether
pamphlets containing information for cancer
patients are written at a level that the cancer
patients can comprehend
31Cancer Pamphlets
32Morals Importance of Graphs
- Look at the data
- Think about the question
- Numerical summaries dont tell the whole story
- median isnt the message - Gould
33Activity 10 Draft Lottery
- Draft numbers (1-366) were assigned to birthdates
in the 1970 draft lottery - Find your draft number
- Any 225s?
34Draft Lottery
35Draft Lottery
- month median
- January 211.0
- February 210.0
- March 256.0
- April 225.0
- May 226.0
- June 207.5
- month median
- July 188.0
- August 145.0
- September 168.0
- October 201.0
- November 131.5
- December 100.0
36Draft Lottery
37Morals Statistics matters!
- Summaries can illuminate
- Randomization can be difficult
38Activity 11Televisions and Life Expectancy
- Is there an association between the two
variables? - So sending televisions to countries with lower
life expectancies would cause their inhabitants
to live longer?
r .743
39Morals Confounding
- Dont jump to conclusions from observational
studies - The association is real but consider carefully
the interpretation of graph and wording of
conclusions (and headlines)
40Activity 6 Revisited (Memorizing Letters)
- Produce, interpret graphical displays to compare
performance of two groups - Does research hypothesis appear to be supported?
- Any unusual features in distributions?
41Lunch!
- Questions?
- Write down and submit any questions you have thus
far on the statistical or pedagogical content
42Exploring RandomnessActivity 12 Random Babies
- Last Names First Names
- Jones Jerry
- Miller Marvin
- Smith Sam
- Williams Willy
43Random Babies
- Last Names First Names
- Jones Marvin
- Miller
- Smith
- Williams
44Random Babies
- Last Names First Names
- Jones Marvin
- Miller Willy
- Smith
- Williams
45Random Babies
- Last Names First Names
- Jones Marvin
- Miller Willy
- Smith Sam
- Williams
46Random Babies
- Last Names First Names
- Jones Marvin
- Miller Willy
- Smith Sam
- Williams Jerry
47Random Babies
- Last Names First Names
- Jones Marvin
- Miller Willy
- Smith Sam 1 match
- Williams Jerry
48Random Babies
- Long-run relative frequency
- Applet www.rossmanchance.com/applets/
- Random Babies
49Random Babies Mathematical Analysis
- 1234 1243 1324 1342 1423 1432
- 2134 2143 2314 2341 2413 2431
- 3124 3142 3214 3241 3412 3421
- 4123 4132 4213 4231 4312 4321
50Random Babies
- 1234 1243 1324 1342 1423 1432
- 4 2 2 1 1 2
- 2134 2143 2314 2341 2413 2431
- 2 0 1 0
0 1 - 3124 3142 3214 3241 3412 3421
- 1 0 2 1 0
0 - 4123 4132 4213 4231 4312 4321
- 0 1 1 2 0 0
51Random Babies
- 0 matches 9/243/8
- 1 match 8/241/3
- 2 matches 6/241/4
- 3 matches 0
- 4 matches 1/24
52Morals Treatment of Probability
- Goal Interpretation in terms of long-run
relative frequency, average value - 30 chance of rain
- First simulate, then do theoretical analysis
- Able to list sample space
- Short cuts when are actually equally likely
- Simple, fun applications of basic probability
53Activity 13 AIDS Testing
- ELISA test used to screen blood for the AIDS
virus - Sensitivity P(AIDS).977
- Specificity P(-no AIDS).926
- Base rate P(AIDS).005
- Find P(AIDS)
- Initial guess?
- Bayes theorem?
- Construct a two-way table for hypothetical
population
54AIDS Testing
Positive Negative Total AIDS No
AIDS Total 1,000,000
55AIDS Testing
Positive Negative Total AIDS
5,000 No AIDS 995,000 Total
1,000,000
56AIDS Testing
Positive Negative Total AIDS 4885 115
5,000 No AIDS 995,000 Total
1,000,000
57AIDS Testing
Positive Negative Total AIDS 4885 115
5,000 No AIDS 73,630 921,370
995,000 Total 1,000,000
58AIDS Testing
Positive Negative Total AIDS 4885 115
5,000 No AIDS 73,630 921,370
995,000 Total 78,515 921,485 1,000,000
59AIDS Testing
Positive Negative Total AIDS 4885 115
5,000 No AIDS 73,630 921,370
995,000 Total 78,515 921,485 1,000,000 P(AIDS)
4885/78,515.062
60AIDS Testing
Positive Negative Total AIDS 4885 115
5,000 No AIDS 73,630 921,370
995,000 Total 78,515 921,485 1,000,000 P(AIDS)
4885/78,515.062 P(No AIDS-)
921,370/921,485 .999875
61Morals Surprise Students!
- Intuition about conditional probability can be
very faulty - Confront misconception head-on
- Conditional probability can be explored through
two-way tables - Treatment of formal probability can be minimized
62Activity 14 Reeses Pieces
63Reeses Pieces
- Take sample of 25 candies
- Sort by color
- Calculate the proportion of orange candies in
your sample - Construct a dotplot of the distribution of sample
proportions
64Reeses Pieces
- Turn over to technology
- Reeses Pieces applet
- (www.rossmanchance.com/applets/)
65Morals Sampling Distributions
- Study randomness to develop intuition for
statistical ideas - Not probability for its own sake
- Always precede technology simulations with
physical ones - Apply more than derive formulas
66Activity 15 Which Tire?
- Left Front Right Front
-
- Left Rear Right Rear
-
67Which Tire?
- People tend to pick right front more than ¼ of
the time - Variable which tire pick
- Categorical (binary)
- How often would we get data like this by chance
alone? - Determine the probability of obtaining at least
as many successes as we did if there were
nothing special about this particular tire.
68Which Tire?
- Let p proportion of all who pick right front
- H0 p .25
- Ha p gt .25
- Test statistic z
- p-value Pr(Zgtz)
- How does this depend on n?
- Test of Significance Calculator
69Which Tire?
- n z-statistic p-value
- 50 1.14 .127
- 100 1.62 .053
- 150 1.98 .024
- 400 3.23 .001
- 1000 5.11 .000
70Morals Formal Statistical Inference
- Fun simple data collection
- Effect of sample size
- hard to establish result with small samples
- Never accept null hypothesis
71Activity 16 Kissing the Right Way
- Biopsychology observational study
- Güntürkün (2003) recorded the direction turned by
kissing couples to see if there was also a
right-sided dominance.
72Kissing the Right Way
- Is 1/2 a plausible value for p, the probability a
kissing couple turns right? - Coin Tossing applet
- Is 2/3 a plausible value for p, the probability a
kissing couple turns right? - Is the observed result in the tail of the what
if distribution?
73Kissing the Right Way
- Determine the plausible values for p, the
probability a kissing couple turns right - The values that produce an approximate p-value
greater than .05 are not rejected and are
therefore considered plausible values of the
parameter. The interval of plausible values is
sometimes called a confidence interval for the
parameter.
74Kissing the Right Way
- How does this compare to estimate margin of
error? - Or the even simpler approximation?
75Morals Kissing the Right Way
- Interval estimation as (more?) important as
significance - Confidence interval as set of plausible (not
rejected) values - Interpretation of margin-of-error
76Activity 17 Reeses Pieces Revisited
- Calculate 95 confidence interval for p from your
sample proportion of orange - Does everyone have same interval?
- Does every interval necessarily capture p?
- What proportion of class intervals would you
expect? - Simulating Confidence Intervals applet
- What percentage of intervals succeed?
- Change confidence level, sample size
77Morals Reeses Pieces Revisited
- Interpretation of confidence level
- In terms of long-run results from taking many
samples - Effects of confidence level, sample size on
confidence interval
78Example 18 Dolphin Therapy
- Subjects who suffer from mild to moderate
depression were flown to Honduras, randomly
assigned to a treatment
78
78
79Dolphin Therapy
- Is dolphin therapy more effective than control?
- Core question of inference
- Is such an extreme difference unlikely to occur
by chance (random assignment) alone (if there
were no treatment effect)?
80Some approaches
- Could calculate test statistic, p-value from
approximate sampling distribution (z, chi-square) - But its approximate
- But conditions might not hold
- But how does this relate to what significance
means? - Could conduct Fishers Exact Test
- But theres a lot of mathematical start-up
required - But thats still not closely tied to what
significance means - Even though this is a randomization test
80
80
813S Approach
- Simulate random assignment process many times,
see how often such an extreme result occurs - Assume no treatment effect (null model)
- Re-randomize 30 subjects to two groups (using
cards) - Assuming 13 improvers, 17 non-improvers
regardless - Determine number of improvers in dolphin group
- Or, equivalently, difference in improvement
proportions - Repeat large number of times (turn to computer)
- Ask whether observed result is in tail of what if
distribution - Indicating saw a surprising result under null
model - Providing evidence that dolphin therapy is more
effective
81
81
82Analysis
- http//www.rossmanchance.com/applets/
- Dolphin Study applet
82
82
83Conclusion
- Experimental result is statistically significant
- And what is the logic behind that?
- Observed result very unlikely to occur by chance
(random assignment) alone (if dolphin therapy was
not effective)
83
83
84Morals
- Re-emphasize meaning of significance and p-value
- Use of randomness in study
- Focus on statistical process, scope of conclusions
85Activity 19 Sleep Deprivation
- Does sleep deprivation have harmful effects on
cognitive functioning three days later? - 21 subjects random assignment
- Core question of inference
- Is such an extreme difference unlikely to occur
by chance (random assignment) alone (if there
were no treatment effect)?
85
85
86Sleep Deprivation
- Simulate randomization process many times under
null model, see how often such an extreme result
(difference in group medians or means) occurs - Start with tactile simulation using index cards
- Write each score on a card
- Shuffle the cards
- Randomly deal out 11 for deprived group, 10 for
unrestricted group - Calculate difference in group medians (or means)
- Repeat many times (Randomization Tests applet)
86
86
87Sleep Deprivation
- Conclusion Fairly strong evidence that sleep
deprivation produces lower improvements, on
average, even three days later - Justification Experimental results as extreme as
those in the actual study would be quite unlikely
to occur by chance alone, if there were no effect
of the sleep deprivation
88Exact randomization distribution
- Exact p-value 2533/352716 .0072 (for difference
in means)
89Morals Randomizations Tests
- Emphasizes core logic of inference
- Takes advantage of modern computing power
- Easy to generalize to other statistics
90Activity 6 Revisited (Memorizing Letters)
- Conduct randomization test to assess strength of
evidence in support of research hypothesis - Enter data into applet
- Summarize conclusion and reasoning process behind
it - Does non-significant result indicate that
grouping of letters has no effect?
91Activity 20 Cat Households
- 47,000 American households (2007)
- 32.4 owned a pet cat
- or the other way around!
- test statistic z-4.29
- p-value virtually zero
- 99 CI for p (.31844, .32956)
92Morals Limits of statistical significance
- Statistical significance is not practical
significance - Especially with large sample sizes
- Accompany significant tests with confidence
intervals whenever possible
93Activity 21 Female Senators
- 17 women, 83 men in 2010
- 95 CI for p
- .170 .074
- (.096, .244)
94Morals Limitations of Inference
- Always consider sampling procedure
- Randomness is key assumption
- Garbage in, garbage out
- Inference is not always appropriate!
- Sample population here
95Activity 22 Game Show Prices
- Sample of 208 prizes from The Price is Right
- Examine a histogram
- 99 confidence interval for the mean
- Technical conditions?
- What percentage of the prizes fall in this
interval? - Why is this not close to 99?
96Morals Cautions/Limitations
- Prediction intervals vs. confidence intervals
- Constant attention to what the it is
97Activity 23 Government Spending
- 2004 General Social Survey Is there an
association between American adults opinion on
federal government spending on the environment
and political inclinations?
98Government Spending
Liberal Moderate Conservative Total
Too Much 1 17 32 50
About Right 27 80 91 198
Too Little 127 158 113 398
Total 155 255 236 646
99Government Spending
- Inferential analysis 3S approach
- 1. Chi-square statistic
- 2. Simulate sampling distribution of chi-square
test statistic under null hypothesis of no
association - Randomly mix up political inclinations, determine
could have been table - Repeat many times and examine what if
distribution of chi-square values under null
hypothesis
100Government Spending
- 3. Strength of evidence
- Is observed chi-square value in tail of
distribution? - Summarize What conclusions should be drawn?
- Very statistically significant
- Not cause and effect
- Ok to generalize to adult Americans
101Government Spending
- What about federal spending on the space program?
More or less evidence of association? Larger or
smaller p-value?
102General Advice
- Emphasize the process of statistical
investigations, from posing questions to
collecting data to analyzing data to drawing
inferences to communicating findings - Use simulation, both tactile and
technology-based, to explore concepts of
inference and randomness - Draw connections between how data are collected
(e.g., random assignment, random sampling) and
scope of conclusions to be drawn (e.g.,
causation, generalizability) - Use real data from genuine studies, as well as
data collected on students themselves - Present important studies (e.g., draft lottery)
and frivolous ones (e.g., flat tires) and
especially studies of issues that are directly
relevant to students (e.g., sleep deprivation)
103General Advice (cont.)
- Lead students to discover and tell you
important principles (e.g., association does not
imply causation) - Keep in mind the research question when analyzing
data - Graphical displays can be very useful
- Summary statistics (measures of center and
spread) are helpful but dont tell whole story
consider entire distribution - Develop graph-sense, number-sense by always
thinking about context - Use technology to reduce the burden of rote
calculations, both for analyzing data and
exploring concepts - Emphasize cautions and limitations with regard to
inference procedures
104Implementation Suggestions
- Take control of the course
- Collect data from students
- Encourage predictions from students
- Allow students to discover/tell you findings
- Precede technology simulations with tactile
- Promote collaborative learning
- Provide lots of feedback
- Follow activities with related assessments
- Intermix lectures with activities
- Dont underestimate ability of activities to
teach materials - Have fun!
105Suggestion 1
- Take control of the course
- Not control in usual sense of standing at front
dispensing information - But still need to establish structure, inspire
confidence that activities, self-discovery will
work - Be pro-active in approaching students
- Dont wait for students to ask questions of you
- Ask them to defend their answers
- Be encouraging
- Instructor as facilitator/manager
106Suggestion 2
- Collect data from students
- Leads them to personally identify with data,
analysis gives them ownership - Collect anonymously
- Can do out-of-class
- E.g., matching variables to graphs
107Suggestion 3
- Encourage predictions from students
- Fine (better) to guess wrong, but important to
take stake in some position - Directly confront common misconceptions
- Have to convince them they are wrong (e.g.,
Gettysburg address) before they will change their
way of thinking - E.g., AIDS Testing
108Suggestion 4
- Allow students to discover, tell you findings
- E.g., Televisions and life expectancy
- I hear, I forget. I see, I remember. I do, I
understand. -- Chinese proverb
109Suggestion 5
- Precede technology simulations with tactile/
concrete/hands-on simulations - Enables students to understand process being
simulated - Prevents technology from coming across as
mysterious black box - E.g., Gettysburg Address (actual before applet)
110Suggestion 6
- Promote collaborative learning
- Students can learn from each other
- Better yet from arguing with each other
- Students bring different background knowledge
- E.g., Matching variables to graphs
111Suggestion 7
- Provide lots of feedback
- Danger of discovering wrong things
- Provide access to model answers after the fact
- Could write answers on board
- Could lead discussion/debriefing afterward
112Suggestion 8
- Follow activities with related assessments
- Or could be perceived as fun and games only
- Require summary paragraphs in their own words
- Clarify early (e.g., quizzes) that they will be
responsible for the knowledge - Assessments encourage students to grasp concept
- Can also help them to understand concept
- E.g., fill in the blank p-value interpretation
113Suggestion 9
- Inter-mix lectures with activities
- One approach Lecture on a topic after students
have performed activity - Students better able to process, learn from
lecture having grappled with issues themselves
first - Another approach Engage in activities toward end
of class period - Often hard to re-capture students attention
afterward - Need frequent variety
114Suggestion 10
- Do not under-estimate ability of activities to
teach material - No dichotomy between content and activities
- Some activities address many ideas
- E.g. Gettysburg Address activity
- Population vs. sample, parameter vs. statistic
- Bias, variability, precision
- Random sampling, effect of sample/population size
- Sampling variability, sampling distribution,
Central Limit Theorem (consequences and
applicability)
115Suggestion 11
116Assessment Advice
- Two sample final exams
- Carefully match the course goals
- Be cognizant of any review materials you have
given the students - Use real data and genuine studies
- Provide students with guidance for how long they
should spend per problem - Use multiple parts to one context but aim for
independent parts (if a student cannot answer
part (a) they may still be able to answer part
(b)) - Use open-ended questions requiring written
explanation - Aim for at least 50 conceptual questions rather
than pure calculation questions - (Occasionally) Expect students to think,
integrate, apply beyond what they have learned. - Sample guidelines for student projects
117Promoting Student Progress
- Document and enhance student learning
- Element of instruction
- Interactive feedback loop
- Diagnostic with indicators for change
- Throughout the course
- To student and instructor
- Encourage self-evaluation
- Multiple indicators
118Student Projects
- Best way to demonstrate to students the practice
of statistics - Experience the fine points of research
- Experience the messiness of data
- From beginning to end
- Formulation and Explanation
- Constant Reference
- statweb.calpoly.edu/bchance/stat217/projects.html
119Resources
120Resources
121Resources
122Resources
- Inter-University Consortium for Political and
Social Research (ICPSR)
123Resources
- www.rossmanchance.com/applets/
- http//statweb.calpoly.edu/csi/
124Resources
- https//app.gen.umn.edu/artist/
125Resources
- http//lib.stat.cmu.edu/DASL/
- www.amstat.org/publications/jse/
- /jse_data_archive.html
126Background Readings
- Guidelines for teaching introductory statistics
- Reflections on what distinguishes statistical
content and statistical thinking - Educational research findings and suggestions
related to teaching statistics - Collections of resources and ideas for teaching
statistics - Suggestions and resources for assessing student
learning in statistics
127Thanks very much!
- Questions, comments?
- bchance_at_calpoly.edu
- arossman_at_calpoly.edu
128My Syllabus Briefly
- W1 Collecting Data
- W2 Graphical/Numerical
- W3 Normal Project 1
- W4 Exam 1 Project 2
- W5 Probability
- W6 Sampling Distributions
- W7 Inference
- W8 Inference
129My Syllabus Briefly
- W9 Two Samples
- W10 Exam II Project 3
- W11 Two variables
- W12 Inference for Regression
- W13 Two-way Tables Project 4
- W14 ANOVA
- W15 Presentations
130Non-simulation approach
- Exact randomization distribution
- Hypergeometric distribution
- Fishers Exact Test
- p-value
- .0127