Experimental Research Methods in Language Learning

1
Experimental Research Methods in Language Learning

• Chapter 2
• Experimental Research Basics

2
Leading Questions

• Do you believe in a cause-effect relationship in
  language learning? Why or why not?
• What do you think are characteristics of an
  experimental research study?
• What kind of research questions do you think
  experimental researchers ask?

3
Experimental Research Design

• A useful research methodology for those studies
  that aim to address a causal-like relationship.
• It allows researchers to strictly control the
  influence of factors that are not of interest by
  setting them constant across groups, but to vary
  the degree of a factor under study across groups
  of learners in order to undestand a causal-like
  relationship.

4
Key Characteristics of Experimental Research

• Causal or causal-like relationships
• Research questions in experimental research
• Independent and dependent variables
• Dichotomous and continuous variables
• Measurement scales of variables
• Constructs in experimental research
• Manipulation and control in experimental research

5
Causal or Causal-like Relationships

• Researchers can examine whether their hypothesis
  about a causal-like relationship is supported by
  empirical data
• Reasons for using causal-like in place of
  causal are that there is no direct proof that
  an experimental finding indicates a causal
  relationship (i.e. We can only make inferences).
• Statistical analysis in experimental research is
  not a method to discover causes.

6
Examples of Research Titles

• Baralt and Gurzynski-Weiss (2011) Comparing
  learners state anxiety during task-based
  interaction in computer-mediated and face-to-face
  communication
• Park (2010) The influence of pretask
  instructions and pretask planning on focus on
  form during Korean EFL task-based interaction
• Takimoto (2006) The effects of explicit feedback
  on the development of pragmatic proficiency

7
Research Questions

• Research questions are used to frame a research
  focus and method to answer them
• Two types of research questions Theoretical
  questions, and practical or pedagogical
  questions.
• Theoretical questions are connected with basic
  research which seeks empirical evidence that can
  inform a new theory, or refine and extend
  existing theories.
• Example What is motivation in L2 learning?

8
Research Questions

• Practical/pedagogical research questions are
  connected with applied research which aims to
  address a practical/pedagogical problem in a
  particular setting or to apply relevant theories
  or recommendations to real practice.
• Example How can teachers help students memorize
  a grammar rule?

9
Examples of Research Questions

• Do adult and child dyads respond differently to
  the amount of implicit negative feedback provided
  to NNSs non-native English speakers during
  task-based interaction? (Mackey, Oliver Leeman
  2003, p.44)
• To what extent is Lexical Focus-on-Form
  beneficial during a focus on meaning activity
  (such as listening comprehension) in terms of
  students receptive vocabulary learning? (Tian
  Macaro 2012, p. 373)

10
Independent and Dependent Variables

• Variable is an aspect or characteristic of
  something that can take different values or
  scores.
• Examples age, gender, first language, length of
  learning, intelligence, English language
  proficiency, motivation and anxiety
• An independent variable is a variable that exists
  freely, and is hypothesized to have an effect on
  other variables that are described as dependent
  variables

11
Independent and Dependent Variables

• Independent variables are factors that influence
  certain behaviors or psychological processes.
• A dependent variable is a variable that changes
  as the independent variable being examined
  changes.
• Example Gass and Mackey (1999) The effects of
  task repetition IV on linguistic output DV

12
Dichotomous and Continuous Variables

• Categorical variables are used to group
  non-overlapping variables such as English
  proficiency levels (beginning, intermediate and
  advanced).
• A dichotomous variable is the simplest type of
  categorical variable. It has only two classes
  (e.g., male or female, pass or fail).
• Continuous variables can be arranged from lowest
  to highest (e.g. age, length of residency)

13
Measurement Scales of Variables

• Nominal scales use numbers to label or classify
  variables into categories
• Ordinal scales are rank-order scales used for
  ranking some quality or ability (e.g. grade point
  average (GPA)).
• Interval scales have the features of both ordinal
  scales and equal distances or intervals (e.g.
  language test scores, personality scores).
• Ratio scales are measurements with all the
  properties of nominal, ordinal and interval
  scales, and also possess a true zero

14
Constructs in Experimental Research revisited

• Constructs are aspects or abstract concepts that
  researchers seek to understand (e.g.
  intelligence, language proficiency, memory,
  language aptitude).
• Constructs cannot be seen directly or be easily
  measured, thereby needing a sound theory to help
  us define and measure them.

15
Constitutive versus Operational Constructs

• A constitutive construct is one defined using the
  general definition of a term (e.g., motivation,
  self-regulation, language learning strategies,
  and self-efficacy).
• An operational construct definition is one
  defined by researchers for a particular study. It
  is often concerned with exactly what a researcher
  means in a research setting and how it can be
  measured using a designed instrument.

16
Manipulation and Control in Experimental Research

• Manipulation is related to how experimental
  researchers manipulate independent variables.
• It aims to rule out other plausible rival
  explanations of the research outcomes.
• Manipulation helps researchers control
  confounding variables as well as to
  systematically vary the independent variable for
  testing its effect.

17
Manipulation and Control in Experimental Research

• Confounding variables are those independent
  variables which are not of interest but can
  interact with the independent variable to
  co-affect the dependent variable.
• Manipulation can also take place when researchers
  hold several conditions for two or more groups of
  comparisons constant, to avoid the potential
  confounding effects.

18
Discussion

• What are the ways in which experimental
  researchers can produce evidence of causality in
  language learning?
• What is random assignment? Why is it essential
  for a true experimental study?
• Can you think of a situation in which you can do
  an experimental study? Think of a situation in
  which you can have control over your research.
  What would be the potential confounding variables
  that can influence your findings?