Problems in Research

1
Problems in Research

• Poor design decisions
• Researcher bias
• Researcher effects on participants
• Things participants may do regardless

psychlotron.org.uk
2
Poor Design

• Problems that stem from a badly designed study
• Poor operationalisation of variables
• Inadequate control of variables
• Threaten the internal validity of the study
• Cause reliability problems

psychlotron.org.uk
3
Poor Design

• Poor operationalisation
• Insufficient care when translating unobservable
  psychological processes into observable
  behavioural ones
• Can end up measuring the wrong thing
• Can generate false positives or false negatives

psychlotron.org.uk
4
Poor Design

• Reliability the extent to which a study or
  measuring tool produces consistent results
• Test-retest reliability
• Inter-observer reliability
• Reliability is maximised by (1) careful
  operationalisation (2) adequate control of
  variables
• NB a reliable study can still be invalid

psychlotron.org.uk
5
Poor Design

• Inadequate controls
• Extraneous variables should be held constant
• Most are not a problem (random effect)
• Those that have a systematic effect (e.g. affect
  one condition more than another) become
  confounding variables

psychlotron.org.uk
6
Researcher Bias

• Emphasising some things at the expense of others
• Self-confirmatory bias tendency for researchers
  to pay more attention to data that confirm their
  expectations
• Reduce the need for subjective judgements by (1)
  careful operationalisation (2) use of double
  blind

psychlotron.org.uk
7
Researchers Participants

• Researcher effects - behaviour of researcher can
  affect that of participants
• Intimidate (anxiety)
• Insult (motivation)
• Cue PPs to behave in particular ways
• Can be reduced through standardisation of scripts
  and procedures

psychlotron.org.uk
8
Researchers Participants

• Interviewer effects characteristics of the
  researcher can affect how PP relates to them
  hence their responses
• Gender
• Social class
• Age
• Ethnicity
• Difficult to predict, impossible to eliminate

psychlotron.org.uk
9
Participants

• are people too people try to make sense of
  their situationhow they do this is not always
  predictable.
• Attempts to please (or displease) the researcher
• Management of self-presentation

psychlotron.org.uk
10
Participants

• Respond to the fact that they are being studied
• Evaluation anxiety
• Feelings about the study the researcher
• Effects on motivation please the researcher or
  confound the results depends on how researcher
  is perceived

psychlotron.org.uk
11
Participants

• May respond to demand characteristics (cues)
  these can enter the research
• Through poor design e.g. allowing PPs to be aware
  of both conditions in an expt
• Through researcher behaviour e.g. unwittingly
  indicating desired behaviour
• Can be reduced through careful choice of design
  standardisation double blind

psychlotron.org.uk
12
Participants

• Generally try to manage how they are perceived by
  others (we all do) can lead to
• Socially desirable responses
• Other distortions of truth in order to manage
  impressions (how depends on who and why)
• Can be reduced through anonymity, confidentiality
  but rarely eliminated

psychlotron.org.uk
13
General Precautions

• Start with the aim, not the procedure
• Pay attention to choice of design, controls,
  operationalisation etc.
• Pilot all procedures and materials
• Use double blind procedures where appropriate
  possible

psychlotron.org.uk