M.Sc. Computing Research Methods 06-Experimental Research - PowerPoint PPT Presentation

1 / 28
About This Presentation
Title:

M.Sc. Computing Research Methods 06-Experimental Research

Description:

M.Sc. Computing Research Methods 06-Experimental Research Adrian J Pullin FML208 pullina_at_hope.ac.uk hopelive.hope.ac.uk/imc/staff/pullina – PowerPoint PPT presentation

Number of Views:121
Avg rating:3.0/5.0
Slides: 29
Provided by: UserL2
Category:

less

Transcript and Presenter's Notes

Title: M.Sc. Computing Research Methods 06-Experimental Research


1
M.Sc. ComputingResearch Methods06-Experimental
Research
  • Adrian J Pullin
  • FML208
  • pullina_at_hope.ac.uk
  • hopelive.hope.ac.uk/imc/staff/pullina

2
Plagiarism Warning
  • The majority of the material in this presentation
    would constitute plagiarism if it were submitted
    for assessment

3
What is Science?
  • Science is best defined as a careful,
    disciplined, logical search for knowledge about
    any and all aspects of the universe, obtained by
    examination of the best available evidence and
    always subject to correction and improvement upon
    discovery of better evidence. What's left is
    magic. And it doesn't work. Randi, James,
    n.d.
  • James Randi has an international reputation as a
    magician and escape artist, but today he is best
    known as the world's most tireless investigator
    and de-mystifier of paranormal and
    pseudoscientific claims.

4
Originality (1)
  • Setting down a major piece of new information in
    writing for the first time
  • Continuing previous original work
  • Carrying out original work designed by supervisor
  • Providing single original technique, observation
    or result in otherwise unoriginal but competent
    research
  • Having many original ideas, methods or
    interpretations performed by others under your
    direction
  • Originality in testing someone elses idea
  • Empirical work that has not been done before
  • Making a synthesis that has not been made before
  • Using already known material but with new
    interpretation

5
Originality (2)
  • Doing something in your country that has
    previously been done in other countries
  • Corollary comparing something in your country
    with the same thing in another country AP
  • Taking a particular technique and applying it in
    a new area
  • Bringing new evidence to bear on an old issue
  • Being cross-disciplinary and using different
    methodologies
  • Looking at areas that people in that discipline
    have not looked at before
  • Adding to knowledge in a way that has not been
    done before
  • Phillips and Pugh, 2000

6
What is Scientific Method?
  • Observe some aspect of the universe
  • Invent a tentative description, called a
    hypothesis, that is consistent with what you have
    observed
  • Use the hypothesis to make predictions
  • Test those predictions by experiments or further
    observations and modify the hypothesis in the
    light of your results
  • Repeat steps 3 and 4 until there are no
    discrepancies between theory and experiment
    and/or observation
  • Wudka, Jose 1998

7
What is a Theory?
  • When consistency is obtained the hypothesis
    becomes a theory and provides a coherent set of
    propositions which explain a class of phenomena.
  • A theory is then a framework within which
    observations are explained and predictions are
    made
  • Wudka, Jose 1998

8
What Is Scientific Method? (Diag)
Wudka, Jose 1998
9
Steps in the Scientific Method
Dolphin L, 1992
10
Scientific Method Text Version
  • Working from previous knowledge
  • Formulate research question
  • State hypothesis
  • Test hypothesis
  • Design experiment
  • Conduct experiment
  • Analyse data
  • (Iterate)
  • Publish results
  • Peer review
  • After Kenyon, P, n.d.

11
Triangulation
  • Note triangulation on previous slides
  • Concept is credible because a range of sources
    state the same thing
  • In this case, a massive range, of which this is
    only a very small sample

12
Why Scientific Method?
  • Unprejudiced
  • Dont have to believe researcher
  • Redo experiment and determine if results are true
    or false
  • Conclusions will hold irrespective of state of
    mind, religious persuasion, state of
    consciousness of investigator and/or subject
  • Faith (belief that does not rest on logical proof
    or material evidence) does not determine whether
    theory is adopted
  • Theory is accepted not on prestige or convincing
    powers of proponent, but on results obtained
    through observations and/or experiments which
    anyone can reproduce
  • (? Do you really believe this? - AP)
  • Results obtained using the scientific method are
    repeatable
  • Most experiments and observations are repeated
    many times
  • If original claims not verified, origin of
    discrepancies is studied
  • Wudka, Jose 1998

13
Caveat
  • Getting the same results is not the same as
    drawing the same conclusions

14
Experimental ResearchHolah, M 2005 next 10
slides
  • Most scientific of all methods
  • Method of choice
  • At least in some areas
  • Provides controlled situations
  • An experiment is a study of cause and effect
  • Involves deliberate manipulation of one variable,
    while trying to keep all other variables constant

15
Laboratory Experiments (1)
  • Try to keep all aspects of the situation constant
    except one
  • The one we are looking at
  • Aspect we vary is called the independent variable
    (IV)
  • Change this in a very precise way
  • The aspect we measure, the dependent variable
    (DV), changes as a result
  • If our thesis is correct
  • DV depends on IV

16
Laboratory Experiments (2)
  • All other variables which might affect the
    results and therefore give us a false set of
    results are called confounding variables
  • Or random variables
  • E.g. 
  • Differences in instructions given by an
    experimenter
  • Could be overcome by standardising instructions
  • Differences between participants, e.g. in their
    age
  • Eliminated as a variable by using a single age
    group

17
Laboratory Experiments (3)
  • Change one variable (the IV)
  • Measuring another (the  DV)
  • Control all others, as far as possible
  • Experimental method allows us to draw conclusions
    with far more certainty than any non-experimental
    method
  • If IV is only thing that is changed then it must
    be responsible for any change in DV

18
Example (Say HCI)
  • Divide participants into 2 groups
  • Experimental group and control group
  • Introduce a change for experimental group and not
    control group
  • A control group is a group for whom the
    experimenter does not change IV
  • Experimental and control groups must be matched
    on all important characteristics
  • Age, sex, experience etc.

19
Advantages of Laboratory Experiments
  • Only means by which cause and effect can be
    established
  • IV is thought of as cause and (DV) as effect
  • Allows for precise control of variables
  • Control group enables experimenter to isolate one
    key variable (IV) and observe its effect on some
    other variable (DV)  
  • Experiments can be replicated
  • Cannot generalise from one set of results
  • Repeating experiment, with same results obtained,
    gives more confidence that theory being tested is
    valid
  • Experimental method consists of standardised
    procedures and measures which allow it to be
    easily repeated
  • Experiments yield quantitative data
  • Can be analysed using inferential statistical
    tests
  • Permit statements about how likely the results
    are to have occurred through chance

20
Limitations of Laboratory Experiments (1)
  • Artificiality experiment is not typical of real
    world situations
  • Most experiments done in laboratories
  • Strange, contrived environments
  • People asked to perform unusual or bizarre tasks
  • Unusual equipment set-up
  • Bias George Miller (1962) estimated 90 of U.S.
    experiments used college students
  • Accessible and cheap
  • Results still tend to be generalised to
    population as a whole

21
Limitations of Laboratory Experiments (2)
  • Not possible to completely control all variables
  • Variables at work of which the experimenter is
    unaware
  • Ethics
  • Experiments with people nearly always involve
    deceiving participants to some extent
  • Educational experiments are still experimenting
    on children
  • (Aside need approval from head teacher/parents
    and possibly a CRB check)
  • Normative data
  • Grouping people together and trying to cancel out
    individual differences to look at a group norm is
    limited in how much it can tell us because it
    ignores what is special about people

22
Field Experiment
  • Experiments in a more natural setting
  • The field
  • IV still deliberately manipulated by researcher
  • Not possible to have such tight control over
    variables in the field
  • Advantage of being far less artificial than the
    laboratory

23
Natural Experiment
  • Take advantage of natural situation to carry out
    investigation
  • Circumstances cannot be manipulate
  • E.g. school may decide to switch to a new CBL
    package
  • Effects compared with similar school using
    different package 
  • Not true experiment
  • Unable to manipulate or control variables
  • Quasi-experiment
  • Is possible to compare two groups
  • Equivalent of experimental and control group
  • Participants are unaware that they are taking
    part in an investigation
  • Not as artificial as a laboratory setting

24
Scientific Method Infallible?
  • Error? - Electrochemically Induced Nuclear Fusion
    of Deuterium
  • Martin Fleischmann, Department of Chemistry, The
    Universityof Southampton, Hants. S09
    5NH,ENGLANDStanley Pons, Department of
    Chemistry, University of Utah, Salt Lake city, UT
    84112 USA
  • Submitted to Journal of Electroanalytical
    Chemistry March 11,1989 in final form March 20,
    1989
  • http//www.totse.com/en/fringe/fringe_science/fusi
    on.html
  • http//encyclopedia.thefreedictionary.com/coldfus
    ion
  • Hoax? - Piltdown Man
  • http//www.clarku.edu/piltdown/Prefacegratitude.
    html
  • http//www.talkorigins.org/faqs/piltdown.html
  • Deliberate fraud? - molecular-scale transistors
  • http//www.socialismtoday.org/70/science.html
  • http//seattlepi.nwsource.com/business/88624_bell2
    61.shtml

25
Scientific Method Protection
  • http//www.sfn.org/NL/1997/March-April/ethics_poli
    cy.html
  • http//www.bps.org.uk/publications/journals/princi
    ples-of-publishing.cfm
  • http//www.liv.ac.uk/research/work/conduct.htm

26
And Finally
  • "There is not the slightest indication that
    nuclear energy will ever be obtainable. It would
    mean that the atom would have to be shattered at
    will." Albert Einstein, 1932

27
ReferencesHarvard format
  • Wudka, Jose 1998 The Scientific Method
    http//phyun5.ucr.edu/wudka/Physics7/Notes_www/n
    ode5.htmlSECTION02120000000000000000 accessed
    24/02/05
  • Holah, M 2005, The Experimental Methods,
    http//www.holah.karoo.net/experimental_method.ht
    m Accessed 22/02/05
  • This is a site aimed at Psychology AS students,
    but the material on experimental method is
    excellent
  • Dopohin L 1991 Steps in the scientific method,
    http//www.ldolphin.org/SciMeth2.html accessed
    24/02/05
  • Phillips, E and Pugh, DS, 2000 How to Get a
    PhD A Handbook for Students and Their
    Supervisors, Open University Press
  • Kenyon P n.d. The Scientific Method
    http//salmon.psy.plym.ac.uk/year1/scimeth3.gif
    accessed 24/02/05

28
See Also
  • Web search on scientific method
  • http//en.wikipedia.org/wiki/Scientific_method
  • http//koning.ecsu.ctstateu.edu/Plants_Human/scime
    th.html
  • Randi, James n.d. http//www.randi.org/
  • But also
  • http//www.dharma-haven.org/science/myth-of-scient
    ific-method.htm
  • Note these are very sloppy references
  • Do them properly in your dissertation
Write a Comment
User Comments (0)
About PowerShow.com