An Introduction to Research Methods for Computer Science

1
An Introduction to Research Methods for Computer
Science

• Paul Cairns
• Dept of Computer Science
• University of York

2
Purpose of Class

• Very brief intro to Research Methods
• working with people
• Your own research needs
• Framing your research

3
Introductions

• Name
• Paul Cairns
• First degree/doctorate
• DPhil in Mathematics, General Topology
• Current research topic (one sentence)
• What does it mean to be immersed in a videogame?

4
What is a research method?
5
Learning RM

• Psychology, social sciences
• Not maths, physics, CS, chemistry
• CS research is teleological
• intended to change things

6
Types of method

• Collection plus analysis
• Quantitative
• Data and results numerical
• Qualitative
• Data anything, results textual

7
Example methods

• Experiments
• Statistics
• Surveys
• Interviews
• Grounded theory
• Case study
• Observations

• Ethnography
• Discourse analysis
• Content analysis
• Focus groups
• Longitudinal studies
• Thematic analysis

8
Today

• Experiments, statistics (quant)
• Grounded theory (qual)
• Case studies (qual)
• Thurs your work, narratives

9
Experiments

• Numerical data
• Sample
• Statistics
• But what for?

10
Experimental argument

• Belief X causes Y
• So change X and measure Y
• But
• people vary!
• other things affect Y
• hard to measure Y
• Experiments pierce through the murk!

11
Experimental design

• Change only X
• Control everything else you can
• Randomise what you cant control
• Measure Y
• Use statistics to see real differences

12
Example

• What are the independent and dependent variables?
• What is the experimental hypothesis?
• What also might affect the dependent?
• How might people differ?
• What might be wrong with the study?

13
Samples

• People who do you experiment
• participants
• Representative of population
• Whats a population?
• Unbiased
• Modest size

14
Inferential statistics

• People vary
• Measures therefore vary
• Is variation systematic or chance?

15
Example

• You have three systems, A,B and C. You show 30
  people all three and to state their preference.
  You get
• 15 prefer A, 7 prefer B and 8 prefer C.
• Which is the most preferred system?

16
Gold standard statistical argument

• Make a prediction
• Assert null hypothesis
• Gather data fairly
• Calculate probability of outcome
• If significant, supports your theory

17
Consequences

• No prediction, no good
• Dont over-test
• No proof of hypothesis

18
Statistical machines

• Turn data into probabilities
• Match machine to the data
• Standard repertoire
• T-test, F-test (ANOVA), correlation, chi-squared,
  Mann-Whitney, Wilcoxon

19
Pros and cons

• Highly supported method
• High validity
• Lot of craft skill
• Small questions
• One experiment is not enough

20
Grounded theory

• Hypothesis seeking
• Interview
• Analysis
• Descriptive theory (grounded in data)

21
Approach

• Some (vague) phenomenon
• Semi-structured interviews
• Analyse, develop theory
• Adapt interviews

22
Example Sarah Faisal

• What is user experience of using an information
  visualisation?
• Interaction and subjective experience are key but
  how?

23
Semi-structured interview

• Schedule of questions
• No order or necessity
• Explore matters arising

24
Devising questions

• Open
• Is user experience like A or B?
• Unbias
• Did you like this system?
• Unembarrassing
• Why did you make that mistake?

25
Coding

• Open coding
• Open to new concepts
• Selective coding
• Grouping, categorising concepts
• Axial coding
• Relating concepts
• Storyline

26
Theoretical sampling

• Choose your interviewees
• Fill out theory
• Explore limitations
• Saturation

27
Tricks for coding

• Microcoding
• Asking questions
• Memoing
• Example

28
Final theory

• Headline
• Good user experience arises from a harmonious
  flow between interaction and internalization
• Description of the data
• Recontextualise in the literature
• Instrumental genesis

29
Validity?

• Is it objective?
• Is it honest?
• Is it verifiable?

30
Homework

• Write a fantasy abstract for a paper or a large
  study in your PhD (200-300 words)
• Must include
• Title of work
• Problem to solve
• Key research findings
• Why this is better than other peoples work
• Fantasy impact (200 words max)

31
(No Transcript)