Data Analysis, Interpretation, and Presentation

1
Data Analysis, Interpretation, and Presentation

• Devin Spivey
• Asmae Mesbahi El Aouame
• Rahul Potghan

2
Objectives

• Difference between qualitative and quantitative
  data and analysis.
• Analyze data gathered from questionnaires.
• Analyze data gathered from interviews.
• Analyze data gathered from observation studies.
• Software packages for data analysis.
• Pitfalls in data analysis, interpretation, and
  presentation.
• Presenting your findings.

3
Qualitative and quantitative

• Quantitative numbers, translated into numbers.
• Qualitative difficult to express in numerical
  terms in a sensible fashion.
• Be careful in translating qualitative
  data into quantitative data.

4
Types of Data
5
Simple quantitative analysis

• Simple quantitative analysis techniques
• Percentages for standardizing data (compare
  large sets of data).
• Averages
• Mean commonly understood average.
• Median middle value of the data.
• Mode most commonly occurring number.
• Initial Analysis finding averages, outliers,
  depict any patterns from the graphical
  representation of data on a spreadsheet.

6
Initial analysis example data gathering

• Evaluation study of an e-commerce website
  identify transactions difficulties faced by
  users.
• Data gathering methods
• Questionnaires.
• Observation of a controlled task.
• data logging.

7
Initial analysis example Visualization of data
8
Initial analysis example Visualization of data

• Outlier
• Removed from the larger data set since it
  distorts the general patterns.
• Interesting case for further analysis.

9
Activity(1)

• The data in the table represents the time taken
  by a group of users to select and buy an item
  from an online shopping website.

10
Activity(2)

• Further investigation
• The values for users N(24) and S(26) are higher
  than the others.
• Trends
• - Users at the beginning of the testing time
  performed faster than those towards the end of
  the testing.
• - O is at the end of the testing but
  performed well.

11
How question design affects data analysis
12
Comparing two products 2 Phone designs
13
Initial analysis from data logs (1)

• Research question investigate how effective
  Massively Multiplayer Online Role-Playing Games
  (MMORPGs) are at encouraging interactivity
  between users.
• Data data logs and video recordings of players
  interactions in SWG.
• Ethnography study to identify the locations
  heavily used by players.

14
Initial analysis from data logs (2)
15
Simple Qualitative Analysis

• Qualitative analysis expresses the nature of
  elements and is represented as themes, patterns,
  stories
• Qualitative data difficult to measure sensibly
  as numbers.e.g. counting number of words to
  measure dissatisfaction
• .

16

• -The first step is to gain an overall impression
  of the data and start looking for patterns.-
  Next comes more detailed work using structured
  frameworks or theories to support the
  investigation.- Patterns may relate to a
  variety of aspects. e.g. behavior, user groups,
  places or situations where certain events happen.

17

• In terms of structure- Unstructured Not
  directed by a script. Rich but not replicable.
  e.g. video material. - Structured Tightly
  scripted. Replicable but may lack richness. e.g.
  questionnaire. - Semi-structured Guided by a
  script but interesting issues can be explored in
  more depth. Provides a good balance between
  richness and replicability.e.g. interviews.

18

• When Should I Use Qualitative Vs. Quantitative
  Research?
• http//www.youtube.com/watch?v638W_s5tRq8feature
  related

19
Three types of qualitative analysis

• 1. Identifying recurring patterns and
  themes2. Categorizing data3. Analyzing
  critical incidents

20
1. Identifying recurring patterns and themes
21
1. Identifying recurring patterns and themes

• Emergent from data, dependent on observation
  framework if used
• Patterns in Quantitative can be find identified
  by graphical representation but for Qualitative,
  requires researcher to be immersed in data
• Studying the data
• Focusing on the study goals
• Keeping clear records of the analysis as it
  progresses and close description of themes or
  patterns that are emerging.
• e.g. Box 8.4, Themes in European culture.

22
2. Categorizing data

• Can be at a high level of detail such as
  identifying stories or themes OR at a fine level
  of detail in which each word, phrase or gesture
  is analyzed.
• Most challenging aspects 1. Determining
  meaningful categories that are orthogonal (do not
  overlap each other in any way).2. Deciding on
  the appropriate granularity for the categories
  (word, phrase, sentence, or paragraph level).
• The categorization scheme used must be reliable
  so that the analysis can be replicated!
• - Categorization scheme may be emergent or
  pre-specified

23
3. Looking for critical incidents

• It is a flexible set of principles that emerged
  from work carried out in the United States Army
  Air Forces where the goal was to identify the
  critical requirements of good and bad
  performance by pilots.
• Two basic principles1. Reporting facts
  regarding behavior is preferable to the
  collection of interpretations, ratings and
  opinions based on general impressions.2.
  Reporting should be limited to those behaviors
  which make a significant contribution to the
  activity
• Helps to focus in on key event
• e.g. Box 8.5, analyzing video material

24
Tools to support data analysis

• Qualitative data analysis tools
• Categorization and theme-based analysis
• N6 can be used to search a body of text to
  identify categories or words for content analysis
• N6 is used to handle large set of data.
• Quantitative analysis of text-based
• data- Focuses on the number of
• occurrences of words or words with
• Similar meanings.

25
CAQDAS

• CAQDAS Networking Project, based at the
  University of Surrey (http//caqdas.soc.surrey.ac.
  uk/)
• support theory-building through the visualization
  of relationships between variables that have been
  coded in the data.
• CAQDAS is useful
• Grounded Theory
• Qualitative Content Analysis
• Ethnography

26
SPSS

• SPSS is Statistical Package for the Social
  Science
• It is used by market researchers, health
  researchers, survey companies, government,
  education researchers, marketing organizations
  and others
• Statistical packages, e.g. SPSS
• You can access, manage, and analyze enormous
  amounts of data with SPSS.
• SPSS offers statistical test for frequency
  distributions, rank correlations, regression
  analysis, and cluster analysis

27
Observer Video-Pro

• The Observer Video-Pro is a system for
  collecting, managing, analyzing, and presenting
  observational data.
• It integrates The Observer software with time
  code and multimedia hardware components.

28
Observer Video-Pro

• The software allows the user to summarize
  research findings in numerical, graphical, or
  multimedia format
• http//www.noldus.com/human-behavior-research/prod
  ucts/the-observer-xt

29
Theoretical Frameworks for Qualitative Analysis

• Structuring the analysis of qualitative data
  around a theoretical framework can lead to
  additional insights that go beyond the results
  found from the simple techniques introduced
  earlier.
• Three frameworks are discussed in this section
  Grounded theory, Distributed Cognition, and
  Activity theory.

30
Grounded Theory

• Grounded theory aims to develop theories from
  systematic analysis and interpretation of
  empirical data, i.e. the theory is grounded in
  the data.
• A grounded theory is developed through
  alternating data collection and data analysis
• Data is first collected and analyzed to identify
  categories, then that analysis leads to the need
  for further data collection, which is analyzed,
  and more data is then collected.
• Data collection is driven by the emerging theory.
• Approach continues until no further insights
  emerge and the theory is well developed.

31
Grounded Theory

• Analysis is mainly to setup Categories.
• Category identification and definition is
  achieved by coding the data, i.e. marking the
  data up according to the emerging categories.
• Coding has three aspects
• Open coding the process through which
  categories are discovered in the data.
• Axial coding the process of systematically
  fleshing out categories and relating them to
  their subcategories.
• Selective coding the process of refining and
  integrating categories to form a larger
  theoretical scheme. Categories are organized
  around one central category that forms the
  backbone of the theory.

32
Grounded Theory

• Researchers are encouraged to draw on their own
  theoretical backgrounds to help inform the study,
  as long as they are alert to the possibility of
  unintentional bias.
• Emphasizes the important role of empirical data
  in the derivation of theory.

33
Distributed Cognition

• The people, environment and artifacts are
  regarded as one cognitive system
• Focuses on information propagation and
  transformation
• A distributed cognition analysis results in an
  event-driven description which emphasizes
  information and its propagation through the
  cognitive system under study.

34
Distributed Cognition

• It is recommended to have a deep understanding of
  the domain under study. Even taking steps to
  learn the trade under study. This could take
  more time than a research team has available.
• Alternatively, it is possible to spend a few
  weeks immersed in the culture and setting of a
  specific domain to become familiar with it.

35
Distributed Cognition

• The framework can reveal where information is
  being distorted resulting in poor communication
  or inefficiency.
• The framework can show when different
  technologies and the representations displayed
  via them are effective at mediating certain work
  activities and how well they are coordinated.

36
Activity Theory

• Activity theory (AT) is a product of Soviet
  psychology that explains human behavior in terms
  of our practical activity with the world.
• AT provides a framework that focuses analysis
  around the concept of an activity and helps to
  identify tensions between the different elements
  of the system.

37
Activity Theory

• AT outlines two key models
• A model that outlines what constitutes an
  activity
• A model that outlines the mediating role of
  artifacts
• AT models activities in a hierarchical way.
• Activity Provides a minimum meaningful
  context for understanding the individual actions.
• Actions behavior that is characterized by
  conscious planning.
• Operations routinized behaviors that require
  little conscious attention.

38
Activity Theory

• Activity is motivated.
• Actions are to accomplish a goal.
• Actions involve operations.

39
Activity Theory

• AT models artifacts in two ways
• Physical
• Abstract
• Physical artifacts have physical properties that
  cause humans to respond to them as direct objects
  to be acted upon. The object usually embody a
  set of social practices.
• A spoon
• Abstract artifacts follow the idea of
  mediation. A fundamental characteristic of
  human development is the change from a direct
  mode of acting to one that is mediated by
  something else.
• A set of rules or symbols

40
Presenting Your Findings

• Only make claims that your data supports.
• The best method for presenting your finding
  depends on the audience, the purpose of the
  study, and the data gathering and analysis
  techniques used.
• Graphical representations (numbers, tables,
  graphs, etc.) may be appropriate for presenting
  your findings.
• Other techniques are
• Rigorous notations
• Using stories
• Summarizing your findings

41
Rigorous Notations

• UML is an example of rigorous notation because it
  uses notations that have clear syntax and
  semantics.

42
Using Stories

• Used as a basis for constructing scenarios.
• May be employed in three ways
• Stories told by the participants
• Stories about the participants
• Stories constructed from smaller anecdotes or
  repeated patterns that are found in the data.

43
Summarizing the findings

• Summarizing the data by presenting the headline
  findings, overviews, and detailed content list.
• Numbers and statistical values can be very
  valuable in a summary.
• If you found 800 out of 1000 users preferred
  design A over design B. This statement would be a
  quick indication of your findings.

44
Summarizing the findings

• Activity
• Consider each of the findings below and the
  associated summary statement about it.
• What is correct or incorrect about each findings
  statement?

45
Conclusion

• The kind of data analysis that can be done
  depends on the data gathering techniques used.
• Qualitative and quantitative data may be
  collected from any of the main data gathering
  techniques interviews, questionnaires, and
  observation.
• Quantitative data analysis for interaction design
  usually involves calculating percentages and
  averages.
• There are three different kinds of averages
  mean, mode and median.

46
Conclusion

• Graphical representations of quantitative data
  help in identifying patterns, outliers, and the
  overall view of the data.
• Qualitative data analysis may be framed by
  theories. Three such theories are grounded
  theory, activity theory, and distributed
  cognition.