Agenda

1
(No Transcript)
2
Agenda

• Overview of Survey Editor
• Important Findings
• Contextual Inquiry / Design
• Heuristic Evaluation and Cognitive Walkthrough
• Initial Prototype
• Think-Aloud Results
• Final Prototype

3
Overview of Survey Editor
4
Project Focus Session
Where does and doesnt the web survey editor
support the process of designing, building,
delivering, executing and revising surveys?
5
Contextual Inquiry Design
What is the process for designing, building,
delivering, executing and revising surveys?
6
Important Findings CI CD
7
Possible Directions

• Focus work re-design on using the survey editor
  as both a design tool and an implementation tool.
• Focus work re-design on creation and reuse of
  questions to implicitly apply social science
  principles.
• Focus work re-design on creating a way to
  visually input and manipulate branching in
  surveys.

8
Selected Direction
Focus work re-design on creation and reuse of
questions to implicitly apply social science
principles.

• Question library design
• Select questions from library, including format
• Guide creation of good questions
• Questions include citations, usage discussions,
  measures of reliability and validity
• Personalized libraries of questions designer has
  used before

9
Heuristic Evaluation

• Heuristic Evaluation sample results include
• Preview function hidden (separated from main
  program)
• No help or documentation
• No undo functionality
• Un-clickable text field with label User enters
  data here

10
Cognitive Walkthrough States
11
(No Transcript)
12
Add New Scales
Search
Search
Details
Explorer
Scale Name Source Licensing and
Permissions Link to licensing Discussion
Reversed Questions
Topic
License
Source
Scale
Scale Sample 1 Topic 1 Source 1
Free Scale Scale Sample 2 Topic 2
Source 2 License Needed Scale Sample 3
Topic 3 Source 3 Free
Scale Scale Sample 4 Topic 4
Source 4 License Needed Scale Sample 5
Topic 5 Source 5
Subscription Scale Sample 6 Topic 6
Source 6 Free Question Scale Sample 5
Topic 5 Source 5
Subscription Scale Sample 6 Topic 6
Source 6 License Needed Scale Sample 7
Topic 7 Source 7
Subscription Scale Sample 8 Topic 8
Source 8 Free Question
How many hours do you sleep at
night? ____________
How long do you commute to work?
____________ How long is your
commute home? ____________
How much time is free time?
____________
Add to Survey
13
Important Findings TA

• Sample UARs that guided iteration of the design
• Placement of information for "reversed" questions
  problematic.
• Details should not show data until an item is
  clicked
• Open circles to indicate Likert on prototype not
  helpful
• User has a difficult time understanding the
  differences between Scale and Topic
• Questions formatted in the prototype were
  incorrect to the user, without indication of
  whether or not they could be edited

14
(No Transcript)
15
Search
Search
Details
Explorer
License
Scale
Time Pressure Free Nasa TLX (Task Load
Index) Free Vitality Free Scale Sample 4
Permission Scale Sample 5
Subscription Scale Sample 6 Free
Question Scale Sample 5 Free
(Members) Scale Sample 6 License
Needed Scale Sample 7 Free Scale Sample 8
Free (Members)
Add to Survey
16
Search
Search
Explorer
Details
License
Scale
Scale Name Vitality Source Ware, J.E. Jr,
Sherbourne, C.D. (1992) The MOS 36-item
Short-Form Health Survey (SF-36). Medical Care,
30, 473-483. Licensing/Permissions Free - Link
to licensing information N/A Discussion Vitalit
y scale measures the amount of energy people feel
like they have. How to use scale Reliability
discussion Cronbachs alpha (combined
measurement of reliability and validity) at time
T1.79. Measurement applies to _____. And can
be edited in the following ways only Question
Type All questions are 5-choice likert
scale. did you feel worn out? reversed did you
feel tired? reversed
Time Pressure Free Nasa TLX (Task Load
Index) Free Vitality Free Scale Sample
4 Permission Scale Sample 5
Subscription Scale Sample 6 Free
Question Scale Sample 5 Free
(Members) Scale Sample 6 License
Needed Scale Sample 7 Free Scale Sample 8
Free (Members)
?
1. Did you feel full of pep?
2. Did you have a lot of energy?
3. Did you feel worn out?
4. Did you feel tired?
?
?
?
?
Add to Survey
17
User Quotes

• "In the tasks I was trying to do, the best thing
  was sort of this...source area the Details
  section on the right. I was surprised that there
  was such a rich knowledge base underneath it.
  That was kind of surprising, and nice."  U3
• "You basically have this database of validated
  scales I don't know any other place where you
  could look them up like that." U3
• "Overall I think this a really neat idea." U1
• And its nice because it says how much it costs
  under license or what I have to do in order to
  use it. Very cool because it can be a big deal
  when using pre-tested questions. Thats very
  neat. U4

18
(No Transcript)
19
Modifications to Methods
20
Solution Overview

• Data drove design solution aims to alleviate
  pain of question design
• (CI) Users have trouble with question formatting
• (CI) Role switching and communication difficult
  between designers and implementation
• (HE) Unclear how to verify formatting