Designing Information Architecture for Search

1
Designing Information Architecturefor Search
Tutorial SIGIR 2001

• Marti Hearst
• University of California, Berkeley
• www.sims.berkeley.edu/hearst
• NSF CAREER Grant, NSF9984741

2
Outline

1. Motivation
2. Search Interfaces
3. Web search vs Site Search
4. Search UIs What works what doesnt
5. Methodology
6. Information Architecture Defined
7. Faceted Metadata
8. Integrating Search into IA via Faceted Metadata
9. Results of Usability Studies
10. Tools
11. Conclusions

3
Contributors to the Research

• Dr. Rashmi Sinha
• Graduate Students
• Ame Elliott
• Jennifer English
• Kirsten Swearington
• Ping Yee
• Research funded by
• NSF CAREER Grant, NSF9984741

4
Motivation and Background
5
Claims

• Web Search is OK
• Gets people to the right starting points
• Web SITE search is NOT ok
• The best way to improve site search is
• NOT to make new fancy algorithms
• Instead

6
The best way to improve search
Improve the User Interface
7
Recent Study by Vividence Research

• Spring 2001, 69 web sites
• 70 eCommerce
• 31 Service
• 21 Content
• 2 Community
• The most common problems
• 53 had poorly organized search results
• 32 had poor information architecture
• 32 had slow performance
• 27 had cluttered home pages
• 25 had confusing labels
• 15 invasive registration
• 13 inconsistent navigation

8
Vividence findings effects on users

• Poorly organized search results
• Frustration and wasted time
• Poor information architecture
• Confusion
• Dead ends
• "back and forthing"
• Forced to search

9
Vividence findings effects on users

• Cluttered home pages
• Creates disinterest
• Wastes time
• No contrast everything has equal weight
• Dont know where to start
• Failure to engage
• No call to action
• Failure to establish navigation
• Layout reflects company organization chart
• Investor centeredness

10
Vividence findings characteristics

• Inconsistent Navigation
• Primary navigation bar is, in fact, really
  secondary
• Un-scalable designs
• Poor transitions between company divisions
• "Junk Drawer" navigation bars
• Random links
• Shoe-horned functions
• Heavy need to hit the "back-button"

11
Vividence Study

• Breakdown of most common search problems
• 41 - of searches encountered no problems
• 20 - had search problems not named below
• 14 - of searches were not advanced enough
• 12 - did not organize results well
• 10 - of searches yielded inaccurate/unrelated
  results
• 9 - were too slow
• 8 - of searches had insufficient instructions
• 7 - engine was too difficult to locate
• 7 - of searches produced too few results
• 7 - of searches were too limiting
• 3 - of searches produced an error message
• 3 - were too difficult to use

12
Other Relevant Studies

• Commercial studies (are not usually scientific,
  do not supply full details)
• CreativeGood.com Holiday 2000 ecommerce report
• UIE, and Jared Spools talks http//world.std.com
  /uieweb
• Scientific studies (often less relevant to real
  web situations)
• Many papers from the CHI proceedings
  http//www.acm.org/dl/
• Papers from Human Factors and the Web
  http//www.optavia.com/hfweb/
• See the extensive bibliography from my textbook
  chapter (in this package).

13
The Philosophy

• Information architecture should be designed to
  integrate search throughout
• Search results should reflect the information
  architecture.
• This supports an interplay between navigation and
  search
• This supports the most common human search
  strategies.

14
The Approach

• Assign faceted metadata to content items
• Allow users to navigate through the faceted
  metadata in a flexible manner
• Organize search results according to the faceted
  metadata so navigation looks similar throughout
• Give previews of next choices
• Allow access to previous choices

15
Advantages of the Approach

• Supports different task types
• Highly constrained known-item searches use one
  interface
• Open-ended, browsing tasks use another interface
• Both types of interface use the same underlying
  structure
• Can easily switch from one interface type to the
  other midstream

16
Advantages of the Approach

• Honors many of the most important usability
  design goals
• User control
• Provides context for results
• Reduces short term memory load
• Allows easy reversal of actions
• Provides consistent view

17
Advantages of the Approach

• Allows different people to add content without
  breaking things
• Can make use of standard technology

18
Web Search vs. Site Search
19
Web Search is Working!

• Survey finds high user satisfaction
• Study by npd group
• http//www.searchenginewatch.com/reports/npd.html

20
Why is Web Search Working?

• Web Search is Successful at Finding Good Starting
  Points (home pages)
• Evidence
• Search engines using
• Link analysis
• Page popularity
• Interwoven categories
• These all find dominant home pages

21
(No Transcript)
22
(No Transcript)
23
Organizing Search ResultsWhat works, What
Doesnt

• There is a lot of prior work on this
• Cha-Cha (Chen et al. 1999)
• Scatter-Gather clustering (Cutting et al. 93,
  Hearst et al. 1996)
• Becoming more prevalent in web search too.
• Teoma
• Vivisimo
• Northern Light

24
Putting Results into Clusters
25
Drilldown what does it mean?
26
Vivisimo same idea
27
(No Transcript)
28
Yahoo lists category matches
29
Web Search Results Grouping

• Drill down one category
• Cannot mix and match categories
• Not clear if it is useful or not
• Can help differentiate different meanings of the
  same word.
• But what about site search?

30
If Web search engines are providing source
selection what happens when the user gets
to the site?
Follow Links or Search
31
Following Hyperlinks

• Works great when it is clear where to go next
• Frustrating when the desired directions are
  undetectable or unavailable

Site Search
Is not getting good reviews
32
An Analogy
hypertext
33
Analogy

• Hypertext
• A fixed number of choices of where to go next
• A glance at the map tells you where you are
• But may not go where you want to go.
• To get from Topeka to Santa Fe, may have to go
  through Frostbite Falls
• Site Search
• Can go anywhere
• But may get stuck, disoriented, in a crevasse!

34
Goal An All-Tertrain Vehicle

• The best of both techniques
• A vehicle that magically lays down track to
  suggest choices of where you want to go next
  based on what youve done so far and what you are
  trying to do
• The tracks follow the lay of the land and go
  everywhere, but cross over the crevasses
• The tracks allow you to back up easily

35
Organizing Search ResultsWhat works what doesnt
36
What works, what doesnt

• There is negative evidence for
• Clustering
• Fancy visualizations
• There is positive evidence for
• Grouping into meaningful, consistent categories
• Relevance feedback
• Depends how you do it
• Showing similar items

37
Kohonen Feature Maps on Text(from Chen et al.,
JASIS 49(7))
38
Study of Kohonen Feature Maps

• H. Chen, A. Houston, R. Sewell, and B. Schatz,
  JASIS 49(7)
• Comparison Kohonen Map and Yahoo
• Task
• Window shop for interesting home page
• Repeat with other interface
• Results
• Starting with map could repeat in Yahoo (8/11)
• Starting with Yahoo unable to repeat in map (2/14)

39
Study (cont.)

• Participants liked
• Correspondence of region size to documents
• Overview (but also wanted zoom)
• Ease of jumping from one topic to another
• Multiple routes to topics
• Use of category and subcategory labels

40
Study (cont.)

• Participants wanted
• hierarchical organization
• other ordering of concepts (alphabetical)
• integration of browsing and search
• corresponce of color to meaning
• more meaningful labels
• labels at same level of abstraction
• fit more labels in the given space
• combined keyword and category search
• multiple category assignment (sportsentertain)

41
Visualization of Clusters

• Huge 2D maps may be inappropriate focus for
  information retrieval
• Cant see what documents are about
• Documents forced into one position in semantic
  space
• Space is difficult to use for IR purposes
• Hard to view titles
• Perhaps more suited for pattern discovery
• problem often only one view on the space

42
Summary Clustering(Based on other studies as
well)

• Advantages
• Get an overview of main themes
• Domain independent
• Disadvantages
• Many of the ways documents could group together
  are not shown
• Not always easy to understand what they mean
• Different levels of granularity
• Probably best for scientists only
• Take heart there is good evidence for
  organizing via categories!

43
The DynaCat System

• Decide on important question types in an advance
• What are the adverse effects of drug D?
• What is the prognosis for treatment T?
• Make use of MeSH categories
• Retain only those types of categories known to be
  useful for this type of query.

Pratt, W., Hearst, M, and Fagan, L. A
Knowledge-Based Approach to Organizing Retrieved
Documents. AAAI-99 Proceedings of the Sixteenth
National Conference on Artificial Intelligence,
Orlando, Florida, 1999.
44
DynaCat
45
DynaCat Study

• Design
• Three queries
• 24 cancer patients
• Compared three interfaces
• ranked list, clusters, categories
• Results
• Participants strongly preferred categories
• Participants found more answers using categories
• Participants took same amount of time with all
  three interfaces

46
Cha-Cha (intranet search)
Cha-Cha A System for Organizing Intranet Search
Results, by Chen, Hearst, Hong, and Lin,
Proceedings of 2nd USENIX Symposium on Internet
Systems, Boulder, CO, Oct 1999.
cha-cha.berkeley.edu
47
Cha-Cha (intranet search)
48
How People Search
49
The Standard Model

• Assumptions
• Maximizing precision and recall simultaneously
• The information need remains static
• The value is in the resulting document set

50
Berry-Picking as an Information Seeking
Strategy (Bates 90)

• Berry-picking model
• Interesting information is scattered like berries
  among bushes
• The user learns as they progress, thus
• The query is continually shifting

51
A sketch of a searcher moving through many
actions towards a general goal of satisfactory
completion of research related to an information
need. (after Bates 89)
Q2
Q4
Q3
Q1
Q5
Q0
52
Search Tactics and Strategies

• Marcia J. Bates, Information Search Tactics,
  Journal of the American
• Society for Information Science, 30, 4, 1979
• Marcia J. Bates, Where should the person stop and
  the information
• search interfaces start?, Information Processing
  Management, 26, 5,
• 1990
• Marcia J. Bates, The Berry-Picking Search User
  Interface Design, User
• Interface Design, Harold Thimbleby,
  Addison-Wesley, 1990
• Marcia J. Bates, The design of browsing and
  berrypicking techniques
• for the on-line search interface, Online Review,
  1989, 13, 5,
• 407431.
• Vicki L. O'Day and Robin Jeffries, Orienteering
  in an information
• landscape how information seekers get from here
  to there, Proceedings of ACM INTERCHI '93, April,
  Amsterdam, 1993
• Gary Marchionini, Information Seeking in
  Electronic Environments, Cambridge University
  Press, 1995.

53
Tactics vs. Strategies

• Tactic short term goals and maneuvers
• operators, actions
• Strategy overall planning
• link a sequence of operators together to achieve
  some end

54
An Important Strategy

• Do a simple, general search
• Gets results in the generally correct area
• Look around in the local space of those results
• If that space looks wrong, start over
• Akin to Shneidermans overview details
• Our approach supports this strategy
• Integrate navigation with search

55
Term Tactics

• Move around a thesaurus
• Look at category labels
• Look at related terms
• Look at parent terms
• Look at child terms
• In older literature, refers to navigating the
  thesaurus itself, as opposed to the items
  themselves.

56
Source-level Tactics

• Bibble
• look for a pre-defined result set
• e.g., a good link page on web
• Survey
• look ahead, review available options
• e.g., dont simply use the first term or first
  source that comes to mind
• Cut
• eliminate large proportion of search domain
• e.g., search on rarest term first

57
Source-level Tactics (cont.)

• Stretch
• use source in unintended way
• e.g., use patents to find addresses
• Scaffold
• take an indirect route to goal
• e.g., when looking for references to obscure
  poet, look up contemporaries

58
Monitoring Strategies

• Check
• compare original goal with current state
• Weigh
• make a cost/benefit analysis of current or
  anticipated actions
• Pattern
• recognize common strategies
• Correct Errors
• Record
• keep track of (incomplete) paths

59
Additional Considerations(Bates 79)

• Need a Sort tactic
• When to stop?
• How to judge when enough information has been
  gathered?
• How to decide when to give up an unsuccesful
  search?
• When to stop searching in one source and move to
  another?

60
Information Architecture
61
A Taxonomy of WebSites
Catalog Sites Web-based Information Systems
Web-Presence Sites Service-Oriented Sites
high
Complexity of Data
low
low
high
Complexity of Applications
From The (Short) Araneus Guide to Website
development, by Mecca, et al, Proceedings of
WebDB99, http//www-rocq.inria.fr/cluet/WEBDB/pr
ocwebdb99.html
62
A View of Website Design

• Information design
• structure, categories of information
• Navigation design
• interaction with information structure
• Graphic design
• visual presentation of information and navigation
  (color, typography, etc.)

Information Architecture
From Sitemaps, Storyboards, and Specifications A
Sketch of Web Site Design Practice as Manifested
Through Artifacts. M.W. Newman and J.A. Landay.
In proceedings of Designing Interactive Systems
DIS '00. August 17-19, 2000.
63
A View of Information Architecture

• Content Items
• Information Structure
• Navigation Structure
• Layout

64
Content Items

• The information items that the site is designed
  to show the user.
• Individual content items can be considered leaves
  in a tree, or base-level items.
• Aggregates of individual (base-level) items can
  be considered to be content items.
• This definition is especially relevant for
  catalog-style sites, for example
• Image collection
• Product selling
• Collection of articles on some topic (medical,
  legal)
• Collection of information about some entity (IRS,
  Park Service)

65
Information Structure

• Independent of the website.
• A set of descriptors which are used to
  characterize the content of a website.
• Consists primarly of a category structure and a
  set of textual labels.
• The categories can have flat, hierarchical,
  faceted or network structure.
• The textual labels include alternative ways of
  expressing the same concepts (synonyms).

66
Navigation Structure

• Defined in terms of the website.
• Site level
• The paths connecting content items throughout the
  site.
• Page level
• The link from one page to others.

67
Example from Walmart.com
68
Content
Navigation Structure
69
Related Items

• Often are content items
• Related to the target by some shared information
  structure
• The particular related items that are shown are
  revealed through the navigation structure

70
The Information Structure

• Consists of a set of descriptors for the content
  items
• Cant really see it directly, since it is
  independent of web site description
• Can see parts of it in the navigation structure

71
A View of Information Architecture
Content Items
Information Structure
Start with an information structure
(categories and labels) and a set of content
items.
72
A View of Information Architecture
Content Items
Information Structure
Prod Camping Brand REI Material Nylon Size
4-person
Each content item is assigned some descriptors
from the information structure.
73
Navigation structure links items or groups of
items.
74
Navigation Structure Differs from Information
Structure

• Example
• Part of the info structure is the product
  hierarchy.
• Some products are assigned more than one spot in
  the hierarchy (e.g., sports and games), thus
  forming a tree structure
• Navigation structure shows a progressive
  disclosure of the hierarchical structure only.

75
Navigation Structure Differs from Information
Structure

• Example
• Main navigation structure is the product
  hierarchy.
• However, lateral links are shown from product
  leaf nodes to other nodes
• (e.g., from a tent to a flashlight and a sleeping
  bag)

76
Navigation Structure Differs from Information
Structure

• The differences can be much more profound
• Examples
• Show only main product categories at top levels
• After a search, show links according to brands of
  items, but only those brands that make sense for
  the items retrieved by the search.

77
Breadcrumbs

• A navigation technique for showing either history
  or contextualizing hierarchy via hyperlinks.
• Two main types
• Hierarchy without history
• Search results at walmart.com
• History across facets (without hierarchy)
• Epicurious path recording.

78
An Important IA Trend

• Generating web pages from databases
• Implications
• Web sites can adapt to user actions
• Web sites can be instrumented
• An essential feature of a design environment is
  to give authors the possibility of evaluating the
  current network against the final adaptive
  system.
• Petrelli, Baggio, Pezzulo, Adaptive Hypertext
  Design Environments Putting Principles into
  Practice, AH 2000

79
Faceted Metadata
80
Metadata data about dataFacets orthogonal
categories
81
Faceted Metadata Biomedical MeSH (Medical
Subject Headings)www.nlm.nih.org/mesh
82
Mesh Facets (one level expanded)
83
Using Mesh Facets

• Some stats
• gt18,000 labels
• avg depth 4.5, max depth 9
• 8 labels/article on average
• How to go from the information structure to the
  navigation structure?

84
Using faceted metadata incorrectly

• Yahoo uses faceted metadata poorly in both their
  search results and in their top-level directory
• They combine region other hierarchical facets
  in awkward ways

85
Yahoos use of facets
86
Yahoos use of facets
87
Yahoos use of facets
88
Yahoos use of facets

• Where is Berkeley?
• College and University gt Colleges and
  Universities gtUnited States gt U gt University of
  California gt Campuses gt Berkeley
• U.S. States gt California gt Cities gtBerkeley gt
  Education gt College and University gt Public gt UC
  Berkeley

89
However, Yahoo does use some metadata well

• Yahoo restaurant guide combines
• Region
• Topic (restaurants)
• Related Information
• Other attributes (cuisines)
• Other topics related in place and time (movies)

90
Yellow geographic region
Green restaurants attributes
Red related in place time
91
Combining Information Types

• Region
• State
• City

• A E
• Film
• Theatre
• Music
• Restaurants
• California
• Eclectic
• Indian
• French

Assumed task looking for evening
entertainment
92
Other Possible Combinations

• Region AE
• City Restaurant Movies
• City Weather
• City Education Schools
• Restaurants Schools

93
Bookstore preview combinations

• topic related topics
• topic publications by same author
• topic books of same type but related topic

94
Problems with Metadata Usage

• Standard approaches
• Paths are hand-edited, predefined
• Not well-integrated with search
• Not tailored to task as it develops
• Not personalized
• Not dynamic

95
Questions we are trying to answer

• How many facets are allowable?
• Should facets be mixed and matched?
• How much is too much?
• Should hierarchies be progressively revealed,
  tabbed, some combination?
• How should free-text search be integrated?

96
Recipe Collection Examples
97
From soar.berkeley.edu (a poor example)
98
(No Transcript)
99
From www.epicurious.com (a good example)
100
(No Transcript)
101
(No Transcript)
102
(No Transcript)
103
Epicurious Metadata Usage

• Advantages
• Creates combinations of metadata on the fly
• Different metadata choices show the same
  information in different ways
• Previews show how many recipes will result
• Easy to back up
• Supports several task types
• Help me find a summer pasta,'' (ingredient type
  with event type),
• How can I use an avocado in a salad?''
  (ingredient type with dish type),
• How can I bake sea-bass'' (preparation type and
  ingredient type)

104
Metadata usage in Epicurious
105
Metadata usage in Epicurious
106
Metadata usage in Epicurious
I
107
Metadata usage in Epicurious
gt
I
108
Metadata usage in Epicurious
gt
I
Select
Prepare
Cuisine
I
109
Recipe Information Architecture

• Information design
• Recipes have five types of metadata categories
• Cuisine, Preparation, Ingredients, Dish, Occasion
• Each category has one level of subcategories

110
Recipe Information Architecture

• Navigation design
• Home page
• show top level of all categories
• Other pages
• A link on an attribute ANDS that attribute to the
  current query results are shown according to a
  category that is not yet part of the query
• A change-view link does not change the query, but
  does change which categorys metadata organizes
  the results

111
Metadata Usage in Epicurious

• Can choose category types in any order
• But categories never more than one level deep
• And can never use more than one instance of a
  category
• Even though items may be assigned more than one
  of each category type
• Items (recipes) are dead-ends
• Dont link to more like this
• Not fully integrated with search

112
Epicurious Basic Search

• Lacks integration with metadata

113
(No Transcript)
114
Information previews

• Use the metadata to show where to go next
• More flexible than canned hyperlinks
• Less complex than full search
• Help users see and return to what happened
  previously
• Reduces mental work
• Recognition over recall
• Suggest alternatives

115
The Importance of Information Previews

• Jared Spools studies (www.uie.com)
• More clicks are ok if
• The scent of the target does not weaken
• If users feel they are going towards, rather than
  away, from their target.

116
Problem with Metadata Previews as Currently Used

• Hand edited, predefined
• Not tailored to task as it develops
• Not personalized
• Often not systematically integrated with search,
  or within the information architecture in general

117
Putting it Together
118
Desiderata for Objects inInformation-Seeking
Workspaces

• Structured
• Fractal
• Queriable
• Navigable
• Historical
• Similarity Engine Compatible
• Contextualized
• Other

From Furnas, G., and Rauch, S., Considerations
for information environments and the NaviQue
workspace. In Proceedings of DL 98.
Pittsburgh,PA, June, 1998.
119
Search Usability Design Goals

1. Strive for Consistency
2. Provide Shortcuts
3. Offer Informative Feedback
4. Design for Closure
5. Provide Simple Error Handling
6. Permit Easy Reversal of Actions
7. Support User Control
8. Reduce Short-term Memory Load

From Shneiderman, Byrd, Croft, Clarifying
Search, DLIB Magazine, Jan 1997. www.dlib.org
120
Analogy Chess

• Chess is characterized by a few simple rules that
  disguise an infinitely complex game
• Another intriguing characteristic the
  three-part structure
• Openings many strategies, new ones all the time,
  many books on this
• Endgame well-defined, well-understood
• Middlegame nebulous, hard to describe
• Our thought search is similar and the middlegame
  is critically underserved.

121
Chess-based view of Info Architecture

• The Opening
• Usually exposes top-level hierarchy or top-level
  facets (or both)
• Usually also has a search component
• This is also the place to expose the main tasks
  that can be accomplished on the site

122
The Opening
123
The Opening
124
The Opening
125
The Opening
126
The Opening
127
Chess-based view of Info Architecture

• The Endgame
• Has become rather well-established in shopping
  sites
• Penultimate page shows a list of items
• Leaf node
• Shows one content item in detail
• Lateral links
• To similar items (same facet)
• To other items that go with it (other facets)

128
The Endgame Penultimate Pages
129
The Endgame Penultimate Pages
130
The Endgame Leaf Nodes
131
Chess-based view of Info Architecture

• The Middlegame
• Hardest to describe/understand
• The berry-picking part of supporting search
• Issues
• How to progressively expose hierarchies?
• How to show multiple facet choices?
• How to integrate with search results?
• How to show history / retain context?

132
Sophisticated Middlegames
zdnet.com Continued next slide
133
Sophisticated Middlegames
134
Sophisticated Middlegames
Walmart.com Continued next slide
135
Sophisticated Middlegames
136
Sophisticated Middlegames
137
Online Grocery Shopping Examples

• In each case, note
• Chess analogy
• What is the opening?
• What is the endgame?
• How is the middlegame handled?
• How are search results integrated?
• How is hierarchical drill-down revealed?
• Are multiple facets allowed?

138
Grocery shopping example
Homerun.com
139
Grocery shopping example
Homerun.com
140
Grocery shopping example
Homerun.com
141
Grocery shopping example
peapod.com
142
Grocery shopping example
peapod.com
143
Grocery shopping example
peapod.com
144
Grocery shopping example
webvan.com
145
Grocery shopping example
webvan.com
146
Summary Grocery Shopping Examples

• A good opening seems to make a big difference
• Familiar metadata helps make the task easier
• Middlegame hierarchy exposure
• One uses cascading menus
• Two use webpage-based drilldown
• Two use metadata to organize search results
• But dont use metadata creatively
• Could organize by recipe, etc.

147
Medical Text Example

• Allow user to select metadata in any order
• At each step, show different types of relevant
  metadata,
• based on prior steps and personal history,
• include of documents
• Previews restricted to only those metadata types
  that might be helpful

148
Ecommerce Examples

• E-commerce sites are farther ahead than
  information collection sites
• However, their problem is usually easier
• Single facet often works fine
• Categories are familiar to users
• Collections are often much smaller
• How to move this to large sites containing more
  abstract information?
• Image collections?
• Text collections

149
Current Search Approach
Can a metadata preview approach do better?
150
Asthma gt Steroids

1. A steroid-induced acute psychosis in a child with
   athsma.
2. Management of steroid-dependent asthma with
   methotrexate.

151
Asthma gt Steroids gt Admin Dosage

1. Dosage levels for asthmatic steroids A survey.

152

• Other paths back up and go forward

153
Advantages of the Methodology

• Supports different types of information seeking
  tasks
• Uses interface idioms known to be usable for
  general users
• Flexible content entry and update
• Allows for non-experts to add new content
  independently
• Makes use of standard DBMS technology

154
Advantages of the Methodology

• Systematically integrates search
• search results reflect the structure of the info
  architecture
• search results retain the context of previous
  interactions
• search results preview next choices
• Gives user control
• Over order of metadata use
• Over when to navigate vs. when to search
• Allows integration with advanced methods
• Collaborative filtering, predicting users
  preferences

155
Advantages

• Users have a feeling of control
• Users can predict what will happen
• Not true of statistical ranking or clustering
• Adding new items to the system changes the
  behavior in understandable ways
• Users have flexibility
• In ordering of operations
• In combining of operations

156
Usability Study epicurious
157
Epicurious Usability Study

• 9 participants so far
• Independent Variables
• 1) Epicurious Interface (Basic vs. Enhanced vs.
  Browse)
• 2) Task type (known-item search vs. browsing for
  inspiration)
• 3) Degree of constraint of query
• 4) Number of results required (1 vs. many)
• Dependent Variables
• 1) Time to find satisfactory recipe(s)
• 2) Navigation path (backtracking, starting over,
  revising queries)
• 3) Satisfaction with results of search
• 4) Satisfaction with individual system features
  (e.g. breadcrumbs, query previews, refine by
  hyperlinks)
• 5) Likelihood of using each interface in the
  future.

158
Epicurious Usability Study

• Participants were asked to
• Do 3 pre-specified searches in advance
• In the lab
• Specify a cooking scenario of interest to them
• Search for 3 recipes for this recipe
• Search for each recipe using each of the
  interfaces
• Complete several structured tasks
• Along the way, answer questions about
• Getting closer or farther away from goal
• Satisfaction with search results
• Satisfaction with the interace

159
Usability Study Preliminary Results, Preference
Data
160
Usability Study Preliminary Results, Preference
Data
161
Usability StudyPreliminary Results Feature
Preference
162
Usability StudyPreliminary Results Quantitative
163
Usability StudyPreliminary Results
Constraint-based Preferences
164
Observed patterns of use of epicurious metadata
browse interface

• choosefacet
• refine
• refine
• back
• scan focus
• choosefacet
• refine
• back refine
• scan focus
• choosefacet
• refine
• refine
• scan focus
• choosefacet
• refine
• refine

• choosefacet
• refine
• refine
• searchword
• choosefacet
• searchword
• scan searchword
• back refine
• scan focus
• choosefacet
• refine
• back back refine
• refine
• refine
• choosefacet
• refine

• choosefacet
• refine
• refine
• back refine
• choosefacet
• scan focus

165
Usability Study Results Summary

• People liked the browsing-style metadata-based
  search and found it helpful
• People sometimes preferred the metadata search
  when the task was more constrained
• But zero results are frustrating
• This can be alleviated with query previews
• People dis-prefer the standard simple search
• More study needed!

166
Application to Image Search
167
Image Search What is the task?

• Illustrate my slides?
• Find a crevasse
• Keyword match works pretty well
• Find inspiration for an architectural design?
• Needs richer search support

168
Faceted Metadata for Image Collection
Planalto Palace Parti Communiste
Francais Pantheon Oscar Neimeyer
Oscar Neimeyer Jacques-Gabriel
Soufflot 20th Century 20th Century
17th 18th C. Brasilia Paris
Paris Stone Curvilinear Stone
Image Architect Period Location Concept
169
Faceted Metadata for Image Collection
Planalto Palace Parti Communiste
Francais Pantheon Oscar Neimeyer
Oscar Neimeyer Jaques-Gabriel
Soufflot 20th Century 20th Century
17th 18th C. Brasilia Paris
Paris Stone Curvilinear Stone
Image Architect Period Location Concept
170
SPIRO Query Form (Original)
171
SPIRO query on Subject church
172
Pilot Study

• Architecture task
• Emphasize images over text
• Use hypertext-style interface as a reasonable
  baseline for comparison
• Find out how much choice is too much
• Find out whether explicit metadata is better than
  implicit more-like-this

173
Evaluation Methodology

• Solicit feedback from architects to determine if
  faceted metadata is helpful and how to present it
• Informal evaluation of paper prototype
• Informal study of a crude live version
• 1 hour one-on-one with 9 architects /grad
  students, 2 tasks (audio recorded) and a survey

174
Results of a pilot study with Archictects
Metadata is Helpful

• Very positive feedback about the general approach
• All 9 participants named the metadata in the
  search results area as their favorite aspect of
  Flamenco
• Metadata was successful at giving hints about
  where to go next
• Perceived as useful These are places I can go
  from here.

175
Results More Metadata Please

• Participants asked for more metadata
• Although there were complaints about the contents
  of the metadata, users still wanted more
• Longer lists of options (more hints)
• Users wanted more control to make very specific
  searches
• Half the participants requested the ability to
  control order of results with metadata
• Juxtapose visible images 2 different ways
• Overview (one image from each project) vs. like
  together ( all images of a project next to each
  other)
• Different than ranking for text retrieval
  (precision, recall), but ordering does matter

176
Results Complaints

• The UI was not successful at clarifying searching
  within results vs. starting a new search
• Only 2 of the 9 participants understood the
  distinction without discussion but they want to
  do both
• The 1/3 of the participants who couldnt find a
  treasure hunt image felt that Flamenco was slow
• Corroborates findings that perceived system speed
  is about finding what you want (Spool 00)

177
New Developments

• A new, sophisticated implementation
• Richer, hierarchical, cleaned up metadata
• Usability Study contrasting four versions
• Single search form
• Multiple facet search form
• Yahoo-style directory-based
• Faceted interface with query previews
• Results TBA

178
(No Transcript)
179
(No Transcript)
180
(No Transcript)
181
Tools

• Our system (all open-source)
• Mysql (has a text search component)
• Python 2.2
• Python-mysql
• Webware (python application server)
• Earlier attempt
• Cold fusion not flexible enough, not enough of
  a programming language

182
A new term Parametric Search

• From an XML glossary
• "A search request submitted to a search or
  database engine delivered with consideration for
  the metadata of the underlying dataset.
• www.sla.org/chapter/ctor/courier/v37/v37n1.pdf

183
Commercial Tools

• This list is NOT comprehensive
• These are NOT recommendations
• General Search
• Inktomi Search/site (formerly Infoseek ultra)
• Specializing in Online Catalogs
• Dieselpoint
• Requisite
• Saqqara
• Question-answering
• Askjeeves
• Primus (formerly Answerlogic)

184
Parametric Search

• A survey of sites using parametric search
• http//www.amp.com/search/default.asp (see
  product family search)
• http//ebiz.zilog.com/
• http//www.sears.com (Dieselpoint)
• http//dieselpoint.com/flashlink.htm (for
  Dieselpoint 2.0 demo)
• http//www.findmro.com (Requisite's BugsEye)
• http//www.cypress.com (Saqqara's one step)
• http//infineon-tech.sacosnet.de/search/index.htm
• http//www.idt.com/tools/parametric.html
• http//www.ti.com/sc/docs/psheets/parms/uarts.htm
  parms
• http//www.gensemi.com/search/productsearch.htm
• http//www.usa.samsungsemi.com/search/
• http//www.gearfinder.com
• http//www.mysimon.com/category/index.jhtml?cbaby
  diaperingbathing

Site list courtesy Mark Detweiler
185
Parametric Search Usage

• Goal is to focus on product group for comparison
  shopping.
• Common Procedure
• Begin with a list of product "families" or
  groups.
• User selects a category, and is prompted to
• 1) select a sub-category from a list of
  hyperlinks or
• 2) select search parameters using a form
• If the number of results is too big, the system
  may prompt the user to refine the search further.
  
• When an acceptable number of results is returned,
  the user sees a list of products which can be
• 1) sorted by various criteria
• 2) selected for display in a comparison table
• 3) viewed individually with more detail.

186
Parametric Search as used on these Sites

• Observations
• Only one facet (appropriate for products?)
• No query previews
• Breadcrumbs rare
• Many allow sorting by attribute to facilitate
  comparison
• Others like this simply moves up the hierarchy

187
Summary and Conclusions
188
Summary

• Web site search needs improvement
• Users want more organized results
• Our approach integrate navigation with search
• Metadata is being mixed and matched in
  interesting ways, but there are no guidelines on
  what works
• We are investigating how to design websites
  containing large sets of items
• Preliminary results indicate that metadata
  organization is useful in some situations
• Depends on the type of search need

189
Advantages of the Methodology

• Supports different types of information seeking
  tasks
• Uses interface idioms known to be usable for
  general users
• Flexible content entry and update
• Systematically integrates navigation search
• Gives user control
• Allows integration with advanced methods

190
Summary

• Our research goals
• Systematically determine what works, with the
  following emphases
• Task-centric
• Integrate metadata with search
• Dynamic previews
• Easily retrace steps
• Develop recommendations that reflect both the
  task structure and the richness of the
  information structure
• In future integrate with more sophisticated
  displays

191
Some Unanswered Questions

• How best show combinations of facets that consist
  of large hierarchies?
• How to use faceted metadata to expand (as opposed
  to refine)?
• How to integrate with relevance feedback (more
  like this)?
• How to incorporate user preferences and past
  behavior?
• How to combine facets to reflect tasks?

192
Thank you!
For more information

• bailando.sims.berkeley.edu/flamenco.html