Web Search Alert OLA Super Conference 2003

1
Web Search Alert OLA Super Conference 2003

• Gwen Harris
• goharris_at_websearchguide.ca

2
Google is a Verb
3
Year of Change

• Subject directories faltered.
• Search engines got better, using features to
  compete (or catch up).
• Meta-search engines (some) tackled the Invisible
  Web.

4
Agenda

• Trends and Overview
• Individual search engines using themAlso some
  Google spin-offs.
• Best of tools. What matters.
• Looking forward.

5
Vote for second best service.

• Altavista
• AlltheWeb
• Altavista
• MSN
• Teoma
• Yahoo
• Vivisimo

• Consider results, features, design, ease of
  use.

6
Search Engine Watch Awards

• Outstanding Search Service Google
• Second place AlltheWeb options and interface
• Other picks Yahoo, Altavista, Teoma, Lycos and
  MSN (trailing badly)
• Source SEW 2002 Awards

7
Subject Directories

• Yahoo and Looksmart very commercial paid
  inclusion, pay per click, sponsored listings.
• Yahoo has been portalized and Googlified. Hard to
  find the directory.
• Open Directory Project (dmoz.org) is tired.

8
Yahoo

• Excellent for news, finance, movies, music, and
  tools, but has strayed from being a directory.
• Search results are from Google with Yahoo items
  marked. Eg venezuela oil.
• Access directory at dir.yahoo.com.
• Watch for Inktomi to replace Google in 2003.

9
Scholarly Directories

• Alternative directories created by librarians
  at university and public libraries
• Resource Discovery Network (UK) www.rdn.ac.uk
• Infomine (US) infomine.ucr.edu growing
• Librarians Index to the Internet (US) lii.org
• Toronto Public Library Virtual Reference Library
  vrl.tpl.toronto.on.ca

10
Search Engine Contests

• Google dominates
• AlltheWeb pushing hard but not on chart
• Teoma carving out a niche
• Hotbot metamorphosed
• Altavista refreshed itself
• MSN keeps on ticking in spite of pop-ups

11
The Standings

• 1. Google 54.7 
• 2. Yahoo 22.1 
• 3. MSN Search 9.5 
• 4. AOL Search 3.7
• 5. Terra Lycos 2.8
• 6. Altavista 2.5
• 7. Askjeeves 1.5
•  

• Percent of visitors to a site from a given search
  engine
• Source Search engine war Yahoo is back
  according to OneStat.com (Jan 27, 2003)

12
Trends

• Indexing pdf and other non-html formats.
• Boolean spreading. AlltheWeb has full capability.
  Teoma limited.
• Fielded searches title, site.
• Dynamic descriptions keyword in context
• Geographic recognition ATW, AV, Google.
• Paid inclusion. Site pays to be deeply and
  frequently indexed. (ATW, Inktomi, AV)
• Skins you pick the look.

13
AlltheWeb

• Indexes Flash, pdf, MS word. Total 2.1 b.
• Boolean Rank
• Title, site, filetype
• Language detection from ISP number.
• Search Tools for the browser.
• More powerful features than Google.

14
(No Transcript)
15
AlltheWeb Exercise 1

• susan orlean orchid thief note snippets and
  descriptions of hits.
• Use search within results - adaptation
• Select boolean -- susan orlean and orchid
  thief and (review or ratings)
• susan orlean and orchid thief rank movie

16
AlltheWeb Exercise 2

• Field Searching
• siteaccessola.org superconference
• titleontario titlelibraries reference
• filetypepdf title"city of toronto" water

17
AlltheWeb Exercise 3

• Advanced Search
• Embedded Media Types Must have tetris game
  Macromedia Flash
• File Format hockey Format flash

18
Altavista

• Added pdf grew to 1.3 billion.
• Prisma analyzes text of first 50 pages to
  identify strong word groupings and associations.
  Easier to refine search.
• Much improved freshness.
• Dropped pop-ups, pop-unders.

19
Web Communities

• Researchers at NEC Web is a self-organizing
  structure of communities of common interest or
  related information. (Mar/02)
• Research group at the University of Iowa
  Smaller, well-linked mini-webs concentrated
  around subjects. (Oct/02)
• Both are designing new mini-web crawlers
• ebizsearch.org (eBusiness) citeseer (CompSci)

20
Teoma

• Tries to identify the communities.
• For a set of results it identifies clusters of
  highly linked pages and derives a topic name.
• Picks out hubs or link centres.
• Ranks the authorities.

21
(No Transcript)
22
(No Transcript)
23
Teoma 2.0

• 500 million pages
• Spell check
• intitle smart growth toronto looks for all
  words in title.
• sitegov.on.ca smart growth
• Accepts OR but no nesting smart growth OR
  sustainable development

24
Hotbot

• One stop searching Fast, Google, Inktomi, Teoma.
  Convenient
• Fewer search features. Lite.
• Can set preferences and change skins.
• OR use Hotbot UK (hotbot.co.uk) only Inktomi.
  Handles Boolean AND OR NOT

25
Inktomi

• Used by Hotbot, MSN, Looksmart, Overture
• Web Search 9 code name for improvements
• 3 billion documents. Includes pdf and e-library
  listings
• Refresh every 10 to 14 days. Refresh paid
  inclusions every 2 days
• Smart summaries editor, custom, generated
• Knows York from New York

26
Googlification

• Alexa, AOL, Netscape, iWON, BBCi in the UK,
  Sympatico in Canada, and makes up most of the
  search results at Yahoo
• 76 of searchers time Google 15 AOL 4
  Yahoo 6 25.4 of 33.5 million hours in August
  (2003) (Source D. Sullivan)

27
Google

• Google wants to organize all the worlds
  information, not just what is digitized. Craig
  Silverstein, Google CTO, quoted in Information
  Today (Jan 2003)

28
Google in 2002

• Increased size to 3.1 billion
• Google News (news.google.com) 4,000 sources,
  automated grouping process for topics and photos.
  
• Froogle (froogle.google.com) find products at
  online stores.
• Google Labs

29
Playing with Google

• Glossary finds definitions for words, phrases,
  acronyms. (Labs)
• Webquotes includes quotes about a site in the
  results. (Labs)
• Touchgraph Google Browser visual presentation
  of related (linked) pages
• Google Fight eg Paul Martin v Jean Chretien

30
Get Visual

• Kartoo (www.kartoo.com) cartographic
  meta-search engine.
• Mouseover to see description.
• Refine by topic.
• See groups dynamically.
• 5 language interfaces.

31
Meta-Search Engines

• Typical features cluster results (Vivisimo),
  offer a choice of a specialty databases.
• Ez2www search settings, clusters, 1,000 meta
  engines, bookmarklets
• Profusion revamped create own group from
  1,000 (?) engines, set up alerts.
• Wondir.org meta search, deep search, people
  variety of resource types.

32
Best Tools (Arbitrary)

• Directories
• RDN (UK)
• LII
• Of Interest
• Wondir.org
• Kartoo.com
• Help with invisible web

• Web Search
• Google
• AlltheWeb
• Hotbot UK
• Teoma
• Vivisimo
• Ez2www
• Profusion

33
What Matters

• Size Google indexes more of accessola.org than
  does ATW
• Relevancy - compare the engines
• Site and title searches essential for narrowing
  search
• Multiple formats especially pdf
• Clustering into topics identifying web
  communities

34
Looking Forward

• Stephen Arnold When Google goes public, the
  crunch for bottom-line performance will change
  todays queen of search into a princess of
  paid-listings.

35
Looking Forward

• Greg Notess (Dec 2002) Google personalization,
  conceptual understanding, fresher database.

36
Looking Forward

• New alliances to compete with Google.
• More for-fee content.
• Continued battle with spam.
• Fresher databases.
• More multimedia search tools.
• Search engines for mini-webs.

37
The End