Applications of Universal Computing

1
Context-Aware Computing
John Canny HCC Retreat 7/5/00
2
Consequences of Ubiquitous Computing and Calm
Technology

• We want to have constellations of devices working
  for us, somehow inferring and supporting our
  activities.

3
Mining Tacit knowledge from activity

• Activity mining involves content and context the
  inter-relatedness of people and information
  objects.

4
ABC Activity-Based Computing

• Activities are clusters with
• Users
• Documents
• Tools
• Realizations can be
• GUIs
• speech, etc.
• Nearness encodes awareness.

5
ABC Inspiration

• Based on activity theory from psychology
  (Vygotsky, Leontev, Engestrom, Kutti).
• Activities are high-level behaviors directed
  toward some end, usually with others.

• Recent studies (Gruen 96) support the value of
  the activity model, but
• activities are usually not distinguishable
  instantaneously, they interleave and overlap.
• Activities can persist over different time frames
  and occur in different places.

6
ABC What is it good for?

• Streamlining interaction disambiguation, menu
  customization.
• Prefetching files, prestarting devices.
• Awareness (who else is working on the project
  now)
• Attention management - graded awareness of other
  activities.
• Pro-active sharing in group work.
• Infering document utility.
• Human expertise location.
• Re-contextualizing documents authorship, roles,
  backgrounds, discipline-specific vocabulary...

7
ABC A lo-fi prototype (Danyel Fisher)
8
ABC Contextual data sources

• Who
• Direct communication 1-1 email, phone, F2F.
• What
• Topical discussions, forums, F2F meetings.
• Document writing, reading, search, markup.
• When
• The current time time windows for activities.
• Where
• A place which has meaning for the users
  activities.

9
ABC Representation
User 1
Mail
User1
Read
Document5
Start
User 2
Project
Device7
Write
Program12
User 3
Document3
Markup
Algorithms are SVD and other pattern analysis
schemes
10
Knowledgescapes (Heyning Cheng)

• Knowledgescapes is a search engine that uses
  activity logs only - it doesnt look at document
  contenthttp//indios.cs.berkeley.edu/knowledgesc
  apes.html
• Activity data is used in knowledgescapes to infer
  document relevance/quality. Results
• Performance is fair with text queries
• Performance is very good with related document
  queries
• In progress Infer user expertise from document
  selections.

11
Knowledgescapes prototype
Search Engine
Query(terms)
Document1
Document2
Information Need
Document3
Document4
Rankings from reading time. Treat as probability
of interest or E(interest)
12
Another inspiration LSA
Word 1
Passage 1
Passage 2
Word 2
Passage 3
Passage 4
Word 3
This structure is resolved by SVD into latent
semanticcategories which better model the
document content.
13
Another inspiration LSA
Word 1
Passage 1
Concept 1
Passage 2
Word 2
Concept 2
Passage 3
Concept 3
Passage 4
Word 3
Decomposition of the linear may from words to
passages into two linear maps, with latent
concepts in between.
14
Content and context mining

• Many successful content analysis schemes are
  based on LSA (Latent Semantic Analysis).
• Widely used context analysis schemes (from
  social network analysis) use similar algorithms.
• We are developing a latent-variable method which
  combines evidence about content and context to
  model activities. We will use it for
• Document prefetching for unseen documents
• Document authority/quality estimates
• Naming activities
• Building personalized thesauri

15
Context and content analysis
User 1
Term 1
Read
Document1
User 2
Write
Term 2
Document5
User 3
Term 3
Document3
Markup
Use document access data and LSA andFactor both
maps into latent categories
16
Sensors for the HCC lab

• Sensor networks
• Microphone arrays
• Camera arrays
• Small scanners
• Output
• Wall projectors, PDAs, the UPM
• Networking using USB and IrDA
• Sensor proxies for outside access via Jini or
  HTTP