Design of Web Agents Inspired by Brain Research

1
Design of Web Agents Inspired by Brain Research

• Maya Dimitrova, Hiroaki Wagatsuma and Yoko
  Yamaguchi
• Institute of Control and System Research
  Bulgarian Academy of Sciences
• Laboratory for Dynamics of Emergent
  Intelligence, RIKEN Brain Science Institute, Japan

2
Content

• 1. Introduction
• 2. Information retrieval studies based on
  cognitive heuristics (problem definition)
• 3. Theta phase coding in the brain and episodic
  memory
• 3.1. The Theta phase model
• 3.2. Relevance to episodic/autobiographical
  memory
• 4. Autobiographical memory in Web context An
  example
• 5. Conclusions and future work

3
Introduction

• Modeling agents own experience their
  reconstructed past - is essential to optimize
  agent behavior (i.e. agents for Web knowledge
  search, retrieval and presentation)
• We propose to base this optimization on recent
  studies and models of the brain focusing on the
  generality of processing in the animal and human,
  not on their difference
• We have focused on episodic memory and the
  underlying hippocampal mechanisms of processing
  as closest to the story-telling approach of
  agent design

4
Information retrieval studies based on cognitive
heuristics

• Problem statement
• Current Web page classifiers perform excessive
  amount of processing to reach simple
  classificatory results
• Whenever the classificatory task is
  multi-dimensional, agent performance deteriorates
  dramatically
• There is the database or storage problem where
  models have always been least analogous to human
  memory
• Human memory processing is fast, heuristical and
  much more efficient in the multi-dimensional
  case, than in the sequential processing case
  (contrary to the machine) etc. (recent finding)
• There is always a point in looking into recent
  brain studies for their fast and accelerated
  development as empirically-based science

5
Information retrieval studies based on cognitive
heuristics (2)

• The current study
• Presents the main ideas of the Theta phase
  precession theory as proposing a memory model,
  different from the mainstream rate-coding models
• Operationalizes episodic memory for the purpose
  of design of memory-optimized Web search agent
• Provides an example to illustrate the described
  process in a human Web search process and some
  relevant conclusions for agent future development

6
The agent (1)

• In earlier work we designed a Java-servlet (a
  wrapper to a search engine) to diagnose the level
  of expertise and detail of a corpus of on-line
  available Web pages
• Weak point what to store in the database and in
  what form

7
The agent (2)

• It employed pattern classification based on
  syntactic analysis for diagnosing the amount of
  detail
• And cognitive heuristics based on memory research
  for the expert level of the text

8
The agent (3)

• In following history of interaction, optimization
  is needed much the way natural cognitive systems
  perform - learn, abstract, remember and forget
• We use the autobiographical memory metaphor to
  apply to such kind of Web agents and the Theta
  phase precession theory to model it

9
Current MATLAB GUI versionof the agent interface

• The difference of the MatLab version from the
  Java version is that the Java-servlet processed
  also the hyperlink structure of the Web page
• The current version is simplified for algorithmic
  and modeling reasons

10
Assumptions of the current study

• Assumptions
• - User interaction with the present-day Web is an
  autobiographical experience (remembering
  sequences of events like personal-life episodes)
• Memorizing events in a human-like memory system
  undergoes constant synthesis as any other
  learning process and therefore can be implemented
  in autonomous agent on the Web

11
Operational definitions in the current study

• Definition of Episodic Memory
• The system of processing episodes of Web search
  and transforming them into autobiographical
  events of successful/unsuccessful Web navigation
  to reach the needed goal
• Every Web page viewed by the user is a complex
  perceptual and semantic stimulus, i.e. an episode
  of Web search
• What is stored in episodic memory from the rapid
  viewing of a succession of pages is a set of
  fragments of the search episode rather than the
  entire pages and their exact order of viewing

12
Hypotheses of the current study

• The theoretical hypothesis is that episodic
  memory extracts meaningful cues from the
  fragments of the page-retrieval episodes, rather
  than random perceptual or semantic cues
• Meaningful we define as having anticipatory
  value (i.e. useful in the future)
• The tested hypothesis is that the mechanisms of
  event synthesis from fragments of page-retrieval
  episodes are the same in personally-significant
  and neutral situations, that is universal for
  learning per se
• These effects are reproduced by both the human
  and the agent in the current study

13
Theta phase coding in the brain and episodic
memory

• Theta phase theory models rhythmic patterns of
  activity of assemblies of neurons in the brain
  and their role in complex cognitive tasks (theta
  rhythm 4-10 Hz)
• Theta phase modulation is associated with
  learning and memory in rats and humans A
  mechanism of pattern compression has been
  observed in hippocampal neuron assemblies, which
  is called theta phase precession
• Modelling hippocampal nerve cirquit by the Theta
  phase precession theory (Yamaguchi, Yamaguchi et
  al.) reinstates complex learning effects observed
  in human episodic memory like object-context
  integration, single-trial learning and time-space
  contextualization

14
Neuro-realistic model of episodic encoding and
retrieval

• (a) Main modules of the hippocampus ECII,
  ECIII, CA3, CA1, DG
• (b) Entorhinal input, emerging from the sensory
  system and representing the vector time series
  produced by behavior
• (c) when the rat runs to the right, the phase
  shift in firing within each theta rhythm cycle
  occurs in place cells 1 to 4, which are activated
  sequentially

15
Properties of the phase model

• (The recurrent connections in the CA3 layer of
  the hippocampus form asymmetric connections so
  that units with prior activations are connected
  to those with subsequent activations
• These activations are a consequence of the
  coincidence between their phase differences and
  the time window of the asymmetric synaptic
  plasticity

16
Model of hippocampal nerve circuit

• (a) mechanism of theta phase coding
• (b) phase shift in firing within each theta
  rhythm cycle

17
Relevance to modeling memory

• Partial cuing triggers pattern completion in the
  CA3 and its transmission to the CA1 and EC. Thus
  a stored pattern in the hippocampus can be sent
  to the cortical area to replay the input pattern
• The temporal difference between encoding and
  retrieval in the network parallels behavioral
  tests
• The retrieval is ten times faster than the
  behavioral input in the encoding stage and the
  retrieval time scale is similar to that of the
  compressed pattern of theta phase precession
• The retrieved activity can cause further synaptic
  plasticity in other cortical areas for memory
  consolidation or memory transfer from the
  hippocampus to the cortex

18
Importance of theta phase precession theory to
the present study

• The emergence of rhythmic patterns in the absence
  of memory to cause its formation is a plausible
  candidate for a neural correlate of the formation
  of complex and durable cognitive effects
  governing future behavior i.e. autobiographical
  memories
• We have aimed to give a relevant example to
  illustrate these effects

19
How to find an Art Gallery on the Web with
incomplete initial information?

• Participants
• - Motivated
• - Neutral
• Search goal
• - Example of new concept learning, similar to
  artificial concept learning Kurenai Kai
• Search task
• - Google search from incomplete initial
  information (The name of the gallery Kurenai
  Kai not known apriory)

20
Results and discussion
Table 1. User performance on Web search with
incomplete initial information about the search
goal

• Our study of Web search has replicated the
  mechanism of performance of the theta phase
  precession model
• The Web paths resemble random search until a
  meaningful path is composed
• We observe that meaningful paths are composed,
  rather than encountered

21
Web search Kurenai Kai

• Left path represents the nature of the search
  with incomplete initial information
• Right path represents the shortest path with
  well-remembered concept term

22
Remembering meaningful fragments of search
episodes
Left path represents episodic memory guided
retrieval based on a set of meaningful cues and
insight The pages to the right represent the
mappings to the respective initial cues (none of
them are identical)
23
Conclusions from the Web search study

• Users base subsequent retrieval of the search
  path on remembering the events of their search,
  which are the meaningful fragments of knowledge
  inside the Web pages, not the exact
  page-retrieval episodes
• We propose that the autonomous Web agent behaves
  in analogy with the event-based
  (autobiographical) strategy that Web users are
  applying for retrieval of useful information,
  which is some state of knowledge with
  anticipatory value (useful in the future)
• Next will be tests of the agent for optimized
  storage of fragments of knowledge contained in
  the Web page, rather than of snapshots of
  sequences of Web pages

24
Fragment extraction and recombination of textual
elements as a process of encoding search events
Agent reinstatement of the search task
25
Temporal evolution of episodic events by the
neuro-cognitive agent
26
Conclusions and Future Work

• We presented one approach towards the design of
  Web agents based on recent knowledge from brain
  research
• We have tried to demonstrate how the theoretical
  framework of theta phase coding endows real-time
  process of memory formation of Web search
  experiences and the retrieval process of
  effective routes to the targets through the
  experienced sites
• The proposed approach focuses on bridging the gap
  between neuroscience and Web technologies
• It is currently being applied to retrieval of
  cardiological knowledge from the Web

27
Acknowledgement

• This work is supported by
• NSRF of the Bulgarian Ministry of Education and
  Science as part of the Research Project ? MI -
  1509/2005
• Multimodal User and Sensor Interface in a
  Computer-Aided System for Cardiological
  Diagnosis and Intervention

28

• Thank you
• for your attention