Dimensions of Scalability in Cognitive Models

1

• Dimensions of Scalability in Cognitive Models
• Research Team
• Carnegie Mellon University - Psychology
  Department
• Dr. Christian Lebiere
• Dr. David Reitter
• Dr. Jerry Vinokurov
• Michael Furlong
• Jasmeet Ajmani

2
Overview

• Goal Scaling up high-fidelity cognitive models
  by
• Composing models
• Abstracting models
• Running large networks of models
• ACT-UP a toolkit view of cognitive architectures
• Same validated functionality, different form
• Lemonade game Reusing and integrating models
• Language learning Scaling up to network
  cognition
• The Geo-Game Bringing it all together
• Platform for experimentation and integration

3
Dimensions of Scaling
4
ACT-R Cognitive Architecture

• Computational implementation of unified theory of
  cognition
• Commitment to task-invariant mechanisms
• Modular organization
• Parallelism but strong attentional limitations
• Hybrid symbolic/ statistical processes

5
Issues with Cognitive Modeling

• High-fidelity cognitive models provide very
  accurate models of all observable dimensions of
  cognition (time, accuracy, gaze, neural) but
• They are computationally intensive as they
  simulate all cognitive processes in full detail
• They are labor intensive to specify all aspects
  of cognitive performance (knowledge, strategies)
• They are specialized to a given task in a way
  that makes them difficult to compose and reuse
• They usually focus on single-agent cognition

6
Scaling Up Cognitive Modeling

• Enable the implementation of more complex
  cognitive models in a more efficient manner
• Scale up the application of cognitive models to
  simulate learning and adaptation in communities
  (e.g., 1,000 models in parallel)
• Enable reuse and composition of cognitive models
  similar to software engineering view
• Facilitate integration of cognitive models with
  other modeling and simulation platforms
• Improve maintenance, update and validation

7
The Approach

• Difficulties ACT-R is heavily constrained
  already, and models are difficult to develop,
  reuse and exchange
• Constraints Architectural advances require
  further constraints, e.g. more representational
  constraints
• Scaling it up Complex tasks, broad coverage of
  behavior, multi-agent cognition and predictive
  modeling may motivate further architectural
  changes
• Solution produce models at a higher abstraction
  level
• Retain and emphasize key cognitive mechanisms
• Abstract purely mechanistic model aspects
• Precisely specify model claims, underspecify/fit
  rest
• Benefits of abstraction in efficiency,
  scalability, reuse

8
Cognitive Strategy
Symbolic
deterministic
Subsymbolic (Learning / Adaptation)
non-deterministic
explains empirical variance
9
Underspecified Models
underspecify
deterministic
specify
non-deterministic
explains empirical variance
10
(Lisp Functions)
11
ACT-UP vs ACT-R 6

• Declarative memory chunks as objects
• Explicit context specification all activation
  computations
• Procedural memory productions as functions
• Explicit conflict set groups utility
  reinforcement learning
• ACT-UP is synchronous with serial execution
• Parallelism in process of being implemented
• Perceptual-motor modules being planned

12
Validation

• Against canonical ACT-R tutorial models data

13
Efficiency

• Sentence production (syntactic priming) model
• 30 productions in ACT-R, 720 lines of code
• 82 lines of code in ACT-UP (3 work-days)
• ACT-R 6 14 sentences/second
• ACT-UP 380 sentences/second

14
Scalability

• Language evolution model
• Simulates domain vocabulary emergence (ICCM
  2009, JCSR 1010)
• 40 production rules in ACT-R
• Complex execution paths could not prototype
• 8 participants interacting in communities
• In larger community networks
• 1000 agents
• 84M interactions (about 1 min. sim. Each)
• 37 CPU hours

15
Related Work

• Douglass (2009 2010) on large declarative
  memories
• Implementation through Erlang threads
• Focus on scalability
• Salvucci (2010) work on supermodels
• Integrating and validating independent models
• Focus on instruction interpretation for
  generality
• Stewart and West (2007) work on Python-ACT-R
• Similar deconstructive view of architecture
• Integration with neural constructs

16
Future Work

• Complete validation against canonical model set
  currently in beta testing full release planned
  for spring 2011
• Possible collaboration with AFRL Mesa on
  implementation of finite-state-based systems
• Potential use in other projects (Minds Eye,
  Robotics CTA)
• Allow optional parallelism where needed and
  desired
• Implement perceptual and motor modules
• Potential implementation in other languages (C,
  Java) to facilitate code-level integration with
  common frameworks
• Reitter, D., Lebiere, C. (2010). Accountable
  Modeling in ACT-UP, a Scalable, Rapid-Prototyping
  ACT-R Implementation. In Proceedings of the 2010
  International Conference on Cognitive Modeling.
  Philadelphia, PA.
• Lebiere, C., Reitter, D. (2010). ACT-UP A
  Cognitive Modeling Toolkit for Composition, Reuse
  and Integration. In Proceedings of the 2010
  MODSIM conference. Hampton, VA.
• Lebiere, C., Stocco, A., Reitter, D., Juvina,
  I. (2010). High-fidelity cognitive modeling to
  real-world applications. In Proceedings of the
  NATO Workshop on Human Modeling for Military
  Application, Amsterdam, NL, 2010.

17
Cognitive principles in cooperative and
adversarial gamesMetacognition transfers via
ACT-UP
Networks (Distributed Knowledge)
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
18
ACT-UP Rapid prototyping/Reuse

• Dynamic Stocks Flows ACT-UP model
• Winning modeling competition entry
• Model written in lt 1 person-month
• Free parameters (timing) estimated from example
  data
• Model generalized to novel conditions
• Reuse of Metacognitive Strategy in the Lemonade
  Stand Game (BRIMS 2010)
• Kevin A. Gluck, Clayton T. Stanley, Jr. L.
  Richard Moore, David Reitter, and Marc Halbrügge.
  Exploration for understanding in model
  comparisons. Journal of Artificial General
  Intelligence (to appear), 2010.
• David Reitter. Metacognition and multiple
  strategies in a cognitive model of online
  control. Journal of Artificial General
  Intelligence (to appear), 2010.
• David Reitter, Ion Juvina, Andrea Stocco, and
  Christian Lebiere. Resistance is futile Winning
  lemonade market share through metacognitive
  reasoning in a three-agent cooperative game. In
  Proceedings of the 19th Behavior Representation
  in Modeling Simulation (BRIMS), Charleston, SC,
  2010.

19
Multi-agent Games

• 2x2 games such as the Prisoners Dilemma
• Evolution of cooperation vs. competition
• Memory-based expectations (Lebiere et al, 2001)
• Adversarial games (Paper Rock Scissors, Baseball)
• Zero-sum competition where predictability is
  fatal
• Sequence-based expectations (Lebiere et al, 1998
  2003)
• Lemonade game (3 players)
• Simultaneous cooperation and competition
• Predictability can be desirable for cooperation

20
The Lemonade Stand Game
Zinkevich (2010, unpublished)

• In each iteration, each of three players chooses
  a location 1..12
• Payoff is proportional to the distance to left
  and right neighbors.
• Hidden moves (blind choice)
• 1 game 100 iterations, then reset (no state
  across games)

21
Basic Strategies

• Random (unpredictable) choose random loc.
• Sticky (predictable) choose same location
• Roll, SquareRoot
• Tournament with those four agents
• Equal performance

22
Strategy Elements

• Offer Cooperation Be predictable
• Predict Learn patterns of opponents
• Maximize Utility Choose highest expected payoff
• Cooperate Pick friendly opponent whose payoff
  is also maximized
• Monitoring analyzing own/others performance,
  keep history

23
Strategies
offer Coop Predict maxPayoff Coop MonitorSelf MonitorOpp
Sticky
StickySmart
StickySharp
CopyCat
Statistician
Cooperator
Strategist
24
Metacognition

• Facility to constantly monitor performance, and
  to adapt behavior accordingly
• Choose the best-performing strategy out of a set
  of strategies (Flavell 1979, Brown 1987)
• Strategy-shifting assumed in Dynamic Stocks
  Flows data (DSF Challenge)

25
General Metacognition

• Prediction of each opponents next move
• Learn from agents history in this game
• Multiple possible representations and
  pattern-matching
• Action Making a move
• Optimize Utility
• Suggest cooperation
• Cooperate
• Hurt the worst adversaries

26
Evaluating Strategies

• Prediction and Action strategies are learned as
  episodes (instances)
• Each prediction strategy per iteration, per
  opponent
• one action strategy per iteration
• Instance-based learning (GonzalesLebiere 2003)
• Objective Prediction quality/Action payoff
• Blending weighted mean (recency, frequency,
  objective as above)

27
Metacognition in Prediction
as in Reitter (2010) - DSF model

• Each prediction strategy suggests a next location
  for each opponent
• All past predictions are stored throughout the
  game ltt,l,pgt (time, actual location, predicted
  probability of that location)

ACT-R Activation (recency, frequency)
Expected success of strategy s and agent a
Episode in memory time t, actual chosen location
l of agent a, predicted probability p for l,a by
strategy s
Metacognition for Actions is similar
28
Evaluation

• Outcome of each strategy depends on configuration
  of players
• Some strategies will cooperate
• Metacognitive strategy is flexible, achieves
  consistently high results
• Bigger circle higher winnings. Darker circle
  consistent results.

29
Tournament
30
Adaptive Multi-Agent Behavior

• Offering cooperation and cooperating with the
  right opponent are crucial to doing well
• Metacognitive layer allows an agent to trump all
  others through generality and adaptivity
• Research questions
• Human performance in cooperative games issues of
  trust, social and cultural biases
• Memory activation and rational retrieval
  expectations as proxy for weighing past strategy
  success limits of metacognition

31
Future Work

• ACT-R/ACT-UPs learning vs. more basic Bayesian
  models is cognitive learning more robust through
  open-endedness?
• Break down current limits of cognitive models
  generality
• Are canonical architectural parameters optimal
  through coevolution for empirical clustering
  factors and degrees?
• Key part of environment is social interactions
• Automatic acquisition of rules, strategies,
  structural representations rather than modeler
  specification
• Metacognition accumulation of micro-strategies
  library into reusable, general-purpose
  metacognitive layer
• Combination of above provide way of breaking out
  of task-specific models and their assumptions
  beyond task-specific parameters,
  representation, strategies

32
Scaling Up Cognitive Models from Individuals to
Large NetworksThe case of communication in human
communities
Dr. Christian Lebiere Dr. David Reitter Carnegie
Mellon University
Networks (Distributed Knowledge)
Communities (Teamwork)
David Reitter and Christian Lebiere. Towards
explaining the evolution of domain languages with
cognitive simulation. Cognitive Systems Research
(in press), 2010.
Dyads (Dialogue)
Individuals
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
33
Interactive Alignment

• Garrod Pickering 2004

from Garrod Pickering, BBS 2004
34
Adaptation in Language

• Rapid decay within 8-10 secondsexperimentally,
  for selected constructions Levelt Kelter
  (1982),Branigan et al. (2000)
• Long-term adaptation effects, which do not decay,
  have also been observed (Comprehension
  Mitchell et al. 1995. Production BockGriffin
  2000)
• ACT-Rs declarative memory decay explains the
  repetition probability decay

Reitter (2008)
(Switchboard corpus)
35
Interactive Alignment
Syntactic and Lexical Adaptation Predict Task
Success! (Reitter Moore 2007)
Lexical Representation
Lexical Representation
from PickeringGarrod, BBS 2004
36
Domain Language Experiment

• Vocabulary Signs as meaning-signifier
  combinationSimple Communication System Lewis
  1989, Hurford 1989, OliphantBatali 1996
• Naming game an idealized transaction between two
  players
• Pictionary a director draws a given target
  concept using elementary drawings a matcher has
  to guess the concept.
• 20 target concepts, repeated
• Director/Matcher receive no explicit feedback

Brad Pitt

• Fay et al., Cognitive Science 34(3), 2010. Kirby
  et al., PNAS 2008 Fay et al. PhilTransRoySoc-B
  2008

37
Pictionary Performance
(empirical)
partner switch (communities)
partner switch (communities)
partner switch (communities)
ID accuracy proportion of signs retrieved
From data by Fay et al. 2010
38
Broad Questions

• How does the architecture of human cognition
  interact with social structure?
• Have the human mind and large-scale social
  structures co-evolved?
• Can modeling predict the kinds of team structures
  that will yield optimal communication and
  collaboration?

39
Pictionary Model in ACT-UP

• Ontology shared betweendirector and matcher
• abstract target concepts
• concrete drawings
• link weight distribution acquired from Wall
  Street Journal collocations
• Director chooses three related drawings to
  convey a target concept
• Choice is conventionalized
• Decision-making and memory retention modeled with
  ACT-UP

Ontology
weighted link
40
Lexicon

• Conventionalized Sign Domain Sign
• Held in memory, retrievable for drawing and
  recognition
• Director and Matcher acquire domain signs
• Director based on uni-lateral decision about how
  to draw the target concept
• Matcher based on un-verified guess
• Domain Signs are used when available otherwise,
  ontology is used

41
Cognitive Architecture

• ACT-R (Anderson 2007)
• Declarative Memory
• learning domain signs
• rational behavior activation recency
  frequency
• contextualization
• stochasticity in access (retrieval activation)

42
Pictionary and Networks
(ACT-UP model)
partner switch (communities)
partner switch (communities)
partner switch (communities)
ID accuracy proportion of signs retrieved 100
rep.
ReitterLebiereJournal of Cognitive Systems
Research, in press
43
Scaling up to Networks
Dr. Christian Lebiere Dr. David Reitter Carnegie
Mellon University
Networks (Distributed Knowledge)
Communities (Teamwork)
Reitter, D., Lebiere, C. (2010). Did social
networks shape language evolution? A multi-agent
cognitive simulation. In Proc. Cognitive Modeling
and Computational Linguistics Workshop (CMCL
2010), Uppsala, Sweden.
Dyads (Dialogue)
Individuals
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
44
Research Questions

• Does network structure affect convergence towards
  a common community vocabulary?
• Or Is declarative memory robust w.r.t. a variety
  of network structures?
• The small-scale, empirical and modeling data
  suggest that extreme networks (fully vs.
  disconnected) arrive at similar performance, but
  converge differently. How? Why?
• Larger communities that differ in their
  connectivity are needed to answer these
  questions.

45
Network Types

• In a network, only network neighbors play the
  naming game
• Social Small-World network (low path length,
  high clustering coefficient, assortatively mixed
  by degree)
• Grid (torus)
• Random Graph
• Organizational Trees
• Controlled mean degree (except trees), number
  of nodes
• Here 512 nodes, mean deg. 6., 50 rep. per
  condition

46
ID Accuracy Neighbors
dyads receive no feedback after each trial
preliminary results
47
ID Accuracy Neighbors
preliminary results
48
ID Accuracy Random Pairs
Indication of convergence towards common
vocabulary across network (measured after round
35)
Small World
Random
Grid
Tree
preliminary results
Tree vs. others n.s. (p0.14, MCMC on LMER
log(IDacc) cond (1sequence))
49
Summary

• Online Linguistic Adaptation is a known
  phenomenon
• syntactic, lexical. Between two and more
  participants.
• Nodes can adapt to their immediate surroundings
• Tree hierarchies function very well when stable,
  but are not robust to structural change
• Tree hierarchies represent contemporary
  organizational hierarchies and generalize typical
  command structures
• Small-World structures are more robust to change.

50
Future Work

• Which advantages do non-tree network
  organizational structures have in situations
  where environment/ground truth changes, where
  adversarial elements are present?
• How can temporal dynamics in network structure
  (gradual ramp-up in connectivity) support
  information convergence (domain vocabulary
  acquisition)?
• Do cognitive models require explicit information
  processing policies in non-tree hierarchies, such
  that accountability and reliability are
  preserved?
• Integration of communication with planning,
  control and decision-making in complex dynamic
  domains.

51
Information Exchange in Networks

• Simulation at cognitive level Language Evolution
  Model
• Simulation with Bayesian LearnersWang et al.
  (CMU Robotics), for a Bayesian Belief Update
  network
• Empirical validation is rare
• Real-time communication networks are rarely
  studied
• Most empirical datasets contain asynchronously
  produced communication, lacking control over
  exchanged information (e.g., Enron or Twitter
  corpora)

52
Human Networks Empirical Experiments with the
Geo Game
Dr. Christian Lebiere Dr. David
Reitter Psychology, Carnegie Mellon
University Dr. Katia Sycara Antonio Juarez Dr.
Paul Scerri Dr. Robin Glinton Robotics Institute,
Carnegie Mellon University Dr. Michael
Lewis University of Pittsburgh
Networks (Distributed Knowledge)
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Complex Tasks, Broad-Coverage Models
Controlled Tasks, High-Fidelity Models
53
MURI Team The Geo Game
CMU Psychology
Pitt
CMU Robotics
Cornell
GMU
MIT
Level 1,3
Level 2
Level 1,2
Level 1
Scaling of cognitive performance and workload
Level 1-2.5
Level 1-3
Level 1,3
Level 1
Task allocation among humans/agents
Probabilistic models of human decision-making in
network situations
Level 1,2
Level 1-2.5
Level 3
Level 2
Level 1-3
Decentralized control search and planning
Level 1,2
Level 2
Level 1,2
Information fusion
Level 1,2
Level 1,3, 4
Level 1-3
Network performance as a function of topology
Level 4
Level 2
Communication, evolution, language
Level 3
Level 2, 3
Adaptive automation
Level 1,2
Level 1
54
The Geo Game

• Information exchange in human networks
• On-line (real-time) communication
• Medium to large networks (15 to 1,000 nodes)
• Defined information needed to execute given task
• Information is spread throughout the network
• Natural language as a means to exchange
  communication
• Often task-specific, controlled language (e.g.
  radio communication)
• Trade-off communication vs. task execution

The Geo Game platform is being developed by CMU
Psychology and CMU Robotics
55
Geo Game Participants Task
56
Geo Game as Platform

• Subjects are organized in a graph
• vertices define communication channelssubjects
  can only communicate with network neighbors
• Currently small-world network
• The Geo Game is a platform
• Current game foraging task
• Other variants trading agents, varied
  information types (stochastic, graded, discrete,
  etc.), other networks (trees, adversarial
  networks)
• Serverweb-browser based system remotely
  deployable

57
Geo Game Push vs. Pull

• Basic manipulation push vs. pull of information
• Relevant to practical domain and theoretical
  cognitive issues
• Push condition
• Post all relevant information - items in cities
  path efficiency
• Maximize information at cost of overloading
  attention/memory
• Pull condition
• Specify needs and only answer/forward relevant
  information
• Minimize overload at cost of opportunities

58
Push vs. Pull Scalability

• If communication aids task success, does this
  effect scale?

One group of 15 participants September 2010
59
Geo Game Time-to-Response

• Question-Answer Pairs time (Q to A) is power-law
  distributed

cf. Barabasi (2010)
60
Publications

• Reitter, D., Lebiere, C. (2009). A subsymbolic
  and visual model of spatial path planning. In
  Proceedings of the Behavior Representation in
  Modeling and Simulation (BRIMS 2009). Best paper
  award BRIMS 2009.
• Reitter, D., Lebiere, C. (2009). Towards
  Explaining the Evolution of Domain Languages with
  Cognitive Simulation. In Proceedings of the 9th
  International Conference on Cognitive Modeling.
  Manchester, England.
• Reitter, D., Lebiere, C., Lewis, M., Wang, H.,
  Ma, Z. (2009). A Cognitive Model of Perceptual
  Path Planning in a Multi-Robot Control System.
  In Proceedings of the 2009 IEEE International
  Conference on Systems, Man, and Cybernetics. San
  Antonio, Texas.
• Reitter, D., Juvina, I., Stocco, A., Lebiere,
  C. (2010). Resistance is Futile Winning
  Lemonade Market Share through Metacognitive
  Reasoning in a Three-Agent Cooperative Game. In
  Proceedings of the Behavior Representation In
  Modeling and Simulations (BRIMS 2010) Conference.
  Charleston, SC
• Reitter, D., Lebiere, C. (2010). Did social
  networks shape language evolution? a multi-agent
  cognitive simulation. In Proc. Cognitive Modeling
  and Computational Linguistics Workshop (CMCL
  2010, at Association for Computational
  Linguistics ACL 2010), Uppsala, Sweden.
• Reitter, D., Lebiere, C. (2010). Accountable
  Modeling in ACT-UP, a Scalable, Rapid-Prototyping
  ACT-R Implementation. In Proceedings of the 2010
  International Conference on Cognitive Modeling.
  Philadelphia, PA.
• Reitter, D., Lebiere, C. (2010). On the
  influence of network structure on language
  evolution. In R. Sun, editor, Proc. Workshop on
  Cognitive Social Sciences Grounding the Social
  Sciences in the Cognitive Sciences (at Cognitive
  Science CogSci 2010), Portland, Oregon.
• Lebiere, C., Reitter, D. (2010). ACT-UP A
  Cognitive Modeling Toolkit for Composition, Reuse
  and Integration. In Proceedings of the 2010
  MODSIM conference. Hampton, VA.
• Lebiere, C., Stocco, A., Reitter, D., Juvina,
  I. (2010). High-fidelity cognitive modeling to
  real-world applications. In Proceedings of the
  NATO Workshop on Human Modeling for Military
  Application, Amsterdam, NL, 2010.
• Reitter, D. Lebiere, C. (in press). Towards
  explaining the evolution of domain languages with
  cognitive simulation. Journal of Cognitive
  Systems Research.
• Reitter, D. Lebiere, C. (in press). A
  cognitive model of spatial path planning.
  Journal of Computational and Mathematical
  Organization Theory.
• Reitter, D. (to appear). Metacognition and
  multiple strategies in a cognitive model of
  online control. Journal of Artificial General
  Intelligence.
• Gluck, K. A., Stanley, C. T., Moore, L. R.,
  Reitter, D., Halbrügge, M. (in press) .
  Exploration for understanding in model
  comparisons. Journal of Artificial General
  Intelligence.

61
Future Work

• The Geo Game can be exploited as an experimental
  platform for years to come
• Communication Alignment of communication
  standards (e.g., vocabulary)
• Knowledge Information type and acquisition and
  its influence on its distribution across the
  network (shared knowledge is not copied
  knowledge)
• Network structure influence on team performance
• Trust and strategy Adversarial networks
• Mixed human/model/agent networks
• Bootstrapping methodology for model validation
• Information filtering for humans