Small Worlds in Semantic Networks

1
Small Worlds in Semantic Networks

• Mark Steyvers
• Josh Tenenbaum
• Stanford University

2

• Real-Life Networks
• Collaboration network for film actors
• Power-grid
• Neural network of worm C. elegans
• WWW

• This Research
• Consider semantic networks
• Do semantic networks have similar properties?
• What network model can predict these properties?
  

3
Path Lengths ClusteringWatts Strogatz 98
Adamic 99 Albert, Jeong, Barabasi, 99
4
Erdös-Réyni graphs
Connect every pair of nodes with probability p
Short path lengths L log( n )
5
Path Lengths Clusteringcompared with
Erdös-Réyni graphs
Watts Strogatz (1998)
gt
L
LErdös-Réyni
gt
gt
CErdös-Réyni
C
Film actors 3.65 2.99 .79 .00027
(n220,000) Power Grid 18.7 12.4 .08 .005 (n4,
900) Neural Network 2.65 2.25 .28 .05
C.elegans (n282) WWW 4.06 4.05 .16 .0012 (n3,
000)
6
Degree Distribution
7
Semantic Networks

• Associative Networks
• WordNet
• Rogets Thesaurus

8
Word AssociationNelson et al. (1999)
Nwords 5,000
9
WordNetGeorge Miller and colleagues
Nwords 120,000 Nsenses 99,000
10
Rogets Thesaurus (1911)
Nwords 29,000 Ncategories 1000
11
Path-lengths Clustering
gt
L
gt
gt
LErdös-Réyni
C
CErdös-Réyni
Word Association 3.04 3.03 .175 .0004
WordNet 10.6 10.6 .745 .0000 (0.51) Roget
s 5.60 5.43 .875 .0004 Thesaurus (0.61)
12
Degree Distribution
13
Growing, Scale-Free NetworksBarabasi Albert 99

• Start with m0 nodes
• Growth At each time-step, add a node with m
  links.
• Preferential attachment connect links to
  existing nodes with probability

14
Apply model on Word Association
(m011, m11, T5018)
Degree Distribution
15
Degree vs. time
Because of preferential attachment, early nodes
get most connections Prediction words acquired
early in life have more connections?
16
Degree vs. Age of Acquisition (rated)
17
Degree vs. Age of Acquisition (objective)
18
Conclusions

• Semantic networks have following global
  characteristics
• Short path lengths
• Clustering
• Power law distribution of the degree of nodes
• Growth model with preferential attachment
  describes global characteristics for several
  real-life networks.
• Open questions
• Can we model memory disorders (e.g. semantic
  dementia) by deleting nodes and/or connections?
• can small worlds also occur in featural
  representations?

19
Frequent words on shortest paths
Rogets Thesaurus
WordNet
Word Association
20
Watts Strogatz (1998)
RANDOMGRAPH
REGULARLATTICE
SMALL WORLD NETWORK
Rewiring Probability
L
C
L Characteristic average minimum path length
Path Length
C Clustering fraction of neighbors that are
connected to each other Coefficient