Modeling------->Experimentation

1
topics

• Modeling-------gtExperimentation
• toolkit
• emotion experiments
• time alignment experiments modeling
• neurological relevance
• body animation

2
The PSL Tools
http//mambo.ucsc.edu/psl/tools

• Adds functionality to RAD
• Facilitates design and implementation of
  perceptual experiments
• Generates stimulus lists, keeps track of
  presentation order
• Facilitates presentation of auditory and visual
  speech stimuli
• Gathers and stores subject response data

3
Experiment Control ObjectPSL Tools

• Generates stimulus list by combining independent
  variables, using factorial or expanded factorial
  designs and randomization
• Block based presentations.
• Handles bookkeeping with a central logfile and
  per-subject data files

4
Stimulus Presentation ObjectPSL Tools

• Presents auditory, visual or bimodal speech
  stimuli
• Synthetic visual speech (Baldi)
• Synthetic auditory speech
• Natural auditory speech, synchronized with Baldi
• Waveform playback
• Additive noise, at variable signal/noise ratio

5
Response Collection ObjectPSL Tools

• Analyze each subjects responses
• Percentage of Judgments
• Confusion Matrix
• Statistical Analysis
• Fitting Models Predictions
• Plotting Results

6
Data Analysis ObjectPSL Tools

• Collect a subjects response using one of
• Button clicks
• Single key presses
• Typing
• Sound recording
• Facilitates measurement of reaction time

7
Broadening the Domain

• Individual Variability
• Individual Differences
• Linguistic Differences
• Ecological Variability
• Domain Differences
• Just audible and visible speech
• Task Differences
• Response Differences

8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
Emotion demo 7.1
14
(No Transcript)
15
(No Transcript)
16
Emotion demo 8.3
17
(No Transcript)
18
Written Text and Auditory Speech

• Letters B or D
• Auditory /bi/ to /di/ Continuum
• Judge What You Heard

19
(No Transcript)
20
Issue of Modularity of Mind

• Different Modules imly
• different processes
• recognizing cows versus perceiving sentences
• FLMP
• similar algorithm across different domains
• different information across different domains

21
Neurological Implementation
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
Long-Term Goal Intelligent Conversational
Character

• Realism of the face and voice
• Natural speech production
• Paralinguistic cues and emotion
• Body and Accompanying Gestures
• Dialog and Backchanneling
• Agents Environment

26
(No Transcript)
27
web site HTTP//MAMBO.UCSC.EDUMassaro, D.
W.Perceiving Talking Faces.MIT Press, 1998.FUN
CD-ROM IN BACKSpeech Toolkit
HTTP//OGI.CSE.EDU/CSLU/TOOLKITHTTP//MAMBO.UCSC.
EDU/PSL/TOOLS