Keyboard Acoustics Emanations Revisited

1
Keyboard Acoustics Emanations Revisited
Li Zhuang, Feng Zhou, J. D. Tygar,
zl,zf,tygar_at_cs.berkeley.edu, University of
California, Berkeley
http//redtea.cs.berkeley.edu/zl/keyboard
Sample Collector
Motivation
Subsequent recognition
wave signal

• Emanations of electronic devices leak information
• How much information is leaked by emanations?
• Apply statistical learning methods to security
• What is learned from sound of typing on a
  keyboard?

Feature Extraction
Before spelling and grammar correction
Keystroke Classifier(use trained classifiers for
each key to recognize sound samples)
After spelling and grammar correction
Language Model Correction
Alicepassword
Feedback-based Training
Recovered keystrokes
Acoustic Information Previous and Ours

• Feedback for more rounds of training
• Output keystroke classifier
• Language independent
• Can be used to recognize random sequence of keys
• E.g. passwords
• Representation of keystroke classifier
• Neural networks, linear classification, Gaussian
  mixtures

• Frequency information in sound of each typed key
• Why do keystrokes make different sounds?
• Different locations on the supporting plate
• Each key is slightly different

Asonov and Agrawal (SSP04) Ours
Requirement Text-labeling Direct recovery
Analogy in Crypto Known-plaintext attack Known-ciphertext attack
Some Experiment Results
Unsupervised Learning
4 date sets (1227mins of recordings)

• Group keystrokes into N clusters
• Assign keystroke a label, 1, , N
• Find best mapping from cluster labels to
  characters
• Some character combinations are more common
• th vs. tj
• Hidden Markov Models (HMMs)

Set 1 () Set 1 () Set 2 () Set 2 () Set 3 () Set 3 () Set 4 () Set 4 ()
Word Char Word Char Word Char Word Char
Initial 35 76 39 80 32 73 23 68
Final 90 96 89 96 83 95 80 92
Feature Extraction FFT Cepstrum
Initial training Supervised learning with Neural Networks Clustering (K-means, Gaussian), EM algorithm
Language Model / HMMs at different levels
Feedback-based Training / Self-improving feedback
3 different models of keyboards (12mins recording)
Keyboard 1 () Keyboard 1 () Keyboard 2 () Keyboard 2 () Keyboard 3 () Keyboard 3 ()
Word Char Word Char Word Char
Initial 31 72 20 62 23 64
Final 82 93 82 94 75 90
Key Observation

• Build acoustic model for keyboard typist
• Non-random typed text (English)
• Limited number of words
• Limited letter sequences (spelling)
• Limited word sequences (grammar)
• Build language model
• Statistical learning theory
• Natural language processing

3 different supervised learning methods in
feedback
Language Model Correction
4/26/2006