Activity Recognition Using Cell Phone Accelerometers

1
Activity Recognition Using Cell Phone
Accelerometers

• Jennifer Kwapisz, Gary Weiss, Samuel Moore
• Department of Computer Info. Science
• Fordham University

2
We are Interested in WISDM

• WISDM WIreless Sensor Data Mining
• Powerful portable wireless devices are becoming
  common and are filled with sensors
• Smart phones Android phones, iPhone
• Music players iPod Touch
• Sensors on smart phones include
• Microphone, camera, light sensor, proximity
  sensor, temperature sensor, GPS, compass,
  accelerometer

3
Accelerometer-Based Activity Recognition

• The Problem use accelerometer data to determine
  a users activity
• Activities include
• Walking and jogging
• Sitting and standing
• Ascending and descending stairs
• More activities to be added in future work

4
Applications of Activity Recognition

• Health Applications
• Generate activity profile to monitor overall type
  and quantity of activity
• Parents can use it to monitor their children
• Can be used to monitor the elderly
• Make the device context-sensitive
• Cell phone sends all calls to voice mail when
  jogging
• Adjust music based on the activity
• Broadcast (Facebook) your every activity

5
Our WISDM Platform

• Platform based on Android cell phones
• Android is Googles open source mobile computing
  OS
• Easy to program, free, will have a large market
  share
• Unlike most other work on activity recognition
• No specialized equipment
• Single device naturally placed on body (in pocket)

6
Our WISDM Platform

• Current research was conducted off-line
• Data was collected and later analyzed off-line
• In future our platform will operate in real-time
• In June we released real-time sensor data
  collection app to Android marketplace
• Currently collects accelerometer and GPS data

7
Accelerometers

• Included in most smart phones other devices
• All Android phones, iPhones, iPod Touches, etc.
• Tri-axial accelerometers that measure 3
  dimensions
• Initially included for screen rotation and
  advanced game play

8
Examples of Raw Data

• Next few slides show data for one user over a few
  seconds for various activities
• Cell phone is in users pocket
• Earths gravity is registered as acceleration
• Acceleration values relative to axes of the
  device, not Earth
• In theory we can correct this given that we can
  determine orientation of the device

9
Standing
10
Sitting
11
Walking
12
Jogging
13
Descending Stairs
14
Ascending Stairs
15
Data Collection Procedure

• Users move through a specific course
• Perform various activities for specific times
• Data collected using Android phones
• Activities labeled using our Android app
• Data collection procedure approved by Fordham
  Institutional Review Board (IRB)
• Collected data from 29 users

16
Data Preprocessing

• Need to convert time series data into examples
• Use a 10 second example duration (i.e., window)
• 3 acceleration values every 50 ms (600 total
  values)
• Generate 43 total features
• Ave. acceleration each axis (3)
• Standard deviation each axis (3)
• Binned/histogram distribution for each axis (30)
• Time between peaks (3)
• Ave. resultant acceleration (1)

17
Final Data Set
18
Data Mining Step

• Utilized three WEKA learning methods
• Decision Tree (J48)
• Logistic Regression
• Neural Network
• Results reported using 10-fold cross validation

19
Summary Results
20
J48 Confusion Matrix
Predicted Class Predicted Class Predicted Class Predicted Class Predicted Class Predicted Class
Walk Jog Up Down Sit Stand
A c t u a l C l a s s Walk 1513 14 72 82 2 0
A c t u a l C l a s s Jog 16 1275 16 12 1 1
A c t u a l C l a s s Up 88 23 323 107 2 2
A c t u a l C l a s s Down 99 13 92 258 1 2
A c t u a l C l a s s Sit 4 0 2 3 270 3
A c t u a l C l a s s Stand 4 1 2 7 1 208
21
Conclusions

• Able to identify activities with good accuracy
• Hard to differentiate between ascending and
  descending stairs. To limited degree also looks
  like walking.
• Can accomplish this with a cell phone placed
  naturally in pocket
• Accomplished with simple features and standard
  data mining methods

22
Related Work

• At least a dozen papers on activity recognition
  using multiple sensors, mainly accelerometers
• Typically studies only 10-20 users
• Activity recognition also done via computer
  vision
• Actigraphy uses devices to study movement
• Used by psychologists to study sleep disorders,
  ADD
• A few recent efforts use cell phones
• Yang (2009) used Nokia N95 and 4 users
• Brezmes (2009) used Nokia N95 with real-time
  recognition
• One model per user (requires labeled data from
  each user)

23
Future Work

• Add more activities and users
• Add more sophisticated features
• Try time-series based learning methods
• Generate results in real time
• Deploy higher level applications activity
  profiler

24
Other WISDM Research

• Cell Phone-Based Biometric identification1
• Same accelerometer data and same generated
  features but added 7 users (36 in total)
• If we group all of the test examples from one
  cell phone and apply majority voting, achieve
  100 accuracy
• Can be used for security or automatic
  personalization
• Interested in GPS spatio-temporal data mining

1 Kwapisz, Weiss, and Moore, Cell-Phone Based
Biometric Identification, Proceedings of the IEEE
4th International Conference on Biometrics
Theory, Applications, and Systems (BTAS-10),
September 2010.
25
Thank You

• Questions?