Inhabitant Behavior Prediction

1
Inhabitant Behavior Prediction

• Crescent
• Texas Christian University
• Department of Computer Science
• 5/29/03

2
What It Is

• Inhabitant behavior prediction is the process of
  discovering patterns in an inhabitant's behavior
  and from these patterns, accurately predicting
  the next action the inhabitant will take.

3
What Is It Good For?

• Energy efficiency
• Increased comfort
• Assisted Living (for Elderly, Physically
  Disabled)
• Security

4
Example Pattern (Routine)

• Inhabitant wakes up at 630, reads weather on
  computer, showers, dresses, gets a cup of coffee
  while watching local news, gets into car, and
  leaves for work.

5
How Behavior Prediction Could Help

• At 630, smart home turns on shower so water is
  warm by the time inhabitant is done reading the
  weather
• Starts coffee pot 5 minutes into inhabitant's
  shower so coffee is fresh and ready by the time
  inhabitant is done dressing
• Turns on TV to local news as soon as the
  inhabitant is in the kitchen
• Turns off all devices/lights in the house once
  inhabitant has left for work

6
Components of a Pattern Recognition System

• A sensor
• A preprocessing mechanism
• A feature extraction mechanism (manual or
  automated)
• A classification algorithm
• A set of examples (training set) already
  classified or described

7
Structure of Information

• Data stored much like records in a data base
• Each piece of information is field in a record
• Each record is one event
• e.g. event (datum0, datum1, , datumn)

8
Types of Data - Essential

• Basic three pieces of information that are
  required to obtain accurate patterns
• Action
• What device did the inhabitant interact with?
• Location
• Where was the inhabitant when the event occurred?
• Time
• What time did event take place?
• What happens if one of these three pieces is
  removed?

9
Types of Data - Additional

• Data that is not required for pattern, but could
  provide additional fidelity to model
• Inhabitant's mood
• Is the inhabitant cranky? Happy?
• Inhabitant's Diet
• Are we having a low blood sugar moment?
• Etc.
• Anything else that impacts a person's behavior.
  The possibilities are endless.
• Additional data could also add extra noise to the
  model, reducing its accuracy

10
Diminishing Marginal Returns on Increasing Event
Size
11
Data Quality

• Data should be as free of noise as possible
• Errant data points will obfuscate patterns or
  create false patterns
• Number of useful patterns in collected data
  depends on inhabitant
• Inhabitant might follow many daily routines,
  providing a number of patterns
• Inhabitant might not follow any routines,
  providing few if any patterns to discover

12
Types of Classifiers - Statistical (StatPR)

• Patterns classified based on an underlying
  statistical model of the features
• The statistical model is defined by a family of
  class-conditional probability density functions
  Pr(xci) (Probability of feature vector x given
  class ci)

13
Types of Classifiers - Neural (NeurPR)

• Classification is based on the response of a
  network of processing units (neurons) to an input
  stimuli (pattern)
• Knowledge is stored in the connectivity and
  strength of the synaptic weights
• NeurPR is a trainable, non-algorithmic, black-box
  strategy
• NeurPR is very attractive since it requires
  minimum a priori knowledge with enough layers and
  neurons, an ANN can create any complex decision
  region

14
Types of Classifiers - Syntactic (SyntPR)

• Patterns classified based on measures of
  structural similarity
• Knowledge is represented by means of formal
  grammars or relational descriptions (graphs)
• SyntPR is used not only for classification, but
  also for description
• Typically, SyntPR approaches formulate
  hierarchical descriptions of complex patterns
  built up from simpler sub patterns

15
Training Sets

• Training set
• Subset of the collected data
• Must be used to train or prime the pattern
  recognition algorithm
• algorithm discovers patterns to look for in later
  sets of data
• In general, the more data used in the training
  set, the more accurate the algorithm will be

16
Collecting Data

• Real world data is hard to collect
• Requires setting up sensors, having people to
  observe
• Collecting large amounts of data takes weeks or
  months
• Data can be generated
• Large amounts of data can be generated in minutes
• Specialized data sets can be easily created
• No need to set up sensors, have people to observe

17
TCU's Data Generator
18
Current Research

• Active LeZi - University of Texas at Arlington
• COORDINATE - Microsoft
• Neural Network House - University of Colorado

19
Active LeZi

• Developed by University of Texas at Arlington
• Syntactic Classifier
• Uses event (device, onoff, time)
• example event (coffee pot, on, 915am)
• Learns patterns by examining sequence of event
  sets as an input string
• Uses blended probabilities to predict
  inhabitant's next action

20
Active LeZi Phrase Trie
21
Active LeZi Prediction

• Probability of 'a' being the next character
  after sequence 'aaa' is encountered 73/115

22
Active LeZi Prediction

• Probability of 'c' being the next character
  after sequence 'aaa' is encountered 12/1150

23
Active LeZi Features

• Simple data requirements
• Only needs device, onoff, time as input
• Domain Independent
• Does not need to know design of smart home
• Boasts 86 accuracy
• Incrementally learns patterns

24
Active LeZi Issues

• Does not yet take into account time of events
• Is Sunday morning's routine the same as Monday
  morning's?
• Requires inhabitant to rigidly adhere to routines
• Skipping parts of routine confuses ALZ

25
Microsoft's COORDINATE

• Examines user calendar, devices associated with
  user, and time to determine user availability
• From this data, COORDINATE assigns a value to the
  user's current level of availability and predicts
  future availability

26
COORDINATE Purpose

• Determine availability of a user in a workplace
  environment
• Increase communication and collaboration between
  coworkers by coordinating availabilities
• Prevent interrupting users at inconvenient times,
  yet provide urgent messages to the proper device
  should a user need to be notified

27
COORDINATE Components

• Data-acquisition
• Data collector run on computers user is likely to
  use
• Detects computer activity, calendar information,
  video, acoustical and position information from
  wireless signal and GPS data.
• Data-coalescence
• Combines data from multiple sources and stores in
  XML-encoded format
• Query Interface
• Generates a query regarding the availability of a
  particular user
• Learning and Inference subsystem
• Takes query and generates Bayesian network
  (statistical classifier) to compute prediction of
  availability, meeting attendance and
  interruptability

28
COORDINATE Components
29
COORDINATE Query Interface
30
COORDINATE Features

• Domain Dependent
• Users must define
• Location/function of devices
• Attendance and interruptability of past meetings
  to train COORDINATE for future prediction
• Users may define availability preferences and
  thresholds
• Meeting attendance determination up to 92
  accurate
• Meeting interruptability prediction up to 81
  accurate
• Incrementally learns

31
Adaptive Control of Home Environments (ACHE)

• Also known as Neural Network House
• Developed by University of Colorado
• Neural classifier
• Attempts to balance inhabitant comfort against
  energy efficiency of home
• Incremental learner

32
ACHE Implementation

• Renovated 3 room schoolhouse
• 22 banks of lights (each having 16 intensity
  levels)
• 6 ceiling fans
• 2 electric space heaters
• water heater
• gas furnace

33
ACHE Inputs

• 75 sensors throughout house monitor
• for each room in the home
• intensity setting of the lights
• status of fans
• status of digital thermostat (which is both set
  by ACHE and can be adjusted by the inhabitant)
• ambient illumination
• room temperature
• sound level
• status of one or more motion detectors (on or
  off)
• the status of doors and windows (open or closed)
• the system receives global information such as
• the water heater temperature and outflow
• outdoor temperature and insulation
• energy use of each device
• gas and electricity costs
• time of day and day of week

34
ACHE Approach

• Based on minimizing costs
• Energy cost vs. inhabitant discomfort cost
• Want to make ACHE as transparent as possible
• Inhabitants should interact with devices (lights,
  thermostats, etc.) as normal
• Communication with system should be channeled
  through common house-hold devices

35
Taming of the ACHE

• No training set data used to prime ACHE
• Training process is ongoing
• User enters room, ACHE senses motion, does
  nothing to minimize energy cost
• If inhabitant turns on lights, ACHE incurs
  inhabitant discomfort cost, and will turn on
  lights to minimal light intensity next time
  inhabitant enters
• If inhabitant again adjusts the lights to a
  higher setting, ACHE incurs further cost
• Once inhabitant stops interfering with lights,
  ACHE assumes setting is satisfactory
• Occasionally ACHE will "test" inhabitant and
  lower light level to see if it is overridden by
  inhabitant
• If inhabitant does not raise lights, lower
  setting is now considered optimal

36
ACHE Makes A Decision

• State transformation represents current state of
  house
• Occupancy model represents current occupied zones
• Predictors are based on neural networks
• Setpoint generator determines the state that the
  house should be in
• Setpoint generator asks device regulators to
  perform the required actions to make adjustments
  to the environment

37
ACHE Issues

• Domain dependent
• Program discontinued after 1999
• No published results regarding accuracy
• Not clear how much of planned project was truly
  implemented
• Deals with small subset of overall prediction
  problem

38
Handling Multiple Inhabitants

• Inhabitants carry device that transmits unique
  signal
• Uniquely identifies inhabitants, device can be
  used for multiple purposes
• - Have to carry a device at all times
• Use face recognition software
• No need for a device
• No privacy
• Monitor by weight
• No device, good privacy
• - Need a false floor in every home

39
Major Inhabitant Behavior Prediction Issues

• Complexity of human behavior is non-trivial
• What am I going to do next?
• Systems must be virtually flawless
• Do you want a house that makes 1 mistake out of
  every 100 actions?

40
Interface or Educate?

• Two ends of the spectrum
• Require inhabitant to specify every action smart
  home should take
• Have smart home learn inhabitant's needs and take
  actions based on learned patterns
• Best solution (as with all good solutions) lies
  somewhere in between

41
One or Many?

• Smart home could be run by one large, global
  system
• Global knowledge about all inputs in the home
• Incredibly complex set of inputs for system to
  monitor and maintain
• Smart home could be orchestrated by smaller
  system contacting countless subsystems
• Lessened burden on overall system
• - Requires devices or zones to be responsible for
  themselves

42
References

• L. Bretzner, I. Laptev, T. Lindeberg, S. Lenman,
  and Y. Sundblad, A prototype system for
  computer vision based human computer interaction
  Technical report ISRN KTH/NA/P-01/09-SE, April
  2001.
• K. Gopalratnam and D. J. Cook, Active LeZi An
  Incremental Parsing Algorithm for Device Usage
  Prediction in the Smart Home , to appear in
  Proceedings of the Florida Artificial
  Intelligence Research Symposium , 2003.
• Ricardo Gutierrez-Osuna. "Introduction to Pattern
  Analysis"Texas AM University http//faculty.cs.ta
  mu.edu/rgutier/courses/cpsc689_f02/l1.pdf
• V. Krueger and R. Gross , S. Baker
  Appearance-based 3-D Face Recognition from
  Video Proceedings of the German Symposium on
  Pattern Recognition (DAGM) , September, 2002.
• Mozer, Michael C. An Intelligent Environment
  Must Be Adaptive http//www.cs.colorado.edu/7Emo
  zer/papers/ieee.html
• Mozer, Michael C. The Neural Network House An
  Environment that Adapts to its Inhabitants
  ftp//ftp.cs.colorado.edu/users/mozer/papers/nnhad
  apt.pdf
• Robert J. Orr and Gregory D. Abowd The Smart
  Floor A Mechanism for Natural User
  Identification and Tracking. June 1, 2003
  http//www.cc.gatech.edu/fce/pubs/floor-short.pdf
• Andy Ward, Pete Steggles, Rupert Curwen, Paul
  Webster, Mike Addlesee, Joe Newman, Paul Osborn,
  and Steve Hodges. The Bat Ultrasonic Location
  System February 10, 2003 ATT Laboratories
  Cambridge. June 1, 2003 http//www.uk.research.att
  .com/bat/