Project

1
Project 2Leveraging the snBenchFace
Detection, Storage, and Display

• Tim Walsh
• Samir Keco

2
Overview

• Problem
• User wishes to track people entering an enclosed
  area such as an office, club, airport, casino,
  etc.

• Solution
• Cameras watch doorway, take snapshot when
  motion is detected
• Face detection is run on snapshot, cropped
  images of faces are stored in database
• Face images in database are displayed on
  automatically updated webpage

3
Data Model
4

• lt?xml version"1.0"?gt
• ltstepgraph id"detect"gt
• ltlevel_trigger id"trigger0"gt
• ltexp id"motionDetection" opcode"sxe.core.image.
  DetectMotion"gt
• ltexp id"frameGrab1" opcode"sxe.core.team.TimSa
  mir.grabframe"gt
• ltsensor id"sens1"gt
• ltdevice type"Image" id"2"/gt
• lt/sensorgt
• lt/expgt
• ltexp id"frameGrab2" opcode"sxe.core.team.TimSa
  mir.grabframe"gt
• ltsensor id"sens2"gt
• ltdevice type"Image" id"2"/gt
• lt/sensorgt
• lt/expgt
• ltvalue id"threshold"gt
• ltsnobject type"snbench/integer"gt1lt/snobjectgt
• lt/valuegt
• lt/expgt

5
Module Dependency Diagram
6
Detecting Faces

• Fine to coarse-grain filtering for skin-color
• Fast but suffers from false positives, not quite
  normalized output
• Makes many assumptions for simplicity
• Pattern analysis or neural net would bring vast
  improvement

7
Storing faces

• Images arrive wrapped in snPair
• Creates a new face.png file using ImageIO.write()
• setBinaryStream() takes the image, converts it
  into a binary bits and then passes it into MySQL

8
Displaying faces

• publishFaces opcode connects to DB, iterates
  through contents, writes changes to
  FaceDisplay.html
• FaceDisplay.html is hosted on Apache 2 web
  server, updates every 10 seconds, uses
  DisplayFaces.php
• DisplayFaces.php (also on Apache 2) connects to
  DB and retrieves specified image

9
Proof of concept limitations

• Face detection missed faces, false positives,
  output not normalized well enough for face
  recognition
• Processing power noticeable lag time between
  detecting motion, taking snapshots, and
  processing them for storage and display
• Cameras only two cameras capturing low
  resolution images
• All easily resolved with more time and resources

10
Schedule

• 25 Jan 12 Feb define problem statement and
  proof of concept, familiarization with snBench
• 12 Feb 10 Mar define data model, requirements,
  and tasks list
• 10 Mar 19 Mar develop face detection module
  outside of snBench, mid term presentation
• 19 Mar 28 Mar establish Windows snBench test
  bed with 1 camera
• 28 Mar 11 Apr test and modify motion detection
  and frame grab opcodes for use in our
  application, convert face detection module into
  snBench opcode
• 11 Apr 30 Apr establish working database for
  snBench, develop opcodes for image storage, test
  face detection opcode in snBench
• 30 Apr 06 May develop opcode/system for
  displaying faces in web browser, move to Linux
  snBench test bed with 2 cameras, write and test
  final STEP program
• 06 May 12 May final presentation, final
  documentation, release java package

11
Division of labor

• Tim Walsh
• Developed face detection algorithm and opcode
• Developed opcode/system for displaying faces on
  the web
• Established Linux snBench test bed
• Maintained website
• Samir Keco
• Established Windows snBench test bed
• Managed test bed database and developed opcodes
  for image storage
• Wrote and tested STEP programs
• Organized Java package and internal documentation

12
Self criticism

• Took too long to decide on problem statement and
  proof of concept pushed back start of writing
  code
• Waited until very late to test final STEP program
  with 2 cameras found bug with Windows and JMF
  and had to switch to Linux at last minute

13
Future Work

• Improve face detection, develop face recognition
  send alerts when matches are found or keep a
  more accurate counter
• Better integration of display system with snBench
  move away from Apache and PHP, develop GUI for
  deleting images from database, changing motion
  sensor threshold, etc.

14
Recommendations for snBench

• Replace Java Media Framework (JMF) with another
  application for interface with cameras
• Enable sxe.Server to work with PHP

15
Our project on the web
http//cs-people.bu.edu/timwalsh/cs511