Grocery Shopping Assistant - PowerPoint PPT Presentation

About This Presentation
Title:

Grocery Shopping Assistant

Description:

Increase independence of people with low vision (specially blind) to perform ... MoZi is a combination of the Mobile Vision System (MoVs) and ZigZag ... – PowerPoint PPT presentation

Number of Views:379
Avg rating:3.0/5.0
Slides: 21
Provided by: Caro345
Category:

less

Transcript and Presenter's Notes

Title: Grocery Shopping Assistant


1
Grocery Shopping Assistant
  • Carolina Galleguillos
  • Pixel-café / June 2 2006

2
Description
  • GroZi project (grocery shopping assistant)
  • Increase independence of people with low vision
    (specially blind) to perform grocery shopping in
    a supermarket or store.
  • Help to plan shopping list, walking path to the
    store and grocery shopping.

3
Motivation
  • 1.3 million legally blind people in the U.S
  • Grocery store are underselling to this market.
  • Blind people are high cost customers.
  • Advance research on object recognition for mobile
    robotics with constrained computing resources.

4
Motivation
  • Characteristics Grocery Store
  • Structured Environment ().
  • Controlled Lightening ().
  • Maintained by staff ().
  • Well indexed ().
  • People moving around aisles (-).
  • Huge amount of products (30K) (-).

5
Motivation
  • Possible existing solutions
  • Seeing-eye dog trained.
  • RFID tags (aisle, shelf, product).
  • Barcode scanning (shelf).
  • Help of sighted guide/customer service.
  • Memorize store layout.
  • Home delivery.

6
Why computer vision?
  1. Limited ability of dogs.
  2. RFID tags bring privacy concerns and heavy
    infrastructure.
  3. Eye safety and mislabeling.
  4. Independence.
  5. Store layout changes constantly.
  6. Autonomy.

7
Our Solution
  • Develop a handheld device that performs visual
    object recognition with haptic feedback.
  • Avail of complementary resources (RFID, Barcode
    scan, sighted guide)
  • We are focusing on the computer vision aspects
    of this problem.

8
MoZi Box
  • General purpose low-cost mobile system geared
    for computer vision applications. MoZi is a
    combination of the Mobile Vision System (MoVs)
    and ZigZag
  • Finite memory Compact Flash (CF) cards ranging
    from 256 MB to 4 GB.
  • Processor speed in the neighborhood of 60-400MHz
  • Frame rate enough snapshots to cover the shelf
    with some overlap (as in panoramic stitching)
    (15fps instead of 30fps?).
  • Color Calibration Macbeth color chart to
    calibrate the color space.

9
Use of the System
  • Creating a Shopping List.
  • Getting to the Grocery Store.
  • Navigating the Store.

10
Shopping List
  • Online Website
  • Website stores data and images of different
    products.
  • Feedback from users.
  • Provides walking path.
  • Prepare shopping list.
  • Download information into Mozi Box.

11
On the way
  • Separate project.
  • Mozi Box with GPS.
  • Visual waypoints.
  • Traffic/Street sign reading.
  • Use in addition to cane and asking sighted
    bystanders.

12
Inside the Store
  • Finding aisle (OCR, RFID, ask).
  • Avoiding obstacles (cane).
  • Finding products (sweep of aisle, spot product,
    barcode check).
  • Checking out (coupon and cash).

13
Obtaining training data
  • Online Images (Web).
  • Collecting from MoZi box (in situ).
  • Collecting from embedded camera near the barcode
    scanner (in situ).
  • Known databases (COIL-100,ETH-80, etc.)(more
    research oriented)
  • Synthetic examples.
  • Active learning.

14
Obtaining training data
  • Active learning problem
  • Find UPC for the corresponding image (labeling).
  • Semi-supervised.
  • Weakly labeled.

15
Obtaining training data
  • Sunshine Store _at_ UCSD.
  • Venue for pilot study.
  • 4K items in stock.
  • 1749 sq. ft. (assignable)
  • We want to scale to a bigger number of products
    (30K).
  • No bakery or vegetables.

16
Object Recognition
  • 2 types of recognition (mn, mltltn)
  • Detection (of objects).
  • Verification (objects detected are in that list).
  • Algorithms
  • SIFT, AdaBoost cascade, Multiclass Adaboost,
    Probabilistic Boosting tree, Color histogram
    matching, etc.

17
Text Detection
  • Standard OCR is unlikely to be sufficient.
  • Low resolution and distortion are main problems.
  • Reading aisle signs, text on shelves.

18
Considerations
  • Occlusion and clutter of products (caused by
    people and shopping carts).
  • Multiple images of same shelf to perform
    hole-fill-in.
  • Cannot fit dominant plane to the front of product
    shelves.
  • Large number of items.

19
Acknowledgements
  • People (UCSD/Calit2)
  • Serge Belongie
  • John Miller
  • Stephan Steinbach
  • Michele Merler
  • Tom Duerig
  • Captions
  • Dennis Metz/ D. Stein
  • X. Chen and A. Yuille

20
  • Questions?
  • Comments?
Write a Comment
User Comments (0)
About PowerShow.com