Perception

1
Perception

• ???? ???? ???? ??????
• ??? 4 ????

2
????? ????? ??????
Perception ?????
Cognition ?????
Sensors ????
Actuators ?????
External World ???? ?????
3
????? ????? ??????

• Perception
• ?? ????? ???? ???? ?? ????? ?????
• Cognition
• ???????? ?????? ???? ??? ????? ???????
• ?????? ??? ??? ?????? ??? ????
• ???? ?? ????? ??????? ??? ?????
• Action
• ????? ??????? ????? ?? ?????

4
?????

• ??? ?? ??????? ????? ?? ????? ?????? ?????? ????
  ?? ????? ??????.
• ?????? ?? ???? ???? ???? ???????? ????? ? ???????
  ??????? ???? ?? ???? ??? ?????? ???? ??? ??????.
• ??? ????? ?? ??????? ?? ???????? ?????? ????
  ??????? ???? ??????. ?? ??? ??? ????????? ????
  ???? ???? ?????? ???? ?? ???? ?????? ??????? ??
  ???? ???? ??????.

5
????? ?? ?? ????? AIBO

• ???? ???? ???? ?? ????? ?????

ERS-110 ERS-210 ERS-220 ERS-311 May 1999
Oct. 2000 Nov.2001 Sept.2001
6
?????? ??? ??????

• 64-bit MIPS RISC CPU
• 32MB SDRAM
• Memory Stick Media for AIBO
• Built in Clock
• PC Card Slot Type 2 (ERS-200 Series)
• Sensors
• CMOS Camera, Stereo Microphone,
• Temperature, Acceleration, Pressure,
• Vibration, IR Distance
• 20 DOF on Legs, Tail, Head, Mouth Ears

7
????? ?????
8
(No Transcript)
9
(No Transcript)
10
?????? ??? ?????? AIBO

• AIBO-ware
• PC Applications for AIBO
• Development
• R-CODE
• OPEN-R SDK
• Aperios / OPEN-R

11
??????? ?? AIBO????? ?????

• Instincts
• Love, curiosity, movement, hunger sleep
• Expressing Emotions
• Happiness, sadness, anger, surprise, fear
• and dislike
• Tonal language, motions, LED
• Learning
• Voice(Tonal language) recognition
• Mimic, Humming, Sing a song
• Object tracking Face recognition
• Take pictures

12
??????? ?? AIBO????? ?????

• ?????? ???? ?? ?????? ?? ?????? ??????? ?
  ???????? ???? ???.
• ?? ???? ????? ?? ?? ???? ???? ?? ??????? ?????.
• ?? ???? ?????? ?????? ?? ?? ?? ??? ???
• ?????? ???? ???? ?? ?????? ???
• ??????? ??? ??? ?? ??? ?????.
• 75 ????? ???? ?? ????? ?????
• ??????? ??????? ????

13
??????? ?? AIBO????? ?????

• ????? ??? ???? ?? ??? ????? ??? ???
• ?????? ??? ???? ? ??????
• ??????? ????? ?? ????? ???
• ??????? ??? ????? ????? ?? ????? ???? ? ???? ??
  ???? ???
• ???? ?? ????? ??? ???.
• ??? ???????? ?????? ????PC ???? ?? ?????? ???
  ????? ?? ?? ????? ??????.

14
?????? ????? ???? AIBO

• Programming AIBOR-CODE
• Features
• Similar to BASIC
• Control statements
• Subroutines
• Variables (16-bit integers)
• System variables support sensor data
• Addition/Subtraction
• Stacks

15
?????? ????? ???? AIBO

• Programming AIBOOPEN-R
• Features
• OO designed OPEN-R API of system
• layer
• Full control of joints displays
• Full access to sensors
• Image data, wave formatted sound data
• Network supports
• High level functions are not supported.
• Gait, Voice Object Recognition, MIDI

16
?? ??????? ?????? ???? AIBO

• AIBO, come here!
• Owner AIBO, come here!
• AIBO Tries to find where its owner is.
• Where are you?
• Owner At the kitchen.
• AIBO Finds the way to the kitchen.

Required Functions Map Construction Way finding
(Search) BFS, DFS, IDS Recognizing the door
17
????? ?? ?????
18
????? ?? ?????
19
???? ???? ???????
20
???? ???? ???????

• ?????
• ???????? ?? ?? ???? ????? ?????? ???? ??????.
  ??? ???? ?????? ????? ???? ??? ????? ?????
  ?????
• ?????
• ???????? ?? ?? ???? ????? ??? ???? ??????
  ?????? ???? ?????? ??? ???? ????? ???
• Passive
• ????? ????? ?? ?? ???? ?????? ????? ??? ?????
  ??????? CCD ? ? ???????
• Active
• ????? ?? ?? ???? ??????? ???? ????? ???? ??
  ????? ???? ?????? ???? ?????.

21
(No Transcript)
22
(No Transcript)
23
?????? ???? ?????? ???????

• Dynamic Range
• ????? ??? ?? ????? ?? ???? ? ????? ?????? ?????
  ?? ????? ?? ?????? ????? ???? ?? ?????? ???? ???
  ????.
• DR Maximum Input/Minimum Input
• ?????? ?? ??? ????? ???? ?????.

24
?????? ???? ?????? ???????

• Range ?? ????? ?????? ????? ?? ?????
• Resolution
• ????? ?????? ??? ?? ?????? ?? ???? ????? ????
  ?????? ???
• Linearity
• ???? ????? ????? ?? ????? ???? ?? ??????? ??????
  ????? ?? ???? ?????
• ?? ????? ??? ?? ????? ??? ???? ?????.
• F(axby) aF(x) bF(y)

25
?????? ???? ?????? ???????

• Bandwidth or Frequency
• ???? ?????? ???? ???? ?? ????? ?? ????? ????
  ?????? ?? ?? ????? ?? ???? ?????. ?????? ????
  ????? ?????? ???? ??? ????? ?? ????? ?? ?? ??????
  ????? ??????.
• Sensitivity
• ????? ??? ?? ?????? ?? ????? ??????? ???? ??
  ?????? ????? ?? ??????? ???? ????? ?? ????? ???
• Output change / Input change

26
?????? ???? ?????? ???????

• Cross Sensitivity
• ????? ??? ?? ?????? ?? ?????????? ????? ?? ??????
  ?? ?????? ???? ?????.
• ?? ??? ??? ??????? ?? ??? ???? ????? ?? ????
  ????? ???????? ???? ????? ???? ????. ??? ???
  ????? ????? ?? ????? ???????? ???? ?? ?????
  ???????? ???? ??? ?????? ????. ?? ????? ???????
  ?? ?? ???? ????? ?? ?? ???? ???? ???? ????? ?????
  ???? ?????? ???.

27
?????? ???? ?????? ???????
28
?????? ???? ?????? ???????
29
?????? ???? ?????? ???????
30
?????? ??? ?? ???? ?????

• ????? ????? ???? ????? ???? ?????? ?? ??? ?????
  ?????? ????? ? ???? ???? ??????? ?? ????? ?????.
  ??? ?????? ???? ?? ???? ????? ?? ?????? ?? ????
  ????? ??? ???? ??? ???? ? ????? ??? ????? ???.
• ????? ?? ??????? ? ?????? ?????
• ??????? ??? ???
• ??? ??? ?? ??? ???? ????
• ?????? ???? ????
• ?? ????? ????? ?????? ?? ???? ????? ?????? ????
  ?? ?????? ? ??????? ????? ? ???? ????? ???.
• ??? ???? ?????? ?????? ??? ???? ???
• ????? ???? ????????? ? ?????? ???? ???? ??? ?????
  ??? ????? ???? ??? ??? ???? ????? ????? ????? ???
  ???.

31
??? ???? ???
32
???? ???? ????? ?? ????

• One of the most basic abilities for a mobile
  robot is that of localization, i.e. to be able to
  determine its own position in the environment.
• ?????? ?????? ???? ?????? ???? ????
• Global Positioning System
• Dead-reckoning localization
• Triangulation Ranging
• ....

33
Dead-reckoning localization

• ??? ??? ????? ?? ??????? ?? ???????? ????? ??? ?
  ?????? ?? ??????? ?? ??????? ? ???? ???????
  ??????? ?????? ? ??? ???? ?? ???? ???? ?????
  ?????? ???? ?????.
• ?????? ?????? ?? ???? ?????? ??????. ??? ??????
  ???? ??? ?? ??? ??? ???? ????.
• ???? ???? ????? ???.
• ?????? ?? ????? ?? ???? ????? ??????? ?????.

34
Dead-reckoning localization
35
(No Transcript)
36
Dead-Reckoning
Accumulated error can be quite big for a period
of time.
(0,0)
Dead-Reckoning
37
Dead-Reckoning
38
????? ??? ? ?????
39
????? ??? ? ?????
40
????? ???

• ????? ??? ??????? ????? ???? ??? ????????? ??
  ????? ??? ??? ???
• ???? ?????? ???? ????? ???? ????? ??????? ?????.
• ?? ???? ????? ??????? ???? ?????? ?? ??????? ????
  ?????? ?? ????? ?? ???? ????.
• ??? ?????? ?? dead reckoning ???????.

Dead reckoning is the process of estimating one's
current position based upon a previously
determined position, or fix, and advancing that
position based upon known speed, elapsed time,
and course.
41
??? ???
42
???????
43
Ground-Based Active and Passive Beacons
44
Introduction to GPS

• What is GPS
• The Global Positioning System (GPS) is a
  worldwide radio-navigation system formed from a
  constellation of 24 satellites and their ground
  stations
• GPS receivers use these satellites as reference
  points to calculate positions and time
• Originally known as Navigation System with Timing
  And Ranging (NAVSTAR)

45
Global Positioning System (GPS)
46
Global Positioning System (GPS)
47
Global Positioning System (GPS)
48
GPS Error Budget

• Source Uncorrected Error Level Ionosphere 0-30
  meters
• Troposphere 0-30 meters
• Measurement Noise 0-10 meters
• Ephemeris Data 1-5 meters
• Clock Drift 0-1.5 meters
• Multipath 0-1 meter
• Selective Availability 0-70 meters

49
????? ??? ?????
50
???????? ?????
51
????? ???????????
52
????? ???????????
53
????? ???????????
54
????? ???????????
55
????? ???????????
56
Ultrasonic Sonar Sensor
57
?????? ???? ????? ?? ??????? ?? ????

• ??????? ?? ???? ???? ?????? ???? ????? ????? ???
  ????? ???? ?? ??????? ?? ??? ????.
• ?? ?????? ??? ????? ?? ??? ??? ???? ??? ???????
  ??? ? ?????? ?? ???? ????? ??? ???? ????? ????
  ?????.
• ????? ?? ??? ???? ???? ???? ??? ? ????? ??? ?????
  ??? ?????.
• ?? ??????? ?? ?? ????? ??????? ?? ???? ???? ?
  ????? ??? ?????? ???? ?? ????? 2 ? ?? 3 ????
  ????? ???? ????? ???? ?? ????? ????.

58
?????? ???? ????? ?? ??????? ?? ????

• Pulsed laser
• ?? ??? ??? ?????? ??????????? ?? ???? ???? ?? ???
  ??? ????? ??? ? ???? ??? ? ????? ?????? ?????.
  ?????? ??????? ?? ????? ??? ?? ???? ?? ??? PS
  ?????? ???? ???.
• Phase Shift
• ?? ??? ??? ?????? ??? ??? ??? ?????? ? ??????
  ?????? ???? ?????. ????? ???? ??? ??? ???? ?? ???.

59
?????? ???? ????? ?? ??????? ?? ????
Phase Shift Measurement
60
?????? ???? ????? ?? ??????? ?? ????
61
?????? ???? ????? ?? ??????? ?? ????
62
?????? ???? ????? ?? ??????? ?? ????
63
?????? ???? ????? ?? ??????? ?? ????
?????? ????? ?? 0.5 ???? ?????? ??? ?? ???? 5
???????? ? ??? ???? ?????? ???? ??? 5cm , 20 m
??? 5 ???? 180 ???? ?? ????? ????? ?????.
64
?????? ???? ????? ?? ??????? ?? ????

• ??????
• ???? ?? ????? ???? ???? ??? ???? ????
• ????? ??? ?????? ????? ???? ??? ?????? ??? ????
  ?????? ? ?? ????? ??? ????? ????? ??????.

65
Triangulation Ranging

• ?? ??? ????? ?? ??????? ?? ???? ????? ?? ??????
  ???? ????? ?????????
• ?????? ?? ???? ???? ?? ???? ??????? ??? ? ??????
  ?? ???? ?????? ?? ?????? ????? ? ?? ????? ???????
  ????? ???? ????? ?????? ???? ?????.

66
Triangulation Ranging
Sensors based on this principle are used in range
sensing up to 1 or 2 m, but also in
high-precision industrial measurements with
resolutions far below 1 µm.
67
Triangulation Ranging
68
Triangulation Ranging
69
Triangulation Ranging
70
???????? ???? ? ????

• ????????? ???? ????? ?? ???????? ??????? ????
  ???? ??? ????? ? ???? ?? ?????? ??????.
• ???? ????????? ?? ?? ??????? ?? ???? ??????
  ??????? ?? ???????? Doppler ???? ??? ????? ????
  ??????.
• ?? ??? ??????? ?? ?? ?? ????? ???? ?
  ?????????????? ??????? ?????.

71
???????? ???? ? ????
72
???? ???
73
?????? ???? ????? ?? ??????? ?? ????
74
?????? ???? ???
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
?????? ???? ????? ?? ??????? ?? ????? ??????

• ?????? ??????? ?? ???? ???
• ?????? ?????? ???? ?? ???? ??? ????? ?????.
• ?????? ?? ???? ?? ?????? ?? ?? ??? ?? ???? ?????
  ?????? ?????? ??????? ??? ??? ???? ??? ?? ?????
  ??? ????? ?? ???? ????? ????? ??.
• ??????? ??????? ??? ??? ?? ?? ????? ?? ???? ???
  ???? ?? ????.
• ?? ??????? ???? ?????? ??????? triangulation ? ??
  ?????? ????? ?? ????? ?????? ???? ???? ?????
  ??????? ??? ??? ??????? ????.

79
???????? ??????
80
Vision Sensors

• CCD (light-sensitive, discharging capacitors of 5
  to 25 micron )
• CMOS (Complementary Metal Oxide Semiconductor
  technology )

81
Visual Range Sensors

• ??? ???????? ?? ???? ?????? ???? ???? ????? ????
  ????? ?????? ???? ????? ?? ??? ?????? ???? ?????
  ???? ??? ?????? ???? ????.
• ??? ?? ??? ?????? ?? ????? ????? ????? ??? ??
  ??????? ????? ???? ???? ???? ???.
• ?? ??? ?????
• Depth from focus/defocus ?? ?????? ??????????
  ?????? ?? ??????? ?????.
• Stereo vision ?? ?????? ??? ?????? ?? ???????
  ?????.

82
?????? ???? ????? ?? ???? ?????/?? ?????

• ???? ???? ?? ??? ??? ??????? ????? ?? ???? ??????
  ???? ?????????? ?????? ???.
• ?? ??? ??? ???? ?? ????? ?????? ???? ??? ???????
  ? ?? ??? ????? ?? ???? ????? ?????. ?????? ??????
  ?? ????? ?????? ? ?? ????? ???? ????? ??????.

83
?????? ???? ????? ?? ???? ?????
84
?????? ???? ????? ?? ???? ?????
85
?????? ???? Sharpness
86
Depth from defocus

• ?? ??? ??? ?? ?? ???? ?? ? ?? ??? ????? ??
  ?????????? ????? ?????? ????? ?????.
• ??????? ????? ?? ????? ?? ????? ?? ???? ????
  ??????? ?????.

87
????? ????? ???

• ??? ?? ?? ?? ????? ?????? Pinehole???? ?????
  ????? ????? ????? ??? ?? ???? x,y?? ??????? ??
  F(x,y) ???? ???? ??? ? ????? ??? ?? ??? ???? ??
  ?? ?? ???? ?? ????? ?????.
• ???? point spread function ????? ??? ????? ?????
• ????? ??? ???? ??? ?? ???? P ?? ?? ????? ?????
  ??? ?? ?? ????? ???? ????? ????? ?????.

88
Point Spread Function

• ?? ??? ????? blur circle ????? ??? ??? ????????
  ??? ??? ???? ????? ??? ????? ?????

89

• Best reported result is 1.3 RMS error in terms
  of distance from the camera when the target is
  about 0.9 m away .
• The key problems are the measurement of
  difference of blurring amount and the calibration
  of the mapping between depth and the difference
  of blurring.

90
Stereo Vision
91
Stereo Vision

• ????? ???? ????? ?? disparity ????
• ???? ??????? ?? ?? ??? ?????? ?????? ?????? ????
• ????? disparity ?? b ????? ?????? ????
• ???? ?? ????? ???? disparity ???? ??? ?????? ????
  ??? ?? ?????? b ????? ????? ??.
• ??? ??? ????? ???? ???? ?? ?? ?????? b ???? ????
  ??? ?? ?? ????? ???? ????.
• ???? ?? ?? ?? ?? ?? ????? ???? ???? ????? ?????
  ?? ??? conjugate ?? ?????? ????? ????? ??? ?? ??
  ?????? ??? ??? ???????? ?? ??? ?? ??? epipolar
  ?????? ????? ??????. ?? ???? ??? ??? ??? ?? ?????
  ???? x ???.

92
Stereo Vision the general case

• ?? ???? ??? ???? ??? ??? ????? ???? ???? ??????
  ?? ?????? ???? ?????.

?????? ?????? ????? ?? ???? ??????????? ????
?????.
We have 12 unknowns and require 12 equations we
require 4 conjugate points for a complete
calibration. Note Additionally there is a
optical distortion of the image
93
???? ?????

• ???? ????? ?? ?? ????? ?? ????? ?????? ???? ????
• How do we solve the correspondence problem?
• ??? Zero Crossing of Laplacian of Gaussian
• ??? ?? ????? ??? ?????? ???? ????? ???? ?????
  ??????.

94
Zero Crossing of Laplacian of Gaussian

• ???? ??? ???? ????? ????? ?? ?????? ??? ???? ? ??
  ?? ?????? ???? ? ????? ?? ?? ?????? ???? ??????.
• ??? ??? ?? ??? ????? ? ??? ????? ????? ??? ???.
• ???? ??? ??? ?? ???????? ????? ????? ?????

95
Zero Crossing of Laplacian of Gaussian

• ???? ????? ????? ????? ??? ?? ???? ?? ?????
  ??????? ?????? ???? ???? ?? ?? ???????? ?????
  ??????

kernel
96
????

• Step / Edge Detection in Noisy Image

97
??? ????

• ???? ???? ??? ???? ????? ?? ??????? ?? ???????
  ????? ??? ????? ????? ???? ???? ?????.
• filtering through Gaussian smoothing

98
????
left images
right images

• ????? ???????? ?? ??? ????? ????? ??? ???? ?????
  ???? spike ???? ? ???? ?? ??? ??? ?? ????? ??.

edge filtered
b1 and b2 vertical edge filtered left and right
image filter 1 2 4 -2 -10 -2 4 2 1 c
confidence image bright high confidence (good
texture) d depth image bright close dark
far
confidence image
depth image
99
Zero Crossing of Laplacian of Gaussian

• ?? ????? ????? ???? ???? ???? ???? ????
  correspondence ??? ?????? ???? ? ?? ??????? zero
  crossing ???????? ????? ????? ??? ???? LoG
• Zero crossing ????? ???? ????? ?? ?? ????? ??? ?
  ????? ?? ?????? ???? ?????? ???.

100
Color Tracking Sensors

• Motion estimation of ball and robot for soccer
  playing using color tracking

101
Representing Colors in an RGB Image
102
How do We Segment a Single Color?

• We need to model it mathematically a priori

103
Simple RGB Color Segmentation
104
Grouping, Clustering Assigning Features to
Features