DLR Activities in IIM and Scene Understanding

1
DLR Activities in IIM and Scene Understanding
Mihai Datcu DLR Oberpfaffenhofen
ESA-EUSC 2004 Theory and Applications of
Knowledge driven Image Information Mining with
focus on Earth Observation - EUSC, Madrid (Spain)
- March 17-18, 2004
2
Why do we need image content exploration ?

• Because we have many complicated images
• 18 TB SRTM X-SAR data, 106 ERS, SPOT, or other
  images, etc.
• HR images (SPOT5, Ikonos, etc.) 20 000 x 20 000
  pixels/image
• Images are multisensor (Multispectral, SAR,
  InSAR, )
• Image time series
• TerraSAR 90 GB/day of SAR data of 1.2 m
  resolution
• Users or distributors need to select and
  understand data

3
What is IIM ?

• IIM is a novel and unique theoretical frame and
  concept for
• Extraction and exploration of the content of
  large volumes of image or other multidimensional
  signals
• Establishing the link between the user needs and
  knowledge and the information content of images
• Communicating at high semantic abstraction
  between heterogeneous sources of information and
  users with a very broad range of interest

4
Overview of DLR activities in IIM
2006 Operation TerraSAR GS
2004 2007 Research
2004 2005 Emerging concepts
2004 - 2006 TerraSAR GS
2002 2004 KES Prototype
2003 2004 KIM Validation
2000 2004 Research
2002 2004 Applications
2000 2002 KIM Prototype
1996 2000 Theory Demo
1992 1996 Theory Concept
5
The partners
NASA
2006 Operation TerraSAR GS
2004 2007 Research
2004 2005 Emerging concepts
2004 - 2006 TerraSAR GS
2002 2004 KES Prototype
2003 2004 KIM Validation
CNES
2000 2004 Research
2002 2004 Applications
ESA
2000 2002 KIM Prototype
ACS
1996 2000 Theory Demo
1992 1996 Theory Concept
ETHZ
6
THE SYSTEMS WE ELABORATED
KIM - Knowledge driven Information Mining in
remote sensing image archives 2001-2002 DLR,
ETH Zurich, ACS Rome, NERSC Bergen KES - EO
domain specific Knowledge Enabled Services
2002-2004 ACS Rome, DLR KIMV - KIM Validation
for EO archived data exploitation support
2003-2004 ACS Rome, DLR Operation with
TerraSAR Payload Ground Segment
7
What are KIM and KES ?

• KIM/KES are the first prototypes of a new
  generation of advanced tools and systems for
• Accessing intelligently and efficiently the
  information content in large EO data repositories
• Better exploration and understanding of Earth
  structures and processes
• Increase the accessibility and utility of EO data

8
How and where to use KIM/KES ?

• Tools to explore very large historical archives
• Tools to explore distributed archives
• Tools to discover and understand high complexity
  data
• Tools to understand high complexity scenes
• Integration in satellite ground segment systems
• Integration as semantic WEB technology
• . . . other fields medical, biometrics, digital
  pictures, . . .

9
Further RD

• Feature extraction
• Meter resolution
• SAR data (E-SAR, Intermap, ERS, TerraSAR)
• optical images (SPOT, Ikonos)
• hyperspectral data (Dedalus)
• change detection and time series
  analysistargets, objects, scene,
  classifications, 3D, time series
• Medium resolution
• optical images (Landsat)
• hyperspectral data (MERIS)objects,
  classifications, time series
• Analysis methods Signal, noise and artefacts
  modelling, Bayesian inference, information
  theory, estimation-detection, time-frequency-scale
  , wavelets, fractals, geometry, topology, robust
  algorithms for large data sets.

10
Further RD
Image compression and visualization - SAR high
compression high visual quality compression -
Integrated image time series analysis and
compression - meter resolution SAR data -
optical images (SPOT) - Colour transforms for
better QL visualization - Integration of image
analysis and synthesis - Realistic visualization
of large DEM/SEM in synergy with optical images
Analysis methods Information theory,
estimation-detection, wavelets, segmentation,
fusion, algorithms for large data sets.
11
Further RD
Clustering massive data volumes - dimensionality
reduction - similarity measures - analysis of
the feature space models - fast and incremental
clustering - grid based - adaptive grid -
rate/distortion analysis - evaluation of the
information content Analysis methods
Information Theory, statistics, the methods and
algorithms are designed for TeraBytes of data
12
Further RD
Semantic learning and image annotation - fast
algorithms for semantic learning of massive data
sets - automatic analysis of confusions for the
image semantics - study and evaluation of
semantic learning with very small training data
sets - methods and bounds evaluation of
unsupervised image semantics generalization Anal
ysis methods Information Theory, stochastic
models, Bayesian, algorithms complexity, the
methods aim at the management of up to106
images, and at the adaptation to the user
conjecture and HCC
13
Further RD

• Learning domain ontology, knowledge represenation
• - stochastic models on graphs and trees for
  knowledge representation
• - interactive learning of categories and domain
  ontologies
• - mining domain ontologies
• communicable knowledge
• reuse knowledge and information
• Analysis methods Information Theory, stochastic
  models, DBMS, the methods aim at the adaptation
  to the user conjecture and HCC

14
Further RD

• Methods and system for IIM systems test,
  evaluation and validation
• objective evaluation
• image quality, estimation methods,
  classification accuracy, data ingestion control.
• accuracy of semantic learning, and probabilistic
  search functions
• subjective evaluation
• performance of relevance feedback
• quality of adaptation to user conjecture
• quality of GUI
• quality of communicability
• Analysis methods Information Theory, stochastic
  models, DBMS, the methods aim at evaluation of
  high complexity systems which manage very large
  data sets, and are based on HCC.

15
Conclusions and further work
We elaborated a novel concept for information
extraction, access and understanding of massive
EO data volumes The concept is implemented in
pre-operational systems KIM KES The systems
are in use to solve actual problems (ESA,EC
projects) We perform tests and validation on
growing volumes of EO data We further plan the
valorisation of the developed technology Integrat
ion with the TerraSAR payload Ground Segment We
plan new RD projects for advancing the High
Resolution We elaborated EO data understanding
16
E.g. HR SAR information extraction
Detection of bridge, cars, metalic structures
Scene understanding