Tutorial on Medical Image Retrieval contentbased image retrieval

1
Tutorial on Medical Image Retrieval-
content-based image retrieval -

• Medical Informatics Europe 2005, 28.8.2005

Henning Müller, Thomas Deselaers Service of
Medical Informatics Geneva University
Hospitals, Switzerland Aachen Technical
University, Germany
2
Overview

• Goals
• Application domains
• System overview
• Some example systems
• The gaps and problems
• The achievements
• The medical domain
• Similarities
• Differences

3
Goals

• Improve the management of multimedia data
• Strongly rising in all fields
• Digital cameras at consumer prizes
• Allows search that is not really possible with
  text
• Feelings, impressions,
• Collaborative filtering
• Abstract forms of trademarks
• Permits to search where there is no text
  available
• Annotation is expensive and also subjective

4
Application domains

• Trademark retrieval
• In addition to text (Vienna code)
• Well for describing objects
• Abstract forms are extremely hard to describe
• Journalist stock photography
• Allows browsing in large archives
• Journalists often do not know what exactly to
  look for
• Sleep over decisions
• Decisions based on rough feeling
• Rough concept
• A few other domains
• CT retrieval, search of web images,

5
System overview
6
System variables

• Visual features
• Distance measures
• Learning methods
• From test datasets
• From user interaction
• Quality of the GUI (2D, 3D, number of clicks, )
• Speed of retrieval, and of indexing
• Interaction models
• Relevance feedback, pos., neg., numbers of images
• Image browsing (PicHunter)

7
QBIC Query by image content

• IBM, commercial product, 1993
• Add on for DB2
• Simple color, texture, layout features
• Very simple feedback

8
Blobworld, 1997
9
PicHunter

• Browsing tool for target image search
• Optimizes information gain from images being
  presented to the user
• Another system is filter image browsing
  (Amsterdam)

10
ASSERT
11
The gaps and problems

• Sensory gap
• The information is lost by representation a real
  world object by the means of a computer, cannot
  really be shortened
• Semantic gap
• Can be shortened when using higher level features
  and learning
• Page zero problem
• We are used to text, but not to visual features
  to start a query
• Evaluation
• No common datasets, no comparison, no prove of
  performance
• Real world use of systems
• Only very few systems, mainly academic world

12
Achievements

• Large number of visual features were developed
• Invariance models for color, size, rotation, etc.
  were developed
• Interaction models and relevance feedback were
  explored
• Probabilistic models for image similarity,
  classification,
• Salient region discovery
• Fast access to indexed features (DBs, Inverted
  files, )
• Feature space reduction methods
• BUT Much was already there in text retrieval

13
The medical domain

• Differences
• Currently images are almost always accessed by
  patient ID, only
• Problems to use images for other tasks than
  directly for healing the patient (laws)
• Images are mainly in gray levels
• Saliency models might not work
• Color invariance models do not work
• BUT images often taken under standardized
  conditions
• Imaging modalities change and can produce very
  different images
• Advances in modalities can be quick

14
Problems with current systems in medicine

• Mostly, needs are defined by MDs
• Large number of publications
• Most systems are developed in computer science
  departments without a direct medical connection
• Only use of datasets for evaluation
• Sometimes the expectations are too high
• Set realistic goals
• Datasets are expensive to get and often protected
  for privacy reasons
• Real world use in clinical practice is inexistent
• Get user feedback

15
The medical domain

• Similarities
• Many of the texture and shape features will work
• General concepts and frameworks are the same
• Databases, access,
• Specializations need to be done but learning
  works as well as for other specialized domains
• Many of the problems that seem specific to the
  medical field apply in other domains as well
• Domain knowledge needs to be integrated

16
Conclusion

• Content-based retrieval can be an important
  concept for multimedia data management
• And the amount of data is rising quickly
• Many good research prototypes exist
• Systems need to be integrated with routine tools
  to get real user feedback
• In the medical fields, computer scientists and
  MDs need to work together on system integration
  and use