The medGIFT project

1
The medGIFT project

• UIN/CUI, 21.9.2004

Henning Müller Service of Medical
Informatics Geneva University Hospitals
2
Overview

• The basis (Viper, GIFT, medGIFT, casimage)
• The changes
• Feature space, interface,
• Image pretreatment
• Background removal
• Combination of visual and textual features
• Specialization
• Lung CT retrieval, lung segmentation
• ImageCLEF

3
casimage

• Radiology teaching file
• gt60.000 images of more than 12.000 cases
  submitted so far
• Integration into PACS environment
• Level/windowing on insertion
• Images in JPEG
• No control of textual input
• Web-based interface
• http//www.casimage.com/

4
medGIFT

• Based on GIFT/Viper
• Slight changes in feature space
• Variations in gray levels and Gabor filters
• More to come
• Link with the casimage teaching file
• Integration into casimage is planned, but
• New php interface
• To display diagnoses
• Correct communication in mrml for parameters
• Project on query-based feature selection (with Qi
  Tian, Singapore)

5
medGIFT user interface
Diagnosis
Link to casimage
6
Background removal

• Many images contain a large background part that
  is basically noise
• Or even worse if colored or textured
• Teaching file contains frame around images

7
Retrieval with/without background
8
Textual/visual combination

• Text of the cases is mixed in French and English
  in bad quality (7, no text at all)
• Pretreatment is necessary
• Removal of XML tags and unimportant fields
• stop word removal
• stemming
• Terms can then be extracted from the remaining
  textual data (in French and English)
• Queries can be executed through free text case
  notes will be ordered by their similarity to the
  query

9
Multi-modal multi-lingual image retrieval

• Text is only available through automatic query
  expansion as initial queries are images
• Visual query
• Text of first result (first three) is taken as
  textual query and as visual query
• Not much is known on the quality of the first
  results
• Case scores are expanded to all images of the
  case and normalized
• Visual and textual results are merged (80
  visual/20 textual, )
• Several steps of query expansion can further
  improve results

10
Lung CT segmentation
11
Specialized retrieval

• Database of annotated cases is needed with marked
  regions and selected slices
• Learning of texture characteristics for
  pathologies (local texture analysis) and healthy
  tissue
• Project on retrieval including
• Slice selection
• Region-based classification of textures
• Proposition of regions that need attention
• Retrieval of cases that are visually similar
• Case-based reasoning, evidence-based medicine

12
imageCLEF

• Casimage database from a radiology teaching file
• 9.000 images from 2.000 cases, web accessible,
  26 queries, TREC methodology
• 18 participants from 10 countries
• Much positive feedback, all will participate
  again
• A medical and a non medical task
• One rather text- and one visual-based
• Bring together the two communities
• Results
• Best systems used textual/visual combination
• medGIFT is best with feedback and in text/visual
  combination, other systems better in visual only

13
Conclusions

• There is an ever-rising amount of multimedia data
  produced in hospitals
• Tools will be needed to manage the data
• Multimedia knowledge management
• Image retrieval needs to integrated into
  applications to get real user feedback
• Evaluation is important to proof performance
• imageCLEF
• Creation of specialized datasets