Kein Folientitel

1
University of Applied Sciences of
Wiesbaden Department of Computer Science Image
Processing Detlef Richter New Applications
of Digital Image Processing in Technology and
Medicine CERN IT Division CH 1211 Genève
23 August 15th, 2003
2
Agenda -------------------------------------------
---------------------------------------------- 1.
Introduction 2. Hardware Development 3. Basic
Algorithms 4. Spectral Sensitivity 5. Image
Sensors 6. Stereo Vision 7. Examples of
Applications 7.1 Automated Assembly 7.2 Sound
Track Restoration 7.3 Computer Based
Learning 7.4 Image Processing of medical
Images 7.5 medical Navigation 8. Summary
3

• 1. Introduction
• --------------------------------------------------
  --------------------------------------
• 1975 Systematic Scientific Development of Image
  Processing
• Problem Availability of sufficient fast ADC
  / DAC Hardware
• 1980 Introduction of Image Processing in
• Industrial Assembly Lines, Production Control,
  Quality Reliability
• 1985 medical Image Analysis
• For Example Production of Middle Class Car
• 1985 West Europe 65 h / Japan 35 h
• General Motors, Eisenach 8 h
• by automation, part of which is done by robotics
  and image processing

4
2. Hardware Development (1)

• Dedicated Image Processing Unit
• Image Display on Video Monitor
• Image Data Transfer to Computer via DMA
• Image Processing Unit Control via C-Bus Extension
• Bottle Necks Transfer Rates of DMA and Computer
  Bus System

5
2. Hardware Development (2)

• Frame Grabber Boards
• Occasional Local CPU on Board
• Improvement of Fast Processing
• Disadvantage of Programming
• Decreasing use of Video Output
• No Separate Video Monitor Necessary
• Image Output and Programming in Different Windows
  on Same Screen
• Bottle Neck Computer Bus System if no Local CPU
  on Board

6
2. Hardware Development (3)

• No Memory on Board
• Image Data and Programs in Same Memory
• Fast Data Transfer via Computer Bus,
• PCI-Bus 1.3 MByte, Transfer Rate Sufficient
  for Transferring 3 Video Frames at Same Time,
• e. g. RGB-Images

7
3. Basic Algorithms ( 1 )

• Modeling of a Scene
• ( A-Priori-Knowledge )
• ?
• Global Preprocessing
• ( Noise Reduction, Edge Filtering, Image
  Transforms etc. )
• ?
• Segmentation
• ( Objects / Background )
• ROI or VOI
• ?
• Extraction of Attributes / Values
• ?
• Analysis of Attributes / Values
• ( Classification )
• ?
• Interpretation, Action

• Binarisation
• Low Pass Filtering ( Noise Reduction )
• 2D High Pass Filtering ( Edge Detection )
• 3D High Pass Filtering ( Shape Detection )
• Hough Transform
• Fourier Transform
• Autocorrelation Function
• Specific Procedures

8
3. Basic Algorithms ( 2 )

• Video Image
• Resolution 786 x 576 Pixels
• 8 Bit Gray Level Resolution

9
3. Basic Algorithms ( 3 )

• Global Filtering
• e. g. 2D-Low Pass Filter ( Median )
• Scan the Image with nn-Matrix
• Apply Matrix for Each Pixel
• Generate New Image Frame
• ( Noise Reduction )

10
3. Basic Algorithms ( 4 )

• Global Filtering
• e. g. 2D-High Pass Filter
• Scan the Image with nn-Matrix
• Apply Matrix for Each Pixel
• Generate New Image Frame
• ( Edge Detection )

11
3. Basic Algorithms ( 5 )

• Global Transformation
• e. g. Hough Transform
• for Lines and Circles
• Transform Each Pixel of an Edge
• into a Hough Matrix
• Analyze the Hough Matrix
• Gain Mathematical Equations
• of Edges

12
3. Basic Algorithms ( 6 )

• Calculate Vanishing Points of Lines
• Define optimal Cut-Out of Source Image
• Define Rectangular Area in Target Image
• Interpolate all Pixels in Target Image by
  Resampling

13
3. Basic Algorithms ( 7 )

• Global Transformation
• e. g. Fourier- Transform
• ( 1D and 2D )
• FFT
• Transform Each Line and/or Row
• from Geometric Domain
• into Frequency Domain
• Analyze Frequency Domain
• Extract Frequency Attributes
• or Change Frequency Attributes
• and Execute Inverse Transform
• Gray Levels with logarithmic enhanced
  Representation

14
4. Spectral Sensitivity ( 1 )

• Visible Electromagnetic Wavelengths 400 nm (
  violet-blue ) to 750 nm ( red )
• Monochromatic Cameras
• Gray Level Video Camera (Luminance Signal Output
  According Integral Sensitivity )
• IR-sensitive Cameras for medical Applications (
  One Channel, Luminance or False Color Output )
• Video Cameras with IR-Long Pass Filter for
  Technical Applications ( One Channel, Small
  Bandwidth )
• Spectral Sensitive Cameras
• One Chip Cameras with R,G,B-Filter Mask,
  FBAS-Output ( Three Channels coded on one Line )
• Three Chip Cameras, R,G,B-Output ( Three Channels
  )
• Landsat Images, IR-Channels ( up to Seven
  Channels )

15
4. Spectral Sensitivity ( 2 )
16
5. Image Sensors

• Video Cameras ( 2D )
• Reflection of Incident Light on Surface
• Monochrome Cameras ( Gray Level, Modified IR
  Level )
• Color Cameras ( One Line FBAS, Three Lines R,G,B
  )
• Infrared Cameras ( monochrome )
• Line Scan Camera ( 1D )
• Roentgen Sensitive Image
• 2D Transparent Shadow of 3D Object
• Large Area Solid Roentgen-Sensitive Devices
• Computer Tomography ( CT ), since 1972
• 2D Slices of 3D Object, Volume Image, X-Ray
  Scattering by High Z-Nuclei
• Magnetic Resonance Tomography ( MRT ), since 1980
• 2D Slices of 3D Object, Volume Image, Light
  Emitting Volume by Hydrogen Nuclei
• Positron Emission Tomography ( PET ), since 1978

17
5.1 Large Area Solid Roentgen-Sensitive Devices

• Large scale Roentgen sensitive sensor
• Sensor elements 200 µ 200 µ
• Resolution 2 K 2 K pixel
• Grey level resolution 16 Bit
• Data 16 MB per Image
• Roentgen energy 80 KeV ? 400 KeV
• approx. 3 frames per s ( 1K1K 7 frames )
• Quality assurance / Production Control /
  Materials Research and Testing
• Medical application

18
5.2 PET Images

• 18F-Fluorodesoxyglucose
• Decay
• p ? n e ?
• e e- ? 2 511 KeV
• Ekin(e) 0,633 MeV
• mean free path in H2O 2,4 mm
• t½ 109,7 min.
• search for diseases before symptoms appear ( e.g.
  Alzheimer disease, micro metastases etc. )
• Problems Mathematical modeling of the
  physiological procedure

19
5.3 US Images

• US-Picture Abdomen
• High Signal to Noise Ratio
• Frequency of Sound 8 to 13 MHz
• Power 10 to 50 mW
• Velocity of Waves Dependent on Tissue
• Measurement of Time with supposed constant
  Velocity
• Ratio of Emitting to Receiving Time 0.1

20
6. Stereo Vision ( 1 )

• Model of Pinhole Camera with Radial Symmetric
  Distortion of the lenses
• Calibration of Intrinsic Mathematical and
  Physical Parameters
• Width to Height Ratio of Pixels
• Intersection of Optical Axes of the Lenses with
  the Surface of the Sensor Chip
• Image width of the lenses
• Coefficients defining Radial Symmetric Distortion
• Position and Orientation of the Cameras

21
6. Stereo Vision ( 2 )

• Use chessboard like pattern of known size
• Apply Hough Transform
• Find the known number of edges of known
  direction
• Calculate all intersections
• Find subpixel precise all corners
• Calculate position and orientation of the camera

22
7.1 Automated Industrial Assembly ( 1 )

• Top View of an Industrial Precise Mechanical
  Production Line
• Problem of Perspective Distortion Recognition
  of exact Positions in x,y-Plane
• Problem of Positioning the Robot Hand for
  Automatic Production and Control within 3D Space
• Parts Recognition, Identification and Measurement
  using Bottom Illumination by IR-Light

23
7.1 Automated Industrial Assembly ( 2 )

• Calibration of Monocular or Binocular Vision
  System (Position, Orientation, Focus Length of
  Camera ) with Respect to the Coordinates of
  Assembling System
• Calibration of Robot Coordinate System with
  Respect to Vision System
• Development of Customer-Specific Recognition
  Algorithms
• Optimizing of Lighting
• Test of Complete System

24
7.2 Sound Track Restoration ( 1 )
25
7.2 Sound Track Restoration ( 2 )

• Reproduction Speed 24 Frames / s
• Sound Track Scanning with Line Camera 6 Frames /
  s
• Scanning of Track Width 512 Pixels per Line
• Gray Level Resolution 8 Bit / Pixel 256 Gray
  Levels
• Nyquist Frequency 24 KHz
• Analog Cut off Frequency 15 KHz
• Frequency Resolution 2000 Lines per Frame
• Data Transfer Rate ( Camera Disk Memory ) 6
  MByte / s
• Required Storage Capacity 24 MByte / 1 Second
  of Sound

26
7.2 Sound Track Restoration ( 3 )

• Left Intensity Code Sound Track of the Speech
  of Albert Einstein 1930 on behalf of the Opening
  Ceremony of the Broadcasting Fair in Berlin (
  Scratches, Spots, Fibers )
• Right Restored Sound Track within ROI,
  Conserving the Authenticity of First Recording
• Converting the Sound Track in Digital Audio Data

27
7.2 Sound Track Restoration ( 4 )

• Left Twofold Double Sided Variable Area Code
  with Faults on Sound Track
• Right Multifold Double Sided Variable Area Code
  with Faults on Sound Track

28
7.3 Computer Based Learning ( 1 )

• Analysis of Time Dependent Color Change caused by
  Chemical Reactions
• Color Sensitive Chemical Indicator
• Time Dependent Addition of Reagents
• Analysis of Colors in Defined Color Space
• ( R, G, B )

29
7.3 Computer Based Learning ( 2 )

• Example Reagent Phenolphthalein, Measurement
  Time 300 s, Sample Intervalls 500 ms
• Create Colored Animation of Chemical Reaction

30
7.4 Image Processing of medical Images (1)

• Methods of Segmentation
• Body Surface Change of Gray Level Distribution
• Vessels, Tracer of Vessels Tube Model with
  slowly varying Diameter and Ramifications
• Filament Model, Transition between both Models
  possible
• - Seed Algorithm with Use of Contrasting
  Injection
• - 3D rays in forward Direction
• - Texture Analysis on Surface of Spheres

31
7.4 Image Processing of medical Images (2)

• Optic nerve 3D interactively directed rays, no
  defined contrast
• Ventricles 3D-growing region, Problem Running
  out in Filament Structures
• Tumors Slowly and unnoticed growing, backing
  down in liquor holes ( ventricles ), first
  complaints if burdening nerves ( optic nerves,
  auditory nerves )
• Threshold based and edge detection based
  Procedures are without success.
• Manual contouring is time consuming but a
  standard procedure.
• New Getting interactive significant numerical
  attrbutes for texture analysis,
• 2D-texture analysis, 2D reconstruction of Tumor
• Disadvantage No proper Contours detectable,
  manual completion necessary
• Liver Definition of liver segments by blood
  vessels ( maximum 5 out of 7 liver- segments are
  resectable )

32
7.5 medical Navigation ( 1 )

• Besides infrared guided systems exist other
  navigational systems
• Mechanical, Stereotactical ( direct contact )
• Electromagnetical ( electromagnetic disturbance
  )
• Laser Guided ( laser beam )

33
(No Transcript)
34
7.5 medical Navigation ( 2 ), Components

• IR-based stereo vision system with two CCD
  cameras with synchronised H-Sync-Signals
• four camera system under development
• conventional calibration of cameras, i. e.
• spatial position and orientation
• focal length
• radial symmetric distortion of the optical lenses

35
7.5 medical Navigation ( 3 ), Tracker

• Tracker for biopsy needles with IR-LEDs, adapter
  for robot based guidance, force / torque sensor
  for feed back
• ( min. 3 LEDs, max. 6 LEDs )
• Wavelength of 895 45 nm
• Longpass filter with cut-off frequency of 830 nm
• CT-volume image data
• ( DICOM III Standard )
• Registration of patients position
• Visualization of navigation

36
7.5 medical Navigation (4 ), Aim of
Brachytherapy

• Irradiation of recurrent or inoperable tumours,
• preserving normal tissue ( cells ),
• avoiding pre-irradiated tissue,
• using biopsy needles with inner diameter of 2.0
  mm and outer diameter of 2.1 mm.

37
using
as radioactive source
External therapy Brachytherapy
Radiation energy 18 MeV 0.6 MeV
Daily applications 1 2
Duration of one application 3 directions, each 10 - 20 s 5 - 15 min
Dose of one appl. 1,8 Gy 4 - 7 Gy
Duration of application 5 - 6 weeks 1 week
Total Dose in Gy 45 Gy 30 - 40 Gy
38
4. Mathematical Procedure -----------------------
--------------------------------------------------
--------------------------------------------------
-- 1. Step Calibrate the stereovision
system 2. Step Calibrate the tracker, getting
a precise tracker model T, i. e. calculate the
3D-coordinates of the LEDs of the tracker noted
by T Ti Ti?? R3, i 1, ..., n Repeat
this step 50 times for getting a higher precision
in the coordinates Use a-priori-knowledge about
the model for matching corresponding points
39
3. Step Calibrate the equation of the biopsy
needle ( assumed to be a linear one ), with
respect to the tracker model with different
lengths of the needle
40
4. Step ( Application ) Find the 3D
coordinates of actual tracker position P P Pi
Pi?? R3, i 1, ..., n Superimpose the
actual tracker position P to the model T with
optimising
getting the precise position and orientation of
the tracker in 3D Calculate the precise position
of the end of the biopsy needle and its
orientation
41
7.5 medical Navigation ( 5 ), Evaluation
High precision of position and orientation
measurement
42

• 7.5 medical Navigation ( 6 ), Visualization
• --------------------------------------------------
  --------------------------------------------------
  -------------------------------
• Definition of a CT tomogram data set intersecting
  plane containing the biopsy channel
• Resolution of CT tomogram data set is not
  homogeneous
• time consuming 3D interpolation would be
  necessary
• Homogenisation of the data set ( isotropic voxels
  ), increasing the data by a factor 5 to 10 and
  using the nearest voxel of intersection plane
  without interpolation

43
7.5 medical Navigation ( 7 ), Example of
Visualization
44
7.5 medical Navigation ( 8 ), Projected Lay-out
45
7.5 Medical Navigation ( 9 ), Registration of
Patient

• 3D-Segmentation of landmarks ( containing IR-LEDs
  ) within the CT-volume-data of the interesting
  region of the body, defining the position of the
  data-cube and of the tumor within the body.
• Segmentation of the landmarks by the stereo
  vision system during medical treatment, defining
  the absolute position of the tumor within the
  CT-data cube.
• Transfer Navigation Data from Vision System into
  CT Volume Date by
• xCT R xCam T
• Visualize Navigation of Biopsy Needle within
  CT-volume Data

46
7.5 medical Navigation ( 10 )

• 8. Procedure of Virtual Navigation and Further
  Steps
• --------------------------------------------------
  --------------------------------------------------
  -----------------------------
• Interactive segmentation of tumor and other risk
  structures by the medical specialist.
• Automatic segmentation of bones.
• Calculation of possible 3D access paths to the
  tumor.

47
(No Transcript)
48
Iteration of verification of the position of the
biopsy needle within the tumour by CT
during application. Automatic positioning of the
biopsy needle by a robot according to the known
access paths for manual interactive application
by physician ( projected ). Automatic
positioning of the biopsy needle by a robot
according according to the known access paths
using online force feed back ( projected ).
49
8. Summary ( 1 )

• Hardware Development
• Basic Algorithms
• Spectral Sensitivity
• Image Sensors
• Stereo Vision
• Examples of applications

50
8. Summary ( 2 )

• Fast Developing Discipline
• Many Known Algorithms Exist for Technical
  Applications ( mainly without any interaction )
• Brand New Applications have to be developed for
  medical Diagnosis and Therapy
• Only very few standard Algorithms Exist for
  medical Applications ( interactions necessary )
• Creativity, Intuition and Experience is
  necessary to Solve Problems
• Many Attainments of other Disciplines (
  Electrical Engineering, Math, Physics, Chemistry
  ) are required