Title: TelePresence Microscopy at CSIRO Minerals
1TelePresence Microscopy at CSIRO Minerals
- Colin MacRae
- CSIRO Minerals - Microbeam Laboratory
- www.minerals.csiro.au/microbeamlab
2Talk outline
- What and why?
- History of TelePresence at CSIRO Minerals
- 3rd generation TelePresence system
- System architecture
- Control interfaces
- Video streaming technology
- Security features
- Conclusions, where to from here?
3What is TelePresence?
- Tele-conferencing
- Video Audio
- Tele-control
- Microscope controls
- On-line data collection analysis
- Tele-control is provided by some manufacturers
- Ideally, TelePresence should be
- Interoperable with many platforms
- Shrink-wrapped software
- Looks professional, with easy to use interfaces
- Fast and responsive
- Stable and secure
4Why do we want TelePresence?
- Internal clients (Brisbane, Hobart, Perth)
- External clients (Melbourne, Sydney, Brisbane)
- For analyses, our clients must either
- Mail / courier samples, and wait for our report
(slow) - Travel to our laboratory (expensive)
- TelePresence can help
- Provide results to client in real-time, and also
allows instant feedback from client - Attract new clients
CSIRO Minerals
CSIRO Minerals, Exploration Mining
BHP-Billiton
CSIRO Marine Research
Comalco RioTinto
5What do we want to connect?
- Scanning Electron Microscopes
- Electron Microprobes Proton Microprobe
- X-ray detectors EDS WDS
- Cathodoluminescence systems
- Optical microscopes
6What do we want to offer
- Clear fast images
- On-line microanalysis
- On-line x-ray spectra
- Large area optical views
- Streaming maps
- Security
- Voice over IP
- Data storage
- Data analysis/presentation software
7What do we need from TelePresence?
- Provision of
- Real-time video from multiple sources (SEM, room
camera) - Streaming EDS spectra and other data (
controls) - Control of microscope stage column (lower
priority)
- Minimal hardware requirements
- PC or laptop (no workstations)
- ADSL or cable modem, 1Mbit/s (not Internet2,
GRID)
- Shrink-wrap software
- Easy to set-up
- Very easy to learn and use, with generic
controls/interface - Very, very stable
- Extraordinary security
- Low maintenance
- Must not affect normal operation of instruments!
8What do we need from TelePresence?
- Provision of
- Real-time video from multiple sources (SEM, room
camera) - Streaming EDS spectra and other data (
controls) - Control of microscope stage column (lower
priority) - Minimal hardware requirements
- PC or laptop (no workstations)
- ADSL or cable modem, 1Mbit/s (not Internet2,
GRID) - Shrink-wrap software
- Easy to set-up
- Very easy to learn and use, with generic
controls/interface - Very, very stable
- Extraordinary security
- Low maintenance
- Must not affect normal operation of instruments!
9TelePresence history
- TelePresence Microscopy 1999
- Nestor J. Zaluzec, Argonne National Laboratory
- Web-based, pervasive
- Web based notebook
- Video with GTS streaming technology
10TelePresence history
- TelePresence Microscopy 1999
- Nestor J. Zaluzec, Argonne National Laboratory
- Web-based, pervasive
- Java system 2001
- Platform independent, safe code
- MJPEG video (TCP-IP)
- Web based notebook with embedded Java applets for
Images, Spectra and maps - Successful, but limited
11TelePresence history
- TelePresence Microscopy - 1999
- Nestor J. Zaluzec, Argonne National Laboratory
- Web-based, pervasive
- Java system - 2001
- Platform independent, safe code
- MJPEG video
- Web based notebook with embedded Java applets for
Images, Spectra and maps - Successful, but limited
- C system 2004 ?
- Widely used language, with broad library support
- MPEG-4 video compression in software, changeable
video size - New, secure architecture
12Video latency
- Definition
- Latency time displayed time captured
- What do we want?
- lt500ms for focussing (or better)
- lt200ms preferred
- Causes of latency
- Video compression / decompression
- Hardware accelerated rendering (done)
- Hardware compression (cheap MPEG-4 hardware
compression coming soon) - Router delays (congestion)
- Ultra-broadband internet (eg GRID)
- Reduce stream size (better codec)
- inter-router delays (speed / distance)
- More bandwidth wont help
- Use faster internet protocol (i.e. TCP ? UDP)
13Another real life problem The Firewall
Firewall
Firewall
Internet
Client network
Our network
port 8080 (UDP), from specified addresses
Server
Client
Video
port 8080 (TCP)
Host
DMZ
14Comparison of platforms
Web browser
Java
C
15System architecture
Password
Client
Host
Server
(TCP socket)
commands data
- JEOL 8500F FEG-Electron Microprobe
- JEOL 8900R
- Electron Microprobe
- FEI Quanta
- FEG-Environmental SEM
- Hitachi S-5000
- In-Lens Cold FEG SEM
- PIXE
- Proton Microprobe Melbourne Uni
Video (UDP stream)
Server
Server
Generic interface
Instrument specific
Drivers
16DirectShow video streaming
- Very easy to configure and re-configure graphs
- Other stream designs are possible, eg
- Saving stream to HDD
- Motion detection triggering
- Adaptable to as yet unforseen requirements
- All components are upgradeable and replaceable
- New codecs can be adopted as they become
available - Future proof
- We didnt have to write any of these components
(only wire them together) - Some DirectShow components are hardware-accelerate
d - Fast video rendering
17TelePresence interfaces
- Video
- Camera control
- Tilt-pan-zoom controls
- Used in conjunction with video window
- EDX spectrometers
- Generic, easy to use interface
- Fast (2-5 updates/s)
- Text streams
- Miscellaneous others
- Stage and Electron optical System
18Video streaming technology
- Microsoft DirectShow
- Widely supported standard with pre-compiled
modules - Hardware-accelerated rendering
- MPEG-4 encoding
- 320 ? 240 at 25 fps is 500kbit/s (ADSL)
- (12 Mbit/s by MJPEG)
- (59 Mbit/s for raw video)
- UDP transmission
- Lowest possible latency over IP (200ms)
19Video cameras
- Canon VC-C1 and VC-C3 cameras
- 450 x 350 pixels at 30 frames/s.
- 90º pan, 25º tilt, 10? zoom
- Good low-light performance
20Bench-top camera
- ELMO camera
- XGA video, 20 frames/s
- www.elmousa.com/presentation
21Client feedback
- Problem Video stream is one-way. How do clients
tell us what they want to see? - Verbal feedback (telephone)
- that thing, over to the left. Up, say, half an
inch - Not very effective
- Graphical feedback
- Video overlay marker
- Visible to all other clients and the operator
- More tools to follow
22Security
- Firewalls
- Blocks all direct communication with servers and
drivers - Host and servers are in separate DMZ network,
not mounted on CSIRO file system - Socket encryption
- Sockets are wrapped with Rijndael cipher, NIST
advanced encryption standard (AES 2001). - Certified for all levels of classification, up to
and including top-secret. - Hackers must break AES to read data, send false
data, or issue commands
23On-line help
24Off-line data analysis
25Elemental Data Set
Ti
Fe
26Three colour map - Chimage
2500um
Chimage ref. MM
27Hand selection of phases
28Phase patched image
29Electron induced cathodoluminescence map with
selected area and smaller inserts showing U, Th,
Yb and Y. X-ray maps produced on the nuclear
microprobe.
MacRae et al. Heavy Minerals 2003, Johannesburg,
South African Institute of Mining and Metallurgy.
pp139-143
30Lines profiles on a spectral map
Distance
Frequency (nm)
340
925
31Summary
- TelePresence allows Microscopy facilities
providing a alternative service to clients - High-quality real-time video is achievable over
cable modem using MPEG-4 codec's and UDP
transmission - Internet security is ensured using firewalls and
encryption of communications - Architecture is extensible and applicable to any
instrument for which DLLs or control code are
available - Off-line data analysis using Chimage
32Acknowledgements
- CSIRO Minerals
- Aaron Torpy
- Nick Wilson
- CSIRO ITS
- Tony Sanderson
- Andrew Smith
- Engineering Firm Cymontkowski
- Marc Cymontkowski
- Further information
- TPM web reference
- http//minerals.tpm.csiro.au/software/teleprese
nce/ - Chimage web reference
- http//www.minerals.csiro.au/em-unit/chimage_5_in
dex.htm
33Questions?