Examples of Grid Usage

1
Examples of Grid Usage Branko Marovic, RCUB -
AMREJ academic network
2
What is driving grid development?
Data and compute intensive sciences are next
generation applications that have extreme needs
but are likely to become mainstream in the next 5
years

• Physics/Astronomy data from different kinds of
  research instruments
• Medical/Healthcare imaging, diagnosis and
  treatment
• Bioinformatics study of the human genome and
  proteome to understand genetic diseases
• Nanotechnology design of new materials from the
  molecular scale
• Engineering design optimization, simulation,
  failure analysis and remote Instrument access and
  control
• Natural Resources and the Environment weather
  forecasting, earth observation, modeling and
  prediction of complex systems river floods and
  earthquake simulation

3
The Vision

• An international network of scientists will be
  able to model a new flood of the Danube in real
  time, using meteorological and geological data
  from several centers across Europe
• A team of engineering students will be able to
  run the latest 3D rendering programs from their
  laptops using the Grid.
• A geneticist at a conference, inspired by a talk
  she hears, will be able to launch a complex
  bio-molecular simulation from her mobile phone

Access to a production quality GRID will change
the way science and much else is done
4
LHC experiments
ATLAS

• Storage
• Raw recording rate 0.1 1 GByte/s
• Accumulating at 5-8 PetaByte/year
• 10 PetaByte of disk
• Processing
• 200,000 of todays fastest PCs

CMS
ALICE
LHCb
5
Network for Earthquake Engineering Simulation
(NEES)
6
SERVOGrid Portal
7
myGrid UK eScience Program Pilot Project
8
myGrid Investigation set of experiments
metadata

• Experimental design components
• Experimental instances that are records of
  enacted experiments
• Experimental glue that groups and links design
  and instance components
• Life Science IDs, URIs, RDF

9
A Grid based Diagnostics and Prognosis System for
Rolls Royce Aero Engines Engine model
10
GENIUS Portal
11
GENIUS graphic workflow editor
12
LCG UI commands

• edg-job-list-match
• Lists resources matching a job description
• Performs the matchmaking without submitting the
  job
• edg-job-cancel
• Cancels a given job
• edg-job-status
• Displays the status of the job
• edg-job-get-output
• Returns the job-output (the OutputSandbox files)
  to the user
• edg-job-get-logging-info
• Displays logging information about submitted jobs
  (all the events pushed by the various
  components of the WMS)
• Very useful for debug purposes

13
Getting proxy certificate

• branko_at_grid02 branko voms-proxy-init
• Your identity /DCORG/DCSEE-GRID/OPeople/OUOB/
  CNBranko Marovic
• Enter GRID pass phrase for this identity
• Creating temporary proxy .........................
  ................................... Done
• Creating proxy ...............................
  Done
• Your proxy is valid until Thu Apr 14 034640
  2005

14
Job description language

• branko_at_grid02 branko cat test.jdl
• Executable "FieldServer"
• Arguments "p 147.91.1.195 4566 0.0.0.0 ANY - -
  ./ ./ 300000000 0 MRVDWLOB"
• StdOutput "std.out"
• StdError "std.err"
• InputSandbox "FieldServer", "MRVDWLOB"
• OutputSandbox "std.out", "std.err"

15
Site matching

• branko_at_grid02 branko edg-job-list-match --vo
  seegrid test.jdl
• Selected Virtual Organisation name (from --vo
  option) seegrid
• Connecting to host ui.ulakbim.gov.tr, port 7772
• 
  
• COMPUTING ELEMENT IDs
  LIST
• The following CE(s) matching your job
  requirements have been found
• CEId
  
• ce.ulakbim.gov.tr2119/jobmanager-torque-seegrid
  
• ce001.grid.bas.bg2119/jobmanager-lcgpbs-seegrid
  
• ce01.grid.acad.bg2119/jobmanager-torque-seegrid
  
• grid1.irb.hr2119/jobmanager-torque-seegrid
  
• 
  

16
Job submission

• branko_at_grid02 branko edg-job-submit --vo
  seegrid -o ID -r ce001.grid.bas.bg2119/jobmanager
  -lcgpbs-seegrid test.jdl
• Selected Virtual Organisation name (from --vo
  option) seegrid
• Connecting to host ui.ulakbim.gov.tr, port 7772
• Logging to host ui.ulakbim.gov.tr, port 9002
• edg-job-submit
  Success
• The job has been successfully submitted to the
  Network Server.
• Use edg-job-status command to check job current
  status. Your job identifier (edg_jobId) is
• - https//ui.ulakbim.gov.tr9000/m2Ocpo8VvA9DZt1j
  98RxGw
• The edg_jobId has been saved in the following
  file
• /home/branko/ID
• 
  

17
Job status check

• branko_at_grid02 branko edg-job-status -i ID
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//ui.ulakbim.gov.t
  r9000/m2Ocpo8VvA9DZt1j98RxGw
• Current Status Ready
• Status Reason unavailable
• Destination ce001.grid.bas.bg2119/jobmana
  ger-lcgpbs-seegrid
• reached on Wed Apr 13 134837 2005
• 
  
• branko_at_grid02 branko edg-job-status -i ID
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//ui.ulakbim.gov.t
  r9000/m2Ocpo8VvA9DZt1j98RxGw
• Current Status Scheduled
• Status Reason Job successfully submitted to
  Globus
• Destination ce001.grid.bas.bg2119/jobmana
  ger-lcgpbs-seegrid
• reached on Wed Apr 13 134912 2005

18
Result retrieval

• branko_at_grid02 branko edg-job-get-output -i ID
• Retrieving files from host ui.ulakbim.gov.tr (
  for https//ui.ulakbim.gov.tr9000/m2Ocpo8VvA9DZt1
  j98RxGw )
• 
  JOB
  GET OUTPUT OUTCOME
• Output sandbox files for the job
• - https//ui.ulakbim.gov.tr9000/m2Ocpo8VvA9DZt1j
  98RxGw
• have been successfully retrieved and stored in
  the directory
• /tmp/jobOutput/branko_m2Ocpo8VvA9DZt1j98RxGw
• 
  

19
Result viewing

• branko_at_grid02 branko cat /tmp/jobOutput/branko_
  m2Ocpo8VvA9DZt1j98RxGw/std.out
• 13.04.105 165832 Field server invoked with
  these parameters
• ./FieldServer p 147.91.1.195 4566 0.0.0.0 ANY - -
  ./ ./ 300000000 0 MRVDWLOB
• Loading database MRVDWLOB
• Format DICOM
• Description
• BrainDWI
• ID MF-0000023
• DOB 1950/07/04
• SEX M
• MR study done on 2005/03/05 at 0000
• Study description fMRI
• Misc. info 1.3.6.1.4.1.5962.1.2.5023.11
  10066623.10209
• Series desc. Diffusion weighted
• By SmithJohn
• Instalation CONSOLE01 (Acme Medical Devices -
  Super Dooper Scanner) at St. Nowhere Hospital
• Dataset voxels 128x128x13
• Voxel size 1.71875x1.71875x5
• Bytes per voxel 2 Max. magnitude estimate 2333

20
Final job status

• branko_at_grid02 branko edg-job-status -i ID
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//ui.ulakbim.gov.t
  r9000/m2Ocpo8VvA9DZt1j98RxGw
• Current Status Cleared
• Status Reason user retrieved output sandbox
• Destination ce001.grid.bas.bg2119/jobmana
  ger-lcgpbs-seegrid
• reached on Wed Apr 13 141601 2005
• 
  
• branko_at_grid02 branko edg-job-get-logging-info
  -i ID
• 
  
• LOGGING INFORMATION
• Printing info for the Job https//ui.ulakbim.gov
  .tr9000/m2Ocpo8VvA9DZt1j98RxGw
• ---
• Event RegJob

21
Submission to several servers

• branko_at_grid02 branko ./submit_job test.jdl
• Selected Virtual Organisation name (from --vo
  option) seegrid
• Connecting to host ui.ulakbim.gov.tr, port 7772
• Submiting to ce.ulakbim.gov.tr2119/jobmanager-tor
  que-seegrid
• Submiting to ce001.grid.bas.bg2119/jobmanager-lcg
  pbs-seegrid
• Submiting to ce01.grid.acad.bg2119/jobmanager-tor
  que-seegrid
• Submiting to grid1.irb.hr2119/jobmanager-torque-s
  eegrid
• branko_at_grid02 branko edg-job-status -i
  test.jdl.jobs.list
• --------------------------------------------------
  ----------------
• 1 https//ui.ulakbim.gov.tr9000/Q5gUKOFTfc0CUf4
  KTWu5Cg
• 2 https//ui.ulakbim.gov.tr9000/Lo7xMlV8SXrn3Jl
  4l8hdEQ
• 3 https//ui.ulakbim.gov.tr9000/95uZFKG4QwQAUDK
  OmY8hxA
• 4 https//ui.ulakbim.gov.tr9000/8FhrKyCPu8vokUH
  QENXkPQ
• a all
• q quit
• --------------------------------------------------
  ----------------
• Choose one or more edg_jobId(s) in the list -
  1-4alla

22
VIVE - Volumetric Image Visualization Environment

• Objective
• Complete interactive analysis of 3D datasets from
  medical imaging devices (CT, MR, PET, SPECT)
• Allow easy remote access over the Internet
• Provide generalized 3D tools for diagnosis,
  surgical planning, and therapy evaluation
• Virtual distant examination based on 3D datasets
• Our basic idea
• Split the system into a client and server part
• Dedicate the server part to processing of
  volumetric data
• Produce a lightweight client for user interaction
  and 3D rendering
• Use Grid to resolve open issues

23
What can VIVE gain from the Grid

• Data transfer and replication
• Abstraction of data within patient electronic
  health record
• Replication of data at the locations close to
  computation elements
• On demand load of necessary parts of dataset -
  usage of spatially segmented formats
• Security
• Data encryption in transfer
• We can enforce anonymisation -removal of personal
  details
• Stripping of data over several nodes
• Not necessary for the regular medical images
• Divide a single field over several WNs using a
  dispatcher and spatially segmented data format
• Functional distribution, thus allocating separate
  machines for texture and surface mesh creation
• Monitoring - Not crucial. May become important
  with
• Several WNs per session
• Several sessions per WN

24
VIVE Gridusage
25
(No Transcript)
26
(No Transcript)