Creating and running an application on the NGS

1
Creating and running an application on the NGS

• Mike Minetermjm_at_nesc.ac.uk

2
Policy for re-use

• This presentation can be re-used for academic
  purposes.
• However if you do so then please let
  training-support_at_nesc.ac.uk know. We need to
  gather statistics of re-use no. of events,
  number of people trained. Thank you!!

3
Acknowledgements

• This presentation re-uses material
• on globus commands from Stephen Pickering
  (University of Leeds)

4
Outline

• A User interface machine and our set-up today
• Commands to be used
• Practical
• Port code and data from desktop/UI to the NGS
  compute nodes
• Compile and run code
• Invoke your application from the UI machine
• Summary

5
Goals

• Youve got your proxy on a UI machine and want to
  know how to port and run an application
• Er.. Whats a UI machine?

6
The UI machine

• The users interface to the grid
• Where you upload your certificate for your
  session
• Where you create proxy certificates
• Where you can run the various commands, including

7
Globus Tools

• Globus Toolkit version 2
• GT 2.4.3 from VDT 1.2
• (VDT comprises several middleware packages used
  in EU, US and UK grids)
• Job submission (GRAM)
• File transfer (GridFTP)
• Shell (GSI-SSH)
• Information Services (MDS/GIIS/GRIS)
• Information providers from GLUE schema

8
Our setup
Tutorial room machines
ssh
UI
Internet
Core NGS nodes
grid-data.rl.ac.uk
9
Secure file copy
UI
NGS compute node
gsiscp copies file using proxy certificate to
allow AA
10
Open shell on NGS CN
UI
NGS compute node
gsissh Can be an X-windows client
Compile, edit, recompile, build SHORT interactive
runs are ok (sequential) Totalview debugger.
11
Establishing contact

• Connecting to the NGS compute node
• gsissh via port 2222
• Modify environment on the NGS account (note
  which node you use and keep using it, or repeat
  commands on other nodes)
• .bashrc
• Used when you connect with gsissh
• Used by globus

12

• gsissh -X -p 2222 grid-compute.leeds.ac.uk
• -X enables forwarding of X11 connections
• -p sets port through which gsissh works
• can set X p 2222 as defaults in config files

13
Run jobs from the UI
UI
NGS compute node
globus_job_run Or globus_job_submit /
globus_get_output
Can pass files with these commands e,g,
parameters for a job.
14
PRACTICAL 2
15
The UI machine

• The users interface to the grid
• Where you upload your certificate for your
  session
• Where you create proxy certificates
• Where you can run the various commands
• That was the good news.
• The bad news install your own (currently!)
• May be your desktop with additional software
  (GT2) if you run UNIX derivative

16
A multi-VO Grid
User Interface
User Interface
Grid services
17
globus_job_run
Job request
I.S.
the gatekeeper reads the gridmapfile to map the
users id from their proxy certificate to a
local account
head-node
Info system
Globus gatekeeper
gridmapfile
Forks process to run command.
Job runs on head-node
18
globus_job_submit
Job request
I.S.
the gatekeeper reads the gridmapfile to map the
users id from their proxy certificate to a
local account
head-node
Info system
Globus gatekeeper
gridmapfile
Job queue PBSPro
Job runs on a cluster node
19
Questions -1

• How do I know which compute node to use?
• Use the Information Service
• You should find that the Core nodes of the NGS
  all run the same software
• In general you are likely to use the same machine
• Is my NGS Compute Node account shared across all
  machines??
• NO You must synchronise your accounts on
  different machines yourself. Your account names
  may be different on each machine. Use GridFTP
  (from portal) or gsi-scp
• You can hold files in the SRB,(Storage Resource
  Broker see tomorrow) and read/write these from
  any compute node

20
Questions -2

• Should I stage an executable? (stage Send it
  to a compute node from my desktop/UI)
• Only if the UI produces a binary-compatible file!
• Safer to
• Check it compiles locally
• Copy to a compute node
• Compile it there

21
Further information

• VDT documentationhttp//www.cs.wisc.edu/vdt/docum
  entation.html
• NGS user pageshttp//www.ngs.ac.uk/users/userguid
  e.html