Introduction for Users

1
Introduction for Users

• Isabel Campos (IFCA-CSIC)
• NA3 Work Package

Integration Meeting, Lisbon, LIP 12-15 November
2
Overview

• User Integration path
• Overview of tools and utilities
• Wiki pages
• Helpdesk
• Job Submission examples
• Resource availability

3
User integration path
User Interfaces
VO services
Certification Authority
Architecture Developers
Operations Team
Application Support
4
User and Operations Team

• Help contacting the certification authority that
  signs a valid certificate
• (EUGRIDPMA)
• int.eu.grid does not sign user certificates
• Runs the Virtual Organization Management
  Services (VOMS)
• Once the user has a certificate must joint a
  Virtual Organization
• (VO) or user community to access resources
• http//i2g-voms.lip.pt8443/vomses
• Certain policy of resource allocation to each VO
• Provides user accounts in the User Interfaces of
  inteugrid
• Login service development, job submission, file
  management services

5
Application Support NA3 and JRA teams

• Application Development Level
• Description of Application Requirements
• Consideration of Middleware adaptation
• Advanced use the Migrating Desktop
• Plugin Development
• Visualization features
• Interactivity Capabilities
• Using i2glogin
• i2glogin Migrating Desktop

6
Overview of tools and utilities wiki

• General Support is provided using wiki page
• https//wiki.fzk.de/i2g
• The wiki is open to users to create sections
• dedicated to own application integrations
• We are open to make it more interactive
• What is missing ?
• What would you like to see, or being able to do?

7
Overview of tools and utilities helpdesk

• Tickets are handled via a Helpdesk tool
• https//i2helpdesk.polgrid.pl
• Users can create an account to login and report
• Non-functioning middleware features
• Site problems at the infrastructure level
• The SA1 team will create a ticket and assign it
• to the responsible team, or site,

8
Job Submission (via classical User Interfaces)

• Submission of batch jobs
• Glite Middleware uses Job Description Language,
  JDL, to describe
• the job to the resource brokers of the Grid
• Submission of the script su3serial.sh

!/bin/sh Debug info echo \ Execution
start date \ Host hostname \ Path pwd
\ Program execution tar xvf
su3serial_exec.tar chmod ux run32 ./run32 gt
run32.log
9
Job Submission (via classical User Interfaces)

• Submission of batch jobs
• JDL description of the Job
• Input/Output Sandboxes are limited to a few
  MegaBytes.

Mandatory attributes Executable
"su3serial.sh" StdOutput "su3serial.out"
StdError "su3serial.err" I/O files to be
staged from/to the User Interface InputSandbox
"su3serial.sh","su3serial_exec.tar"
OutputSandbox "su3serial.out","su3serial.err","
run32.log"
10
Job Submission (via classical User Interfaces)

• Submission i2g-job-submit myjob.jdl
• isabel_at_i2gui01 i2g-job-submit su3serial.jdl
• Selected Virtual Organisation name (from proxy
  certificate extension) ifusion
• Connecting to host i2grb01.ifca.es, port 7772
• Logging to host i2grb01.ifca.es, port 9002
• 
  
• JOB SUBMIT OUTCOME
• The job has been successfully submitted to the
  Network Server.
• Use i2g-job-status command to check job current
  status. Your job
• identifier (i2g_jobId) is
• - https//i2grb01.ifca.es9000/1k_qksI5SCyVvW-c
  3UcK2A
• 
  

11
Job Submission (via classical User Interfaces)

• Status i2g-job-status ltjobidgt
• i2g-job-status https//i2grb01.ifca.es9000/1k_qk
  sI5SCyVvW-c3
• 
  
• BOOKKEEPING INFORMATION
• Status info for the Job https//i2grb01.ifca.es
  9000/1k_qksI5SCyVvW-c3
• Current Status Running
• Status Reason Job successfully submitted
  to Globus
• Destination ce.i2g.cesga.es2119/jobmanag
  er-lcgpbs-ifusiongrid
• reached on Wed Dec 20 161125 2006
• 
  
• When status Done, get your output
• i2g-job-get-output ltjobidgt
• Brings to your machine everything specified as
  OutputSandbox

12
Job SubmissionMPI Jobs
JDL for a parallel job using 8 processors in a
single cluster
Jobtype Parallel Subjobtype
OpenMPI Executable
truchas Arguments
broken_dam.inp NodeNumber
8 StdOutput std.out StdError
std.err InputSandBox
my_scripts.sh,gioc,broken_dam.inp Output
SandBox std.out, std.err Environment
I2G_MPI_PRE_RUN_HOOK./my_scrips.sh, I2G_
POST_RUN_HOOK./my_scripts.sh
13
Job SubmissionMPI Jobs
JDL for a parallel job using 48 processors in
multiple clusters
Jobtype Parallel Subjobtype
PACX-MPI Executable
truchas Arguments
broken_dam.inp NodeNumber
48 StdOutput std.out StdError
std.err InputSandBox
my_scripts.sh,gioc,broken_dam.inp Output
SandBox std.out, std.err Environment
I2G_MPI_PRE_RUN_HOOK./my_scrips.sh, I2G_
POST_RUN_HOOK./my_scripts.sh
14
Job Submission MPI Jobs

• Example 2 Gromacs

JobType "Parallel" SubJobType "OpenMPI"
NodeNumber 4 VirtualOrganisation
"icompchem" Executable "mdrun" Arguments
"-v -s full -e full-o full -c after_full -g flog
-N 4" StdOutput "std.out" StdError
"std.err" InputSandbox "speptide.top","after_
pr.gro","full.mdp",my_hooks.sh" OutputSandbox
"std.out","std.err" Environment
"I2G_MPI_PRE_RUN_HOOK./my_hooks.sh",
"I2G_MPI_POST_RUN_HOOK./my_hooks.sh"
15
Job Submission MPI Jobs

• The HOOKS mechanism my_hooks.sh

Environment variables export
OUTPUT_SElfn/grid/icompchem/test export
OUTPUT_VOicompchem Here comes the pre-run
actions pre_run_hook () PATHPATH/VO_ICOMP
CHEM_SW_DIR/gromacs-3.3/bin grompp -v -f full -o
full -c after_pr -p speptide -np 4 Here
comes the post-run action post_run_hook ()
echo "pack the data and bring it to an Storage
Element" tar cvzf OUTPUT_ARCHIVE lcg-cr
--vo OUTPUT_VO -d OUTPUT_HOST -l
OUTPUT_SE/OUTPUT_ARCHIVE file//PWD/OUTPUT_AR
CHIVE return 0
16
Job Submission Interactive Jobs

• Submission of interactive jobs
• Interactive jobs use i2glogin to make bridge
  between
• the user machine and the execution node

JobType "normal" Interactive
True Executable "/bin/sh" Arguments
"" InteractiveAgent "i2glogin" InteractiveAg
entArguments "-r -p 21015193.136.90.35 -t
-c" InputSandbox "/opt/i2g/bin/i2glogin"
17
Job Submission Interactive Jobs

• Submission of interactive jobs
• Interactive jobs use i2glogin to make bridge
  between
• the user machine and the execution node
• On Job Execution we can connect with job
• i2gui01 /opt/i2g/bin/i2glogin p
  1015193.136.90.35
• lflip23 /home/imain179 gt exit
• exit
• Connection closed by foreign host.

18
Job Submission Open MPI Interactive Jobs

• It is also possible to submit OpenMPI interactive
  jobs

Type "job" JobType
"Parallel" SubJobType "openmpi"
NodeNumber 4 VirtualOrganisation
"imain" RetryCount 0 Executable
"cpi" StdOutput
"cpi.out" StdError
"cpi.err" InteractiveAgent
"i2glogin" InteractiveAgentArguments "-r -p
21015193.136.90.35 -t -c" Interactive
true InputSandbox "/opt/i2g/bin/i2glog
in","cpi" OutputSandbox
"cpi.out","cpi.err"
19
Job Submission Open MPI Interactive Jobs

• It is also possible to submit OpenMPI interactive
  jobs
• When the job is running we can connect with the
• master process of MPI

i2g-ui02 /opt/i2g/bin/i2glogin -p
21015193.136.90.35 Pre MPI_init . Pre
MPI_Comm_size Pre MPI_Comm_rank Pre
MPI_Get_processor_name Process 3 of 4 is on
lflip26.lip.pt pi is approximately
3.1415926544231239, Error is 0.0000000008333307 wa
ll clock time 0.001492 Connection closed by
foreign host.
20
Job SubmissionMigrating Desktop (tutorial)

• All the previous can be done using Migrating
  Desktop
• Platform independent (Java based) interface
• All you need is a browser Java enabled
• Linux, Windows, Solaris, MacOS,
• Avoids dealing with JDL scripts
• More than just a graphical interface
• Single sig-on mechanism
• Supports local and grid file management
• Run interactive applications using Java plugins
  or VNC

21
Resource availability

• We have a number of open tools monitorizing the
• int.eu.grid infrastructure
• https//wiki.fzk.de/i2g/index.php/General_Infras
  tructure