How to run GILDA tutorials

1
How to run GILDA tutorials

• Riccardo Bruno INFN.CT
• 30.06.2008 04.06.2008

2
GILDA t-Infrastructure

• https//gilda.ct.infn.it

• Reasons why people should need a
  t-Infrastructure
• Introduce the Grid paradigm to new scientific
  communities
• Provide expertise on application porting
• Test of new releases of Grid services
• Development on Grid (new services)
• Test of middleware co-existence and
  interoperability
• Provide support
• Deliver training
• Provide training materials

3
GILDA Tutorials

• Tutorial for Users
• Tutorial for Site Administrators
• Grid Schools/Tut. for App. developers
• Customized events
• Tutorial for Trainers

4
Some concepts

• The virtualization
• GILDA uses VMWare to virtualize PCs/Grid Services
• VMWare (http//info.vmware.com/)
• An alternative software is Xen
  (http//www.xen.org/)
• GILDA Virtual Services
• Several virtual services are ready to use at
• https//gilda.ct.infn.it/VirtualServices.html
• The most important Virtual Service is the virtual
  User Interface
• https//gilda.ct.infn.it/GILDAVM/gLite-UI-VM3.0.ta
  r.gz
• The UI PnP (https//gilda.ct.infn.it/UIPnP.html)
• Sometimes it is useful to install a UI on an
  existitng Linux distro
• No root privileges are necessary
• Easy to install and configure
• Grid2Win (http//grid2win.gilda-forge.ct.infn.it/n
  ew/index.php)
• Recently GILDA supports/tests a Windows porting
  of gLite

5
Tutorial for users

• Provides mostly Training on Grid
  middleware
• But GILDA also supports the multi-middleware
  co-existence gLite, GTK4, UNICORE, Condor,
  OMII-UK (OURGrid, GTRS)
• GILDA offers tools developed by supported
  projects like Grid2Win, SecureStorage, GSAF,
  gLibrary/DRI
• The aim of the tutorial is to let student
  APPRECIATE the service offered by the GRID and
  let him able to operate on it

We will focus on training for gLite
6
Tutorial for Users Requisites

• Audience Requisites
• Basic Unix/Linux commands knowledge (command
  line)
• Basic concepts of networking (ssh, scp, etc.)
• Technical Requisites
• A computing room equipped with computers that can
  be used by students or a simple class room where
  students can bring their own laptops
• To access to the Grid, 3 possibilities
• Use of a dedicated UI server
• Use of UIs running on virtual machines
• Use of the UI PnP on Linux Accounts
• Use of the GILDA glite-tutor and glite-tutor2 UI
  servers
• A network connection with the outside Internet
  bandwidth of at least 8 Mbit/s
• a FastEthernet (10/100 Mbit/s) and/or a Wi-Fi
  (IEEE802.11b/g802.11b/g at 11/54 Mbit/s) local
  area network
• a big screen and a video-projector
• a sound system (microphone amplifier loud
  speakers) if the room is big.

Sites with outside connections smaller than 2
Mbit/s are strongly discouraged to candidate
themselves to host grid tutorials.
7
Requesting tutorial for users 1/3

• From GILDA web site https//gilda.ct.infn.it

8
Requesting tutorial for users 2/3
9
Requesting tutorial for users 3/3

• From GILDA site admin. a email will come
  containing
• A tarball file containing usercertificates
• location1/ location1_userkey.pem
  location1_usercert.pem location1_userkey.p
  12 location2/ cityname2_userkey.pem
  location2_usercert.pem location2_userkey.p1
  2 locationn/ locationn_userkey.pem
  locationn_usercert.pem locationn_userkey.p
  12
• The account names and passwords associated to
  students
• Normally follow simple rules related to the
  account numbers

ATTENTION! The number of users is limited to max
80 persons Accounts will be created on GILDA UI
servers glite-tutor.ct.infn.it glite-tutor2.ct.in
fn.it Specified IP address in the request form
will be enabled for incoming connections to both
machines The PASSPHRASE is common to ALL
generated user certificates It is location
(all capital letters) GILDA tutorial
certificates normally expire a week after the end
of the tutorial. Users might want to continue to
use GILDA have to follow the instructions https/
/gilda.ct.infn.it/users.html
10
Tutorial for Users The UI

• A single UI server
• Usage of UIs running on Virtual Machines

The Grid
The UI server
The student room
The Grid
The student room
11
Tutorial for Users glite-tutor

• GILDA relies on two UI servers
• glite-tutor.ct.infn.it
• glite-tutor2.ct.infn.it

The student room
glite-tutor.ct.infn.it
The Grid
Users /home/
GENIUS Web portal
glite-tutor2.ct.infn.it
12
Tutorial for Users Exercises

• Students need to make practice on
• GILDA maintains a tarball file containing lot of
  exercises ready to be used by students
• Its a good practice to copy exercises on ALL
  students home directories.
• The tarball of exercises is available
  onhttps//gilda.ct.infn.it/RPMS/GILDA-TAR-APPLIC
  ATIONs.tar.gz
• For each student account just execute
• wget https//gilda.ct.infn.it/RPMS/GILDA-TAR-APPLI
  CATIONs.tar.gz --no-check certificate
• tar xvfz GILDA-TAR-APPLICATIONs.tar.gz

ATTENTION! Install the tarball file into the
HOME directory The classic and simplest JDL
example among exercises is under
gLite/Other/hostname.jdl Other examples come
from supported application portings made by the
GILDA Team Ohter examples show different gLite
middleware capabilities, like MPI Jobs Use of
APIs Use of well known software packages like
OpenFoam, Matlab, etc.
13
Tutorial for Users Testing the UI

• In case the students UI is centralized, it is a
  good practice to make stress tests on the UI in
  order to discover potential problems.
• GILDA developed several stress tests for WMS and
  Storage and Cataloguing services.
• Stress tests may discover problems on resource
  overloading, network latencies, certificates,
  etc.
• Its a good practice to perform at least once a
  stress test over the UI hosting students
  accounts.

ATTENTION! To request the stress test scripts we
kindly invite you to send a email to grid-prod
mailing list grid-prod.ct.infn.it
14
Tutorial for Users Other scenarios

• A local RB serving the site

The Grid
The local UI server
The student room
The local RB server
The local Grid Portal
15
Tutorial for users Student PCs

• Two different solutions
• PCs running Windows
• Office suite PDF viewer (Adobe)
• SCP Client
• SSH Client
• PCs running Linux
• X Interface
• Office suite PDF viewer (xpdf)
• Suggestions (available free software)
• Windows
• MS Office (if licenced) or OpenOffice
  (http//www.openoffice.org/)
• SCP WinSCP (http//winscp.net/eng/docs/introducti
  on)
• SSH Putty (http//www.chiark.greenend.org.uk/sgt
  atham/putty/)
• LinuxLike solution with CygWin
• Linux
• Office package OpenOffice (http//www.openoffice.
  org/)

In this tutorial followin SSH client will be used
16
User tutorial Running the tutorial

• Every day ask students to sign a list containing
  at least
• Name, Surname, Institute, Country, email
• Tutors will describe theory first
• For practice sesssions Tutors may choice among
• Use of dedicated slides to explain exercises
• Use of the GILDA Wiki Pages (recommended)
• https//grid.ct.infn.it/twiki/bin/view/GILDA/WebHo
  me
• During practice sessions ALL participating
  tutors will support students requesting help
• At the end of the tutorial dont forget to ask
  students to fill the Feedback Form! (see later)
• At the end of the tutorial provide the Training
  Task Leader with the list of students and the
  feedback values.

17
Tutorial for site administrators ..

• Provides mostly Training on Grid
  Middleware
• GILDA actually supports the multimiddleware
  co-existence gLite, GTK4, UNICORE, Condor,
  OMII-UK (OURGrid, GTRS)
• The aim of this tutorials is to explain how to
  INSTALL and MAINTAIN grid sites.

We will focus on training for gLite
18
Tutorial for S.Adm Requisites

• Audience Requisites
• Advanced Unix/Linux commands knowledge (command
  line)
• Unix/Linux administration knowledge
• Good knowledge of networking and relative
  services (DNS, Firewall, etc.)
• Technical Requisites
• A network connection having outside Internet
  bandwidth of at least 8 Mbit/s
• a FastEth. (10/100 Mbit/s) and/or a WiFi
  (IEEE802.11b/g - 11/54 Mbit/s) LAN
• A computing room equipped with desktops that can
  be used by students or a simple class room where
  students can bring their own laptops
• At least one machine (either real or virtual,
  with Xen or Vmware) each 2 users. Machines must
  have direct and inverse resolution, so either 
      - Machines with public IP, direct/reverse
  solved by public DNS (or)     - Machines with
  private IP solved by a local DNS ("pointed"  by
  students)
• Real machines should have at least 1 GB of memory
  per processor (dual processor machines are
  preferred) and at least 40 GByte of internal
  hard-disk
• The machines must be at the disposal of tutors at
  least 1 week before the tutorial
• A big screen and a video-projector (a 2nd
  video-projector for the machine to install would
  be better)
• A sound system (microphone amplifier loud
  speakers) if the room is big.

Sites with local network slower than 100 Mbit/s
are strongly discouraged to candidate themselves
to host grid tutorials.
19
Tutorial for S.Adm Request

• The requesting procedure is the same as for users
  

ATTENTION! The number of users is limited to max
30 persons Specified IP addresses in the request
form will be enabled for incoming connections to
both GILDA UI servers GILDA host certificates
for tutorials normally expire after the end of
the tutorial. Users might want to continue to use
GILDA and have to follow this procedure https//
gilda.ct.infn.it/sites.html Requested GILDA
host certificates have one year of validity
20
Tutorial for S.Adm Preparation

• Dedicate a PC hosting a web server and make it
  accessible by students
• Create a mirror of the INFN Grid software
  repository (instructions will follow on this)
• http//igrelease.forge.cnaf.infn.it/doku.php?iddo
  ctipsmrepo
• Create a directory seen by the web server and
  copy on it
• GILDA Host certificates
• GILDA User certificates
• The java JDK 1.5.0 RPM (Java Development Kit 5.0
  Update XX)
• The tarball of GILDA exercises
• A directory containing all slides (sometime
  indico is slow or unreachable or just contain
  last minute slides not yet uploaded into the
  Agenda).
• VM Image with BASIC installation (SL4
  pre-requisites)

21
Tutorial for S.Adm Scenario

• A Dedicated PC for each service
• Usage of Virtual Machines

Any Grid Service
The student room
The student room
22
Tutorial for S.Adm Final Exercise

• A key point in tutorial for site administration
• Each student installs a service storing the VM
  image file
• Let installed machine cooperate as a Local grid
  site ( )

The student room
23
Tutorial for S.Adm Run the Tut.

• Every day ask students to sign a list containing
  at least
• Name, Surname, Institute, Country, email
• Tutors will describe theory first, then the
  installation of the grid service will follow
• Students will install the service in a dedicated
  and separated slot in the Agenda.
• Students may also have a look on GILDA wiki
  pages
• https//grid.ct.infn.it/twiki/bin/view/GILDA/GILDA
  SiteInstall
• During practice sessions ALL participating
  tutors will support students requesting help
• At the end of the tutorial dont forget to ask
  students to fill the Feedback Form! (see later)
• At the end of the tutorial provide the Training
  Task Leader with the list of students and the
  feedback values.

24
Grid schools

• Aims to support students in porting their own
  applications
• Requires a big effort in terms of human and
  physical resources
• A survey of potential application must be done in
  advance
• A selection of posted application must be done in
  accordance of well defined selection and
  evaluation criteria

25
Grid Schools Requisites

• Audience Requisites
• Researchers having User tutorial requisites
• The Application responsible must partecipate to
  the GridSchool
• Technical Requisites
• A network connection having outside Internet
  bandwidth of at least 8 Mbit/s
• a FastEth. (10/100 Mbit/s) and/or a WiFi
  (IEEE802.11b/g - 11/54 Mbit/s) LAN
• A computing room equipped with desktops that can
  be used by students or a simple class room where
  students can bring their own laptops
• At least 10 real or virtual (with Xen or Vmware)
  machines. These machines must have public IP
  addresses and must be registered on a local DNS
  with direct and inverse name resolution.
• Real machines should have at least 1 GB main
  memory per processor (dual processor machines are
  preferred) and at least 40 GByte of internal
  hard-disk
• The machines must be installed by tutors at least
  1 week before the tutorial
• A big screen and a video-projector (a 2nd
  video-projector for the machine to install would
  be better)
• A sound system (microphone amplifier loud
  speakers) if the room is big.
• Appropriate Surveillance, Fire estinguishing
  systems, Cooling fans and Electricity power must
  be granted during the school

Sites with outside connections smaller than 2
Mbit/s are strongly discouraged to candidate
themselves to host grid schools.Sites with local
network slower than 100 Mbit/s are strongly
discouraged to candidate themselves to host grid
tutorials.
26
Grid Schools Request

• There is a standardized way to organize a
  GridSchool, like tutorials for users and site
  admin tutorials
• Normally Grid Schools are planned at the
  beginning of a project due to their huge request
  of resources
• Planning are normally done on TB Meetings or
  during Workshops.
• In order to collect applications a survey must be
  submitted to the community
• The list of potential applications must be
  provided to NA2 and the TB in order to be
  evaluated

ATTENTION! The number of application is limited
to max 15 The number of days shouldnt be less
than 10 (working days) Applications will be
installed on GILDA or other Organizations (EELA-2)
27
Grid Schools Preparation

• A local grid site must be installed by tutors one
  week before the school
• Schools requires 2 rooms
• One room dedicated to Grid Services
• One room dedicated to Students
• Computer room should be deployed far from the
  student room in order to reduce heating and noise
• Student room can be arranged in different
  fashions (U/O-shape, classroom shape, etc.)
• Student PCs must be installed exaclty like in a
  user tutorial

28
Grid Schools Scenario
29
Grid Schools ROC on Duty

• Normally Grid Schools are a very good opportunity
  to train site managers to the ROC on Duty
  tutorial
• The Regional Operation Center (ROC) has the
  responsability for providing the Grid Services.
  This responsibility includes
• Middleware certification
• Middleware deployment
• Day to day operations
• User support
• This tutorial AIMS to explains the ROC activities
  and Policies adopted by the GILDA supported
  project.

30
Customized events

• Sometime a generic tutorial does not fit training
  requests coming from the hosting institute
• In such case GILDA has to provide a focused
  training
• Examples
• Training focused on HPC and MPI Jobs
• Application developers days
• Training on GILDA services SecureStorage, GSAF,
  gLibrary/DRI

31
Customized events Requisites

• Due to the unhespected nature of the target topic
  to cover requisites may be very different
• A good starting point maight be to refer to
• Same requisites of Tutorial for Users
• Same requisites of Tutorial for
  SiteAdministrators

32
Customized events Request

• There is a standardized way to organize a
  Customized event, like tutorials for users and
  site admin tutorials
• Normally Customized events are planned during the
  lifetime of the supported project
• Planning are normally done on TB Meetings or
  during Workshops.
• Customized events normally need more time to be
  organized due to the unhespected nature of the
  target topic

33
Training for Trainers

• GILDA currently supports many projects EELA-2,
  EGEEIII, COMETA, etc.
• Projects actually cover many continents Europe,
  Asia, Latin America, etc.
• Training means costs for the project, then the
  need of Local Tutors.
• GILDA ever encourages the creation of local team
  of tutors
• Training for Trainers is the opportunity to
  become a GILDA Local Tutor.
• The aim is to form Local Tutors

34
Training for Trainers Requisites

• Audience Requisites
• Same requisites of Site Administration Tutorial
• Knowledge of gLite or a previous participation to
  any GILDA Tutorial
• Technical Requisites
• A network connection having outside Internet
  bandwidth of at least 8 Mbit/s
• a FastEth. (10/100 Mbit/s) and/or a WiFi
  (IEEE802.11b/g - 11/54 Mbit/s) LAN
• A computing room equipped with desktops that can
  be used by students or a simple class room where
  students can bring their own laptops
• At least 10 real or virtual (with Xen or Vmware)
  machines. These machines must have public IP
  addresses and must be registered on a local DNS
  with direct and inverse name resolution
• Real machines should have at least 1 GB of memory
  per processor (dual processor machines are
  preferred) and at least 40 GByte of internal
  hard-disk
• The machines must be at the disposal of tutors at
  least 1 week before the tutorial
• A big screen and a video-projector (a 2nd
  video-projector for the machine to install would
  be better)
• A sound system (microphone amplifier loud
  speakers) if the room is big

Sites with outside connections smaller than 2
Mbit/s are strongly discouraged to candidate
themselves to host grid schools.
35
Training for Trainers Request

• There is a standardized way to organize a
  Training for Trainers event, like tutorials for
  users and site admin tutorials
• Normally Training for Trainers events are planned
  at the beginning of the supported project
• Planning are normally refined on TB Meetings or
  during Workshops.

36
Common operations

• Following actions are common for any kind of
  Training Event

37
Common The Agenda

• The training event need to be published
• GILDA uses CERN Indico Agenda System
• EELA-2 Project (http//indico.eu-eela.org/)
• EGEEIII Project (http//indico.cern.ch/categoryDi
  splay.py?categId41)

ATTENTION For EELA-2 project the preferred
timetable default style is Indico Style Please
do not forget to ask Tutors to provide slides,
Then upload them into the Agenda
38
Common The feedback form

• The training event need a Feedback Form
• Why the feedback form
• Improve the quality of training material
• Improve the quality of training
• Obtain suggesions from students
• Report the quality of training to the funding
  institute
• The FF just collects student feedbacks
• Normally one feedback per Talk
• 6 degree of choice
• Poor, Insufficient, Fair, Good, Very Good,
  Excellent
• 1 comment field

ATTENTION! There is no actually a official
request form for the Feedback Form To request a
FeedbackForm just send a email TO
rita.ricceri_at_ct.infn.it CC grid-prod_at_ct.infn.it

39
Common Images

• Few images of GILDA tutorials

ATTENTION! Please do not forget to bring your
camera! Photos are very useful for many cases
like Creating project booklets Designing Web
sites Creating document reports .
Grid SchoolComputer Room
User Tutorial
Grid SchoolStudent Room
40
Questions