Building Grid Enabled Portals Using GridPort 3

1
Building Grid Enabled Portals Using GridPort 3

• TACC Technology Transfer
• October 25, 2004
• Eric Roberts
• Maytal Dahan
• Akhil Seth
• Tomislav Urban
• David Walling

2
Goals

• This training class is intended for current and
  future GridPort 3 users
• GridPort 3.0.2 is the current release
• Participants should leave knowing how to build
  and install GridPort3 and use it to develop
  grid-enabled portals
• Required Skills
• Basic Unix and Java Programming Skills
• Basic Understanding of Grid Tools such as Globus

3
Agenda

• 900 1000 AM
• What is the Grid and Grid Computing?
• Globus Toolkit
• GridPort 3
• 1015 Noon
• GridPort Components Architecture, Informational
  Services, Interactive Services
• Portal development with GridPort
• 100 200 PM
• Lab Installing, Configuring, and Testing
  GridPort
• Lab Informational Administration Client
• 215 400 PM
• Lab Build and Deploy a Portlet Interface
• Training Evaluation

4
What is Grid and Grid Computing?
5
What is Grid and Grid Computing?

• There are several competing definitions for The
  Grid and Grid computing
• These definitions tend to focus on
• Implementation of Distributed computing
• A common set of interfaces, tools and APIs
• Some stress the inter-institutional aspect of
  grids and Virtual Organizations
• The Virtualization of Resources abstraction of
  resources

6
What is Grid and Grid Computing?

• Grid computing promises a standard, complete
  set of distributed computing capabilities
• There is a lot of hype around grid computing
• Traditional users need to get work done now!
• Some CS researchers see it as a fad
• But there is real-world value!
• In e-science and e-business

7
What is Grid and Grid Computing?

• Grid computing must provide basic functions
• resource discovery and information collection
  publishing
• data management on and between resources
• process management on and between resources
• common security mechanism underlying the above
• process and session recording/accounting
• Current grid computing tools such as Globus
  provide most of the above at some level
• The current capabilities are incomplete
• New web service based-standard will help current
  tools become interoperable.

8
Globus Toolkit
9
What is Globus Toolkit 3 (GT3) ?

• Open source toolkit for building grids
• Low-level Middleware and grid services
• Migrating from OGSI to WSRF
• Current Globus Toolkit implements the Open Grid
  Services Infrastructure (OGSI)
• Set of WSDL specifications defining standard
  interfaces, behaviors, and schema for grid
  computing
• GT3 is an implementation of OGSI (grid services)
• This is being replaced by Web Services Resource
  Framework (WSRF)
• Similar functionality to OGSI but adhering to Web
  services standards
• Includes Globus 2 services (pre-OGSI)

10
GT3 Components

• 4 Major components to GT3
• Security (GSI)
• Data management (Grid FTP, RFT)
• Resource Management (GRAM, MMJFS)
• Information Services (Index Services)
• More information can be found at
  http//www.globus.org

11
Security

• Globus uses Grid Security Infrastructure (GSI)
• GSI Authentication has the following components
• Certificate Authority (CA)
• Certificate
• Private Key
• Proxy
• http//www.globus.org/security/

12
Security Certificate Authority

• Institution or 3rd party organization that grants
  certificates
• The role of the CA in this process is to
  guarantee that the individual granted the unique
  certificate is, in fact, who he or she claims to
  be.
• Every institution or virtual organization (VO)
  would have their own CA to create certificates
  for their users.
• TACC has a CA where users can get certificates
  and create portal accounts for the TACC User
  Portal

13
Security Certificate and Key

• Grid certificate is like a passport
• Allows single sign on authentication
• Certificate /.globus/usercert.pem
• Key /.globus/userkey.pem
• Globus uses these certificates to authenticate
  users

14
Security Proxies

• Proxy gives you single sign on capabilities for
  12 hours by default (i.e. the proxy lifetime)
• Log on to any grid resource for which you have
  authorization
• Users have a Distinguished Name (DN) which is
  their certificate identity
• The DN must be in a Globus grid-mapfile on the
  remote resources users want to access
• Maps a users DN to a system account on a grid
  resource

15
Data Management

• GridFTP
• GridFTP is a high-performance, secure, reliable
  data transfer protocol optimized for
  high-bandwidth wide-area networks.
• GSI Security on control channels and data
  channels
• Built on FTP
• Put, Get, Third Party Transfer
• Currently available through GridPort
• Reliable File Transfer (RFT) is a grid service
  implementation of GridFTP.
• Not currently available through GridPort

16
Resource Management

• Master Managed Job Factory Service (MMJFS)
• Job Submission
• Batch Job submission submit a job to a local
  scheduler and return a job handle
• Job Handles URLs known as Grid Service Handles
  (GSH)
• Interactive Job Submission execute a command
  and wait for results
• Job Status query a jobs status using the GSH

17
Information Services

• Data is gathered using informational grid
  services
• GT3 uses Index Services to aggregate data from
  informational grid services
• Individual grid services can register with an
  Index Service
• Index Services can be arranged hierarchically

18
Bridging Globus and GridPort

• GridPort 3 wraps most of the Globus components
  mentioned in order to provide a consistent and
  simple interface for rapid client development
• GridPort 3 adds some of its own services that
  aggregate and/or enhance core Globus functionality

19
GridPort 3
20
GridPort

• High-Level middleware introduced between the
  low-level grid services such as Globus and the
  interface layer such as portals.
• This layer can fundamentally transform the ease
  and speed with which user-interface developers
  can bridge the gap between end-users and a grid.

21
Software Stack
22
GridPort 3 Status

• Current Release GridPort 3.0.2
• uses GT 3.0.2
• GridPort 3.1 release scheduled for SC04
• GT 3.2.1 compatibility
• Additional functionality - SRB, Advanced File
  Transfer, and more
• GridPort 3 is included in NMI GRIDS-R5 and
  NPACKage 1.x
• http//www.gridport.net - download, project
  description, user guide, mailing list, etc.

23
GridPort-Globus Component Mapping

• Security
• GridPort repository and MyProxy support
• Data Management
• File Management
• Resource Management
• Single Resource
• Batch Job Submission
• Command Execution
• Grid
• Meta-scheduler Batch Job Submission
• Information Services
• GridPort Information Repository (GPIR)
• Aggregated services
• Job Sequencer

24
Globus Compatibility

• Currently planning support for GT 3.2.1 in the
  GridPort 3.1 release
• Will evolve to WSRF after GT4 is released
  February 05

25
GridPort Components Informational Services,
Interactive Services
26
Informational Services GPIR

• Clients, such as portals, will frequently need
  information regarding the state of a grid
• GridPort Information Repository (GPIR) is used to
  store and retrieve information about the grid
• GPIR is a relational database (PostgreSQL) based
  system
• Query and Ingestion functionality is accessible
  via a Web Services interface

27
GPIR

• Stores two broad types of data
• Dynamic grid status data
• Load, Node, Resource Status, Jobs, NWS, MOTD
• Human-centric data
• Virtual organizations (VO), Resource data,
  Contacts, Sites, Institutions, Departments

28
GPIR Interfaces

• Web Services
• Query
• Allows WS-based queries of grid data
• Reads from the database
• Ingester
• WS-based input of dynamic grid data (e.g. Job
  Data)
• Writes to the database
• Administration Client
• Web-based GUI for the maintenance of
  human-centric grid data

29
Getting Information into GPIR

• In order to take advantage of GPIR, it must be
  populated with grid data
• Ingester Web Service
• Dynamic grid data such as machine load will be
  populated via the Ingester web service
• The Ingester is written to via Information
  Providers that run on or poll the resources
  themselves
• Administration Client
• For infrequently changing data such as resource
  names, use the web-based GPIR Admin client

30
Administration Client

• Go to the web client and edit data directly

31
Information Providers and the Ingester WS

• Sends XML data (job, node, motd, etc.) to GPIR
  web services
• Run as cron jobs at the desired periodicity
• Sample information providers are included with
  the GridPort download in the examples directory
• The IP address of any provider writing to the
  Ingester Web Service must be registered with GPIR
  (called a client) for a given resource in order
  to provide data for that resource
• This registration can be achieved through the
  administration client

32
GPIR Web Services
Information Providers
Resources
Clients
dB
Portals
Perl Client
Portlets
edu.tacc. gridport.gpir
Java Client
Ingester WS
Query WS
MDS
GPIR PostgreSQL
Other Middleware
OGSA (Future)
Web Scraping
SOAP-XML
HTTP
Other
JDBC
33
GPIR

• GridPort comes with sample providers to populate
  GPIR with dynamic data
• Sample job, node, load providers for LSF, PBS,
  LoadLeveler
• Located in the gridport3/examples/providers
• Also sample with Ingester client written in Perl
  but clients can be written in any language that
  has web services support

34
Interactive Services

• Authentication
• Using the GridPort repository
• Using MyProxy
• File Management
• File Listing
• File Transfer
• Resource Management
• Single Resource
• Batch Job Submission
• Command Execution
• Grid
• Meta-scheduler Batch Job Submission using CSF
• Sequencer

35
Interactive Services - Authentication

• What does logging in mean in GridPort?
• GridPort can get a proxy to authenticate the user
  during a session
• GridPort creates a session for the user
• 2 ways to authenticate a user in GridPort
• GridPort Repository
• MyProxy

36
Interactive Services - Authentication

• Using the GridPort Repository
• Repository has 2 main tasks
• Store a users credentials
• These credential files are stored according to
  GridPort username.
• Example ltusernamegt_cert.pem and
  ltusernamegt_key.pem
• User logs in to GridPort with username and
  private key password
• Maintain state of a user
• When a user logs in a session ID is created
• The proxy for the session is named by the session
  ID
• A session file for maintaining state is created
• The session file is named with the session ID

37
Interactive Services - Authentication

• Using MyProxy (http//grid.ncsa.uiuc.edu/myproxy/)
  
• MyProxy is an online credential repository
• User stores a proxy on a MyProxy server for a 7
  day period
• User can log in to GridPort to by retrieving
  credentials stored in MyProxy
• It is up to the user to store credentials in
  MyProxy
• This is for more grid savvy users who dont want
  to give GridPort a copy of their credentials
• What happens when a user logs in to GridPort
  with MyProxy?
• GridPort checks out credentials from MyProxy
• Generates a session file for this user

38
Sample Authentication Code

• Using the GridPort Repository
• Authentication authObj new Authentication()
• String sessionId authObj.gsiLogin(username,
  passphrase)
• GSSCredential cred authObj.getCredential()
• Using Myproxy
• Authentication authObj new Authentication()
• String sessionId authObj.myproxyLogin(myproxyhos
  t,myproxyusername, myproxypassphrase)
• GSSCredential cred authObj.getCredential()

39
Interactive Services File Management

• File Listing
• Used to list the files in a users home directory
  or any subdirectory
• File Transfer
• User can execute a GridFTP put transfers a file
  from the GridPort server to a remote resource
• User can execute a GridFTP get transfers a file
  from a remote resource to the GridPort server
• User can execute a GridFTP third party transfer
  transfers a file between two remote resources

40
Sample File Management Code

• // Previous Login
• Authentication authen new Authentication()
• authen.gsiLogin(username, passphrase)
• credential authen.getCredential()
• ...
• // File Listing
• FileListing listing new FileListing(resource.ta
  cc.utexas.edu, credential)
• FileInfo files1 listing.listFiles()
• listing.closeConnection()
• // File Put
• FileTransfer.put(resource.tacc.utexas.edu,
  /tmp/localGridPortPutTest, /tmp/GridPortPutTest
  , credential)
• // File Get
• FileTransfer.get(resource.tacc.utexas.edu,
  /tmp/GridPortGetTest, /tmp/localGridPortGetTest
  , credential)
• // Third Party file Transfer
• FileTransfer.thirdPartyTransfer(resource1.tacc.ut
  exas.edu,
  /tmp/GridPortThirdPartyTest1,
  
  resource2.tacc.utexas.edu,
  
  /tmp/GridPortThirdPartyTest2,

41
Interactive Services Single Resource Management

• Command execution
• Allows a user to execute a command on a remote
  resource
• Returns results immediately
• Sample Command Execution Code
• String command /bin/ls
• String args -l
• String serviceUrl SERVICEURL
• Command Execution cu new CommandExecution(creden
  tial, serviceUrl)
• cu.setExecutable(command)
• cu.setArguments(args)
• cu.execute()
• String stdErr cu.getStderr()
• String stdOut cu.getStdout()

42
Interactive Services Single Resource Management

• Batch job submission
• Allows a user to submit a batch job to a
  scheduler on a remote resource
• Returns a job handle which can be used to obtain
• Job status
• Job results

43
Batch Job Submission Sample Code

• // instantiate GridPort batch job object
• BatchJob job new BatchJob(credential)
• // set the required parameters for the batch job
  submission
• job.setArguments(hello)
• job.setStdOut(/home/user/testjob.out)
• job.setStdError(/home/user/testjob.err)
• job.setDirectory(/home/user)
• String serviceUrl ltURL_TO_REMOTE_MMJFSgt
• String executable /bin/echo
• // submit the batch job and capture the returned
  job handle
• String jobhandle job.submit(serviceUrl,
  executable)
• // create a new BatchJob object using the job
  handle
• // that was obtained during submission of the job
• submittedJob new BatchJob(credential,
  jobHandle)
• // get the status of the job
• String status submittedJob.getStatus()

44
Interactive Services Grid Resource Management

• Grid Resource Batch Job Submission
• Platform Computing developed an Open Source
  Community Scheduler Framework (CSF)
• CSF is a meta-scheduling framework implemented as
  an OGSI grid service which schedules jobs across
  a grid
• The CSF meta-scheduler schedules the job on a
  particular resource based on a scheduling
  plug-in.
• How to obtain CSF
• From Platform Computing http//www.platform.com/pr
  oducts/globus/
• GT3 source download in the scheduler directory
  http//www-unix.globus.org/toolkit
• GridPort can submit batch jobs to CSF which will
  pick an appropriate resource and submit the job

45
Sample Grid Job Submission Code

• // create a new CSF Job passing in a credential
• CsfJob csfjob new CsfJob(credential)
• String jobHandle csfjob.submit(serviceUrl,
  queue, executable, arguments, directory, count,
  stdin, stdOut, stdErr)
• //Get Job Status
• CSfJob submittedJob new CsfJob(credential,
  jobHandle)
• String stat submittedJob.getStatus()
• //Cancel a job submitted to CS
• CsfJob submittedJob new CsfJob(this.cred,
  this.jobHandle)
• boolean jobCancel cu.jobCancel()
• submittedJob.destroyJobService()

46
Interactive Services - Sequencer

• Allows a user to construct a sequence of steps
• File transfer
• CSF job submission
• File transfer from the host CSF chose (if this
  step occurs directly after a CSF job submission)
• Interface exposed as a web service
• Uses a Sequencer session to maintain state and
  authentication of the sequence even after the
  user has logged out of the portal
• Tentative plans to add batch job submission and
  command execution
• This requires a working instance of GPIR

47
Sample Job Sequencer Code

• //Create a new sequence passing in the sessionID
• SequenceBuilder sb new SequenceBuilder(sessionID
  )
• //Add a file transfer step to the sequence
• this.sb.addGridFtpStep(transferFrom, transferTo,
  sourcefile, destfile)
• //Add CSF Step to the sequence
• this.sb.addCsfStep(factory, queue, executable,
  args, dir, count, stdIn, stdOut, stdError)
• //Transfer CSF output file to another host
• this.sb.addCsfFtpStep(toHost, csfOutputFile,
• destFile )
• //Now we have built the sequence submit it
• sequenceID sq.submit()
• //Get status of sequence
• String status sq.getStatus()

48
GridPort Architecture
49
Interfaces to GridPort Portals and More
50
What is an Interface?

• Definition
• The point of interaction or communication
  between a computer and any other entity, such as
  a printer or human operator.
• GridPort has many different types of interfaces
• API
• Portal
• Web Service

51
Why are GridPort Interfaces Useful?

• Need a method for exploiting the functionalities
  of GridPort
• Can have multiple types of interfaces to access
  the same functionality but in different ways
• A well designed interface should make complex
  tasks easier to understand and execute

52
Who Uses the GridPort Interfaces?

• Users of the GridPort interfaces include
• API Application or User Interface developer
• Portal Users
• Web Service Applications

53
API

• Application Programming Interface
• Users of GridPort developing applications require
  a well-defined interface that they can program to

54
Portal

• Easily accessible by anyone with a Web browser
• Low resource overhead for the user
• Access from different types of devices (desktop,
  laptop, PDA, cell phone)
• Centrally managed by portal development team

55
Web Service

• Web services allow distributed applications to
  communicate with each other in a platform- and
  programming language-independent manner.
• Web services and clients must both speak SOAP
  protocol

56
Portal Development with GridPort 3
57
Objectives

• Discuss what a portal is and why they are useful
• Look at the Jetspeed portal framework in detail
• Discuss the demo portal that is packaged with
  GridPort 3.0
• Build and deploy a grid portlet

58
Portal Overview
59
What is a Portal?

• A portal is a gateway to a set of distributed
  services that can be accessed from a Web browser
• A portal provides a consistent environment for
  accessing services

60
How does a portal relate to GridPort?

• A portal is one example of an interface
  application that uses GridPort.
• Exposes services through a Web-based interface.
• A GridPort based portal is grid-enabled in that
  it can communicate with other grid middleware

61
Grid-enabled portals

• GridPorts Role
• Grid Middleware aids developers by easing use of
  low level grid tools
• Simplifies portal development which
• lowers the barrier of entry for grid portal
  developers
• Allows them to easily build portals and
  applications that access grid resources

62
Jetspeed A Portal Framework
63
What is Jetspeed?

• Open-source enterprise portal framework that uses
  Java and XML.
• It provides basic portal capabilities such as
• Security
• Customization
• Localization
• Personalization
• Jetspeed uses a technology called portlets
• We will be using Jetspeed version 1.x for this
  tutorial

64
Portal Architecture
65
Portal Role

• The role of the portal framework is to
• aggregate the dynamic content produced by the
  portlets
• organize that content into some layout
• ultimate control over how content is displayed
• renders the complete portal page

66
What is a portlet?

• A portlet is a container managed Web component
  that processes client requests and produces
  dynamic content
• Unlike servlets, portlets do not have direct
  interaction with Web clients.
• Web clients interact with a portal through a
  request/response mechanism enforced by a portlet
  container which also manages the lifecycle of the
  portlets.
• Generally, portlets have a clean separation of
  content from presentation which is managed by one
  or more Java classes that contain the application
  logic

67
Portlet Modes

• There are three basic portlet modes
• View
• Normal user interaction with the portlet
• Edit
• Customize behavior of the portlet
• Help
• Help information about the portlet

68
Portlet Window States

• There are three basic window states classified by
  the portlets real estate within the portal page
• Normal
• Portlet is sharing space with other portlets on
  the portal page
• Maximized
• Portlet may use more space than when in Normal
  state and may display more detailed content
• Minimized
• Portlet may use less space than when in Normal
  state and may display less or no content

69
General Portal Page Layout
70
Jetspeed Customizer

• Tool for manipulating layout from within the
  portal
• Provides the capability to
• Add, remove and move portlets in a particular
  pane
• Apply different skins to each pane
• Rename a pane
• Add, remove and move panes

71
(No Transcript)
72
Portal Registries

• Portal entities defined in the Portal Registry
• portlets
• controllers (layout portlets)
• controls (portlet decorators)
• skins
• media-types (HTML, WML)
• Security Constraints
• Registry entries are defined in XML file(s) with
  a .xreg extension and reside in the
  WEB-INF/conf directory

73
Portlet Registry Types

• Instance
• Contains all information to instantiate a portlet
• Abstract
• Cannot instantiate
• missing required information
• useful for abstracting properties for groups of
  similar portlets
• Ref
• References another registry entry (of any type)
• cascade references until abstract or instance
  entry is encountered
• overrides all parameters in entry and ATTEMPT to
  instantiate the portlet
• From Jetspeed Tutorial

74
Registry Example Portlet
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltportlet-entry name"CommandExecution"
hidden"false" type"ref"
parent"Velocity" application"false"gt
ltmeta-infogt lttitlegtCommand
Executionlt/titlegt ltdescriptiongtAllows
a user to execute a command on a remote
resource.lt/descriptiongt lt/meta-infogt
ltclassnamegtorg.apache.jetspeed.portal.portlets.
VelocityPortletlt/classnamegt ltparameter
name"template" value"CommandExecution"
hidden"true" cachedOnName"true"
cachedOnValue"true"/gt ltparameter
name"action" value"gp3x.CommandExecutionAction"
hidden"true" cachedOnName"true"
cachedOnValue"true"/gt ltmedia-type
ref"html"/gt lturl cachedOnURL"true"/gt
lt/portlet-entrygt lt/registrygt
75
Skins

• Skins are a way of changing the look and feel of
  the entire portal using Cascading Style Sheets
• Skin settings are particular to each user
• Examples of different skins

76
(No Transcript)
77
Registry Example Skin
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltskin-entry nameorange-grey"
hidden"false"gt ltproperty
name"text-color" value"000000"
hidden"false"/gt ltproperty
name"background-color" value"ffffff"
hidden"false"/gt ltproperty
name"title-text-color" value"000000"
hidden"false"/gt ltproperty
name"title-background-color" value"eeeeee"
hidden"false"/gt ltproperty
name"title-style-class" value"TitleStyleClass"
hidden"false"/gt ltproperty
name"highlight-text-color" value"ffffff"
hidden"false"/gt ltproperty
name"highlight-background-color" value"000000"
hidden"false"/gt lt/skin-entrygt lt/registrygt
78
Portal Layout Structure PSML

• Layout in Jetspeed is specified using the Portlet
  Structure Markup Language (PSML) written in XML
• PSML files are used to specify different layouts
  for different users, groups and roles that the
  portal supports
• Static PSML files are loaded into a DB at portal
  runtime
• Jetspeed Customizer manipulates PSML structure in
  DB
• Upon portal shutdown PSML is serialized back to
  static PSML

79
PSML and Registry Entries

• The following types of registry entries can be
  referenced in a PSML file
• portlets,
• controllers (layouts),
• controls (decorators),
• skins,
• media-types
• security

80
PSML Example Referencing Registry Entries
lt?xml version"1.0" encoding"UTF-8"?gt ltportlets
id"100"gt ltsecurity-ref parent"default"/gt
ltcontroller name"CardPortletController"gt
ltparameter name"parameter" value"pane"/gt
lt/controllergt ltcontrol name"TabControl"/gt
ltskin nameorange-grey"/gt ltportlets
id"301"gt ltsecurity-ref
parent"default"/gt ltcontroller
name"ColumnController"/gt ltmetainfogt
lttitlegtCommand Executionlt/titlegt
lt/metainfogt ltentry id"3011"
parentCommandExecution"/gt lt/portletsgt
lt/portletsgt
81
Profile Requests

• Mapping of requests to template and PSML
  resources
• Fallback algorithm used to locate the appropriate
  templates and PSML
• Algorithm searches from more specific to more
  general
• Country code
• Language
• Media type
• User type
• Can localize your portal for many languages
• Appropriate language will display based on origin
  of the request

82
Templates

• Using templates allows portlets to inject
  dynamic data into template interfaces which are
  then rendered into their final state
• Jetspeed supports Velocity and JavaServer Page
  (JSP) template engines
• Velocity templates do not allow as much freedom
  for putting application logic into the actual
  template code like JSP (which is a bad thing ?
  from a design perspective)
• We will be building a Velocity based portlet in
  this tutorial

83
Built-in Portlet types

• There are many pre-defined portlet types that are
  available with Jetspeed
• AbstractPortlet (all other portlets extend this
  basic portlet type)
• Velocity uses Velocity templates for displaying
  dynamic content
• CustomizerVelocity like a Velocity portlet with
  customization support built-in
• JSP uses JSP pages for displaying dynamic
  content
• WebPagePortlet formats and displays content
  from and existing web page
• HTML displays an HTML page
• WML displays a WML page (e.g. on a handheld
  device)
• RSS formats and displays a Rich Site Summary
  (RSS) feed
• XSL formats and displays content that has been
  transformed using an XSL template
• LinkManager manage links

84
Extending Built-in Portlet Types

• Define your own portlets by extending a basic
  portlet types
• Can only extend one basic portlet type (i.e.
  single inheritance)
• Specify instance, abstract, or reference portlet
  types in portlet registry

85
Security Model

• Users
• Users of the portal
• Permissions
• Permissions to use portlet modes and states
• Groups
• Groups of users who generally have the same needs
  for using the portal
• Roles
• A user can have multiple roles which controls
  access to portal resources (e.g. user, admin)

86
Portal Services - Turbine

• Built-in services
• Jetspeed is built on top of the Turbine Servlet
  framework
• Turbine provides
• Security (login)
• Presentation (velocity JSP)
• Persistence (Database)
• HTML Form Validation
• Logging
• Services (Fulcrum)
• Caching
• Scheduling
• Services are available for use inside of
  portlets.
• See Turbine documentation for defining your own
  services

87
Future Directions Portals

• Sun has defined a standard Portlet API through
  Java Specification Request (JSR) 168 which
  defines the contract between portlets and the
  portlet container
• We will be embracing this standard and will
  eventually re-implement our portlets using JSR168
  technology
• JSR168 is still in its infancy so we expect a
  timeline of about 4-6 months _at least_

88
Future Directions Interfaces to GridPort

• GridPort could be used for interfaces other than
  portals.
• We are currently talking about Java Swing
  application interfaces, Command Line interfaces
  and more Web service interfaces

89
References

• Jetspeed Portal Framework
• http//portals.apache.org/jetspeed-1/
• Turbine Web Application Framework
• http//jakarta.apache.org/turbine/
• JSR 168 Portlet Specification
• http//www.jcp.org/en/jsr/detail?id168
• Pluto Project
• http//portals.apache.org/pluto/

90
Lab Section
91
Goal of the Lab

• Install and Configure GridPort including GPIR
  administration client and a GridPort demo portal
• GridPort web site has an installation guide that
  can walk you through the GridPort installation
  http//www.gridport.net
• Slides from the training class will be available
  on the web side under publications presentations

92
Installing GridPort
93
Laptop Instructions

• Login to Laptops
• Username training
• Passphrase taccguest
• Accounts will only be active today
• Start a New Terminal
• right click on desktop and click on New
  Terminal
• Disable Terminal Sound
• On the terminal window go to Edit -gt Profiles
  Menu
• Edit the Default Profile
• Uncheck the Terminal Bell Checkbox on the General
  Tab page
• To save hit Close on all the subsequent windows
• Change directory to /usr/local/src/gridport3-train
  ing-software
• Navigate to new terminal window
• cd /usr/local/src/gridport3-training-software
• This directory has all the required tar balls
  from the appropriate web sites.

94
Installing Required Tools

• Java J2SE 1.4.1 or higher
• http//java.sun.com/downloads/index.html
• Already Installed run which java to see the
  location
• Apache Ant 1.5 or higher
• http//ant.apache.org/bindownload.cgi
• Already Installed run which ant to see the
  location
• Globus 3.0.2 must be installed on the resources
  you want to interact with.
• http//www-unix.globus.org/toolkit
• Platform CSF must be installed on a resource and
  configured to submit jobs to your resources.
• http//www.platform.com/products/globus/

95
Required Tools

• For GPIR and Job Sequencer
• PostgreSQL 7.3
• http//www.postgresql.org/
• PostgreSQL JDBC Driver
• http//jdbc.postgresql.org/
• JBOSS 3.2.x with Tomcat 5.x
• http//jboss.org
• For Demo Portal
• Apache Tomcat 4.1.x
• http//jakarta.apache.org/site/binindex.cgi

96
Users during Install Process

• Main users
• GridPort User This is the user who will act as
  the main gridport user. We recommend this user
  have limited access and that permissions are made
  correctly. For our lab the gridport user will be
  your lab user training.
• Root User This is required to install some of
  the prerequisite tools
• Postgres User This is a database user. It runs
  the database and database related tasks.

97
Installing GridPort

• Untar the GridPort Binary Install in /usr/local
• cd /usr/local
• tar xvfz /usr/local/src/gridport3-training-softw
  are/gridport3.src.tar.gz
• This will create a gridport3 directory in
  /usr/local
• GridPort directories
• as contains application server files for GPIR
• config contains GridPort configuration
  information
• db contains database related information
• examples Demo portal, GPIR schemas, and
  providers
• lib contains all the jar files needed for
  GridPort
• schema schema files that globus uses

98
Installing GridPort

• Setup GridPort environment
• cd gridport3
• Use vi or your most comfortable text editor to
  create a gridport3.env file
• vi gridport3.env
• export GP_HOME/usr/local/gridport3
• export JAVA_HOME/usr/java/j2sdk1.4.2_04
• export ANT_HOME/usr/local/apache/apache-ant-1.6.1
  
• export PATHPATHANT_HOME/bin
• source gridport3.env
• Note This is for bash shell modify to handle
  your environment

99
Installing GPIR PostgreSQL setup

• Installing PostgreSQL
• We have already untarred and compiled postgreSQL
  source to save time. If we didnt you would
  normally untar the postgres sql directory and
  compile it.
• Note PostgreSQL doesnt need to be installed on
  the same machine, it can be installed on a DB
  machine

100
Installing GPIR PostgreSQL setup

• PostgreSQL
• cd /usr/local/src/gridport3-training-software/po
  stgresql-7.4.1
• ./configure ALREADY DONE
• gmake ALREADY DONE
• su
• gmake install
• /usr/sbin/adduser postgres
• mkdir /usr/local/pgsql/data
• chown postgres /usr/local/pgsql/data
• su - postgres
• /usr/local/pgsql/bin/initdb -D
  /usr/local/pgsql/data
• /usr/local/pgsql/bin/postmaster -D
  /usr/local/pgsql/data gtlogfile 2gt1
• /usr/local/pgsql/bin/createdb test
• /usr/local/pgsql/bin/psql test
• (Type \q to exit the database)
• exit
• exit (return to training user)

101
Installing GPIR PostgreSQL setup

• Setup PostgreSQL Environment
• Open gridport3.env in the GridPort home directory
• cd /usr/local/gridport3
• vi gridport3.env
• Append the following
• export PGSQL_HOME/usr/local/pgsql
• export PGDATAPGSQL_HOME/data
• source gridport3.env

102
Installing GPIR PostgreSQL setup

• Create GPIR tables and User (pgsql user)
• su
• su postgres
• cd PGSQL_HOME/bin
• ./createdb GPIR
• ./psql GPIR
• GPIR CREATE USER gridportdbuser WITH
• GPIR PASSWORD gridportdbpass
• GPIR \q

103
Installing GPIR PostgreSQL setup

• Setup Connections (pgsql user)
• Open the server to TCP/IP connections
• gt vi PGDATA/postgresql.conf
• Line 30 tcpip_socket false
• Uncomment and change value to true. Line should
  look like this
• tcpip_socket true

104
Installing GPIR PostgreSQL setup

• Setup IP Connection (pgsql user)
• NOTE Since we are using the DB locally we dont
  need this step. If DB is on another machine you
  will need this entry!
• gt vi PGDATA/pg_hba.conf
• Add entry to the bottom
• host GPIR gridportdbuser ltIPADDRgt ltIPMASKgt trust

105
Installing GPIR PostgreSQL setup

• Restart PostgreSQL (postgres user)
• PGSQL_HOME/bin/pg_ctl stop m fast
• PGSQL_HOME/bin/pg_ctl start
• (hit enter)
• exit
• exit

106
Installing GPIR JBoss setup

• Install JBoss
• Untar JBoss
• cd /usr/local
• tar xvfz src/gridport3-training-software/jboss-3
  .2.4.tar.gz
• This will create a jboss-3.2.4 directory in
  /usr/local
• This has 3 server configurations all, minimal,
  default we will use default
• Everything here done as user training

107
Installing GPIR JBoss setup

• Setup environment
• Add the following to
• cd GP_HOME
• vi GP_HOME/gridport3.env
• Append the following
• export JBOSS_HOME/usr/local/jboss-3.2.4
• export JAVA_OPTS-Xms256m -Xmx256m
• (the java opts isnt necessary but we found it to
  be useful)
• source gridport3.env

108
Tomcat Configuration and Installation

• Install Tomcat
• cd /usr/local
• unzip src/gridport3-training-software/jakarta-to
  mcat-4.1.30.zip
• This will create a /usr/local/jakarta-tomcat-4.1.3
  0
• Setup Environment
• cd GP_HOME
• vi gridport3.env
• Append the following
• export CATALINA_HOME/usr/local/jakarta-tomcat-4.1
  .30
• source gridport3.env

109
Tomcat Configuration and Installation

• Copy GP_HOME/lib/xml/xalan.jar to
  JAVA_HOME/jre/lib/endorsed
• Note You may have to create the endorsed
  directory. If you installed Java as root you may
  need to become root to copy to the Java endorsed
  directory using the su command.
• gt su (optional)
• gt mkdir JAVA_HOME/jre/lib/endorsed
• gt cp GP_HOME/lib/xml/xalan.jar
  JAVA_HOME/jre/lib/endorsed

110
Configuring GridPort
111
Authentication Configuration

• Setup /etc/grid-security as root
• su
• cd /etc
• tar xvf /usr/local/src/gridport3-training-softwa
  re/grid-security.tar
• This will create an /etc/grid-security directory
  with the following
• certificates/
• 9a1da9f9.0
• 9a1da9f9.signing_policy
• exit (return to training user)

112
Modify GridPort Properties

• Open GP_HOME/build.properties
• gtvi GP_HOME/build.properties
• Modify the following entries
• jboss.location/usr/local/jboss-3.2.4
• jdbc.driver.location/usr/local/src/gridport3-trai
  ning-software/pg74.1jdbc3.jar
• gridport.db.passphrasegridportdbpass
• Gridport.adminClient.usernamegridportadmin
• Gridport .adminClient.passphrasegridportadmin

113
Modify GridPort Properties

• gridport.repository.location/usr/local/GridPortRe
  pository
• catalina.home/usr/local/jakarta-tomcat-4.1.30
• csf.factory.urlhttp//laredo.tacc.utexas.edu1508
  0/ogsa/services/metascheduler/JobFactoryService
• csf.queuenormal
• csf.hostbuda.tacc.utexas.edu
• file.management.machinesbuda.tacc.utexas.edu,blan
  co.tacc.utexas.edu,bandera.tacc.utexas.edu
• portal.adminemailaddress

114
Install GridPort

• Run the GridPort Install Target
• This installs the GridPort tables in GPIR,
  configures and installs the GPIR administration
  client, and deploys the demo portal to tomcat
• gt cd GP_HOME
• gt ant install.all

115
Start JBoss

• Startup JBoss from the GridPort home directory.
  This will run JBoss, create a log file in your
  home directory and run the process in the
  background.
• Note You MUST start JBoss from the GridPort
  installation directory so that GridPort will
  correctly access configuration information!
• gt cd GP_HOME
• gt /bin/sh JBOSS_HOME/bin/run.sh gt
  /jboss-logfile 2gt1
• Bring up the administration client and the GPIR
  web services to ensure they are running.
• GPIR web services http//localhost8080/gridport/
  webservices
• GPIR Administration client http//localhost8080/
  gridport

116
Startup Tomcat

• Start Tomcat from the GridPort home directory
• gt cd GP_HOME
• gt chmod 744 CATALINA_HOME/bin/
• gt /bin/sh CATALINA_HOME/bin/startup.sh
• Ensure that Tomcat is up by making sure the
  demonstration portal comes up.
• Open a browser to http//localhost9080/portal
• There are two log files that are important when
  the portal is not behaving as expected. The
  first is for the Tomcat web server log located in
  CATALINA_HOME/logs/catalina.out
• The second log file is for Jetspeed and is
  located in
• CATALINA_HOME/webapps/portal/WEB-INF/log/jetspeed
  .log

117
Authentication Configuration

• To create a GridPort account put sample
  certificate and key in stored credentials
  directory
• cp /.globus/usercert.pem /usr/local/GridPortRep
  ository/storedCredentials/training_cert.pem
• cp .globus/userkey.pem /usr/local/GridPortRepos
  itory/storedCredentials/training_key.pem

118
Authentication Configuration

• Setup Myproxy
• For simplicity we have already setup Myproxy.
• Myproxy Host pecos.tacc.utexas.edu
• Username training
• Password trainingpass

119
Testing GridPort
120
Command Line Testing

• GridPort uses JUnit for testing
• Setup ant to use JUnit
• su
• cp GP_HOME/lib/common/junit.jar ANT_HOME/lib
• exit
• Tests are run and they create XML and HTML result
  files in GP_HOME/test/reports directory
• Need to add parameters to tests.properties for
  the commands to test

121
Add localhost as GPIR client

• The IP address of the machine sending data to
  GPIR must be registered with GPIR
• Go to http//localhost8080/gridport and log in
• Go to Admin -gt Client -gt Add
• Name GP Training Client
• IP Address 127.0.0.1
• Hostname localhost
• Location Training Class
• Description Training class client
• Scripts GP sample Install
• Select all the resources
• Hit Submit

122
Command Line Testing

• Copy Tests.properties from /usr/local/src/gridport
  3-training-software/ to GP_HOME
• Open tests.properties
• List of Tests
• Configuration
• Authentication
• File Listing
• File Transfer
• Batch Job
• CSF Job
• Command Execution
• Job Builder
• Ingester Client
• Query Client

123
Command Line Testing

• Run the command line tests
• gt ant test.all
• Results get stored in XML and HTML in the
  tests/reports directory

124
GPIR Administration Client
125
Testing Administration Client

• It is important to test GPIR, JBoss, and the
  Administration Client.
• Testing the Administration Client
• Go to http//localhost8080/gridport
• Log in to the Administration Client and make sure
  it is working. It should have a sample VO in it.

126
Using the GPIR Administration Client

• Assuming that JBoss is running, the admin client
  will be at http//localhost8080/gridport
• Log in using the username and password you
  created during the JBoss install
• We are going to
• Create a Virtual Organization
• Create a Resource and add it to the virtual
  organization

127
Creating a Virtual Organization (VO)

• Click on the sign to the left of the Admin
  link in the left-hand navigation menu
• In the list under Admin click on VOs. This
  should display a list of VOs in the main part of
  the screen. You will probably have only a single
  sample VO listed.
• Click on the add link at the bottom of the VOs
  list.
• Fill in the add vo form (at least a name is
  required) and click on the submit link.
• You should now have the VO that you just added in
  the list of VOs

128
Creating a new Resource

• List the compute resources by clicking on the
  sign to the left of the Resources link under
  the Admin list and then clicking on Compute.
• The add resource form is fairly extensive.
  Fill out the name, hostname, and ip address
  fields.
• The vos checkboxes allow you to add the
  resource you are creating to one or more virtual
  organizations.
• The clients checkboxes allow you to designate
  which clients will be allowed to update data for
  the resource you are creating via the GPIR
  Ingester web service
• Click on submit when you are done.

129
Admin Client Demo

• We have now added a new resource and associated
  it with the VO that we created earlier.
• All of the lists available in GPIR, such as
  Institutions, Contacts, and Sites, are editable
  in a similar manner.
• This provides a simple way to maintain your grid
  information.

130
Admin Client Questions?

• Play with adding and editing other entities such
  as manufacturers and models, etc.
• Questions? Problems?

131
Demo Portal
132
Portal Lab

• Build and deploy a grid portlet
• Add and configure the portlet in the portal layout

133
Create a New Portal Account

• cd to GP_HOME
• Start Tomcat and pull up http//localhost9080/por
  tal
• Click the Create New Account link
• Fill in the form with your user information
• Username is training
• Password is the grid credential passphrase for
  the training certificate
• First Name, Last Name, Email Address
• Press the Create New Account button
• You should now be logged into the portal
• Log out of the portal by clicking the Logout
  link
• Log back into the portal with your username and
  password to verify
• Whether creating a new account or just logging in
  you are authenticating with GridPort and a proxy
  credential is created on your behalf and stored
  in your session
• Logout of the portal.

134
Portal Authentication

• Since most grid tasks involving Globus Toolkit
  use GSI authentication the most important thing
  we need in order to interact with Globus is a
  proxy credential.
• In the demo portal a proxy credential is created
  upon login and stored in the users session so
  that it is available at any time to any portlet
  application within the portal for that user.
• The GridPort demo portal provides just one
  example of how to put a proxy credential into a
  users sessions. You may have different needs
  for authenticating users.

135
Building a Portlet

• Building a portlet is simple. Building a
  grid-enabled portlet is more difficult. However,
  with the GridPort API at our disposal it makes
  life easier.
• This part of the tutorial will show you how to
  build a grid-enabled portlet within the GridPort
  demo portal
• Given a hostname our grid portlet will perform a
  simple command execution on a remote host to
  print some version information about that
  resource.

136
Generate a Template Portlet Application

• GridPort comes with a utility for creating and
  deploying Velocity-based portlets.
• Create a new portlet using the following Ant
  target
• ant new-portlet
• Name the portlet MyGridPortlet and give it a
  description
• This will create the necessary portlet directory
  structure in GP_HOME/examples/portal/portlets
• We will be editing the following files
• GridHelloWorld.xreg (configuration)
• GridHelloWorldAction.java (action)
• GridHelloWorld.vm (template)
• Add MyGridPortlet/lib directory

137
A Word About Velocity Portlets

• All Velocity portlet actions should extend the
  VelocityPortletAction class
• The following methods are available to override.
• buildNormalContext (view mode)
• buildConfigureContext (edit mode)
• buildMaximizedContext (maximized state)
• What about buildMinimizedContext?
• A portlet in a minimized state doesnt get
  rendered so there is no sense in providing code
  for it.

138
Velocity Context and Turbine RunData

• The Velocity Context is the context object used
  to share information between the template and the
  action
• Populate the context in the action with key-value
  pairs and access them in the template using a
  ltkeygt syntax
• The Turbine RunData contains portal runtime
  information
• E.g. session information
• A Velocity portlet uses both of these as well
  see

139
Registry Configuration
lt?xml version"1.0" encoding"UTF-8"?gt ltregistrygt
ltportlet-entry nameMyGridPortlet"
hidden"false" type"ref" parent"Velocity"
application"false"gt ltmeta-infogt
lttitlegtMyGridPortletlt/titlegt
ltdescriptiongtdemo grid portletlt/descriptiongt
lt/meta-infogt ltclassnamegtorg.apache.jetspeed.po
rtal.portlets.VelocityPortlet lt/classnamegt
ltparameter name"template" valueMyGridPortlet
" hidden"true"/gt ltparameter name"action"
valueMyGridPortletAction" hidden"true"/gt
ltmedia-type ref"html"/gt ltcategorygtMyPortletslt
/categorygt lt/portlet-entrygt lt/registrygt
MyGridPortlet.xreg
140
buildNormalContext get the users name
open GP_HOME/examples/portal/portlets/MyGridPortl
et/ \ \src/org/apache/jetspeed/modules/actions/Gr
idHelloWorldAction.java --------------------------
------------------------------------ // Add this
import to existing imports import
org.apache.turbine.om.security.User ... // Add
these methods to the class body protected void
buildNormalContext(VelocityPortlet
velocityPortlet, Context context, RunData
runData) context.put("full-name"
,getName(runData)) private String
getName(RunData runData) User user
runData.getUser() String fullName
user.getFirstName() user.getLastName() r
eturn fullName
MyGridPortletAction.java
141
Print the users name
open GP_HOME/examples/portal/portlets/MyGridPortl
et/ \ /templates/GridHelloWorld.vm -------------
-------------------------------------------------
GridHelloWorld.vm Hello ltbgtfull-namelt/bgt ltbrgt
ltbrgt
MyGridPortlet.vm
142
Deploy
Stop Tomcat CATALINA_HOME/bin/shutdown.sh Start
Tomcat CATALINA_HOME/bin/startup.sh

• Shutdown Tomcat
• cd GP_HOME
• Run the following Ant target to compile and
  deploy your portlet web application
• ant deploy-portlet -Dportlet.nameMyGridPortlet
• Start Tomcat
• Point your browser to http//localhost9080/portal
  
• IMPORTANT Always logout of the portal before you
  shutdown Tomcat

143
Add the portlet to your layout

• Click on the edit (pencil) icon in the upper
  right-hand part of the portal page to enter the
  Jetspeed Customizer
• Click the Add Pane button
• Give your new pane a name My Grid Portlets
• Click on the Apply button
• Click on My Grid Portlets link in the list of
  panes
• Click the Add Portlet button
• File portlets by category MyPortlets
• Click the check box next to the MyGridPortlet
  portlet and click Apply
• Click Save and Apply
• Click Apply
• Click on the tab named My Grid Portlets

144
Making the GridPort call in the Action
// Add these imports to existing imports import
org.ietf.jgss.GSSCredential import
org.ietf.jgss.GSSException import
edu.tacc.gridport.job.CommandExecution import
edu.ta