NPACI AHM03 Tutorial

1
NPACI AHM03 Tutorial 10Grid Computing Portals

• Mary Thomas
• Texas Advanced Computing Center
• The University of Texas at Austin
• and
• NPACI
• Presented at the NPACI AHM 2003, San Diego, CA

2
Goals

• Introduce basic portal technologies and concepts
• Provide enough knowledge to go out and begin
  process of evaluating/understanding technologies
• Provide enough knowledge to build a computing
  portal based on GridPort/Perl/CGI
• Not going to teach you how to install Grid
  software assume you can do this is you have
  someone who takes care of this
• Audience application developers or scientists
• Not a sales pitch will address realities and
  tradeoffs

3
Outline

• Motivation for Computational Grid Portals
• Understanding Portals and Web Technologies
• Grid Portal Portals Web the Grid
• Emerging Technologies
• The GridPort Toolkit
• GridPort-Based Portals
• Future Directions

4
Motivation for Computational Grid Portals
5
Why Use Portals for Computational Science?

• Computational science environment is complex
• Users have access to a variety of distributed
  resources (compute, storage, etc.).
• Interfaces, OSs, Grid tools to these resources
  vary and change often
• Environment changes
• Relocation/upgrade of binaries
• Policies at sites sometimes differ, allocations
  change
• Using multiple resources can be cumbersome
• Grid adds complexity for programmers and users
  portals can simplify if done well.
• Best Fit Community Models ? a lot of users to
  justify development effort

6
Portals Provide Simple Interfaces

• Portals are web based and that has advantages -
• Users know understand the web
• Can serve as a layer in the middle-tier
  infrastructure of the Grid
• Integrate various Grid services and resources
• Users can be isolated from resource specific
  details
• Single web interface isolates system
  changes/differences
• Not and end-all solution - several
  issues/challenges here
• Performance, scalability
• Tradeoffs

7
Simple Computational Grid
LSF

• Resource View
• Full Functionality
• Very Complex

8

• Portal View
• Hides Complexities
• Limits User Functionality

9
Grid Portal Portals Web the Grid
10
What is the Web?
11
Web Server Technologies

• Web Servers
• Run on a machine, and clients access the process
• Common Versions
• Netscape (http//www.netscape.com)
• Apache (http//www.apache.org) - open source
• OSs Windows, Unix, MacIntosh, Linux, etc.
• Web Programming Languages
• Server Java, Javascript, Python, PHP, Perl
• Client HTML, Javascript
• Protocols/Components HTTP, CGI, Servlets,
  Applets, Jetspeed/portlets
• Security HTTPS, SSL, Encryption, Cookies,
  Certificates

12
Web Clients

• Multiple display devices
• Desktop workstations, PCs, PDAs, cell phones,
  pagers, other wireless devices, televisions
• Various viewing tools
• Browsers Internet Explorer, Netscape Navigator,
  Opera
• Visually Impaired tools
• OSs Windows/WinCE, Unix, Mac, Linux, Palm, etc.
• Web Programming Languages
• HTML, Javascript
• Perl, Java for scrapers
• Security
• HTTPS, SSL, Encryption, Cookies
• Certificates

13
Portal Features and Capabilities

• Web sites that provide centralized access to a
  set of resources yahoo, msnbc, google
• Characterized by richer features than set of HTML
  pages
• Personalization
• Security/authentication/authorization
• What you see often changes based on what you are
  looking for (e.g. adds)
• Navigation/choices
• Gateway for Web access to distributed
  resources/information
• Hub from which users can locate all the Web
  content that they commonly need.

14
Classes of portals

• Horizontal or mega-portals
• information from search engines and the ISP's
  (yahoo)
• everybody comes in, sees the same thing
• allow personalization to some degree
• Vertical
• portals that are customized by the system.
• the system recognizes who you are, and gives you
  a different view of the university or the company
  that you're going to build.
• More specialized (amazon, travelocity, etc.)
• Intranet
• inside a company that give particular people the
  information that they need

15
Scientific Web Portals

• Use The Grid
• Three standard Types
• User Portals
• simplify users ability to interact with and
  utilize a complex, often distributed environment
• direct access to resources (compute, data,
  archival, etc.)
• Application Interfaces
• Enables scientists to conduct simulations on
  multiple resources
• Customized Portals
• Users can roll out their own portals by writing
  web pages using standard HTML or Perl/CGI,
  Java/JSP, etc.
• NOTE Open Grid Services Architecture (OGSA) is
  changing all of this (will address later)

16
Conceptual Web Grid
17
Reality GridPort Architecture
18
Portal Technology Choices

• Commercial more and more are adopting Grid
  (OGSA)
• Sun Java Servlets, iPlanet
• IBM WebSphere
• MSFT .NET
• AVAKI (A. Grimshaw, Legion) ? ahead of the curve
• Special interest groups
• JetSpeed (emerging grid portal standard),
  uPortal,
• RD within Grid community
• GCE-RG Common Portal Architecture portlets
  (future)
• GridPort Toolkit (http//gridport.npaci.edu)
• GPDK (http//www.nlanr.org ) Java based
• GridSphere (Ed Seidel, GridLab)
• Gateway (Fox)
• CCA (Gannon, large DOE project)
• GRADS (large NSF Project)

19
The GridPort Toolkit
20
GridPort 2.0

• Part of NPACkage
• Perl/CGI
• Easy to install
• Dynamic
• Multiportal architecture
• Account data
• manage certs/keys, session info for users
• Grid Globus, SRB, NWS, etc.
• Thin client

21
PACI HotPage

• Access portal to all resources
• Information Portal to all users
• Secure access for authorized users
• PACI Grid Software used
• Globus Toolkit(GRAM, GSI, GRIS, GIIS), SRB,
  MyProxy, NWS
• Built with the GridPort Toolkit
• GP 2.0 Perl/CGI
• Services provided
• Resource information/status
• job control
• data collection management,
• command execution
• personalization

22
Portal Accounts

• Portal accounts are not the same as resource
  accounts.
• valid Grid user on resource, need allocations
• processes run under own account with same access
  and privileges as if they had logged onto
  resource
• Portal users must have a digital certificate
  signed by a known Certificate Authority (CA)
• And must get DN into mapfile
• Accounts for NPACI users obtained via an on-line
  web form
• Can generate a certificate - certificate and key
  are placed in a secure repository

23
GridPort 3.0 GCE Portal

• Expanded CE Layer thin client, GCE Shell,
  Portals, Portlets, Apps, etc.)
• Distributed grid and web services (OGSA)
• Workflow interaction between components
• Component Approach
• need OOPs capability ? Java
• Python, PHP/Perl
• XML, database at core

24
Grid Technologies Employed

• Globus GT 2.x (NMI R1, R2 (also earlier versions)
• GT 3.0 in next version
• Security
• GSI is key enabling infrastructure
• Globus Grid Security Infrastructure (GSI), SSH
• MyProxy for remote proxies
• Job Execution
• Globus/GRAM Gatekeeper (key)
• used to run batch, interactive jobs and tasks on
  remote resources
• Scheduler Platform Computing (LSF,
  Multicluster)
• Integration with SGE, AVAKI, others (Texas grid)
• Queing systems include PBS and others

25
Grid Technologies Employed (cont.)

• Information Monitoring Services
• Globus MDS 2.2, GIIS, GRIS
• NWS, data from LSF, United Devices, etc.
• Web service based GIS archival system Grid-IAS
• Custom information provider scripts
• Grid Monitoring System (Java enhanced version of
  NCSA)
• File Management
• GridFTP --gt key technology
• SDSC Storage Resource (SRB)
• for file collection management
• Sun SAN between TACC/Campus

26
Emerging TechnologiesOGSAJetspeed/Portlets
27
Web Services

• Architecture mechanisms for
• dynamic service discovery (UDDI)
• Separation of implementation from function (WSDL)
• Knowx protocol (SOAP/HTTP, SOAP/RPC)
• Service provider encapsulates implementation
  details
• Client doesnt need details, just where/how to
  send request
• Commercial world developing P2P web services
• In some ways, Globus/GRAM is a web service
• Advantage language independent, so can run on
  any system
• Community pursuing Python, Java, C at this time

28
Open Grid Services Architecture

• IBM and Globus team integrated key concepts of
  Grid and web
• Taking Grid community to next level services
  are interoperable
• protocol based rather than implementation
• PROTOCOLSexamples telnet, ftp, ssh
• telnet
• Login
• password

• Grid
• Security (PKI, GSI)
• persistence stateless web is gone track task,
  user info, etc.
• Handles to instances
• Web
• HTTP transport layer
• Simple Object Access Protocol (SOAP)
• XML
• Web Services Description Language (WSDL)

29
OGSA Component Approach Workflow

• Grid Web services components
• Standard interface
• Dynamic composition and exchange of data

30
JetSpeed and Portlets

• New direction for grid computing portal community
  based on Apache and open source
• Uses Java plug-in software behind web servers
• Builds dynamic web pages based on client request
• Executes set of components (Java Portlets)
• Composites them into a web page
• Returns page to user
• Portlets exchanged by sharing code
• WSDL will be employed

31
Portlet-based Tools and Technology

• Provided Capability
• Management of user proxy certificates
• Remote file Management via Grid FTP
• Collaborations tools -News/Message systems
• Event/Logging service
• Access to OGSA services
• Specialized Application Factories
• Access to directory services and Metadata tools

See http//www.extreme.indiana.edu/xportlets
32
Jetspeed Gridport 2.0

• Demonstrates advantage of using Jetspeed portlet
  classes to create URL connections to Perl/CGI
  code which make calls to Gridport.
• This is the method we use to wrap Gridport with
  Jetspeed.
• Needed for backwards compatibility of existing
  NPACI portals
• Only minor modification made to Gridport
• Perl modules - authentication
• pass Jetspeed session datap-set Gridport cookies
• Current Progress
• Gridport Login/Logout
• Globus Run

33
Jetspeed Advantages

• Overall portal customization
• Java Portlet mini code perform tasks.
• Can install someone elses portlets
• Individual user customization
• This will fulfill a need for users to tailor
  their portal interface to their liking.
• Open Source
• Always being debugged, re-released.
• One downside of Open Source is that documentation
  is limited. But, tight user/developer community
  provides some assistance.
• Template interfaces such as Velocity and JSP
  allow for presentation layer to be separated from
  java program layer.

34
Wrapping Gridport

• By utilizing Jetspeed portlet classes we can
  create URL connections to Perl/CGI scripts which
  make calls to Gridport.
• There is some minor modification that needs to be
  made to the Gridport Perl modules.
• This is mostly passing session information in
  from Jetspeed and forcing Gridport to use that
  instead of its own.
• Current Progress
• Gridport Login/Logout
• Globus Run

35
http//gridtest.tacc.utexas.edu2080/jetspeed
36
Services grid computing portals (OGSA)

• Grid-Specific
• Security
• Account and allocation management
• Information, discovery and monitoring
• Resource scheduling and management
• Data and collection management
• Application support

• Portal specific
• view customization, user session management, and
  portal logging.
• Groups, roles, sharing, access control
• collaboration and communication systems -
  chat/instant messaging services, whiteboards,
  calendars, newsgroups, citation browsers
• Ubiquitous access browsers, cmd line, cell, pda.

37
Recommended Technologies

• Information Monitoring Services
• Globus MDS 2.2, GIIS, GRIS
• NWS, data from LSF, United Devices, etc.
• Web service based GIS archival system Grid-IAS
• Custom information provider scripts
• Grid Monitoring System (Java enhanced version of
  NCSA)
• File Management
• GridFTP --gt key technology
• SDSC Storage Resource (SRB)
• for file collection management
• Sun SAN between TACC/Campus
• Portals
• Java, Jetspeed, portlets, CC
• Web services (in addtn to grid)
• Database back end
• XML

• Grid
• OGSI/OGSA Globus 3.0
• Globus GT 2.x (NMI R1, R2 (also earlier versions)
• Security
• GSI is key enabling Techn.
• Grid Security Infrastructure
• MyProxy for remote proxies
• Job Execution
• Globus GRAM Gatekeeper (key)
• used to run batch, interactive jobs and tasks on
  remote resources
• Scheduler Platform Computing (LSF,
  Multi-cluster)
• Integration with SGE, AVAKI, others (Texas grid)
• Queuing systems PBS, LSF, etc

38
GridPort-Based Portals
39
Variety of GridPort Applications

• Current applications in production
• NPACI/PACI HotPages https//hotpage.npaci.edu
• Telescience (Ellisman)
• https//gridport.npaci.edu/Telescience
• Protein Data Bank CE Portal (Phil Bourne)
• https//gridport.npaci.edu/CE
• LAPK Portal Pharmacokinetic Modeling (live demo
  of Pharmacokinetic Modeling Portal)
• https//gridport.npaci.edu/LAPK
• GAMESS
• https//gridport.npaci.edu/GAMESS

40
PACI HotPage

• Access portal to all resources
• Information Portal to all users
• Secure access for authorized users
• PACI Grid Software used
• Globus Toolkit(GRAM, GSI, GRIS, GIIS), SRB,
  MyProxy, NWS
• Built with the GridPort Toolkit
• GP 2.0 Perl/CGI
• Services provided
• Resource information/status
• job control
• data collection management,
• command execution
• personalization

41
Telescience Access to Instruments/Data

• Uses GridPort to Integrate Telescience
  technologies with the Grid
• Access to instruments
• Globus job control
• SRB data collections
• Migrating to BIRN

42
NBCR GAMESS Portal

• Community Model
• XML database
• Access variety of compute resources
• Couples to proprietary visualization rendering
  services
• Part of multi-portal system

43
Future Directions

• Portals Workshop on Friday (open to all)
• Goal is to bring NPACI Portal developers users
  together
• GridPort 2.2 part of NPACKage, comliant with NMI
  program (NSF)
• Perl has no GSI security capabilities moving
  away
• Developing Jetspeed/Portlet solutions for
  GridPort
• Planning on pyGlobus version pyGridPort
• Collaborating with U. Mich, Indiana, Argonne,
  NCSA to develop grid portlet repository
• Developing GridPort GCE
• OGSA, Java/portlets, GCEShell interfaces

44
GridPort Project Team

• GridPort Project represents collaboration efforts
  spanning the PACI Program
• Mary Thomas, Jay Boisseau, Maytal Dahan, Eric
  Roberts, Tomislav Urban (TACC)
• Cathie Mills, Steve Mock, Kurt Mueller (SDSC)
• Charles Severance, Joseph Hardin (U. Mich)
• Dennis Gannon, Goeffrey Fox, Marlon Pierce
  (Indiana)
• Argonne/ISI Globus development team
• And input from other Institutions
• NASA/IPG
• GGF/GCE Research Group

45
References

• Related AHM Sessions
• Tutorial 7 SRB
• Tutorial 9 Grid Portals
• Parallel Session 2 (Weds) Grid Experiences
• Workshop on Portals (Friday)
• GridPort Toolkit Contact Mary Thomas
  (mthomas_at_tacc.utexas.edu)
• Project Websites http//gridport.npaci.edu
• Download http//gridport.npaci.edu/download
• HotPages
• https//hotpage.npaci.edu, https//hotpage.paci.or
  g