Reflections on Production Grids Enterprise and Research

1
Reflections on Production Grids Enterprise and
Research

• Frederica Darema ltfdarema_at_nsf.govgt
• Dynamic Data Driven Application Systems (DDDAS)
• Division of CCF
• Fillia Makedon ltfmakedon_at_nsf.govgt
• Office of Cyberinfrastructure
• National Science Foundation
• Global Grid Forum, Athens,Greece

2
Outline

• Production Grids for Science and Engineering
• Research applications
• Challenges
• Enterprise applications
• Non homogeneous architectures
• Heterogeneous solutions
• Interoperability
• Servicing the user
• Service oriented architectures
• Balance optimality with flexibility
• Enterprise applications driving grid computing
• Utility computing
• Resource Brokers
• Coordinate distributed resources and users

3
Production Grids for Science and Engineering

• Primary task to enable large-scale scientific
  research
• Not just a test-bed for software development, or
  experimental computer science.
• Severe constraints in construction because
• access to resources subject to stringent
  requirements of security and high quality of
  service.
• Production grid resources may not be for Grid use
  by design.
• Production Grid middleware cannot control policy
  on such Grids, it must co-operate with the site
  policy and resource management systems.
• Challenge in dealing with non computational
  resources, (e.g., telescopes and experimental
  facilities) that do not have a full operating
  system, and may have policy and management
  requirements that are very different from each
  other and from a computational node.

4
Research applications

• Life science problems
• E.g., protein folding requires enormous
  quantities of multiprocessing power
• self-deploying multiprocessing support is
  essential (e.g. SDSCs Rocks allow for affordable
  human resources).

5
Challenges

• Unlike supercomputers, clusters, servers and P2P,
  grids have heterogeneity, flexibility and
  reliability and not a single control
• A grid virtualizes resources
• Resources may be quite different in character and
  capabilities
• May come and go without warning as their
  availability changes over time.
• Massive volumes of data and increasing
• in e-business Web site operations, customer
  relationships, management applications, financial
  services
• Need to mine millions of rows of data.
• IT professionals can obtain serious computing
  power from low-cost components.

6
Non homogeneous architectures

• Production Grids have multiple architectures,
  clusters of workstations, specialized
  massively-parallel machines, clusters of
  shared-memory nodes, and machines with vector
  rather than scalar processors
• Different operating systems and no common set of
  software
• Grid middleware used to combine them in a virtual
  organization
• Constraints exist in providing reliability and
  high quality of service and availability
• Important to Monitor the Grid
• Establish a set of Core Grid Functions

7
Heterogeneous solutions

• Heterogeneous solutions in business force users
  to remember many logins, such as,
• Oracle E-Business Suite for financial and order
  management
• Siebel for customer relationship management
  (CRM)
• SAP for inventory management
• PeopleSoft for Human Resources and multiple
  applications.

8
Interoperability

• Interoperability is Key to extending Grids across
  organizational boundaries.
• Issues
• Diverse hardware resources (supercomputers,
  clusters, databases and devices)
• Application software, services, files and data
  archives need to work with diverse hardware
• Complex middleware for job submission, software
• Application projects that require access to
  resources in multiple grid systems
• Data as important as computation data-handling
  and transfer issues must be part of core
  functionality.
• This is accomplished in three ways
• 1) A set of uniform core software services that
  manage and provide access to heterogeneous,
  distributed resources,
• 2) a widely deployed infrastructure, and
• 3) higher level services like the Data Grid tools

9
Servicing the user

• Information services
• Help user discover, track and provide information
  on Grid resources
• vital to Grid Infrastructure
• Example How does a user identify and locate the
  information that is important to her?
• Obstacles in extracting useful and meaningful
  information
• Dynamically changing heterogeneous resources
• Large amounts of disparate information about the
  Grid
• Enterprise Portal a single source of interaction
  with corporate information and for day-to-day
  business.
• key components of Business Process Management
  (BPM), Enterprise Application Integration (EAI),
  and Business Activity Monitoring (BAM)
  initiative
• Single sign-on (SSO) for users to access multiple
  applications.

10
Service oriented architectures

• Evolving applications
• from monolithic, closed systems to modular, open
  systems with well-defined interfaces.
• Middleware complexity remains an obstacle to
  meeting business demands (e.g., Oracles solution
  with the Application Server)
• service-oriented architectures to design and
  integrate with existing legacy systems and
  business applications.
• Like web services attract enterprise users and
  applications on demand, grids can deliver
  computational resources on demand.

11
Balance optimality with flexibility

• Do not ask whether a grid is the right model for
  any particular task but what is the latency and
  bandwidth requirements of an application
• Some problems require a large shared memory best
  provided by a multiprocessor server with the
  fastest possible interconnections.
• Other problems, best handled on a cluster or
  grid using low-cost compute nodes with affordable
  Ethernet connections.
• balance between optimality for one task and
  flexibility for many tasks

12
Enterprise applications driving grid computing

• Supercomputing on sale
• cost-push of a commodity technology, rather than
  the demand-pull of problems worth solving at any
  price.
• The bang for the buck good price/performance
  ratio in building a grid of high-density x86
  blade servers running Linux-based operating
  systems.
• Grid raw computing power
• Example off the shelf solutions entering the
  market
• Oracles Real Application Clusters technology,
  with its Cache Fusion architecture
• Globus Toolkit, an open-source "service factory"
  framework providing state maintenance and
  discovery tools
• Utility grid computing

13
Utility computing

• In the real world, grids provide utilities (e.g.,
  gas, electricity or water) on-demand to consumers
  who will pay for them.
• "Utility computing is a model of how you pay for
  computing resources.
• It's purchasing computer resources in a
  pay-per-use model.
• Grid and utility are complementary concepts.
• The grid is the infrastructure for sharing
  resources.
• Utility is the concept of paying for what you
  need.
• Appeals to government agencies that experience
  seasonal spikes in demand, which require more
  power but may not justify purchasing -- or the
  agency simply can't afford -- new hardware.

14
Resource Brokers

• Dealing with dynamic, multi-institutional virtual
  organizations A set of individuals and/or
  institutions defined by set of sharing rules form
  a virtual organization.
• Social and policy issues Negotiate
  resource-sharing arrangements among a set of
  participating parties (providers and consumers)
  and then use the resulting resource pool for some
  purpose.
• Resource Brokering Strategies
• Sharing not only file exchanges but also access
  to computers, software, data, and other
  resources, as needed by a collaborative
  problem-solving application
• Highly controlled sharing with resource providers
  and consumers defining clearly and carefully just
  what is shared, who is allowed to share, and the
  conditions under which sharing occurs.

15
Coordinate distributed resources and users

• Tools to integrate and coordinate resources and
  users that are not centrally controlled
• Address distributed issues security, policy,
  payment, membership, etc
• Refine multi-purpose grid protocols and
  interfaces by addressing authentication,
  authorization, resource discovery, and resource
  access.
• Make protocols and interfaces standard and open
  in order to deliver nontrivial qualities of
  service
• relate to response time, throughput,
  availability, and security, and/or co-allocation
  of multiple resource types to meet complex user
  demands
• Make the utility of the combined system
  significantly greater than that of the sum of its
  parts.

16
Enterprise Grid
17

• What is EG
• Whats difference with GRID
• Open problems
• Future Issues

18
What is EG

• A GRID technology focuses on Enterprise level.
• Some samples
• Alchemi a framework for EG
• Entropia
• Grid MP
• SETI _at_ home
• Condor in enterprise

19
Whats difference with GRID

• Different priorities to traditional Grid
  community
• EG does not require best efforts as GRID
• Still focus on Mission critical application
• Be economic
• Better performance

20
Open Problems

• Privacy protection
• How to share computation but not privacy
• Security in Enterprise-level
• How to get high performance

21
Reference

• Enterprise Grid Alliance www.gridalliance.org