Virtual Workspaces in the Grid

1
Virtual Workspaces in the Grid

• Kate Keahey
• keahey_at_mcs.anl.gov
• Argonne National Laboratory
• Ian Foster, Tim Freeman, Xuehai Zhang, Daniel
  Galron

2
The Grid Metaphor
Grid Computing is much harder heterogeneous
and multi-dimensional
3
The Missing Link in Grid Computing

• We need to define mechanisms for and dynamic
  deployment and management of remote environments

• Requirements
• Flexibly define an environment
• The more we can customize it, the more useful it
  is
• Deploy and manage such environments
• Can such environments be deployed securely?
• How fast/dynamic can this deployment be?
• How can I control resources allocated to such an
  environment?

4
Virtual Workspaces

• Virtual Workspaces environments that can be made
  available dynamically the Grid with
  well-understood properties
• Examples
• A TeraGrid node with well-defined software
  environment and adjustable access and sharing
  policies
• A physical cluster booted to a desired
  configuration (e.g. Cluster on Demand)
• An ATLAS node dynamically configured using
  Pacman
• A virtual machine configured to represent a
  specific environment whose resource consumption
  can be controlled

5
Virtual Machines as Workspaces

• Virtual Machines
• Highly customizable software configuration
• Enforcement properties
• Grid 2004 paper Dynamic environments in the
  Grid
• F. Cappello lab Comparison of different
  hypervisors
• Pausing, serialization, migration
• Performance

SOSP 2003 paper Xen and the Art of
Virtualization
6
Workspace Template Aspects

• Environment Aspect (workspace meta-data)
• Generic information
• Name, time to live, etc.
• Software partition information
• Software description OS, OSG configuration,
  application partition, etc.
• Software meta-data is bundled with the actual
  software and attested by its issuer
• Services ssh, GRAM, pre-configured job
• Deployment independent
• Resource allocation request (deployment time)
• Memory, disk, networking, etc.
• See GGF JSDL standard
• On deployment the actual resource allocation
  information becomes available

7
Atomic Workspaces and Virtual Clusters

• Atomic workspace
• One or more homogeneous workspaces
• The only differences are in names
• Cluster/aggregate workspace
• A set of interdependent heterogeneous workspaces
• Example a headnode and a set of worker nodes
• Interdependencies of metadata are expressed
  through tags and pointers

8
Deploying Workspaces in the Grid

• Define workspace environment
• Manage workspace
• Negotiate workspace deployment characteristic

Workspace Wizard (VW Factory)
manage workspace environment
Workspace Management Service (VW Repository)
workspace metadata
Workspace Service (VW Manager)
terminate workspace deployment
manage activities within the workspace
9
Current Implementation

• Current prototype using Globus Toolkit 4
• Leveraging standard Grid Service features such as
  lifetime management
• Workspace Wizard
• Returns workspace meta-data
• Very rudimentary implementation
• Workspace Serivce
• Create takes workspace meta-data and a
  deployment descriptor
• Manage
• renegotiate resource allocation (moving towards a
  WS-Agreement model)
• Also traditional Grid Service management TTL,
  etc.
• Destroy
• Different options pause, shutdown or destroy

10
How dynamic is the deployment?

• Automatic
• Protocol-based
• Moving towards better articulation of migration
• Renegotiation of resource allocation
• How fast is this deployment?
• Deployment of workspace for EMBOSS suite
• Manual 45 minutes
• Based on pre-configured Vmware VMs 6 minutes
• Based on pre-configured Xen VM
• How much overhead does workspace deployment add
  over what we have today?

11
Workspace Service Individual Workspaces

• GRAM job execution
• GRAM job execution in a paused Xen VM
• job execution in a booted Xen VM (pre-configured
  job)

• Using a paused VM allowed us to save on
  initiation time

12
Workspace Service Virtual Clusters
13
Deploying Workspaces Across Technologies

• Basic node configuration (/-boot from image)
• Cluster on Demand, PXE, bcfg
• On the order of many minutes (30 minutes)
• Refining configuration, creating access
• Dynamic account with workspace service (mostly GT4 request processing time)
• Refining Installation 2 hours to configure an
  ATLAS node using Pacman
• Virtual machines
• Deploying images
• Xen 100 ms
• VMware Workstation several seconds

14
Nested Workspaces
15
Computational Grids
16
Conclusions

• We need mechanisms for dynamically deploying and
  managing environments in the Grid
• Workspaces are a fundamental building block of a
  Grid environment
• Workspaces are implemented using wide variety of
  technologies
• VMs are a highly promising one a computon for
  the Grid
• Workspace aspects
• Deployment-independent environment definition
• Deployment-time policy and enforcement
  negotiation
• Many challenges remain
• Security and deployment issues
• Protocols, protocols, protocols
• Leveraging the opportunities