Grid Computing in North Carolina: Past and Present

1
Grid Computing in North CarolinaPast and
Present

• SURA Cyber-infrastructure Workshop
• Georgia State university
• January 6, 2005
• MCNC Grid Computing and Networking Services
• Phil Emer
• Chuck Kesler

2
MCNCs Role in Grid

• MCNC is a service provider
• Manages production infrastructure
• For RE community across NC
• So to us, grid is
• Infrastructure
• an access method
• A service delivery platform
• MCNC is the experiment support center for the
  National Lambda Rail

3
NC Research and Education Network
Duke (GbE)
NCSU (GbE)
Raleigh
RTP
7609
7609
Qwest Internet
Level3 (GbE)
Abilene
Greenville
Greensboro
UNC-CH (GbE)
ECU ECSU CMST
UNC-G NCAT ASU
OC48 SRP Ring counter-rotating ring lt50ms
reroute Fully active redundancy
Winston-Salem
Fayetteville
FSU UNCP
WFU WSSU NCSA
UNC-C
Wilmington
Charlotte
UNCW
Qwest
Asheville
OC12 SRP Ring counter-rotating ring lt50ms
reroute Fully active redundancy
Level3
UNCA WCU
Greenville
4
Toward Grid

• We have many administrative domains
• We have a network of distributed points of
  presence
• We provide access to shared resources which are
  distributed
• So conditions are favorable for attaining a state
  of grid-ness
• What we need is an exercising application
• NC BioGrid!

5
Grid Computing in North CarolinaPast and
Present

• Chuck Kesler
• jckesler_at_mcnc.org
• January 2005

6
The Grid Revolution in NC

• Proving ground for Grid
• Successful prototype apps
• Catalyst for collaboration
• International recognition

NC BioGrid
2002 2003 2004 2005
7
Why Bio Grid? (circa 2002)
Its about staying ahead of the curve...

• Moores law has allowed labs to keep ahead of
  data, but sequence data is now outpacing
  processing capability
• Biotech and pharma industries are highly
  competitive and capital intensive
• Getting ahead and staying ahead of the
  competition will require the creation of new and
  unique capabilities

8
The NC BioGrid Partnership

• NC Biotech Center
• Provided the catalyst through the NC Genomics
  Bioinformatics Consortium
• MCNC
• Provided the funding and dedicated staff
• Sun
• Donated infrastructure hardware
• Established Sun Center of Excellence in
  Bioinformatics
• IBM
• Donated human capital (application developers)

• Triangle Universities
• Focal point for the collaboration
• Brought early adopters to the table
• Created collaborative working groups

9
NC BioGrid Accomplishments

• In the Summer of 2002, installed a dedicated
  testbed for evaluating grid middleware and
  developing grid applications for bioinformatics
• Testbed spanned multiple administrative domains
  with systems located at MCNC, NC State, UNC-CH
  Duke, and included representative heterogenity of
  hardware and OS platforms found at those sites
• Employed best of breed approach to grid
  middleware deployment
• Working groups met up to twice a month during
  2002-2003
• Created several pilot applications using the
  testbed

10
NC BioGrid MiddlewareBest-of-Breed Approach
11
NC BioGrid - Data Grid

• Avaki 4.0 Data Grid
• Federation of data providers across the WAN
• Provides a global name space for user home
  directories, shared project spaces, databases,
  and applications
• Ability to have results from canned SQL queries
  show up as files in the global name space
• Variety of access methods
• Web-based user interface
• NFS and CIFS through local data grid access
  servers to provide access at the native OS level
• Simple deployment
• No kernel mods required
• Each site can run a share server to distribute
  their local home and project directories to the
  grid
• Web-based management interface

12
NC BioGrid - Compute Grid

• Globus Toolkit
• NSF Middleware Initiative (NMI) V2 (Globus 2.4.3)
• Provides gatekeeper functionality for
  submitting jobs through to the local cluster
  manager
• Provides GridFTP support for file transfer
• Provides MDS to track grid resource
  characteristics
• MCNC provides infrastructure services
• Certificate Authority (initially based on the
  Globus SimpleCA)
• GIIS (master resource directory for the grid)

13
NC BioGrid - Web Portal

• CHEF/OGCE a grid portal framework
• Implements web-based interfaces for managing job
  submissions, file access, and online meetings
• Originally developed as a distance learning tool
• MyProxy security credential repository
• Provides the portal with a mechanism for
  accessing and using Globus security credentials

14
Portal Example
15
NC BioGrid Proof of Concept Applications

• Parameter Space Study with BLAST
• BLAST compares a target gene sequence against a
  known genome to find similarities
• Grid BLAST distributed 1,000 target sequences
  across the grid for comparison
• IBM Extreme Blue Project
• Built a grid interface to BioPerl libraries
• UNC-CH/IBM QSAR Application
• Grid-enabled version of a drug compound screening
  application
• Finds compounds that have promising biological
  activity characteristics that should receive
  further research

16
The Grid Revolution in NC

• Apply NC BioGrid lessons
• Cluster and SMP resources
• Research platform for GTEC
• Core component in NCGrid

MCNC Enterprise Grid
NC BioGrid
2002 2003 2004 2005
17
The MCNC Enterprise Grid
Global Grid Resource DB (GIIS)
Users
Campus Grids
Portals
(FIREWALL)
Interactive Nodes / Grid Gatekeeper / GridFTP
LSF Master Job Scheduler
32-CPU SGI Altix Linux SMP Server
128-CPU IBM Linux Cluster (64 nodes)
8-TB Storage
18
The Enterprise Grid and MCNCs Services Strategy
GTEC, NLR, ANR and other Innovation Initiatives
DEPLOYMENT
State-wide Grid Services
Enterprise Grid Services
Value-add Information Systems Services
Self-serve Data Center Services
Information Security Services
Data Archival Services
Hosting Infrastructure
Grid Computing
Information Assurance
DATA CENTER
NCREN
19
The Grid Revolution in NC

• State-wide partnership
• Leverage lessons learned
• Grid education training resource
• Enable first mover applications

NC Grid Initiative
MCNC Enterprise Grid
NC BioGrid
2002 2003 2004 2005
20
NC Grid A Grid for Grid Developers(for now, at
least)

• Provide a development testbed that spans the
  state
• Multi-institutional resources
• MCNC offers the Enterprise Grid as a resource
• MCNC is also developing a grid appliance, which
  can be easily deployed and remotely supported as
  a campus or department point of presence on the
  grid
• Currently the community is working together to
  determine the middleware stack
• GT4 vs. GT3
• OGCE vs. GridSphere
• CA architecture
• Data grid strategy
• Platform standards
• etc...

21
A Sampling of CurrentGrid Projects in NC

• GridNexus at UNC-W
• Workflow builder for grid applications
• www.gridnexus.org
• SCOOP
• UNC-CH, RENCI, and MCNC
• Portal and grid infrastructure for running ADCIRC
  model
• BioPortal
• RENCI at UNC-CH
• Grid Computing CS Course
• Offered by WCU to campuses across the state via
  NCREN video service
• First offered in Fall 2004 (30 students), to be
  offered again in Fall 2005