Consolidating Data Management with Enterprise Storage and Backup Systems

1
Consolidating Data Management with Enterprise
Storage and Backup Systems

• Barry Ribbeck Director, Systems, Architecture
  Infrastructure, Rice University
• Hubert Daugherty Manager, Digital Media
  Services, Rice University
• Susie McClure Systems Analyst, Rice University

2
Rice University

• Carnegie R1 Designation
• Ranked in the top 25 schools in US
• 5000 Students
• 1100 Faculty
• 1500 Staff

3
Enterprise Storage and Backups

• Enterprise - a distributed resource provided
  centrally
• Storage - a place to put your stuff
• Backup - Describes what you are doing when you
  are not moving forward. When it works, it can be
  incorporated into your career progressions plan.

4
A Large New Network Project

• Provided a funding opportunity
• Defined storage as part of the critical need
• Provided the required network infrastructure for
  enterprise storage (Data can live anywhere)
• A fresh look at technologies for enterprise
  storage and backups

5
Data Assessment Findings

• Many distributed resources managed inconsistently
• No consistent backup strategy
• Few economies of scale
• Data management plan buy more disks
• Growth Rate of digital storage needs was
  increasing
• No differentiation of data retention or DR
  requirements
• Old, outgrown and dying backup hardware

6
The Available Resources

• Very talented and knowledgeable staff
• Funding
• Support from administration
• Understanding from the campus

7
The Challenges

• Changing EVERYTHING at once
• In order to transition, we would have to keep 2
  systems operational with the same amount of staff
  who are already over taxed with all of the other
  changes
• New skill sets required to maintain new resources

8
Goals

• If possible, the changes MUST have a positive
  visible affect on the customers
• Storage must be able to scale (1/2 PB in 7 years)
  with no step function costs.
• Support for Information Lifecycle Management
  (ILM)
• Must support a Multi-tiered storage model
• Must be a model capable of integrating multi
  vendor storage
• Support costs defined for 5 years
• Staged backups with only full backups to Tape

9
RFP to Storage Vendors

• SAN with NAS Gateway design defined
• Invited 7 vendors to respond to the RFP
• Very wide dispersion of cost vs benefit
• Bluearc Titan carried the day with a big iron
  model and 50 TB of initial storage

10
Defined Storage Usage

• Email Mailbox storage (NO QUOTAS!)
• Client Home directories and shares
• 5000 students, 1000 faculty, 1500 staff, 1000
  guests (with quotas)
• Database storage Oracle, MSQL, Postgress
  (Sakai, Coldfusion,etc..)
• Video Archives
• Research Data Scratch space

11
Backups

• Remember Katrina Rita?
• Tape is like Listerine
• We needed a MAID to clean up our backup processes
• What about Disruptive Technologies?
• Hardware choices
• Copan Systems Sun/StorageTek
• Software choice
• Commvault

12
Storage Details

• San Storage components
• Positives
• Standards Based
• Multiple vendors enable competition
• Negatives
• Difficult to modify SAN architecture
• Expensive network components
• Nas Gateway
• Provides storage abstraction and management

13
Systemic Performance
Which System Elements Need Additional
Optimization?
14
Systemic Performance
Surprise, its the regular stuff!
15
Systemic Performance
Dual 6509 frames with 2 dedicated 10G interlinks.
One for layer 2 and one for layer 3.
16
Physical / Logical Architecture
17
What we Learned

• The integration of a lot of new technology took
  longer than anticipated even though all of the
  vendors supported well understood standards like
  NDMP.
• Outsourcing the integration would have been a
  mistake due to the complexity and the need for in
  house knowledge.

18
Bluearc Lessons and Vendor Report

• Clients cant compete
• Outages MUST be coordinated with everything that
  uses the system
• Clustered Titan nodes are a requirement
• Vendor
• Great support very responsive, good technical
  people
• Work well with other vendors
• Problems we have worked through
• NDMP, Replication, LDAP, still some to things to
  address, getting them to understand Enterprise in
  HE

19
Copan Lessons and Vendor Report

• Good Product very reliable
• Good support team, works well with other vendors
• Good hardware, very robust, are meeting their
  claims. 1 drive and 1 shelf failure in 1 year.
• Software is solid for what we use it for

20
Sun Lessons and Vendor Report

• Very good support
• Very good product.
• Robot and disk drives failed and were replaced
  and were quickly replaced under support contract.

21
Commvault Lessons and Vendor report

• Software difficult to get installed even with
  their professional services
• Support is lacking in depth but they are very
  willing to work with us.
• Windows backups work very well
• Support works well with other vendors
• Integration was very difficult and time consuming
• Software patches were very frequent and
  problematic
• Support tracking system is terrible
• NDMP support is very challenging
• When the product works, it works well
• Documentation was poor

22
Contact Information References

• Barry Ribbeck Barry.R.Ribbeck_at_rice.edu
• Hubert Daugherty hd_at_rice.edu
• Susie McClure smcclure_at_rice.edu
• www.bluearc.com
• www.copansystems.com
• www.sun.com (Storagetek)
• www.commvault.com
• www.inphase-technologies.com