CSS Storage Services

1
CSS Storage Services

• CD/CSS
• Central Services Activities Coordination Meeting
  9/19/2006

2
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

3
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

4
Storage Services - Priorities

• Reliability
• Data integrity
• Service availability
• Manageability
• Cost
• Performance

5
Storage Services
IP

FCP
Backups
AFS Global Access lt25MB/s
NFS/CIFS Campus Access 200-300MB/s
SCSI Local to FCC gt200MB/s per lun
SAN
Fabric-A
Fabric-B
NDMP
Tier 1 RAID 5 Fibre-Channel
Tier 2 RAID 5/6 SATA
6
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

7
Tiered Storage Pool

• Redundant components
• Tier 1 - RAID 5 Fibre Channel disk
• High I/O applications
• Single disk fault tolerant
• Tier 2 RAID6 SATA disk (some RAID5)
• Low to medium I/O applications
• Double disk fault tolerant

8
Direct SAN Attach - FCP

• Block-based access (SCSI)
• Must be in FCC (500m limit)
• 2Gbps Fibre-channel network
• Multi-attach for 24x7 service
• Support for Linux, Solaris, IRIX, Windows

9
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

10
NAS Services - AFS

• Global filesystem
• Throughput lt 25MB/s
• Secure
• Communication can be encrypted
• Kerberos authentication
• Directory ACLs
• Limited to 8GB volumes

11
NAS Services NFS/CIFS

• Campus-wide access only
• CIFS off-site via VPN possible
• Throughput
• 200MB/s write
• 300MB/s read
• HA cluster
• Many virtualization services

12
NAS Service Overview

• 2 Titan-21 NAS Gateways (Physical)
• 2 Enterprise Virtual Servers (EVS)
• Each EVS has its own IP address
• Each EVS provides NFS, CIFS and FTP services
• An EVS can run on either Titan (HA Failover
  Capability)
• All storage provided by SAN
• 4 Gb/s connection to 256 Gb/s LAN core
• high performance for all customers

13
NAS Overview

• Storage Pools
• Composed of LUNS (units of physical storage
  provided by SAN)
• File Systems
• Multiple file systems reside in a pool
• Different block sizes (large or small files)
• Virtual Volumes (ViVols)
• Multiple ViVols reside in a pool
• Key to thin provisioning

14
NAS Virtualization Services

• Zero downtime capacity upgrades
• Thin Provisioning
• Snapshots
• Policy Based Data Migration

15
Capacity Upgrades

• DAS Capacity Upgrade
• Move server to a new rack to fit new chassis
• Install new RAID channels to support new RAID
  chassis
• Install new RAID chassis
• Create new file systems
• Migrate / Rebalance data
• Downtime (Lots !!)

16
Capacity Upgrades

• NAS Capacity Upgrade
• Increase size of Vivol (best case)
• Add LUNs to storage pool (worst case)
• No need to move server
• No need to install new RAID channels
• No need to create new filesystems
• No need to migrate and rebalance data
• Downtime (Zero !!)

17
Storage Provisioning

• DAS Style Storage Provisioning
• Each Project/Dept has its own server
• Each server has its own disks
• Each server has its own free space pool
• Storage provisioning predictions are often wrong
• Some projects, plenty of available excess(sum of
  total free space across all servers TB)
• Others, heavily usedstarved for space
• New project new disk purchase
• Sorry . Sharing not possible

18
Storage Provisioning

• NAS/SAN Style Storage Provisioning (Thin)
• All Projects/Services share the sameSAN
  connected storage
• ViVol parameters can be tuned to over provision
  physical storage
• New Project, no problem
• Get started now
• Defer storage purchase until later
• Easier to defer and lump storage purchases
• Better Prices

19
Snap Shots
\.snapshot
\Live-Files
20
SNAP Shots

• Why snapshots are cool
• Users can recover their own data immediately
• No need to submit helpdesk ticket

21
Policy Based Data Migration (PDM)

• Ability to automatically migrate data from
  expensive tier 1 to tier 2 disk
• Does not interrupt users
• Various Migration Criteria
• Age
• Size
• Type (avi, jpg, tiff ..)
• Various Analysis Schedules

22
NAS Status

• Windows
• Backups, Printer Migrations
• Soon
• CSS Home directories
• PSEEKITS soon
• VMSSTREAMER (data only)
• CDSERVER (printers first)
• PPDSERVER (printers first)
• VMSSERVER1 (printers, www services first)
• Others

23
NAS Status

• NFS
• CMS, OSG, Scientific Linux
• Many customers, Some unexpected growth (new NAS
  on order)
• Can I use the NAS for my project
• YES Its a site service
• Costs
• Small projects . None
• Large projects . share portion of disk, tape
  costs (no management costs)

24
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

25
Strategic Direction
Connection
Access
Virtualization
Dynamic Resize
Dynamic Tuning
Thin Provisioning
Thin Provisioning
Automated Tiering
Automated Tiering
Snapshots
Storage Pool
RAID Storage
26
Additional Information

• http//computing.fnal.gov/nasan
• Questions?

27
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

28
Backups

• TiBS True Incremental Backup System
• Only 1 full network backup
• All other backups are incremental
• Lightens client workload
• Use previous backups to create new fulls
• No per-client connect costs

29
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

30
TiBS Supported Systems

• Servers
• IRIX, Linux, Mac OSX, Solaris, Windows
• Client code must be installed
• NAS appliances via NFS

31
TiBS Unsupported

• Sparse Files (databases)
• Needs to be dumped to flat file
• Compressed files as filesystem
• Swap files
• Desktops
• Raw devices/block-level backups

32
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

33
TiBS How it works
Disk Cache
Full
Mon
Wed
Full
M-T Merge
M-T-W Merge
New Full Merge
Mon
Tue
Previous Full
Previous Full
34
TiBS Tape Rotation

• Full - 84 days (mirrored)
• Retained 1 year
• Monthly 28 days (mirrored)
• Retained 1 year
• Weekly 7 days
• Retained 14 weeks
• Incremental Daily
• Retained 56 days

35
TiBS - Recoverability

• 1 Full tape lost
• Use other mirror to recover
• Both Full tapes lost
• Use current monthly and previous full to recreate
  full
• Both Full tapes and both month tapes lost
• Use current weekly and previous month to recreate
  new current month
• Use new current month with previous full to
  recreate new current full

36
TiBS Actual size of data

• Multiple copies of data on tape
• Mirroring
• Retention policies
• For one year, size ratio is 141
• Tape vs. disk for static data. More for
  dynamic data.

37
Agenda

• SAN/NAS services
• Priorities
• SAN
• NAS
• Strategic Direction
• Backups - TiBS
• Supported
• How it works
• Stats

38
Stats and Facts

• 415 servers being backed up
• BSS, CMS, CSS, DCG, DIR, DSG, ESE, ESH, EPP,
  FESS, MIPP, QCD, REX
• 6770 AFS volumes backed up
• 15 TB of data 1 full backup
• Clients can recover from bare metal
• Limited availability for new customers
• Tape resources

39
More Information

• http//computing.fnal.gov/site-backups
• Questions?