Backups

1
CIT 470 Advanced Network and System
Administration

• Backups

2
Topics

1. Backup Decisions
2. Types of Backups
3. Backup Hardware
4. Backup Software
5. Snapshots and CDP
6. Cron
7. Backup Security

3
Backup Decisions

• Why?
• Why are you backing up data? What would happen
  if you lost data and didnt back up? What types
  of data do you have?
• What?
• What to back upentire system, or specific
  filesystems? What OS to backup? What other
  things to backupMBR, LVM?
• When?
• When is the best time to backup? How often?
• Where?
• Where will backup occur? Where to store backup
  volumes?
• Who?
• Who is going to provide backup system? Who will
  do backups?
• How?
• How are you going to do backups? Tape, mirrors,
  off-site, etc.

4
Why Backups?

1. Accidental deletions.
2. Hardware failures.
3. Data corruption.
4. Security incidents.
5. Plan for the worst.
6. System catches fire.
7. Fire spreads to replicated systems.
8. Sprinklers destroy backup system in data ctr.

5
What to Backup?

• Backups for your backups
• Be able to restore your backup server (software
  backup volume db) in case its down.
• Which peripherals?
• How many drives per server?
• What is the capacity of each drive?
• How were they partitioned?
• Drive partitions must be same as before disaster
  for restore from backup to work.
• fdisk l
• lvmcfgbackup

6
Filesystem / Data Types

• Operating System
• Standard OS image on server.
• Software
• Software config files specific to server.
• Data
• Data files specific to server.

7
Backing Up Selected FS

• Saves media space and network traffic
• But OS is small compared to data today.
• Harder to administer
• Must remember or document which fs to backup for
  each server.
• Easier to split up between volumes
• Can easily distribute backup of a server across
  different backup volumes on a per fs basis.
• Worst case
• Forget to backup an important filesystem.

8
Backup Entire System

• Complete automation
• Create script to parse /etc/fstab and LVM, then
  backup every disk filesystem.
• Do this once, then it works on all servers.
• Worst case
• Increase network traffic by a few percent.

9
Backup Types

• Image level
• Backup raw disk partition or entire disk.
• Back up every byte on drive, used or not.
• Use compression to eliminate GBs of zeros.
• Cannot restore individual files.
• Filesystem level
• Backup files within filesystem.
• Backup tool must understand filesystem.
• Can restore files, no backup of unused blks.

10
Backup Types

• Full backup
• Complete copy of all files from a particular
  time.
• Backup slow, requires high capacity.
• Restore fast, simple.
• Incremental backup
• Storage of changed files since last backup.
• Backup fast, may store multiple per tape.
• Restore slow, complex (requires multiple tapes)

11
Capacity Planning Space

• Partition 40GB
• Full backup every week.
• Daily incremental backups.
• 50 full now, grows 2GB per day
• Tape capacity needed
• Day 1 40GB
• Day 2 2GB
• Day 7 12GB
• Day 8 40GB

12
Capacity Planning Time

• Fileserver 4TB
• Full backup must finish overnight (8 hours)
• Tape drive 40MB/s 144 GB/hr 1.15TB/night
• Need 4 tape drives running simultaneously.
• Additional concerns
• Network performance btw file backup
  servers.
• Does any capacity need to be reserved for
  restores?
• Actual performance vs. manufacturer specs.

13
Capacity Planning Media

• How much media do you need?
• Determined by policy and schedule.
• How long are full backups kept?
• How often are incrementals recycled?
• How often are tapes moved off-site?

14
Choosing a Backup Drive

1. Reliability
2. Flexibility
3. Transfer speed
4. Time-to-data
5. Capacity
6. Compatibility
7. Cost

15
Reliability

• MTBF
• Remember that drives fail faster than MTBF claims
  early and late in lifespan.
• Talk to people who have used system.
• Duty cycle
• Expected usage per day.
• 40 duty cycle 10 hours per day
• MTBF based on listed duty cycle.
• Hard drives are more reliable than tape.
• Closed system protects from contaminants.

16
Flexibility

• Flexibility means
• Able to respond to different data rates.
• Can be used in different ways.
• Tapes arent very flexible
• Typically require a standard data rate.
• Slower/faster rates result in I/O errors.
• Can only read/write sequentially.
• Disks are very flexibile
• Random access medium.
• Can change data rates rapidly.
• Combine with RAID or LVM for capacity or perf.
• Virtual tape software allows disks to appear as
  tape.

17
Transfer Speed

• Compare native sustained transfer rates
• Transfer rates often assume compression
• Burst or synchronous rates are temporary,
  best-case scenarios.
• Disks are much faster than tape.

18
Time to Data

• Time to Data
• How long to load a volume
• Seek to appropriate place on volume
• Start reading data.
• Load time can include
• Time to manually find and load tape.
• Time for tape robot to locate and load tape.
• Most restores are for a few files.
• User deletions.
• Time to Data matters more than Transfer Rate

19
Capacity

• Want one backup to fit on single volume.
• Easy to manage than backups across multiple
  volumes.
• Tape capacity grows slower than disk capacity.
• Cost
• Lots of small backups to a single volume.
• Reduces number of volumes to purchase store
• High capacity is faster
• Fewer volume switches when backing up.

20
Compatibility

• Want to be able to restore everything.
• If new system incompatible with old, must
  transfer old backups to new format.
• Use a single format for easy of management.

21
Backup Media

• Flash Memory
• Very expensive, small media, personal use only.
• Super floppies
• ZIP 750MB, small capacity, high /GB media.
• Optical
• CD-R cheap drives, slow small (650MB).
• DVD-R cheap drives, slow small (4.7GB).
• Ultra Density Optical (UDO) expensive but larger
  (60GB).
• Hard disk
• Large capacity (1TB), bulky, low /GB media.
• Tapes
• Large capacity (800GB), low /GB media expensive
  drives.

22
D2D2T

• Backup data first to disk
• Take advantage of fast disk speeds.
• Complete backups within nightly window.
• Copy backup disks to tape
• Copy backup data from disks to tape.
• Disks arent in production, so this can be slow.
• Reuse backup disks each night.

23
Current Tape Formats

• LTO (Linear Tape Open)
• Uses Ultrium format tapes.
• LTO4 800GB capacity, 120 MB/s, 49s seek
• Super DLT (Digital Linear Tape)
• Backward compatible with DLT formats.
• DLT-S4 800GB capacity, 60 MB/s, 79s seek
• SuperAIT (Advanced Intelligent Tape)
• SAIT-1 500GB capacity, 30 MB/s, 45s seek

24
Common Tape Features

• Form Factor
• 5.25 FH SCSI drives
• Media are 1/2 wide tape stored in cartridges.
• Hardware compression
• Usually cited as 21, some cite higher.
• Depends heavily on nature of data stored.
• Future Roadmaps
• Plans to double capacity in next few years.

25
Hardware vs Software Compression

• Software Compression
• Compress data via software before writing.
• Can use high compression tools like 7zip, bzip2.
• Lowers amount of data xfer across network.
• Higher CPU usage.
• Hardware Compression
• Compress using specialized hardware on tape.
• Does not require additional CPU usage.
• Increases throughput of drive.

26
Tape Autochangers

• Stackers
• Works sequentially through a stack of tapes.
• Autoloader / Jukebox
• Provides random access to set of tapes.
• Library / Silo
• Multiple drives w/ random access to set of tapes.
• May incorporate bar code reader, ethernet, etc.

27
Backup Software

• OS Provided (backup individual systems)
• cpio, dump, tar, ntbackup
• Open source (backup servers)
• AMANDA
• Bacula
• Commercial (backup servers)
• Tivoli Storage Manager (IBM)
• Veritas Storage Manager

28
Provided Backup Software

• UNIX/Linux
• tar
• cpio
• dump
• dd
• Windows
• ntbackup
• Windows Restore
• MacOS
• ditto

29
Windows Restore Points

• Auto backup of registry critical files
• Must have System Restore enabled.
• Done automatically can create manual too.
• Useful when software install has corrupted.
• Recovering Windows with a Restore Point
• Boot into safe mode.
• Select Restore my computer to an earlier time
• Choose date from list of restore point dates.
• If that doesnt work, reboot, try an older one.

30
Backup Software
Feature tar cpio dump
List files on vol Slowsearch entire backup (tar t) Slowsearch entire backup (cpio it) Fastindex at front (restore t)
Incremental backup Easy (-newer) Must use find Easy (set level)
List files as backed up tar cvf cpio v Only after backup
Compatibility Multi-platform Multi-platform with ASCII header Readable between some platforms.
Backup specific files Yes Yes Whole fs only
Stop reading tape after restore file is found No No Yes
Likelihood file exists in TOC but not in backup Low Low Medium (TOC is made before backup)
Recovery from I/O errors No No Yes
31
Open Source Backup Tools

• Amanda
• Single master backup server.
• Simple, fast, uses native backup tools.
• Bacula
• Client/server backup system.
• More advanced features but slower than Amanda.
• BackupPC
• Web-based so works with any client OS.
• Backs up PCs and laptops to disk on server.

32
Snapshots

• Virtual read-only copy of filesystem
• Snapshot has same contents that filesystem has
  when snapshot was made.
• Snapshot uses pointers to data and copy-on-write
  to avoid making a copy of entire fs.
• Snapshots require 1 of fs size, depending on
  updates.

33
Snapshot Applications

• Quick restore times
• Snapshots take seconds to create, restore from.
• Wont help you if disk or other hardware fails.
• Snapshot lifetime could be short as 1m or as long
  as a few days for this purpose.
• Staging for backups
• Snapshot filesystem before starting backup.
• Files on snapshot do not change during backup.
• Snapshot lifetime is how long it takes to backup.

34
LVM Snapshots

• LVM can create snapshots of logical volumes.
• lvcreate -L500M -s -n backup /dev/db/db1
• Creates snapshot (-s) volume named backup
• Volume is snapshot of /dev/db/db1 LV
• Can make 500M of changes.
• What happens if gt500M of changes?
• LV cant receive copies of old data if changes
  are made to original logical volume.
• Ensure 500M is more than enough space for changes
  made during lifetime of snapshot.

35
Continuous Data Protection

• Copy every file change to backup server.
• Stores changes in a log like RCS or a database.
• Can restore to any point of time.
• Near-CDP
• Snapshots replication.
• Does not provide a log, so can only restore to
  saved snapshots, not to any change like CDP.

36
Backing up Virtual Machines

• Back up VMs as physical machines
• Connect VM to your standard backup system.
• Have to configure backups for each VM.
• Back up VM files
• Can back up all VMs on host at once.
• VM files are constantly changing, so either
• Suspend VM
• Snapshot filesystem with VM files

37
Automation

• The key to efficiency and reliability.
• Use cron instead of manually backing up.
• Single tapes require manual media change.
• Tape libraries automate this process.
• Other automated tasks
• Monitoring (up/down, disk space, security)
• Logs (rotation, monitoring)
• File distribution

38
Cron

• Performs tasks at scheduled times.
• Crontab files specify schedule of tasks
• root /etc/crontab
• users /var/spool/cron/crontabs/
• Cron may log activities and errors.
• Timing limitations
• Runs tasks (if any) every minute.
• Does not perform scheduled tasks if system down.
• May or may not perform tasks on DST transition.

39
Crontab

• Format
• minute hour day month weekday user command
• Examples
• 30 4 0 root yum y update
• 3 root (cd /var/www make)
• 20 1 root /usr/local/rot-logs

40
Managing Automated Tasks

• Divide by time Hourly, daily, weekly, monthly
  tasks
• Crontab uses run-parts meta-script
• 17 root run-parts --report
  /etc/cron.hourly
• 25 6 root run-parts --report
  /etc/cron.daily
• 47 6 7 root run-parts --report
  /etc/cron.weekly
• 52 6 1 root run-parts --report
  /etc/cron.monthly
• Add crons by placing script in time directory.
• Add random delay if all hosts share same crontab.

41
Backup Security

• Tape security
• Tapes contain all of your important data.
• Data isnt secure unless tapes are secure.
• Solutions tape vault, encrypted tapes.
• Backup server security
• Has read access to all important data.
• If backup server isnt secure, data isnt secure.
• Solutions integrity checking, least privilege
• Restore process
• Who can request files to be restored?
• Where will restored file be placed? What will
  its ACL be?

42
References

1. AIT, AIT Forum, http//www.aittape.com/index.html,
   2006.
2. Lynne Avery, A Brief History of Tape, Exabyte
   white paper, http//www.kontron.com/techlib/whitep
   apers/A_brief_history_of_tape.pdf, 2000.
3. Aeleen Frisch, Essential System Administration,
   3rd edition, OReilly, 2002.
4. LTO, http//www.ltotechnology.com/newsite/index.ht
   ml, 2006.
5. Peter McGowan (ed), Quantum DLTape Handbook,
   http//downloads.quantum.com/sdlt320/handbook.pdf,
   2001.
6. Evi Nemeth et al, UNIX System Administration
   Handbook, 3rd edition, Prentice Hall, 2001.
7. Shelley Powers et. al., UNIX Power Tools, 3rd
   edition, OReilly, 2002.
8. W. Curtis Preston, Backup Recovery, OReilly,
   2007.
9. Quantum, Tape Storage Automation,
   http//www.dlt.com/storage/whitepapers/quantum/dlt
   /Tape20Storage20Automation.pdf
10. The Tao of Backup, http//www.taobackup.com/
11. Wikipedia Contributors, Magnetic Tape,
    http//en.wikipedia.org/wiki/Magnetic_tape, 2005.
12. Elizabeth Zwicky, Torture Testing Backup and
    Archive Programs, ftp//ftp.berlios.de/pub/star/t
    estscripts/zwicky/testdump.doc.html, 1991.