DISASTER RECOVERY

1
DISASTER RECOVERY

• KASBO SPRING CONFERENCE
• 2009

2
When Disaster Is On Its Way

• Monitor weather forecast for projected
  arrival/severity/duration
  
• Notify Disaster Recovery Vendor of impending
  disaster
  
• Provide notice of potential need for recovery
• Schedule timing of their backup ref. storms
  arrival
  
• Request archival of backup
• Note contact persons information keep copy off
  site

3
When Disaster Is On Its Way

• Time in-house backup options 1 and 5
• Use tapes with some but very little use
• Verify backup options 2 and 6
• Archive off site
• Log off and turn off all electronic machines
  except server
  
• Unplug if not on uninterrupted power source (e.g.
  Tripp-Lite)
  
• Charge all cell phones

4
When Disaster Is On Its Way

• Discuss with staff accept input
• Decide on priorities of responsibilities and
  deadlines who does what
  
• Formulate what-if scenarios
• Backup all critical Microsoft files to flash
  drives share the location
  

5
When Disaster Is On Its Way

• Collect home and cell numbers for all department
  personnel
  
• Share with all staff
• Keep on person
• Consider personal obligations of staff in event
  of disaster
  
• Recognize that some staff may be hit harder than
  others due to home location
  

6
When Disaster Is On Its Way

• Decide on action plan and method of notification
• If power is lost, land lines are down
• If cell phone is dead, what is the alternative?
• If disaster will hit close to A/P check run
• process checks ahead of time with appropriate
  date and postage
  
• Designate staff member to mail on appropriate
  date
  

7
When Disaster Is On Its Way

• If disaster will hit close to pay date
• Process all checks and transmit direct deposits
• Decide on delivery procedure to staff
• Communicate this procedure with all staff
• Determine process for collecting time records
  from all locations
  
• Communicate process with building administrators
• Deliver tax deposits in advance to bank with
  agreement to process on given date
  

8
When Disaster Is On Its Way

• Stage all tax transmittals prior to arrival of
  disaster
  
• Establish back-up plan for reconciliation and
  delivery of payroll vendor checks

9
When Disaster Strikes

• Ride out the storm in a safe place
• Notify Disaster Recovery Vendor advise of
  status, severity, required assistance
  
• Estimate down time and impact on priorities
• Notify staff of impending implementation of
  action plan
  
• Implement action plan based upon priorities and
  deadlines
  

10
When Disaster Strikes

• Determine storms personal impact on staff to
  determine availability
  
• Set up alternate site to proceed with priorities
• Initiate Vendors Disaster Recovery
• Be flexible!
• Maintain daily contact with staff to advise of
  status

11
What We Did

• Limit any in-office time
• Temperature
• Lights
• Collection of needed materials
• At-home Disaster Recovery
• Multiple users
• Neighboring district assistance
• Value of personal relationships and contacts

12
What We Did

• Moving server, printer, folder/sealer
• Weight
• Bulk
• Disconnect/reconnect
• Change in IP address
• IT assistance with off-site small network set up
• Generator for postage meter
• Frequent monitoring of freezers and refrigerators

13
What We Learned

• Old advice take to another district and youre
  ready to go this doesnt work!
  
• Availability of 4-wheel drive a must
• Off-site location with heat/power
• Laptops!
• User-friendly at-home DSL service
• Consider availability of unlimited DSL usage
  (other family members needs)
  

14
What We Learned

• Generators computer, RS 6000, district server,
  postage meter you probably do not have nearly
  enough
  
• If state network is down, youre down
• When facing critical deadline before the
  disaster, stay as late as necessary to finish it
  
• Have a backup plan for critical postage, such as
  adequate petty cash
  

15
What We Learned

• If you spool a function for later access with an
  outside vendors product (e.g. PSST), they cannot
  recover it unless previously pulled up with their
  product
• Munis can access the backup file and email it,
  but it is not usable without Munis software
  
• Back up all critical Microsoft files and email
  contacts on flash drive
  
• Keep on person
• Consider encryption

16
What We Learned

• Dont rely on H-drive server is probably down
• If youre going to another location, take box of
  all needed supplies copy paper, stapler, pens,
  note paper, post-its, forms, instructions,
  mailing envelopes and addresses, check signature
  diskettes
• Dont mooch
• Take IP address, if you dont know it

17
What We Learned

• Be considerate of other districts time and
  generosity
  
• Determine a backup location to the backup
  location ref. heat and power
  
• Need for availability of brute force for moving
  heavy equipment
  
• Dont rely on other department personnel they
  may have other priorities
  

18
What We Learned

• Remember that KDE staff may be impacted/unavailabl
  e as well
  
• Keep your cell phone charger with you charge
  whenever possible
  
• Heavy-duty extension cords for outside
  generators
  
• Flashlights and batteries
• Empty refrigerators

19
What We Learned

• Alternate supplier for gasoline for generators
• Alternate supplier for diesel to start up buses
  (at least one hour)
  
• Establish backup source for drinking water - A/P
  vendor set up
  
• Determine need for hot tap water ref. lack of
  power to water heaters
  

20
What We Learned

• Stay on top of deadlines and plan accordingly
• Once power is restored, allow 6-8 hours for
  inside temperature to reach an acceptable level
  
• Gradually bring up other equipment
• Restore IP address on RS6000

21
What We Learned

• Notify Disaster Recovery vendor
• Do not permit staff to update RS6000 database
  until Disaster Recovery has been completed
  
• Move Disaster Recovery updated backup to RS6000
  allow several hours
  
• Find a restaurant thats open and get a hot lunch
  and/or hot beverage to warm up
  

22
What We Learned

• Use of cell phone cameras is not allowed no
  blackmail material!
  
• Maintaining your sense of humor is critical
• Remember that other people may be without heat
  and lights for a lot longer than you, so they may
  have lost their sense of humor