2005 FNAL Computer Security Peer Review and

1
Fermi Computer Incident Response

• Michael Diesburg
• Fermi Computer Incident Response Team
• (FCIRT)
• March 22, 2005

2
Computer Security Plan

• FCIRT operating rules are laid out in CSPP
• CSPP defines 3 levels of incidents
• Non-issue
• SMOKE System Managers OKurance Evaluation
• FIRE FCIRT Incident Response Emergency
• CSPP defines reporting procedures to keep line
  management informed during serious incidents
  (FIRES)
• Defines authority of FCIRT to take control of
  resources and manpower during FIREs

3
FCIRT Operation

• Response consists of several parts
• Containment
• Removal from network, network blocks
• Classification
• No incident, SMOKE, FIRE
• Investigation
• Determine root cause that allowed breakin
• Restoration to service
• Define action necessary to return to service
• Assessment
• Evaluate procedures with CST, relevant parties

4
Incident Classification

• Non-Incident
• No actual compromise or significant threat to any
  Fermi resources
• SPAM from outside FNAL
• Non active virus laden files reported by AV
• Normal, ineffective probing from off site
• Instances of inappropriate use that are actually
  line management issues (and not visible from
  outside). I.e. when the computer is only an
  incidental tool to the actual infraction

5
Incident Classification

• SMOKE
• Security issues exist which need to be addressed
  but are limited in scope and threat level
• Known viruses confined to single system and not
  in a Critical System (Major Application)
• Configuration deficiencies which require
  attention of management personnel
• Suspicious but not obviously threatening activity
  which requires further investigation by system
  managers or FCIRT

6
Incident Classification

• FIRE
• Defined by scope and visibility
• Any computer activity which threatens the
  operation of the laboratory
• Threat to Critical System (Major Application)
• Threat of data loss
• Possibility of public embarrassment (activity
  noted by outsiders)
• Disgruntled employees (TEPCSR)
• Need to invoke authority
• Illegal activity or involvement of outside
  agencies

7
Serious Incidents

• Rate of FIREs 1/month
• 2001 18
• 2002 7
• 2003 9 ( 1 TEPCSR)
• 2004 12 ( 3 TEPCSR)
• 2005 None to date (1)

8
Overview of Incidents - 2003

• 01-17 csdserver1
• Problem Web defacement
• Alert CIAC, published on zone-h.org
• Root Cause Unpatched application (Frontpage)
• 01-25 SQL Server
• Problem root compromise
• Alert mail to abuse_at_fnal.gov
• Root Cause Unpatched application (MS SQL)
• 04-15 Samba Server
• Problem root compromise
• Alert Autoblocker, FBI
• Root Cause Unpatched application (samba).
  Update failed. Threat noted on 7th, patched on
  8th , entered on 13th
• 05-13 beamssrv1
• Problem root compromise
• Alert Internal user, drives not available
• Root Cause weak password/TS/stolen laptop. Not
  resolved

9
Overview of Incidents - 2003

• 07-07 DOE-hep.hep.net
• Problem Web defacement
• Alert Internal user
• Root Cause Inappropriate configuration/pilot
  error
• 07-18 Cisco IOS router exploit
• Problem Significant vulnerability (no
  compromise)
• Alert CIAC
• Root Cause NA
• 07-23 Unauthorized file distribution
• Problem Unauthorized distribution of IP
• Alert External, MPAA
• Root Cause Unpatched application (MS SQL)
• 08-19 W32/Sobig virus
• Problem Widespread infection
• Alert Internal alert, CIAC
• Root Cause Unpatched machines. Late AV
  signatures. Pilot error
• 08-19 Nachi/Welchia
• Problem Widespread infection
• Alert Internal

10
Overview of Incidents - 2004

• 02-15 Infected systems (Netsky)
• Problem Widespread infection
• Alert Internal
• Root Cause Unpatched machines. Late AV
  signatures. Overloaded mail gw not scannin
  plain/txt files. Missing some zip files. Pilot
  error
• 03-20 Black Ice
• Problem Widespread compromise
• Alert Autoblocker
• Root Cause Unpatched application (black Ice)
• 04-08 Web site hacked
• Problem Web site defacement
• Alert Internal
• Root Cause Unpatched application (Serv-U ftp)
• 04-21 bdcry026
• Problem root compromise
• Alert Internal, admin noted unauthorized logins
• Root Cause Unpatched application (MS SQL)
• 05-13 Increased inbound scanning
• Problem Off-site connectivity impacted by scans.
  Routers overloaded
• Alert Autoblocker

11
Overview of Incidents - 2004

• 08-04 fcdfdata28
• Problem root compromise
• Alert External via inside channels
• Root Cause Pilot error (password stored in file
  on remote system)
• 08-09 tdpc355
• Problem Widespread infection
• Alert Internal
• Root Cause Unpatched machine, Pilot error. Late
  AV signature
• 09-01lss-recfacility
• Problem root compromise
• Alert Internal, Autoblocker
• Root Cause Unpatched application (MS SQL)
• 09-22 laptop-sopczak
• Problem Widespread infection
• Alert Autoblocker
• Root Cause Pilot error. Rode in on laptop
• 09-28 trojan on fcdflnx3
• Problem User level compromise
• Alert CIAC

12
Incident Reporting

• For all serious incidents as described above, the
  CSPP defines reporting procedures
• Inform CSExec
• Inform relevant system manager(s)
• Inform relevant Division Head
• Inform Public Information office if there is
  significant visibility from off-site
• Inform CIAC, DOE, CI and IG if incident meets
  their reporting guidelines

13
Summary of Incident Causes

• 2003
• Unpatched systems 5(4)
• Bad configuration 1
• Update failure 1
• Unknown 1
• 2004
• Unpatched systems 6(4)
• Bad configuration 1
• Pilot error 3
• Other 1
• First two categories in above list should be
  preventable. Possibly only 4 in first category
  for each year.

14
Preventative Actions

• Unpatched vulnerabilities are a clear problem
• Two pronged approach
• Identify and scan for critical vulnerabilities
• Ongoing problem as new holes emerge
• Time is of the essence
• See R. Reitz talk
• Patch/Configuration management
• May not cover non mainstream apps, but should
  cover biggest holes
• Need to identify stealth installations
• See M. Kaletka talk
• Pilot error will always be with us
• Ongoing educational efforts
• Computer Security Awareness Day
• Articles in Fermi Today
• Its a 00.05 problem