NPTF SEPTEMBER SESSION

1
NPTF SEPTEMBER SESSION

2
Meeting Schedule

• April 6 (planning session)
• May 4 (strategy session)
• July 20 (strategy session-reducing costs)
• September 21 (PNP support model, DNSSEC,
  security/ID management, service monitoring,
  wireless, vLANs)
• October 19 (cancelled)
• November 16 (rate setting)

3
Agenda

• PennNet Phone support model
• DNSSec
• Security/ID management
• Service monitoring
• Next generation wireless
• vLANS

4
PennNet Phone Support Model

• Michael Palladino

5
PennNet Phone Support

• Back Ground
• Service initially deployed to IT Support staff
  with the assumption that a technical background
  was needed
• Service initially supported by Local Support
  Providers (LSP)
• Whats Changed
• The community said why change what is working
• Traditional phone support in schools and centers
  done mostly by non-LSPs
• Service matured technical background is not
  needed to order or support PennNet Phone
• Traditional Telephone Support Providers (TSP) are
  now supporting PennNet Phone
• Recommendation
• Schools/Centers should identify a TSP to be
  responsible for ordering and supporting telephone
  services
• The TSP may be a staff member that currently
  supports traditional phone services or an LSP
  for those departments wishing to consolidate
  support services
• It is your choice. Do what is best for you.

6
PennNet Phone Support

• Installation requests should be made at
  http//www.upenn.edu/computing/isc/networking/orde
  rforms.html. ISC requests 10 business days
  notice for all voice installation requests.
• Support Requests should be made using the web
  services available at http//www.upenn.edu/computi
  ng/voice/help/repair.html. ISC Network
  Operations will respond to the ticket within 4
  hours with a resolution provided within one
  business day.
• More information available at the first quarterly
  PennNet Phone SIG. Wednesday, September 23 _at_
  100PM. 3401 Walnut Street Suite 337A.

7
DNSSEC

• Shumon Huque

8
DNSSEC Why discuss?

• Needed part of Internet security architecture
• Will take a long time to fully deploy
• But A lot of recent publicity
• Dan Kaminskys attack
• Active deployment plans at various levels
• DNS Root
• Educause
• Penn

9
DNSSEC at a glance

• DNS Security Extensions
• A system to verify the authenticity of DNS data
• Helps detect spoofing, misdirection, cache
  poisoning, etc.
• Some potential secondary benefits
• Storing cryptographic keys in the DNS
• SSHFP, IPSECKEY, CERT, DKIM, etc.

10
. (root)
roots pubkey
3
.edu
www.upenn.edu set DO bit
referral to .edu DS, RRSIG
edu pubkey
2
recursive resolver
4
5
referral to upenn.edu DS, RRSIG
(has roots pubkey)
6
upenn.edu
www.upenn.edu
1
8
upenn pubkey
answer 1.2.3.4 RRSIG
7
endstation (uses DNS stub resolver)
11
DNSSEC EDU Announcement

• Educause, Verisign US Dept of Commerce
• Announced on Sep 3rd that .EDU will deploy DNSSEC
  by March 2010
• http//www.educause.edu/AboutEDUCAUSE/PressReleas
  es/SecurityofeduInternetDomaintoI/178963

12
Educause Announcement
EDUCAUSE and VeriSign announced today the
initiation of a project to enhance Internet
reliability and stability. By the end of March
2010, the project will deploy a security system
known as Domain Name Security Extensions (DNSSEC)
within the .edu portion of the Internet, which
EDUCAUSE manages under a cooperative agreement
with the U.S. Department of Commerce. When the
project is completed, institutions whose domain
names end in .edu will be able to incorporate a
digital signature into those names to limit a
variety of security vulnerabilities. The Domain
Name System (DNS) is the part of the Internet
that translates names such as "educause.edu" into
numeric addresses (for example, 198.59.61.90).
All Internet applicationsfrom electronic mail to
online bankingdepend on the accuracy and
integrity of this translation. Over the years,
Internet security experts have discovered a
variety of ways that DNS translation may be
compromised. The DNSSEC security system limits
the problem by allowing owners of domain names to
provide a digital signature that adds an extra
level of authentication to the translation
process.
13
DNS Root Signing

• Planned deployment by end of 2009
• http//www.icann.org/en/announcements/announcement
  -2-03jun09-en.htm
• http//www.nist.gov/public_affairs/releases/dnssec
  _060309.html
• Other top level domains deployed or plans in
  progress (ORG, GOV, COM, NET, etc)

14
DNSSEC at Penn

• MAGPI (Internet2 GigaPoP) deployed it in 2006!
• Penn (UPENN.EDU) was done this summer
• For details, see presentation at Internet2 Joint
  Techs meeting
• http//events.internet2.edu/2009/jt-indy/agenda.cf
  m?gosessionid10000653

15
Information Security Updates

• Dave Millar

16
Compromises Down, DMCA mixed
Uptick in compromises in FY09 was a bit of a
surprise.
17
Worms dont account for much of the uptick.

• FY09 worms
• Nachi - 15 machines
• Conficker - 14 machines
• FY08 Worms
• Storm 11 machines

18
Not caused by any one School/Center
19
Phishing attacks continue to succeed
20
Threat Assessment

• Systems are being well-managed, though the uptick
  in FY09 would seem to justify additional focus in
  the coming year on mitigation
• Patch management
• Least privilege
• Targeted phishing attacks are a significant
  threat against PennKeys.
• Continue to focus on education and awareness.
• Lost/stolen mobile data is a very credible
  threat.
• Continue to focus on education (dont store
  sensitive data) and mobile data encryption.

21
Past Initiatives

• SPIA Cohort 3
• 33 Schools/Centers now engaged
• Considerable risk reduction
• Phishing awareness
• Almanac tips
• Online training
• Online Privacy and Security Training
• Optional
• Available on Knowledgelink
• PennGroups
• Supports authorization/access control
• Grant access by individual/need to know, or
  group/role
• http//www.upenn.edu/computing/penngroups/
• PennKey ASAP
• Streamlined PennKey support for alumni
• Supports remote identity verification
• No need to appear on campus in-person
• 636 PennKeys issued since inception

22
Past Initiatives

• SecureShare
• Secure file exchange for Penn Faculty and Staff
• 1666 people have used it since inception
  (5/14/2009)
• Replace ISS scanner with NeXpose
• Self-service vulnerability scanning on demand to
  supplement scheduled critical host scans
• Very comprehensive Windows, Mac, BSD, AIX, SQL
  Server, MySQL, Oracle, PostgreSQL, Apache, IIS,
  SQL Injection, Cross-site scripting,
• http//www.upenn.edu/computing/security/scanner
  /
• Security Liaisons
• Representative from each School/Center
• Work to build awareness locally
• Authentication Logging
• Capturing PennKey authentication logs
• Developing anomaly detection

23
Initiatives in Progress

• RT-IR (Target FY10)
• Incident tracking system to replace current
  homegrown application
• Tightly integrated with Penn applications
• SPIA Cohort 4 (Target FY10)
• Five new Schools/Centers
• More flexible schedule
• Hard Drive Encryption for Laptops (Target FY10)
• PGP selected
• Central service available, with key escrow
• Cloud Computing Guidance, Policy and Approved
  Services (Target FY11)
• Examples Google Apps, Mozy
• Recommending that confidential data may only be
  kept on third party with approved contract
• Levels of Assurance (Target FY11)
• Offer two or three levels of identity assurance,
  suitable to application requirements
• Varying levels of ID proofing and protocol
  strength

24
Strengthening PennKey

• Deke Kassabian

25
Initiatives in Progress Penn WebLogin

• Penn WebLogin provides a more secure, cost
  effective architecture than Websec
• Built upon CoSign and Shibboleth, Internet
  standards with broad deployment in Higher Ed.
• CoSign available to the University since November
  2008. An August 2009 upgrade to CoSign 3.0
  addressed a security vulnerability
• Websec to be decommissioned in December 2009
• Only 27 of Websec applications have migrated to
  WebLogin
• ISC providing proactive support to assist Schools
  and Centers with migration efforts

26
Initiatives in Progress Penn WebLogin

• Next Steps
• Continue to provide IT Directors monthly status
  updates on School and Center migration progress
• Continue to provide technical assistance for
  conversions at no charge but staff availability
  may be thin as we approach the deadline
• Continue to provide training sessions through
  October and November
• Continue to provide rapid support to implementers
• Decommission Websec December 2009

27
Initiatives in Progress Shibboleth

• Shibboleth is an open source, Internet2 web
  authentication service
• Works along with CoSign as a part of Penn
  WebLogin
• Supports secure, federated authentication
• Wide adoption in higher ed
• Limited pilot deployment in production through
  end of 2009, with five applications scheduled
• Penn is registered with InCommon for support of
  federated authentication to external service
  providers
• General availability of Shibboleth with self
  provisioning by first quarter 2010

28
Initiatives in Progress Shibboleth

• Next Steps
• Complete the Shibboleth provisioning for the five
  pilot participants
• Publish documentation
• Implement automated provisioning through the
  WebLogin Management Console
• Define process for registering Service Providers
  for external, federated users

29
Initiatives in Progress Two Factor
Authentication

• Pilot Implementation of second authentication
  factor for users attempting to access Penn
  resources through WebLogin
• Completed technology analysis and selected pilot
  vendors
• Received evaluation kit for RSA SecurID (One Time
  Password token)
• Purchased limited licenses for PhoneFactor
  (Tokenless two-channel phone based solution)
• Purchased pilot hardware

30
Initiatives in Progress Two Factor
Authentication

• Next Steps
• Deploy hardware and implement limited test
  environment for evaluation of local applications
• Finalize the selection of the pilot application
• Coordinate with pilot application development
  team configuration and architecture requirements
• Deploy in production environment for pilot to run
  through end of FY2010
• Perform final evaluation including
• Technology Evaluation
• Security Evaluation
• Supportability Model
• Total Cost of Ownership

31
Service Monitoring

• Deke Kassabian

32
Service Monitoring

• The model ISC NT Service Metrics
• Leverage Nagios, Open Source monitoring tool
• Public view http//status.net.isc.upenn.edu/
• Current service status, as well as historic
  uptime reports
• Commodity hardware, free software
• All testing done from a non-server (user) network
• We use a combo of Nagios/Spectrum/Attention. We
  would decouple the latter two and use other Open
  Source software for paging and voice

33
Service Monitoring

• Proposed Features
• Redundant, high availability
• Host / switch / anything with an IP
• Default monitors available FTP, HTTP (and/or
  URL), PING, SMB (Windows), SSH, 
• Alerts via mail, SMS, voice to contact(s) of your
  choice configurable schedules
• Current and historical data, or log
  retrieval/shipping for local analysis

34
Service Monitoring

• Challenges
• Driven by interest,  many customers already run th
  eir own monitoring
• Delegated access, isolation of customer
  access/data
• Customization too little/not enough value too
  much/not enough time
• Possible cost models per node (pay for what you
  use) per org (unlimited)
• Costs
• Fixed hardware and staff time
• Could sustain with 5-6 customers, 1000/org/year
  (unlimited host monitoring)
• Custom monitoring scripts (TM), custom reports
  (TM)
• Redundant hardware affects costs interest in
  lower SLA?
• 15K capital for systems/infra, 4-year lifecycle,
  15hrs staff time/year about 5600/year to run.

35
Alternate Option Monitor-the-Monitor

• For customers with monitoring already in place
• 24x7 monitoring and alerting, but lower SLA
  (daytime maintenance, etc.)
• Simple service tests PING, possibly HTTP or
  other TCP services
• No customized monitoring
• Email alerts only
• Lightweight reporting email/SCP logs and you
  process
• Use existing NT infrastructure to keep costs
  down
• 200/node/year

36
Alternate Option ISC Partners with Vendor

• Small project to identify vendor with suitable
  offering of broad campus interest
• Agent on host or agentless depending on
  requirements
• May rely on infrastructure outside PennNet
• Leverage number of contract customers for better
  pricing for a central service
• One size may not fit all

37
Wireless

• Mark Wehrle

38
Wireless Current Status

• Wireless PennNet Retirement Completed
  06/30/2008
• AirPennNet-Guest Network in Operation starting
  July 1, 2008
• Completed per subnet IP ranges to provide
  scalability and management
• Coordinated with LSPs to set IP ranges for
  AirPennNet and AirPennNet-Guest Networks
• Consolidation of all Wireless Networks
• AirPennNet expansion (SAS and SEAS buildings)
• AirSAS retired and replaced with AirPennNet and
  AirPennNet-Guest.
• SEAS has AirPennNet and AirPennNet-Guest
• AirPennNet with native 802.1x authentication
• Over 1400 APs have common log-on campus-wide
• Results in 70 Campus Covered

39
Wireless Current Status

• AirPennNet website completely reworked
• Coverage maps, FAQ, Technical information
• Continue with wireless expansion per customer
  demand in FY10
• Project to Evaluate and Select Next Generation
  Wireless Hardware
• Good trade in costs and strong negotiations
  helped to keep under our projected monthly
  support costs for FY10
• Design of Campus User Rapid/Self Service to
  Enable Guest Access

40
Next Generation Wireless

• Advantages include
• Speed up to 100mbs
• Uses new and improved MIMO technology equates to
  more bits per second per hertz of bandwidth and
  link reliability or diversity which reduces
  signal fading
• Performance
• Ability to support legacy 802.11b clients without
  downgrading higher speed clients on same access
  point
• Provides framework for QoS (Quality of Service)
  for next generation applications over wireless
  Voice over WLAN, video streaming, location
  services
• Enables client mobility and eliminates client
  roaming tendency problems between APs from
  other wireless subnets

41
Next Generation Wireless

• Advantages include
• Operational Efficiencies
• Potential savings in staff time (installation,
  management support)
• Dynamic wireless coverage and signal strength
• Coverage adjustment upon AP failure, automatic AP
  configuration
• Rogue AP detection and elimination
• Ability to stage 802.11n roll out

42
NG Wireless Upgrades

• Controller-based Architecture
• N1 Topology
• 1 Master Controller, 3 Slave Controllers
• Master Controller Manages Configurations and
  Failover
• 1435 APs to upgrade in approximately 140
  Buildings
• Wireless LANs (WLANs) Targeted by School/Center
  Department
• Joint effort to establish upgrade schedules
• Wholesale Upgrades by WLAN (e.g. must swap all
  APs in same subnet)
• Physical Replacement of the AP done by Union
  Contractors
• ISC NT Ops takes care of all background work and
  onsite testing with LSP
• To Date over 50 (730) of the APs are upgraded
  in 72 buildings.

43
NG Wireless Buildings (Completed)
44
NG Wireless AP Upgrade Timeline

• Admin 8 AP(s) in 1 Building(s)
• EIS 8 Estimated upgrade in Q3 FY10
• Annenberg 17 AP(s) in 1 Building(s)
• ANB 17 Estimated upgrade in Q3 FY10
• Business-Services 1 AP(s) in 1 Building(s)
• BOK 1 Estimated upgrade in Q2 FY10
• CCEB 8 AP(s) in 1 Building(s)
• MKE 8 Estimated upgrade in Q2 FY10
• DRIA 30 AP(s) in 8 Building(s)
• DUN 4 Estimated upgrade in Q2 FY10
• FKF 6 Estimated upgrade in Q2 FY10
• GYM 6 Estimated upgrade in Q2 FY10
• HOL 2 Estimated upgrade in Q2 FY10
• HTC 2 Estimated upgrade in Q2 FY10
• MPY 1 Estimated upgrade in Q2 FY10
• PAL 3 Estimated upgrade in Q2 FY10
• WTM 6 Estimated upgrade in Q2 FY10

• Dental 33 AP(s) in 3 Building(s)
• EVN 24 Estimated upgrade in Q3 FY10
• LEV 1 Estimated upgrade in Q3 FY10
• SCH 8 Estimated upgrade in Q3 FY10
• Design 20 AP(s) in 3 Building(s)
• AFC 4 Estimated upgrade in Q2 FY10
• MEY 12 Estimated upgrade in Q2 FY10
• MGN 4 Estimated upgrade in Q2 FY10
• FRES 3 AP(s) in 1 Building(s)
• GEO 3 Estimated upgrade in Q2 FY10
• Finance 6 AP(s) in 2 Building(s)
• FBA 2 Estimated upgrade in Q2 FY10
• FKB 4 Estimated upgrade in Q2 FY10
• GSE 8 AP(s) in 1 Building(s)
• GEB 8 Estimated upgrade in Q2 FY10
• Hillel 7 AP(s) in 1 Building(s)
• HSH 7 Estimated upgrade in Q2 FY10

45
NG Wireless AP Upgrade Timeline

• Museum IT 9 AP(s) in 1 Building(s)
• MUS 9 Estimated upgrade in Q4 FY10
• Nursing 14 AP(s) in 1 Building(s)
• NEB 14 Estimated upgrade in Q2 FY10
• SOM 61 AP(s) in 8 Building(s)
• ACH 7 Estimated upgrade in Q2 FY10
• BLK 13 Estimated upgrade in Q2 FY10
• BRB 8 Estimated upgrade in Q2 FY10
• BRC 21 Estimated upgrade in Q2 FY10
• CRB 5 Estimated upgrade in Q2 FY10
• EAP 2 Estimated upgrade in Q2 FY10
• MEB 1 Estimated upgrade in Q2 FY10
• MLA 4 Estimated upgrade in Q2 FY10
• SP2 1 AP(s) in 1 Building(s)
• POB 1 Estimated upgrade in Q2 FY10
• University Square 2 AP(s) in 1 Building(s)
• FKB 2 Estimated upgrade in Q2 FY10

• SAS 182 AP(s) in 18 Building(s)
• CAS 2 Estimated upgrade in Q4 FY10
• CHM 28 Estimated upgrade in Q4 FY10
• CJS 5 Estimated upgrade in Q4 FY10
• DRL 31 Estimated upgrade in Q4 FY10
• ESA 5 Estimated upgrade in Q4 FY10
• FEL 4 Estimated upgrade in Q4 FY10
• GDD 9 Estimated upgrade in Q4 FY10
• IST 11 Estimated upgrade in Q4 FY10
• LDY 14 Estimated upgrade in Q4 FY10
• LOG 8 Estimated upgrade in Q4 FY10
• LUA 3 Estimated upgrade in Q4 FY10
• MCN 15 Estimated upgrade in Q4 FY10
• MEL 4 Estimated upgrade in Q2 FY10
• MUS 9 Estimated upgrade in Q4 FY10
• PSY 10 Estimated upgrade in Q4 FY10
• SLC 1 Estimated upgrade in Q4 FY10
• STI 5 Estimated upgrade in Q4 FY10
• WMS 18 Estimated upgrade in Q4 FY10

46
NG Wireless AP Upgrade Timeline

• VPUL 6 AP(s) in 1 Building(s)
• SFR 6 Estimated upgrade in Q2 FY10
• Vet 44 AP(s) in 9 Building(s)
• CAH 4 Estimated upgrade in Q3 FY10
• HTD 1 Estimated upgrade in Q3 FY10
• MYR 1 Estimated upgrade in Q3 FY10
• ROS 7 Estimated upgrade in Q3 FY10
• SSM 1 Estimated upgrade in Q3 FY10
• VHP 10 Estimated upgrade in Q3 FY10
• VRB 14 Estimated upgrade in Q3 FY10
• VSB 4 Estimated upgrade in Q3 FY10
• WID 2 Estimated upgrade in Q3 FY10

• Wharton 140 AP(s) in 6 Building(s)
• CPN 3 Estimated upgrade in Q3 or Q4 FY10
• HNT 70 Estimated upgrade in Q3 or Q4 FY10
• LFR 4 Estimated upgrade in Q3 or Q4 FY10
• SCC 26 Estimated upgrade in Q3 or Q4 FY10
• SDH 29 Estimated upgrade in Q3 or Q4 FY10
• VAN 8 Estimated upgrade in Q3 or Q4 FY10
• Writing 1 AP(s) in 1 Building(s)
• LSW 1 Estimated upgrade in Q2 FY10

47
NG Wireless Upgrades

• Plans for FY09 and FY10
• Currently running two association methods
• DynamicWEP (Open/WEP) (Old standard client
  config)
• WPA (WPA/TKIP) (FY10 standard client config)
• Need to remove DynamicWEP in favor of WPA2
• How many clients are still running DynamicWEP?
  In Progress
• WPA (WPA/TKIP)
• WPA2 (WPA2/AES) (FY11 standard client config)
• This will allow for deployment of 802.11n
• Association rates up to 300Mbs
• Requires WPA2/AES
• IP Multicast support
• On FY11 PennConnect DVD

WEP - Wired Equivalent Privacy WPA/WPA2 Wi-Fi
Protected Access TKIP Temporal Key Integrity
Protocol AES Advanced Encryption Standard
48
Controller Wireless Topology
IP Mobility between wLAN
All Wireless Traffic sent over IPSEC Tunnel to
Local Controller
All Wireless Traffic sent over IPSEC Tunnel to
Local Controller
Master Manages Configs Backs Up Local Controllers
Primary gateway for all wireless networks
Secondary gateway for all wireless networks
49
Proposed Wireless Guest IP Funding Model

• Goal To enable proper IP ranges for AirPennNet
  and AirPennNet-Guest, and to ensure use of
  AirPennNet as primary wireless network
• Key Concepts
• AirPennNet-Guest was designed for visitors and
  for devices incapable of supporting 802.1x.
  (network has restrictions and is less secure)
• Also allows for some guest access to campus wLANs
  that are paid for by other Schools/Centers
• Policy Current policy allows for 10 of IP range
  for AirPennNet networks be subsidized for IP
  range in AirPennNet-Guest networks. Schools or
  centers will pay for IP costs greater than 10 of
  AirPennNet IP range.
• Proposed Full Subsidy of all IP Address for
  AirPennNet-Guest Aggregate Cap of 30 to still
  encourage use of AirPennNet. Review at NPTF each
  fiscal year.

50
Proposed Wireless Guest IP Funding Model

• Current Cost impact to CSF FY10
• 6500 IPs assigned for AirPennNet in FY10 (Does
  not include Resnet)
• 2200 IP addresses assigned for AirPennNet-Guest
  (34 of AirPennNet IP ranges in use today)
• 10 cost of 650 IPs equals 650x1.67x1213k per
  year.
• Remaining 1550 IPs are billed out
  (1550x1.67x1231k)
• We propose starting the new model as of January
  1, 2010.
• Potential cost impact to CSF FY11
• 8000 IPs assigned for AirPennNet projected (23
  Growth)
• 30 cost of those IPs equals 2400x1.52x1244k
  per year.
• This cost could be added to the CSF for FY11 and
  not billed directly to schools.

51
vLANs

• Mark Wehrle

52
vLans

• How many are there?
• 144 Private vLANs in various buildings
• 5060 ports out of 48,600 ports (10.4 are vLAN
  ports)
• Why do we charge?
• Increase complexity
• Network designs (in planning and upgrades)
• Technical management overhead (all labor)
• Troubleshooting more difficult between subnets in
  buildings
• Can we lower the charge?
• Factors affecting this decision are scope of the
  vLAN (entire building)
• Number of vLANs in the building
• Total percentage of vLAN ports vs. regular ports
• Could spread vLAN costs across all ports (cost
  exercise and report at later NPTF)
• Should vLANs behind a firewall cost less?
• Depends on factors above?
• Entire buildings could be considered as reduced
  overall vLAN cost in specific SLA (assumes all
  ports behind firewall)