SODA:

1

• SODA
• Service-On-Demand Architecture for Application
  Service Hosting Utility Platforms
• Dongyan Xu, Xuxian Jiang
• Lab FRIENDS
• (For Research In Emerging Network and Distributed
  Services)
• Department of Computer Sciences
• Center for Education and Research in Information
  Assurance and Security (CERIAS)
• Purdue University

2
Outline

• Motivations and goals
• Related work
• Research components of SODA
• Summary and on-going work

3
Motivations

• Vision of utility computing
• Computation utility
• Storage utility
• Application service hosting
• Conference management
• e-Campaign
• Digital government
• Serving the underserved communities
• IT function shadowing for disaster recovery
• Virtual enterprise, collaboratory, and community

4
Our Goal

• To build a value-added application service
  hosting platform based on shared infrastructure,
  achieving
• On-demand creation and provisioning
• Virtualization
• Isolation
• Protection
• Accountability
• Privacy

5
Related Work

• Utility computing architectures
• VERITAS, HP UDC, IBM Oceano
• Grid platforms
• Computation Globus, Condor, Legion, NetSolve,
  Harness, Cactus
• Storage and data SRB, NeST, Data Grid,
  OceanStore
• Shared infrastructure
• PlanetLab, Emulab
• Active services
• Active Service Grid, Berkeley Active Service
  Framework, CANS (NYU), Darwin, WebOS

6
Related Work

• Resource isolation
• GARA, QLinux (UMass), Virtual service (UMich),
  Resource Container, Cluster Reserves (Rice)
• Virtualization technologies
• Virtual super computer (aggregation) NOW, HPVM
• Virtual OS, isolation kernel (slicing) VMWare,
  Xen (Cambridge), Denali (UW), UML, UMLinux,
  Virtual Private Server (Ensim)
• Grid computing on VM Virtuoso (Northwestern),
  Entropia
• Virtual cluster Cluster-on-Demand (Duke)

7
SODA

• Service-On-Demand Architecture for application
  service hosting utility platforms
• Research components of SODA
• General architecture
• Protection, intrusion detection, logging
• Confined and VM-based overlay
• Market-driven planning and management

8
Outline

• Research components of SODA
• General architecture
• Security and protection
• Confined VM-based overlay
• Property planning and management

9
Detailed Information

• Xuxian Jiang, Dongyan Xu, "SODA a
  Service-On-Demand Architecture for Application
  Service Hosting Utility Platforms", Proceedings
  of The 12th IEEE International Symposium on High
  Performance Distributed Computing (HPDC-12),
  Seattle, WA, June 2003.

10
Overview of SODA
AS
Virtual service node
AS
SODA Host (physical)
11
Virtualization Key Technique

• Two-level OS structure
• Host OS
• Guest OS
• Strong isolation
• Administration isolation
• Installation isolation
• Fault / attack Isolation
• Recovery, migration, and forensics
• Virtual service node
• Application service (AS)
• Guest OS
• Internetworking enabled

12
Service Requests From Clients
Service Requests From Clients
Service Switch for S
Service Switch for S
Service S
SODA Daemon
Service S
SODA Daemon
Service S
SODA Daemon
Service S
Virtual servicenode
Guest OS
Guest OS
Guest OS
Guest OS
Host OS
Host OS
Host OS
SODA Master
SODA Agent
Service Creation Requests From ASP
13
On the Same SODA Host
WWW service Honeypot
14
Host OS and Guest OS

• Guest OS based on User-Mode Linux (UML), an
  open-source virtual OS (different from UMLinux
  and VServer )
• By Jeff Dike, http//user-mode-linux.sourceforge.n
  et
• Running in user space of host OS
• Separate kernel address space
• Physical memory usage limit
• Host OS Linux (linux-2.4.19, enhanced)
• CPU fair share scheduler (for CPU isolation
  between virtual service nodes)

15
Experiment CPU Isolation
VM1 CPU-intensive VM2 IO-intensive VM3
Web
Original Linux Scheduler
Enhanced Linux Scheduler
16
On-Demand Service Priming

• Performed by SODA Daemon
• Customization of guest OS (cook to order )
• Active service image downloading
• Automatic bootstrapping of virtual service node

17
Service Bootstrapping Time
Linux Configuration Image size Time (seattle) Time (tacoma)
Rootfs_tomrtbt_ 1.7.205 15 MB 2.0 sec. 3.0 sec.
Rootfs_base_1.0 29.3 MB 3.0 sec. 4.0 sec.
Root_fs_lfs_4.0 400 MB 4.0 sec. 16.0 sec.
Root_fs.rh-7.2-server.pristine.20021012 253 MB 22.0 sec. 42.0 sec.
18
Slow-Down (w/o optimization)
Application level
System call level (clock cycles)
19
Outline

• Research components of SODA
• General architecture
• Security and protection
• Confined VM-based overlay
• Property planning and management

20
Detailed Information

• Xuxian Jiang, Dongyan Xu, Rudolf Eigenmann,
  "Protection Mechanisms for Application Service
  Hosting Platforms", Proceedings of IEEE/ACM Int'l
  Symposium on Cluster Computing and the Grid
  (CCGrid 2004), Chicago, IL, April 2004.
• Xuxian Jiang, Dongyan Xu, "Collapsar A VM-Based
  Architecture for Network Attack Detention
  Center", to appear in Proceedings of the 13th
  USENIX Security Symposium (Security '04), San
  Diego, CA, August 2004.

21
Security and Protection

• Virtual switching and firewalling
• IDS in guest OS kernel
• Untamperable logging (blackbox-ing)

22
Virtual Switching and Firewalling
Virtual machine (with IP addr.)
Guest OS
Guest OS
Guest OS
Host OS
Firewall
SODA host (Invisible on Internet)
23
Kernort IDS in Guest OS Kernel

• Problems with traditional IDS
• Encrypted traffic (e.g. ssh) makes NIDS less
  effective
• App-level IDS process will be killed, once a
  machine is compromised
• Log may be tampered with
• Fail-open
• Related projects
• Backtracker (Michigan)
• VMM-based retrospection (Stanford)
• Forensix (OHSU)
• ESP (Purdue CERIAS)
• Open-source projects Snort, Saint Jude

24
Kernort

• VM-based IDS
• Deployed in each VM
• Inside guest OS kernel a unique vista point
• Customizable without affecting host OS
• Clearer view
• Untamperable logging (saved to SODA host)
• Renewable signature (read from SODA host)
• Fail-close instead of fail-open

25
Kernort IDS in Guest OS Kernel
Guest OS
Guest OS
IDS
IDS
26
Kernort

• Components
• Kernort sensor
• Event-driven (system call and packet reception)
• Renewable signature set
• Matching against a small signature set (Top 20
  most wanted)
• Kernort blackbox
• Untamperable logging
• Privacy preservation of ASes
• Analyzer
• Exhaustive signature matching
• Detection of complex attack patterns
• Session replay

27
Kernort
Virtual machine
Host OS
Kernort (shaded areas logs)
28
Real-Time Alert
29
Session Re-play
30
Impact on Performance
31
Impact on Performance
32
Outline

• Research components of SODA
• General architecture
• Security and protection
• Confined VM-based overlay
• Property planning and management

33
Detailed Information

• Xuxian Jiang, Dongyan Xu, "vBET a VM-Based
  Emulation Testbed", Proceedings of ACM Workshop
  on Models, Methods and Tools for Reproducible
  Network Research (MoMeTools, in conjunction with
  ACM SIGCOMM 2003), Karlsruhe, Germany, August
  2003.
• Xuxian Jiang, Dongyan Xu, "VIOLIN Virtual
  Internetworking on OverLay INfrastructure",
  Department of Computer Sciences Technical Report
  CSD TR 03-027, Purdue University, July 2003.
• Xuxian Jiang, Dongyan Xu, A Middleware
  Architecture for Confined Virtual Machine
  Overlays", in preparation, March 2004.

34
Traditional Overlay Network

• Problems with traditional overlays
• Open for attacks
• Attacks from the outside (i.e. Internet) against
  overlay nodes
• Attacks from an overlay node against the outside
• Difficult to manage
• An overlay across multiple administration domains
• A host participate in multiple overlays
• Difficult to enforce overlay topology and traffic
  volume
• VPN does not solve the problems

35
Traditional Overlay Network
36
VM-based Overlay

• The case for VM-based overlay
• Multiple overlays on shared infrastructure
• On-demand creation
• Confinement and isolation
• VM introduces new network administration
  complexity
• What is this new machine that has suddenly
  appeared in my domain?
• Where is the machine that was in my domain
  yesterday?
• How much network connectivity should a VM have?
• How many IP addresses for VMs?

37
Confined VM-based Overlay

• In addition to VM, we need VN for VMs
• VN a highly overloaded term (VPN, X-bone)
• What is new Confined and VM-based overlays
• Applications
• Multi-institutional collaborations
• Philanthropic (volunteer) computing systems
• Network emulations

38
Confined VM-based Overlay
VM
VM
VM
2Mbps
2Mbps
Virtual infrastructure
1Mbps
39
Key Properties

• Confined overlay topology and traffic
• No attack possible from inside the overlay to the
  outside world
• Virtual IP address space
• No need for application modification and
  re-compilation

40
A More Generic Picture
VIOLIN Virtual Internetworking on OverLay
INfrastructure
41
vBET an Example of Confined Overlays on Demand

• An education tool for network and distributed
  system emulation
• Fidelity-preserving setup
• Maneuverable network entities
• Real-world network software
• Strict confinement (network security experiment)
• Flexible configuration
• Not constrained by device/port availability
• No manual cable re-wiring or hardware setup
• Simultaneous experiments
• Cost-effective

42
vBET
vBET Features

• Can be deployed in n 1 vBET servers
• Efficient startup and tear-down of emulated
  entities
• Strong network virtualization
• IP address space
• Virtual routers, switches, firewalls, end-hosts,
  links
• Communications confined by virtual topology
• Dynamic addition, deletion, migration,
  configuration of network entities

43
vBET GUI
44
Sample Emulation OSPF Routing
45
Emulation of OSPF Routing
Demo video clip athttp//www.cs.purdue.edu/jia
ngx/vBET/videos/vbet_ospf.avi
46
Sample Emulation Distributed Firewalls
47
Screenshot
48
Sample Emulation Chord P2P Network
49
Screenshot
50
Outline

• Research components of SODA
• General architecture
• Security and protection
• Confined VM-based overlay
• Property planning and management

51
Property Planning and Management

• Tenant selection
• Among a set of potential tenants (ASes), which
  ones to host? (for maximum revenue, resource
  utilization, security)
• SODA provider selection
• Among a set of SODA providers, which one should
  be chosen to host an AS?

52
Property Planning and Management

• Examples of bad planning
• Many PDA transcoding ASes in an area with a small
  PDA user population
• AS not requiring client registration and log-in
  (potential DDoS attacks)
• Majority of ASes exhibiting similar demand
  characteristics such as

53
Property Planning and Management

• AS profiling
• Resource requirement
• Security/authentication
• Demand characteristics
• Market analysis
• Competing ASes, market size/growth/expected share
• ASes correlation (80 of clients requesting AS X
  also request AS Y )
• Trading/pricing of SODA machine slices

54
Property Planning and Management

• Forming alliance of SODA providers

55
Property Planning and Management

• Forming alliance of SODA providers

C
C
C
P
P
P
56
Summary

• Virtualization a key enabling technology in
  realizing utility computing vision
• Hosting utility is more complex than computation
  utility (host tenants clients)
• SODA achieves
• On-demand service creation
• Service virtualization, isolation and confinement
• Protection, accountability, privacy
• Overlay isolation and confinement

57
Ongoing Work

• VM/service migration, shadowing, recovery
• Service profiling, accounting, auditing
  (resources, security)
• Market-driven planning, provisioning, and
  management (SODA ecology)
• Deployment and evaluation (Purdue Bindley
  Bioscience Center)

58
Thank you.
For more information dxu, jiangx_at_cs.purdue.edu
http//www.cs.purdue.edu/dxu AOL keywords
Purdue SODA Friends