Georgia%20Tech

1
Virtual Machines Supporting Changing Technology
and New Applications

• Georgia Tech
• November 2006
• J. E. Smith

2
Introduction

• Why virtual machines?
• They allow transcending of standardized
  interfaces
• (which sometimes are an obstacle to innovation)
• They enable innovation in flexible, adaptive
  software hardware, security, network computing
  (and others)
• They involve computer architecture in a pure
  sense
• Virtualization technologies will be a key part of
  most future computer systems

3
Outline

• Virtualization
• Virtual Machine Architecture
• Virtual Machine Implementation
• Computer Architecture Applications
• Co-Designed VMs
• Private Virtual Machines

4
Abstraction

• Computer systems are built on levels of
  abstraction

• Higher level of abstraction hide details at lower
  levels
• Example files are an abstraction of a disk

5
Virtualization

• Similar to abstraction
• Except
• Same level of detail
• Construct Virtual Disks
• As files on a larger disk
• Map state
• Map operations
• VMs do the same thing with the whole machine
• Key concepts Map state Map Operations

6
The Family of Virtual Machines

• There are lots of virtual machines
• IBM VM/370
• Java
• VMware products

Including things not called virtual machines
IA-32 EL HP Dynamo
Transmeta Crusoe

• The subjects of virtual machines and
  emulators have been treated as entirely separate.
  they have much in common. Not only do the
  usual implementations have many shared
  characteristics, but this commonality extends to
  the theoretical concepts on which they are based
• -- Efrem G. Wallach, 1973

7
Machines

• Different perspectives on what the Machine is
• OS developer
• Compiler developer
• Application programmer
• Instruction Set Architecture
• ISA
• Major division between hardware and software

Application
Programs
Libraries
Operating System
Execution Hardware

• Application Binary Interface
• ABI
• User ISA OS calls

• Application Program Interface
• API
• User ISA library calls

Memory
Translation
System Interconnect
(bus)
I/O devices
Main
and
Memory
Networking
8
System Virtual Machines

• ISA level
• Provide a system environment
• VMM manages guest OS apps
• Persistent
• Examples IBM VM/360, VMware, Transmeta Crusoe

guest
guest
guest
guest
guest
guest
process
process
process
process
process
process
Guest OS2
Guest OS
VMM
VMM
HOST PLATFORM
virtual network communication
9
Process Virtual Machines

• ABI level
• Runtime manages guest process
• Guest processes may intermingle with host
  processes
• Not persistent
• Guest and host OSes are often the same
• Dynamic optimizers are a special case
• Examples IA-32 EL, FX!32, Dynamo

guest
host
process
process
runtime
guest
guest
host
process

process
process
runtime
runtime
create
HOST OS
file sharing
Disk
network communication
10
High Level Language Virtual Machines

• Constructed at API level
• User higher level virtual ISA
• OS abstracted as standard libraries
• A form of process VM

11
Virtual Machine Architectures
Process VMs
System VMs
different
different
same ISA
same ISA
ISA
ISA
Classic
Whole
Dynamic
Multi
OS VMs
System VMs
Translators
programmed
Systems
Hosted
Co-Designed
Dynamic
HLL VMs
VMs
VMs
Binary Optimizers
12
VM Technology State Mapping

• VM SW re-maps virtual state to real state
• Recall virtual disk
• Registers to registers
• Registers to memory
• Memory to memory
• Memory to disk

Host
Register
Host Registers
Space
Guest Registers
Runtime
Data
Runtime
Code
Host ABI
Guest Data
Address
Space
Guest Code
13
VM Technology Operation Mapping

• VM SW re-maps operations on state
• Instruction-level state changes
• Emulation
• Protected state changes
• OS operations
• Done under VMM control
• Key concepts Emulation and Control

14
VM Technology Emulation

• Interpretation
• Software loop decodes and dispatches each
  instruction

interpreter
source code
routines
"data"
accesses
dispatch
loop
15
VM Technology Emulation

• Binary translation and code caching
• Translate blocks of instructions at a time
• Hold translated blocks in code cache
• This was a key enabler for VMware success

binary translated
target code
source code
binary
translator
16
VM Technology Emulation

• Staged Emulation
• Emulation techniques invoked in staged manner
• Based on performance tradeoffs

17
Code Caches

• Contain
• Basic blocks
• Superblocks (one entrance, multiple exits)
• Optimized Superblocks
• Used in many VMs
• Dynamic binary translators Intel IA-32 EL,
  Compaq FX!32
• Dynamic binary optimizers Dynamo family
• Co-designed virtual machines Transmeta, IBM
  DAISY
• High performance Java virtual machines
• System VMs with inefficiently virtualizable
  ISAs
• Sandboxing secure VMs (x86 DynamoRIO)

18
Code Caching with Chaining

• Chaining of blocks in code cache minimizes VM
  overhead

Code Cache
Superblock
Dispatch table lookup code
Superblock
Superblock
Superblock

19
VM Technology Control

• Interpretation
• Fine grain control
• Every dynamic instruction inspected before
  execution
• Binary translation and code caching
• Coarser grain control
• Every static instruction inspected before
  execution
• Jumps to VM SW can be inserted anywhere
• Protection levels
• Very coarse grain control
• Every resource-related instruction trapped by
  protection system
• Otherwise, use interpretation/translation
  techniques
• Used in system VMs

20
Resource Control in System VMs
Application

• Traps and interrupts ( sys calls)
• Transfer to VMM
• VMM determines appropriate Guest OS
• VMM transfers to Guest OS
• Guest OS return to user app.
• Transfer to VMM
• VMM bounces return back to Guest app.
• Resource sensitive instructions
• Trap to VMM
• VMM checks correctness
• VMM reads/modifies guest resource
• Returns to Guest

system call/trap
Guest OS
privileged operation
next instruction
virtual vector location
system return
VMM
check privileges
perform operation
return
vector location
21
VMs and Computer Architecture

• Use virtualization to give computer architects a
  layer of software
• Beneath all conventional software
• Maintains vision of hardware as seen by
  conventional software
• Performance optimizations via Co-designed VMs
• VM SW can alter/enhance architecture via
  emulation
• Resource management Private Virtual Machines
• VM SW can manage microarchitecture resources

22
Co-Designed Virtual Machines

• Separate the hardware/software interface from the
  ISA level of abstraction
• Restore the ISA to its natural place
• ? as an Implementation ISA that reflects actual
  hardware
• Support existing ISAs
• ? as a Virtual ISA
• Let processor designers use both
• hardware and software
• A form of system VM

23
VM Technology -- Concealed Memory

• VM software resides in memory concealed from
  all conventional software
• This software is available to hardware designer

Code
ICache
Cache
concealed memory
Hierarchy
VM Code
Processor
VM Data
Core
Source ISA Code
DCache
conventional
Hierarchy
Source ISA Data
memory
24
Co-Designed VMs

• Of interest to both architects and
  micro-architects
• Offers opportunities for performance, power
  saving, fault tolerance and other
  implementation-dependent features
• Allows transcending conventional ISAs
• Dont confuse them with VLIW!
• Early examples IBM Daisy and Transmeta Crusoe
• pioneers are the ones with arrows in their
  backs

25
Another Way of Doing Things
conventional
Func.
Unit
Translation
Func.
Cache
Processor
Main Memory
Unit
Unit
Hierarchy
Pipeline
(form uops)
. ..
Func.
Unit
Main Memory
dynamic translation
Software
Func.
Translator
Unit
. ..
Processor
Cache
Code Cache
Translation
Hierarchy
Pipeline
Unit
(form uops)
Func.
Unit
26
Fused Microarchitecture

• Fuse dependent pairs of micro-ops to macro-ops
• Current Intel approach
• Use co-designed SW to achieve wider-scale fusing
• Process execute fused macro-ops as single
  Instructions throughout the entire pipeline
• Allows pipelined wake-up/select issue logic

27
Fusible Instruction Set

• RISC-ops with unique features
• Fuse bit per instruction fuses two dependent
  instructions
• Dense instruction encoding, 16/32-bit ISA design
• Special Features to Support the x86 ISA
• Condition codes
• Addressing modes
• Aware of long immediate displacement values

Core
32
-
bit instruction formats
21
bit Immediate
/
Displacement
10
b opcode
F
bit immediate
F
16
/
Displacement
10
b opcode
5
b Rds
11
b Immediate
/
Disp
10
b opcode
5
b Rds
5
b Rsrc
F
16
bit opcode
5
b Rds
5
b Rsrc
5
b Rsrc
F
Add
-
on
16
-
bit instruction formats for code density
F
5
b op
10
b Immd
/
Disp
F
5
b op
5
b Rds
5
b Rsrc
F
5
b op
5
b Rds
5
b Rsrc
Fusible ISA Instruction Formats
28
Fusing Profile

• About 50 of operations are fused
• Only 5-10 of non-fused are single-cycle ALU ops

29
Performance
Base Code Cache
fusing
shorter pipe
3-1 ALU
70
60
50
40
Nomarlized IPC speedup ()
30
20
10
0
-10
175.vpr
252.eon
254.gap
164.gzip
176.gcc
181.mcf
300.twolf
256.bzip2
186.crafty
255.vortex
Harmonic
197.parser
253.perlbmk
30
Virtual Private Machines

• Multi-core systems will have many hardware-level
  shared resources
• Multi-threaded processors
• Multi-level shared caches
• Shared memory ports
• Spares for fault tolerance
• And a number of important implementation
  dependences
• Non-uniform memory delays
• Power optimization features
• Fault tolerance features

31
Virtual Private Machines

• Co-design a MicroVisor to provide software with
  Virtual Private Machines
• Insulates conventional software from complicated
  implementation-dependent features
• Provides performance virtualization
• Unlike classic VMs
• Quality of Service (QoS)
• Performance Isolation

System
System
VM
VM
Conventional
OS
Functional
Hypervisor (VMM)
Virtualization
VPM
VPM
Performance
MicroVisor
Virtualization
Multi-Core Hardware
32
MicroVisor

• Virtualizes performance, not functionality
• Co-Designed software to support hardware resource
  management
• Concealed/isolated from all conventional software
  
• Much larger than microcode, does not consume
  processor chip real estate
• Uses conventional instructions (extended) so
  there is more likelihood of some cross-system
  portability

33
Real-izing Processors/Memory

• Separate Real Processors from Physical Processors
• OS assigns processes to Real Processors
• MicroVisor maps real processors to physical
  processors
• MicroVisor also maps real memory to physical
  memory
• Extend to cache memories

34
Virtual Private Machines

• MicroVisor maps high level requirements to
  hardware configuration
• Requires mechanisms to provide microarchitecture
  level QoS

Main Memory
Main Memory
Memory Controller
Mem
Mem
Mem
Mem
Controller
Controller
Controller
Controller
L2 Cache
L2 Cache
L2 Cache
L2 Cache
L2 Cache
Interconnection Net
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
L1 Cache
Proc. 0
Proc. 1
Proc. 0
Proc. 0
Proc. 0
Proc. 0
Threads
Threads
Thread 0
Thread 0
Thread 0
Thread 0
0 1
2 3
VPM 0
VPM 1
VPM 2
VPM 3
35
Applications

• Performance optimization
• Deal with NUMA
• Provide QoS and performance isolation in
  multi-threaded systems
• Power management
• Adjust resources to match power constraints
• Requires inferring demand for resources
• in contrast to conventional OS
• Fault tolerance
• Detected fault triggers MicroVisor
• Diagnose, reconfigure, re-map memory/processors

36
Summary

• Many types of VMs
• But common virtualization technologies
• An important system component
• Should be studied/taught as a discipline on its
  own
• Alongside OS, Application SW, HW
• Many avenues for computer architecture research
• Co-designed VMs
• Virtual Private Machines
• Adaptive microarchitecture
• Fault-tolerant implementations
• Primitives for supporting efficient VMs