Xen 3.0.3 with VMX

1
Xen 3.0.3 with VMX
Presented by psboy (Dec 7th,2006)
2
Scheme
- Introduction to Xen 3.0.3 with VMX -
Install/Build Domain 0 (Privileged
Domain) - Install Domain U (Unprivileged
Domain) - Operate Xen Virtual Machine / Xen VMX
Server - Q A
3
Introduction to Xen 3.0.3 with VMX
What is Xen?

• Xen is an open-source hypervisor that enables
  increased server utilization and server
  consolidation by enabling multiple operating
  system images to simultaneously run on a single
  physical server. Xen provides resource guarantees
  to virtual servers to ensure that application
  layer SLAs are met, including CPU, memory and I/O
  guarantees.
• Xen is the industrys fastest and most secure
  infrastructure virtualization software
  technology, and has been endorsed and adopted by
  over 20 of the industrys major vendors,
  including AMD, Dell, Egenera, Hewlett-Packard,
  IBM, Intel, Mellanox Technologies, Network
  Appliance, Novell, Red Hat, Sun Microsystems,
  TopSpin, Unisys and Voltaire.

4
Introduction to Xen 3.0.3 with VMX

• Xen is licensed under the GNU General Public
  License (GPL2), and is available free for
  download. It is offered by XenSource and other
  vendors as a supported enterprise software
  product.
• Xen was created in 2003 at the University of
  Cambridge Computer Laboratory in whats known as
  the Xen Hypervisor project led by Ian Pratt.
  Additional team members include Keir Fraser,
  Steven Hand, and Christian Limpach. This same
  team founded XenSource with experienced Silicon
  Valley technology entrepreneurs Nick Gault and
  Simon Crosby.

Who create Xen?
5
Introduction to Xen 3.0.3 with VMX
What are the business benefits of adopting Xen?

• Increased server utilization Data centers can
  achieve dramatic improvements in server
  utilization and enable server consolidation,
  reducing wasted capital costs and personnel
  management expenses.
• Reduced burden on IT By enabling server
  consolidation, Xen reduces the cost, complexity,
  and personnel time required for data center
  server management. XenSources virtualization
  infrastructure software solutions provide both
  Xen based virtualization for servers and
  management, and control and automation tools that
  allow IT organizations to operationalize
  virtualization in a seamless, low cost fashion.
• Lowest TCO Today virtualization has yet to enter
  the heart of the data center. There are two
  reasons poor performance of proprietary
  hypervisors and their tremendous cost. Xen is the
  best performing hypervisor, and its free. That
  radically changes the economics of the adoption
  of virtualization.

6
Xen 3.0 features
Introduction to Xen 3.0.3 with VMX

• Support for up to 32-way SMP guest
• Intel VT-x and AMD Pacifica hardware
  virtualization support
• PAE support for 32 bit servers with over 4 GB
  memory
• x86/64 support for both AMD64 and EM64T

Xen 3.0.3 features

• New easy-to-use CPU scheduler including weights,
  caps and automatic load balancing
• Much enhanced support for unmodified ('hvm')
  guests including windows and legacy linux systems
• Support for sparse and copy-on-write disks
• High performance networking using segmentation
  off-load

7
Xen Networking
Introduction to Xen 3.0.3 with VMX
8
Logical network cards connected between dom0 and
dom1
Introduction to Xen 3.0.3 with VMX
9
Introduction to Xen 3.0.3 with VMX
Illustration on network-bridge and vif-bridge
10
Intel Virtualization Technology
Introduction to Xen 3.0.3 with VMX
Xen 3.0 architecture
11
Introduction to Xen 3.0.3 with VMX
VMCS
Shadow Page Table
12
Introduction to Xen 3.0.3 with VMX

• For the IA-32 architecture, a VMCS structure is
  created for each CPU in a HVM domain
• Instructions such as CPUID, MOV from/to CR3, MOV
  to CR0/CR4, RDMSR, WRMSR, HLT, INVLPG, MOV from
  CR8, MOV DR, and MWAIT are intercepted as VM
  exits.
• Exceptions/faults, such as page fault, are
  intercepted as VM exits, and virtualized
  exceptions/faults are injected on VM entry to
  guests.
• External interrupts unrelated to guests are
  intercepted as VM exits, and virtualized
  interrupts are injected on VM entry to the
  guests.
• Read shadows are created for the guest CR0, CR4,
  and time stamp counter (TSC). Read accesses to
  such registers will not cause VM exit, but will
  return the shadow values.

13
Introduction to Xen 3.0.3 with VMX

• The virtual MMU module supports all page table
  formats that can be used by the guest OS
• For IA-32
• it supports 2-level page tables with 4 KB page
  size for 32-bit guests.
• For IA-32 Physical Address Extension (PAE)
• it supports 2-level page tables with 4 KB page
  sizes for 32-bit guests.
• it supports 3-level page tables with 4 KB and 2
  MB page sizes for 32-bit PAE guests.
• For Intel EM64T
• it supports 2-level page tables with 4 KB page
  size for 32-bit guests.
• it supports 3-level page tables with 4 KB and 2
  MB page sizes for 32-bit PAE guests.
• it supports 4-level page tables with 4 KB and 2
  MB page sizes for 64-bit guests.

14
Introduction to Xen 3.0.3 with VMX
Device IO Virtualization The primary function of
the device model is to wait for an I/O event
from the HVM guest and dispatch it to the
appropriate device emulation model. Once the
device emulation model completes the I/O
request, it will respond back with the result.
A shared memory between the device model and the
Xen hypervisor is used for communication of I/O
request and response.
I/O Device virtualization model
15
Install Domain 0 (Privileged Domain)

• Before install Domain 0
• Unmodified operating systems are supported.
  Support is provided for running Windows on VT or
  AMD-V hardware.
• Xen supports 32-bit processors with and without
  Physical Address Extension (PAE), 64-bit
  processors, and Symmetric Multiprocessing (SMP)
  guest operating systems.
• Enterprise Linux guest operating systems
  supported Red Hat Enterprise Linux 3.5, 4.1,
  SUSE Linux Enterprise Server 9.2 Also Fedora
  Core 4, CentOS 4, SUSE Open Linux 10, Debian and
  others.
• Current test with Xen 3.03 for example , linux
  kernel 2.6.16.29.

16
Install Domain 0 (Privileged Domain)

• System requirement
• Cpu need support VMX /SVM
• You can use command cat /proc/cpuinfoto see if
  your cpu have vmx/svm flag
• Chipset need support VMX
• Bios need to enable VMX /SVM
• Intel VMX default is disable.
• AMD SVM default is enable.
• You can try to use Xen Test CD first before test
  Xen HVM.

17
Install Domain 0 (Privileged Domain)

• Several ways to install Domain 0
• Install from binary tarball
• Quick for installation but no preferred modules
  contain in official release
• Install from RPMs
• Quick for installation but no preferred modules
  contain in official release
• Install from source
• Kernel and module can customize and more flexible
• Dom0 kernel and hypervisor tuning is available

18
Install Domain 0 (Privileged Domain)

• Step 0.1
• Download Xen 3.03 source from http//bits.xensourc
  e.com/oss-xen/release/3.0.3-0/src.tgz/xen-3.0.3_0-
  src.tgz
• Download linux-2.6.16.29.tar.bz2 from
  http//www.kernel.org
• Step 0.2
• Install RHEL4 AS U4 32bit.(Full install)
• Recommend set your system resolution to 1024x768
  for later install Domain U.
• Dont use whole disk as / (left some space for
  guest os )

19
Install Domain 0 (Privileged Domain)

• Step 1
• Extract xen-3.0.3_0-src.tgz.
• gunzip xen-3.0.3_0-src.tgz.
• tar zxvf xen-3.0.3_0-src.tar.
• cd xen-3.0.3_0-src
• Copy linux-2.6.16.29.tar.bz2 to xen-3.0.3_0-src
  directory.
• Dont extract linux-2.6.16.29.tar.bz2 !!

20
Install Domain 0 (Privileged Domain)

• Step 2 Compile and install Xen and Dom0 kernel
• Check if your system need PAE support.When system
  memory over 4G,you will need PAE support for high
  address memory.
• Use XEN_TARGET_X86_PAEy to make parameter.
• XEN_TARGET_X86_PAEy make world install
• Use the following command if you dont need PAE
  support.
• make world install

21
Install Domain 0 (Privileged Domain)

• If you want to rebuild or clean source directory
  before compile , use the following command.
• make clean world install
• If you want to customize your dom0 kernel config
  , use the following command.
• make linux-2.6-xen-config CONFIGMODEmenuconfig
• make linux-2.6-xen-build
• make linux-2.6-xen-install

22
Install Domain 0 (Privileged Domain)

• You can build dom0 and domU by KERNELSlinux-2.6-
  xen0 linux-2.6-xenU parameter.
• KERNELSlinux-2.6-xen0 linux-2.6-xenU make
  world install
• To rebuild without change and clean source tree ,
  use the following command.
• make dist
• make install

23
Install Domain 0 (Privileged Domain)

• Step 3 Make initial ram disk for domain 0
• If you want to use initial ram disk , your kernel
  configuration should contain BLK_DEV_RAMy and
  BLK_DEV_INITRDy setting , and BLK_DEV_RAM can
  not be compile as module.
• depmod 2.6.16.29-xen
• mkinitrd f withata_piix withscsi_mod
  /boot/initrd-2.6.16.29-xen 2.6.16.29-xen
• -f overwrite existing image file
• --withmodule_name load module in the initial
  ram disk

24
Install Domain 0 (Privileged Domain)

• Step 4 Modify grub.conf
• Grub config need to modify for apply compiled xen
  kernel and dom0 kernel
• Add entry in grub.conf such like below
• title Xen 3.0.3
• root (hd0,0)
• kernel /xen-3.0.3-0.gz consolevga
  dom0_mem262144
• module /vmlinuz-2.6.16.29-xen ro
  root/dev/VolGroup00/LogVol01 rhgb consoletty0
• module /initrd-2.6.16.29-xen

25
Install Domain 0 (Privileged Domain)

• Step 5 Disable tls library
• mv /lib/tls /lib/tls.disabled
• Step 6 Reboot system and choose Xen 3.03 from
  grub boot menu
• Step 7 Check your Domain 0 Xend service is
  running
• chkconfig xend on
• service xend start

26
Install Domain U (Unprivileged Domain)

• Before install Domain U
• Check your hardware (bios) support VMX
• xm infogrep xen_cap
• xen_caps xen-3.0-x86-32p hvm-3.0-x86-32
  hvm-3.0-x86_32p
• xm dmesggrep i vmxon
• (XEN) VMXON is done
• Each time you start a guest domain , Xen will
  random select a Mac 00163EXXXXXX if you
  didnt set Mac in your guest domain config file.

27
Install Domain U (Unprivileged Domain)

• Install windows based guest OS
• (Install windows2003 RC2 as example)
• Step 1 Create lvm partition for guest OS
• lvcreate L 10000 n w2k3 VolGroup00
• Step 2 Use xmexample.hvm as sample and modify it.
• cp /etc/xen/xmexample.hvm /etc/xen/w2k3.hvm
• Modify below entries in w2k3.hvm
• memory 512
• name w2k3
• disk phy/dev/VolGroup00/w2k3,hda,w
  ,phy/dev/hdb,hdbcdrom,r
• cdrom /dev/hdb
• boot d
• sdl1

28
Install Domain U (Unprivileged Domain)

• Step 3 Put windows 2003 RC2 cd into cdrom
• If /dev/hdb does not exist,use command mknod
  /dev/hdb b 3 64 to create /dev/hdb device node.
• Step 4 Use xm command power on guest os
• xm create /etc/xen/w2k3.hvm
• Step 5 Follow the popup window screen and
  continue install guest os
• Press any key when boot from windows 2003 cdrom.
• After first time install complete and reboot ,
  change boot d to boot c.
• You can use iso file emulate cdrom

29
Install Domain U (Unprivileged Domain)

• Install linux based guest OS
• (Install sles10 as example)
• Step 1 Create lvm partition for guest OS
• lvcreate L 10000 n sl10 VolGroup00
• Step 2 Use xmexample.hvm as sample and modify it.
• cp /etc/xen/xmexample.hvm /etc/xen/sl10.hvm
• Modify below entries in sl10.hvm
• memory 512
• name sl10
• disk phy/dev/VolGroup00/sl10,hda,w
  ,phy/dev/hdb,hdbcdrom,r
• cdrom /dev/hdb
• boot d
• vnc1
• vncviewer1

30
Install Domain U (Unprivileged Domain)

• If you have multiple nic devices , you need to
  modify xend-config.sxp , sl10.hvm ,
  script/mynetwork-script .
• Modify sl10.hvm
• vif typeioemu , bridgexenbr0,typeioemu ,
  bridgexenbr1
• Add mynetwork-script to /etc/xen/script
• !/bin/sh mynetwork-script
• dir(dirname 0)
• dir/network-bridge _at_ vifnum0 netdeveth0
• dir/network-bridge _at_ vifnum1 netdeveth1
• Modify xend-config.sxp (search network-script)
• (network-script mynetwork-script)

31
Install Domain U (Unprivileged Domain)

• Step 3 Put SLES10 cd into cdrom
• If /dev/hdb does not exist,use command mknod
  /dev/hdb b 3 64 to create /dev/hdb device node.
• Step 4 Use xm command power on guest os
• xm create /etc/xen/sl10.hvm
• Step 5 Follow the popup window screen and
  continue install guest os
• Type linux when boot from SLES10 cdrom.
• After first time install complete and reboot ,
  change boot d to boot c.
• Suggest to adjust resolution to 800x600 , use
  vga0x314
• You can use iso file emulate cdrom
  

32
Operate Xen Virtual Machine / Xen VMX Server

• Booting domain 0
• service xend start
• Booting guest domain
• xm create /etc/xen/xmexample.hvm
• You need modify your guest domain config file
  before start it
• Stop guest domain
• Shutdown by init 0 command in guest OS
• xm destroy domain-name (Or xm shutdown
  domain-name)

33
Operate Xen Virtual Machine / Xen VMX Server

• Booting domain 0
• service xend start
• Booting guest domain
• xm create /etc/xen/xmexample.hvm
• You need modify your guest domain config file
  before start it
• Stop guest domain
• Shutdown by init 0 command in guest OS
• xm destroy domain-name
• XM tool (use this tool to operate guest domain)
• xm help

34
Operate Xen Virtual Machine / XenEnterprise Server
Basic Management Commands xm list which lists
all domains running in rows of the following
format The meaning of each field is as
follows name The descriptive name of the
virtual machine. domid The number of the domain
ID this virtual machine is running in.
memory Memory size in megabytes. vcpus The
number of virtual CPUs this domain has.
35
Operate Xen Virtual Machine / XenEnterprise Server
state Domain state consists of 5 fields
r running b blocked p paused s shutdown
c crashed cputime How much CPU time (in
seconds) the domain has used so far.
36
Operate Xen Virtual Machine / XenEnterprise Server
xm top Monitor a host and domains in
real time xm dmesg Read and/or
clear Xends message buffer xm info Get
information about Xen host xm uptime Print
uptime for a domain
37
Operate Xen Virtual Machine / XenEnterprise Server
More detail guest domain config file
entry vif List of MAC addresses (random addresses
are assigned if not given) and bridges to use for
the domain's network interfaces, e.g. vif
'mac00163E000011, bridgexen-br0',
'bridgexen-br1' to assign a MAC address and
bridge to the first interface and assign a
different bridge to the second interface, leaving
xend to choose the MAC address. disk List of
block devices to export to the domain e.g. disk
'phyhda1,sda1,r' exports physical device
/dev/hda1 to the domain as /dev/sda1 with
read-only access. Exporting a disk read-write
which is currently mounted is dangerous - if you
are certain you wish to do this, you can specify
w as the mode. memory Memory size in
megabytes. extra Extra string to append to the
kernel command line (if any)
38
Question and Answer

• Thank You