Xen

XEN: installation

see notes on Debian wiki
aptitude install xen-linux-image-2.6.32-5-xen-amd64 xen-tools xen-utils
backup /etc/xen/xend-config.sxp
edit /etc/xen/xend-config.sxp, and uncomment line "(network-script network-bridge)" or for a non-default interface ethX:

(network-script 'network-bridge netdev=ethX bridge=xenbr0')

reboot
aptitude remove --purge linux-image-2.6.32-5-amd64 linux-image-2.6-amd64
/etc/default/grub => GRUB_DEFAULT=2
update-grub
reboot
cat /proc/xen/capabilities #(to check xend is running)

To install libvirt & get some additional tools:

apt-get install libvirt-bin
Edit /etc/xen/xend-config.sxp and set

(xend-unix-server yes)

XEN: guest creation

If you've a proxy it's much easier to have it defined in the environment variables:
- Edit /etc/environment

http_proxy=http://myproxy:port

- Logout/login
Edit the config file for creating images (/etc/xen-tools/xen-tools.conf) to your needs:

--- xen-tools.conf.orig 2010-05-30 22:42:25.000000000 +0200
+++ xen-tools.conf      2010-08-17 17:47:35.000000000 +0200
@@ -52,7 +52,7 @@
 # LVM volume group here instead
 #
 ##
-# lvm = vg0
+lvm = vg0
 
 
 #
@@ -125,12 +125,12 @@
 #  Disk and Sizing options.
 ##
 #
-size   = 4Gb      # Disk image size.
+size   = 8Gb      # Disk image size.
 memory = 128Mb    # Memory size
 swap   = 128Mb    # Swap size
 # noswap = 1      # Don't use swap at all for the new system.
 fs     = ext3     # use the EXT3 filesystem for the disk image.
-dist   = `xt-guess-suite-and-mirror -s` # Default distribution to install.
+dist   = squeeze  # Default distribution to install.
 image  = sparse   # Specify sparse vs. full disk images.
 
 #
@@ -166,7 +166,7 @@
 #
 # Uncomment this if you wish the images to use DHCP
 #
-# dhcp = 1
+dhcp = 1
 
 
 ##
@@ -184,7 +184,7 @@
 # Uncomment the following line if you wish to interactively setup
 # a new root password for images.
 #
-# passwd = 1
+passwd = 1
 
 #
 # If you'd like all accounts on your host system which are not present
@@ -254,10 +254,10 @@
 #
 #  You may specify the things to use here:
 #
-# serial_device = hvc0 #default
+serial_device = hvc0 #default
 # serial_device = tty1
 #
-# disk_device = xvda #default
+disk_device = xvda #default
 # disk_device = sda
 #

Example:

xen-create-image --hostname=myguest1 --memory=1G --swap=1G --vcpus=20
WARNING strangely the default root password was disabled with a '*' in /etc/shadow so impossible to log in. I had to mount manually the lv (/dev/vg0/myguest1-disk) to remove the '*'
One may prefer to setup the password interactively when creating the image with xen-create-image ... --passwd (or setup passwd=1 in /etc/xen-tools/xen-tools.conf as I've now shown in the above diff file)
lvm> lvcreate -L 1000G -n myguest1-data vg0
/etc/xen/myguest1.cfg: add data volume:

root        = '/dev/xvda2 ro'
disk        = [
                  'phy:/dev/vg0/myguest1-disk,xvda2,w',
                  'phy:/dev/vg0/myguest1-swap,xvda1,w',
                  'phy:/dev/vg0/myguest1-data,xvda3,w',
              ]

xm create -c /etc/xen/myguest1.cfg
Hook the data drive as /home
- mkfs.ext4 /dev/xvda3
- In /etc/fstab: /dev/xvda3 /home ext4 noatime,nodiratime,errors=remount-ro 0 1
- mount /home

XEN quickref

Xen Tools, Xen Shell

Guest images

xen-create-image --hostname=mynewguest
- creation of config file and base filesystem
xen-delete-image
xen-list-images
xen-update-image
- Does apt-get update; apt-get upgrade in the guest, !!! only if guest is NOT running

Guest start/stop

xm create /etc/xen/mynewguest.cfg
- actual starting of guest
xm shutdown mynewguest
xm destroy mynewguest

Guests info

xm list
- Show running xen guests
xm info
- Show xen version and many other info
xm uptime
xm top
xm vcpu-list
xm network-list mynewguest
xm log
xm dmesg
brctl show
virsh nodeinfo
virsh vcpuinfo mynewguest
virsh dominfo mynewguest

Guest management

xm console mynewguest
- CTRL+] to detach
xenscreen
- attaches one screen to all domains
- ctrl-a \ to quit properly, or ctrl-a d to detach
xm block-attach mynewguest phy://dev/vg0/myguest1-data2 /dev/xvda4 w
- to attach some device to a running guest
- not tried: "w!" for shared read/write, when the same device is attached to two different VMs
xm block-list mynewguest
xm block-list mynewguest --long
xm block-detach mynewguest 12345

XEN tuning

Autostart

If you want xen1.example.com to start automatically at the next boot of the system, then do this (if XENDOMAINS_AUTO=/etc/xen/auto in /etc/default/xendomains):

mkdir -p /etc/xen/auto
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

Suspend or halt

By default it will attempt to save running hosts when dom0 is halted. This means a "suspend-to-disk" of all domU in dom0 filesystem (/var/lib/xen/save). To prevent it and do regular shutdown of domU, edit /etc/default/xendomains and change:

#XENDOMAINS_SAVE=/var/lib/xen/save
XENDOMAINS_SAVE=""

Wallclock

Since Xen 4, wallclocks are separated and need to be sync, e.g. in each domU (& dom0):

apt-get install ntpdate

Edit /etc/default/ntpdate

NTPDATE_USE_NTP_CONF=no 
NTPSERVERS="....."

Add cron job /etc/cron.daily/ntpdate

#!/bin/sh
/usr/sbin/ntpdate-debian

And

chmod 755 /etc/cron.daily/ntpdate

Misc notes

Boot & DHCP

I've still sometimes some issues at boot time when the xenbr0 bridge doesn't catch its IP...
In those cases I find dhclient still hooked to the non-existing bnx1 IF (which was renamed pbnx1 by XEN). I've to kill it and start a similar dhclient on IF xenbr0.
Finally I renamed the default interface as eth0 and called the xen bridge script without parameters: (network-script network-bridge)

Allocating too much memory

I tried to create two guests with each 12Gb of RAM while the host had only 16Gb in total. I would have expected a nice handling of the error but it froze xend and I could even not shutdown properly.
And, oh, BTW, no, xen doesn't support memory overcommitment

Mount a DomU disk on Dom0

I had to transfer a large amount of data from a USB disk to a DomU. Easiest is to:

stop DomU (maybe it's possible to umount and detach block while running, I didn't try)
look into /etc/xen/mydomu.cfg for the disk you want, here

disk        = [
                  ...
                  'phy:/dev/vg0/sagemath-data,xvda3,w',
              ]

Hot plug block device to Dom0

xm block-attach 0 phy:/dev/vg0/sagemath-data xvda3 w

Mount it

mount /dev/xvda3 my-tmp-mount-point/

When done:

umount /dev/xvda3
xm block-detach 0 xvda3

Xen

Contents