Difference between revisions of "Xen"

From YobiWiki
Jump to navigation Jump to search
m (Reverted edits by Etegohy (Talk) to last revision by PhilippeTeuwen)
m
Line 1: Line 1:
  +
===XEN: installation===
−
''Converted with [[MediaWiki#HTML-WikiConverter|HTML::WikiConverter::MediaWiki]] from my old phpwiki site''
  
  +
* see [http://wiki.debian.org/Xen notes on Debian wiki]
−
----
  
  +
* aptitude install xen-linux-image-2.6.32-5-xen-amd64 xen-tools xen-utils
−
=== Install of Xen Source (paravirtualizer) on Debian Sarge ===
  
  +
* backup /etc/xen/xend-config.sxp
−  
  +
* edit /etc/xen/xend-config.sxp, and uncomment line "(network-script network-bridge)" or for a non-default interface ethX:
−
==== Install of Xen ====
  
  +
(network-script 'network-bridge netdev=ethX bridge=xenbr0')
−  
  +
* reboot
−
I used the binary distribution of xen 2.0.7 altough I also tried to compile a custom kernel as I had question about hardware support which I didn't find in the pached kernel. I posted it on the xen-users ML, no answer yet.
  
  +
* aptitude remove --purge linux-image-2.6.32-5-amd64 linux-image-2.6-amd64
−  
  +
* /etc/default/grub => GRUB_DEFAULT=2
−
To be documented (note are offline)
  
  +
* update-grub
−  
  +
* reboot
−
update grub's menu.lst and rename /lib/tls in /lib/tls.disabled as it is not natively supported by xen and need a real emulation which will reduced performance a lot (this step has to be done in all unpriviledged guest too (enforce it ?)
  
  +
* cat /proc/xen/capabilities #(to check xend is running)
−  
−
==== Xen topology ====
  
−  
−
Xen uses a para-vitualisation hypervisor running in ring 0 (x86 CPU). <br />The kernel runs in ring1 and user-app in ring3. <br />Hardware management is done inside a privileged kernel named domain0 (guest OSes are called domainX). <br />Xen uses a xend daemon in dom0 to manage network, cpu, memory and export standard devices to the domX guest (generally eth0, a root fs and some swap space). <br />xend also manages bridging the real eth0 and sets up a xen-br0 device <br />xend is not running by default on dom0 at startup
  
−  
−
Invoking
  
   
  +
===XEN: guest creation===
  +
* If you've a proxy it's much easier to have it defined in the environment variables:
  +
** Edit /etc/environment
  +
http_proxy=http://myproxy:port
  +
** Logout/login
  +
* Edit the config file for creating images (/etc/xen-tools/xen-tools.conf) to your needs:
  +
<source lang=diff>
  +
--- xen-tools.conf.orig 2010-05-30 22:42:25.000000000 +0200
  +
+++ xen-tools.conf 2010-08-17 17:47:35.000000000 +0200
  +
@@ -52,7 +52,7 @@
  +
# LVM volume group here instead
  +
#
  +
##
  +
-# lvm = vg0
  +
+lvm = vg0
 
 
−
update-rc.d xend defaults 80
  
−  
−
does the job as dmesg says
  
−  
 
 
−
eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
  
−
eth0: Promiscuous mode enabled.
  
−
device eth0 entered promiscuous mode
  
−
xen-br0: port 1(eth0) entering learning state
  
−
xen-br0: topology change detected, propagating
  
−
xen-br0: port 1(eth0) entering forwarding state
  
−  
−
and
  
−  
−
 
−
ns1:/home/skycode# xm list
  
−
Name Id Mem(MB) CPU State Time(s) Console
  
−
Domain-0 0 251 0 r---- 13.0
  
−  
−
So domain0 is now running (in a small 4Go partition, the rest of the disk is left for LVM)
  
−  
−
setting-up LVM for the hosts
  
−  
−
 
−
apt-get install lvm2
  
−  
−
Partitioning the free space left on the disk
  
−  
−
 
−
ns1:/home/skycode# fdisk /dev/hda
  
−
 
−
The number of cylinders for this disk is set to 9964.
  
−
There is nothing wrong with that, but this is larger than 1024,
  
−
and could in certain setups cause problems with:
  
−
1) software that runs at boot time (e.g., old versions of LILO)
  
−
2) booting and partitioning software from other OSs
  
−
(e.g., DOS FDISK, OS/2 FDISK)
  
−
 
−
Command (m for help): p
  
−
 
−
Disk /dev/hda: 81.9 GB, 81964302336 bytes
  
−
255 heads, 63 sectors/track, 9964 cylinders
  
−
Units = cylinders of 16065 * 512 = 8225280 bytes
  
−
 
−
Device Boot Start End Blocks Id System
  
−
/dev/hda1 1 486 3903763+ 83 Linux
  
−
/dev/hda2 * 487 608 979965 82 Linux swap / Solaris
  
−
 
−
Command (m for help): n
  
−
Command action
  
−
e extended
  
−
p primary partition (1-4)
  
−
p
  
−
Partition number (1-4): 3
  
−
First cylinder (609-9964, default 609):
  
−
Using default value 609
  
−
Last cylinder or +size or +sizeM or +sizeK (609-9964, default 9964):
  
−
Using default value 9964
  
−
 
−
Command (m for help): p
  
−
 
−
Disk /dev/hda: 81.9 GB, 81964302336 bytes
  
−
255 heads, 63 sectors/track, 9964 cylinders
  
−
Units = cylinders of 16065 * 512 = 8225280 bytes
  
−
 
−
Device Boot Start End Blocks Id System
  
−
/dev/hda1 1 486 3903763+ 83 Linux
  
−
/dev/hda2 * 487 608 979965 82 Linux swap / Solaris
  
−
/dev/hda3 609 9964 75152070 83 Linux
  
−
 
−
Command (m for help): t
  
−
Partition number (1-4): 3
  
−
Hex code (type L to list codes): 8e
  
−
Changed system type of partition 3 to 8e (Linux LVM)
  
−
 
−
Command (m for help): p
  
−
 
−
Disk /dev/hda: 81.9 GB, 81964302336 bytes
  
−
255 heads, 63 sectors/track, 9964 cylinders
  
−
Units = cylinders of 16065 * 512 = 8225280 bytes
  
−
 
−
Device Boot Start End Blocks Id System
  
−
/dev/hda1 1 486 3903763+ 83 Linux
  
−
/dev/hda2 * 487 608 979965 82 Linux swap / Solaris
  
−
/dev/hda3 609 9964 75152070 8e Linux LVM
  
−
 
−
Command (m for help): w
  
−
The partition table has been altered!
  
−
 
−
Calling ioctl() to re-read partition table.
  
−  
−
reboot :-)
  
−  
−
Create physical volumes
  
−  
−
pvcreate /dev/hda3&lt;/pre&gt;
  
−
add it to a volume groupvgcreate volg0 /dev/hda3
  
−
Volume group "volg0" successfully created
  
−  
−
Create 2 partition (root and swap for the first test domain)
  
−  
−
ns1:/home/skycode# lvcreate -L 2G -n test0root volg0
  
−
Logical volume "test0root" created
  
−
ns1:/home/skycode# lvcreate -L 256M -n test0swap volg0
  
−
Logical volume "test0swap" created
  
−
ns1:/home/skycode# mke2fs -j /dev/volg0/test0root
  
−
mke2fs 1.37 (21-Mar-2005)
  
−
Filesystem label=
  
−
OS type: Linux
  
−
Block size=4096 (log=2)
  
−
Fragment size=4096 (log=2)
  
−
262144 inodes, 524288 blocks
  
−
26214 blocks (5.00%) reserved for the super user
  
−
First data block=0
  
−
16 block groups
  
−
32768 blocks per group, 32768 fragments per group
  
−
16384 inodes per group
  
−
Superblock backups stored on blocks:
  
−
32768, 98304, 163840, 229376, 294912
  
−
 
−
Writing inode tables: done
  
−
Creating journal (8192 blocks): done
  
−
Writing superblocks and filesystem accounting information: done
  
−
 
−
This filesystem will be automatically checked every 38 mounts or
  
−
180 days, whichever comes first. Use tune2fs -c or -i to override.
  
−
ns1:/home/skycode# mkswap /dev/volg0/test0swap
  
−
Setting up swapspace version 1, size = 268431 kB
  
−
no label, UUID=451e4e02-5d9f-4524-922a-c7432e844b21
  
−
ns1:/home/skycode#
  
−  
−
Install a minimal debian sarge with debootstrap and back-it up for futur installs
  
−  
−
 
−
ns1:/home/skycode# debootstrap --arch i386 sarge /mnt/xen_install http://www.uk.debian.org/debian/
  
−
Lots of output ...
  
−
ns1:/home/skycode# cd /mnt/xen_install/
  
−
ns1:/mnt/xen_install# mkdir /var/xen_images
  
−
ns1:/mnt/xen_install# tar -jpcf /var/xen_images/debian-sarge-pre-config.tar.bz2 .
  
−  
−
Now it's config time
  
−  
−
fstab
  
−  
−
<nowiki>
  
−
ns1:/mnt/xen_install/etc# cat fstab
  
−
# Fstab for virtual domain root is exported by dom0 as sda1
  
−
# And swap as sda2
  
 
#
  
#
  +
@@ -125,12 +125,12 @@
−
# Exxoss - brewed on March 24th 2006
  
  +
# Disk and Sizing options.
−
 
  +
##
−
/dev/sda1 / ext3 defaults 0 1
  
  +
#
−
/dev/sda2 swap swap defaults 0 0
  
  +
-size = 4Gb # Disk image size.
−
proc /proc proc defaults 0 0</nowiki>
  
  +
+size = 8Gb # Disk image size.
−  
  +
memory = 128Mb # Memory size
−
hostname
  
  +
swap = 128Mb # Swap size
−  
  +
# noswap = 1 # Don't use swap at all for the new system.
−
 
  +
fs = ext3 # use the EXT3 filesystem for the disk image.
−
ns1:/mnt/xen_install/etc# cat hostname
  
  +
-dist = `xt-guess-suite-and-mirror -s` # Default distribution to install.
−
test0
  
  +
+dist = squeeze # Default distribution to install.
−  
  +
image = sparse # Specify sparse vs. full disk images.
−
hosts
  
−  
−
 
−
ns1:/mnt/xen_install/etc# cat hosts
  
−
127.0.0.1 localhost
  
−  
−
Network interfaces
  
−  
−
<nowiki>
  
−
ns1:/mnt/xen_install/etc# cat network/interfaces
  
−
# Used by ifup(8) and ifdown(8). See the interfaces(5) manpage or
  
−
# /usr/share/doc/ifupdown/examples for more information.
  
 
 
 
#
  
#
  +
@@ -166,7 +166,7 @@
−
# Exxoss - brewed march 24th 2006
  
−
# virtual domain interface config file
  
−
# dom0 export a NIC as eth0
  
 
#
  
#
  +
# Uncomment this if you wish the images to use DHCP
  +
#
  +
-# dhcp = 1
  +
+dhcp = 1
 
 
−
# Loopback device
  
−
auto lo
  
−
iface lo inet loopback
  
 
 
  +
##
−
# eth0 device
  
  +
@@ -184,7 +184,7 @@
−
auto eth0
  
  +
# Uncomment the following line if you wish to interactively setup
−
iface eth0 inet static
  
  +
# a new root password for images.
−
address 192.168.1.111
  
  +
#
−
netmask 255.255.255.0
  
  +
-# passwd = 1
−
gateway 192.168.1.1</nowiki>
  
  +
+passwd = 1
−  
−
sources.list
  
−  
−
<nowiki>
  
−
ns1:/mnt/xen_install/etc# cat apt/sources.list
  
−
#deb file:///cdrom/ sarge main
  
 
 
−
deb </nowiki>http://ftp.debian.skynet.be/ftp/debian/ stable main contrib
  
−
deb-src http://ftp.debian.skynet.be/ftp/debian/ stable main contrib
  
−
 
−
deb http://security.debian.org/ stable/updates main contrib
  
−  
−
lib/tls (again)
  
−  
−
 
−
ns1:/mnt/xen_install/etc# cd ..
  
−
ns1:/mnt/xen_install# mv lib/tls lib/tls.disabled
  
−
 
−
umount /mnt/xen_install
  
−  
−
Configure xen for the new dom
  
−  
−
in /etc/xen/test0.conf
  
−  
−
<nowiki>
  
−
ns1:/etc/xen# cat /etc/xen/test0.conf
  
 
#
  
#
  +
# If you'd like all accounts on your host system which are not present
−
# Test Xen domU config file
  
  +
@@ -254,10 +254,10 @@
 
#
  
#
  +
# You may specify the things to use here:
−
 
  +
#
−
name="test0"
  
  +
-# serial_device = hvc0 #default
−
memory=256
  
  +
+serial_device = hvc0 #default
−
kernel="/boot/vmlinuz-2.6-xenU"
  
  +
# serial_device = tty1
−
nics=1
  
  +
#
−
disk=</nowiki><span>[%27phy%3Avolg0 'phy:volg0][%27phy%3Avolg0/test0root%2Csda1%2Cw%27%2C%20%27phy%3Avolg0 /test0root,sda1,w', 'phy:volg0]<span class="named-wikiunknown"><u>test0swap,sda2,w'</u>[%27phy%3Avolg0/test0root%2Csda1%2Cw%27%2C%20%27phy%3Avolg0/test0swap%2Csda2%2Cw%27?action=create [[Image:uww.png|?]]]</span></span>
  
  +
-# disk_device = xvda #default
−
root="/dev/sda1 ro"
  
  +
+disk_device = xvda #default
−
ns1:/etc/xen#
  
  +
# disk_device = sda
−  
  +
#
−
==== xm (xend ctl tool) commands ====
  
  +
</source>
   
  +
Example:
−
Create a new domain &lt;pre&gt;xm create /etc/xen/test0.conf -c&lt;/pre&gt;
  
   
  +
* xen-create-image --hostname=myguest1 --memory=12288 --swap=12288 --vcpus=20
−
Logout from console to dom0 : Ctrl-]
  
  +
* WARNING strangely the default root password was disabled with a '*' in /etc/shadow so impossible to log in. I had to mount manually the lv (/dev/vg0/myguest1-disk) to remove the '*'<br>One may prefer to setup the password interactively when creating the image with xen-create-image ... --passwd (or setup passwd=1 in /etc/xen-tools/xen-tools.conf as I've now shown in the above diff file)
  +
* lvm> lvcreate -L 1000G -n myguest1-data vg0
  +
* /etc/xen/myguest1.cfg: add data volume:
  +
root = '/dev/xvda2 ro'
  +
disk = [
  +
'phy:/dev/vg0/myguest1-disk,xvda2,w',
  +
'phy:/dev/vg0/myguest1-swap,xvda1,w',
  +
'phy:/dev/vg0/myguest1-data,xvda3,w',
  +
]
   
  +
* xm create -c /etc/xen/myguest1.cfg
−
list runing domains &lt;pre&gt; xm list &lt;/pre&gt;
  
  +
* Hook the data drive as /home
  +
** mkfs.ext4 /dev/xvda3
  +
** In /etc/fstab: /dev/xvda3 /home ext4 noatime,nodiratime,errors=remount-ro 0 1
  +
** mount /home
  +
==XEN quickref==
  +
[http://xen-tools.org/software/xen-tools/ Xen Tools], [http://xen-tools.org/software/xen-shell Xen Shell]
   
  +
See also [http://www.howtoforge.com/virtualization-with-xen-on-debian-lenny-amd64-p2 here] and [http://www.howtoforge.com/xen_tools_xen_shell_argo here]
−
stop a runing domain &lt;pre&gt; xm shutdown domain &lt;/pre&gt;
  
   
  +
* xen-create-image --hostname=mynewguest
−
==== Tests ====
  
  +
* xen-delete-image
  +
* xen-list-images
  +
* xen-update-image # apt-get update; apt-get upgrade in the guest, !!! only if guest is NOT running
  +
* xm create /etc/xen/mynewguest.cfg
  +
* xm list # Show running xen guests
  +
* xm info # Show xen version and many other info
  +
* xm console mynewguest
  +
** CTRL+] to detach
  +
* xm shutdown mynewguest
  +
* xm destroy mynewguest
  +
==XEN tuning==
  +
If you want xen1.example.com to start automatically at the next boot of the system, then do this:
   
  +
ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto
−
what happens if I try to allocate more meory to a domain than I have available
  
   
  +
By default it will attempt to save runing hosts when dom0 is halted. This means a "suspend-to-disk" of all domU in dom0 filesystem (/var/lib/xen/save). To prevent it and do regular shutdown of domU, edit /etc/default/xendomains and change:
−
it fail rturning an error code (12)
  
  +
#XENDOMAINS_SAVE=/var/lib/xen/save
  +
XENDOMAINS_SAVE=""
   
  +
==Misc notes==
−
set the cpu time limit
  
  +
===Boot & DHCP===
  +
I've still sometimes some issues at boot time when the xenbr0 bridge doesn't catch its IP...
  +
<br>In those cases I find dhclient still hooked to the non-existing bnx1 IF (which was renamed pbnx1 by XEN). I've to kill it and start a similar dhclient on IF xenbr0.
  +
<br>Finally I renamed the default interface as eth0 and called the xen bridge script without parameters: (network-script network-bridge)
   
  +
===Allocating too much memory===
−
use cpu_weight in config file like cpu_weight=5
  
  +
I tried to create two guests with each 12Gb of RAM while the host had only 16Gb in total. I would have expected a nice handling of the error but it froze xend and I could even not shutdown properly.
  +
<br>And, oh, BTW, no, [http://blog.coparsystems.com/?p=36&more=1&c=1&tb=1&pb=1 xen doesn't support memory overcommitment]

Revision as of 08:31, 7 April 2011

XEN: installation

  • see notes on Debian wiki
  • aptitude install xen-linux-image-2.6.32-5-xen-amd64 xen-tools xen-utils
  • backup /etc/xen/xend-config.sxp
  • edit /etc/xen/xend-config.sxp, and uncomment line "(network-script network-bridge)" or for a non-default interface ethX:
(network-script 'network-bridge netdev=ethX bridge=xenbr0')
  • reboot
  • aptitude remove --purge linux-image-2.6.32-5-amd64 linux-image-2.6-amd64
  • /etc/default/grub => GRUB_DEFAULT=2
  • update-grub
  • reboot
  • cat /proc/xen/capabilities #(to check xend is running)

XEN: guest creation

  • If you've a proxy it's much easier to have it defined in the environment variables:
    • Edit /etc/environment
http_proxy=http://myproxy:port
    • Logout/login
  • Edit the config file for creating images (/etc/xen-tools/xen-tools.conf) to your needs:
--- xen-tools.conf.orig 2010-05-30 22:42:25.000000000 +0200
+++ xen-tools.conf      2010-08-17 17:47:35.000000000 +0200
@@ -52,7 +52,7 @@
 # LVM volume group here instead
 #
 ##
-# lvm = vg0
+lvm = vg0
 
 
 #
@@ -125,12 +125,12 @@
 #  Disk and Sizing options.
 ##
 #
-size   = 4Gb      # Disk image size.
+size   = 8Gb      # Disk image size.
 memory = 128Mb    # Memory size
 swap   = 128Mb    # Swap size
 # noswap = 1      # Don't use swap at all for the new system.
 fs     = ext3     # use the EXT3 filesystem for the disk image.
-dist   = `xt-guess-suite-and-mirror -s` # Default distribution to install.
+dist   = squeeze  # Default distribution to install.
 image  = sparse   # Specify sparse vs. full disk images.
 
 #
@@ -166,7 +166,7 @@
 #
 # Uncomment this if you wish the images to use DHCP
 #
-# dhcp = 1
+dhcp = 1
 
 
 ##
@@ -184,7 +184,7 @@
 # Uncomment the following line if you wish to interactively setup
 # a new root password for images.
 #
-# passwd = 1
+passwd = 1
 
 #
 # If you'd like all accounts on your host system which are not present
@@ -254,10 +254,10 @@
 #
 #  You may specify the things to use here:
 #
-# serial_device = hvc0 #default
+serial_device = hvc0 #default
 # serial_device = tty1
 #
-# disk_device = xvda #default
+disk_device = xvda #default
 # disk_device = sda
 #

Example:

  • xen-create-image --hostname=myguest1 --memory=12288 --swap=12288 --vcpus=20
  • WARNING strangely the default root password was disabled with a '*' in /etc/shadow so impossible to log in. I had to mount manually the lv (/dev/vg0/myguest1-disk) to remove the '*'
    One may prefer to setup the password interactively when creating the image with xen-create-image ... --passwd (or setup passwd=1 in /etc/xen-tools/xen-tools.conf as I've now shown in the above diff file)
  • lvm> lvcreate -L 1000G -n myguest1-data vg0
  • /etc/xen/myguest1.cfg: add data volume:
root        = '/dev/xvda2 ro'
disk        = [
                  'phy:/dev/vg0/myguest1-disk,xvda2,w',
                  'phy:/dev/vg0/myguest1-swap,xvda1,w',
                  'phy:/dev/vg0/myguest1-data,xvda3,w',
              ]
  • xm create -c /etc/xen/myguest1.cfg
  • Hook the data drive as /home
    • mkfs.ext4 /dev/xvda3
    • In /etc/fstab: /dev/xvda3 /home ext4 noatime,nodiratime,errors=remount-ro 0 1
    • mount /home

XEN quickref

Xen Tools, Xen Shell

See also here and here

  • xen-create-image --hostname=mynewguest
  • xen-delete-image
  • xen-list-images
  • xen-update-image # apt-get update; apt-get upgrade in the guest, !!! only if guest is NOT running
  • xm create /etc/xen/mynewguest.cfg
  • xm list # Show running xen guests
  • xm info # Show xen version and many other info
  • xm console mynewguest
    • CTRL+] to detach
  • xm shutdown mynewguest
  • xm destroy mynewguest

XEN tuning

If you want xen1.example.com to start automatically at the next boot of the system, then do this:

ln -s /etc/xen/xen1.example.com.cfg /etc/xen/auto

By default it will attempt to save runing hosts when dom0 is halted. This means a "suspend-to-disk" of all domU in dom0 filesystem (/var/lib/xen/save). To prevent it and do regular shutdown of domU, edit /etc/default/xendomains and change: 

#XENDOMAINS_SAVE=/var/lib/xen/save
XENDOMAINS_SAVE=""

Misc notes

Boot & DHCP

I've still sometimes some issues at boot time when the xenbr0 bridge doesn't catch its IP...
In those cases I find dhclient still hooked to the non-existing bnx1 IF (which was renamed pbnx1 by XEN). I've to kill it and start a similar dhclient on IF xenbr0.
Finally I renamed the default interface as eth0 and called the xen bridge script without parameters: (network-script network-bridge)

Allocating too much memory

I tried to create two guests with each 12Gb of RAM while the host had only 16Gb in total. I would have expected a nice handling of the error but it froze xend and I could even not shutdown properly.
And, oh, BTW, no, xen doesn't support memory overcommitment

Retrieved from "https://wiki.yobi.be/index.php?title=Xen&oldid=7056"