Server Dell PowerEdge T410

Manuals

here, online version of the main manual here with chapter on LED status here

Install notes

BIOS

- Activate Virtualization support
- Power Management -> OS Control
- Report kbd errors: NO

Debian Squeeze

Still testing at time of writing but frozen so almost stable ;-)

Installing squeeze from daily image (17/08/2010)
- Complains about missing bnx2/bnx2-mips-09-5.0.0.j3.fw & bnx2/bnx2-rv2p-09-5.0.0.j3.fw
  - Get it on another machine and put it on USB stick: aptitude install firmware-bnx2 ; cp /lib/firmware/bnx2/bnx2-*-09-5.0.0.j3.fw /media/XXXX
  - Actually I had also an additional Intel network card so I could have skipped this step...
- Partitionning:
  there is a first primary partition for Dell Utilities (FAT16), I left it just in case...;
  there is also a second primary partition: 2Gb of FAT32, empty, so I deleted it.
  partition #2: primary 10Gb ext3 / label=ROOT
  partition #3: primary 16Gb swap
  partition #5: logical 2Tb physical volume for LVM
  Configure LVM / ok to write table / Create VG with sda5, called vg0 (will be used by XEN)
- Tasksel: SSH & standard utilities
- Reboot
SSH: PermitRootLogin no / ssh-copy-id ...
Install mc, screen, sudo (adduser xxx sudo)
Renaming interfaces (always easier to remember what's plugged where...)
- à la ifrename
  - aptitude install ifrename
  - Create /etc/iftab then reboot

intelG mac 00:1b:21:xx:xx:xx
intelD mac 00:1b:21:xx:xx:xx
bcm1   mac 84:2b:2b:xx:xx:xx
bcm2   mac 84:2b:2b:xx:xx:xx

- à la udev
  - actually we don't need ifrename because udev is now doing the same, see /etc/udev/rules.d/70-persistent-net.rules, you can simply change the NAME key
- Be sure the old name is not used anywhere else in your config, typically in /etc/network/interfaces

XEN: installation

see notes on Debian wiki
aptitude install xen-linux-image-2.6.32-5-xen-amd64 xen-tools xen-utils
backup /etc/xen/xend-config.sxp
edit /etc/xen/xend-config.sxp, and uncomment line "(network-script network-bridge)" or for a non-default interface ethX:

(network-script 'network-bridge netdev=ethX bridge=xenbr0')

reboot
aptitude remove --purge linux-image-2.6.32-5-amd64 linux-image-2.6-amd64
/etc/default/grub => GRUB_DEFAULT=2
update-grub
reboot
cat /proc/xen/capabilities #(to check xend is running)

XEN: guest creation

If you've a proxy it's much easier to have it defined in the environment variables:
- Edit /etc/environment

http_proxy=http://myproxy:port

- Logout/login
Edit the config file for creating images (/etc/xen-tools/xen-tools.conf) to your needs:

--- xen-tools.conf.orig 2010-05-30 22:42:25.000000000 +0200
+++ xen-tools.conf      2010-08-17 17:47:35.000000000 +0200
@@ -52,7 +52,7 @@
 # LVM volume group here instead
 #
 ##
-# lvm = vg0
+lvm = vg0
 
 
 #
@@ -125,12 +125,12 @@
 #  Disk and Sizing options.
 ##
 #
-size   = 4Gb      # Disk image size.
+size   = 8Gb      # Disk image size.
 memory = 128Mb    # Memory size
 swap   = 128Mb    # Swap size
 # noswap = 1      # Don't use swap at all for the new system.
 fs     = ext3     # use the EXT3 filesystem for the disk image.
-dist   = `xt-guess-suite-and-mirror -s` # Default distribution to install.
+dist   = squeeze  # Default distribution to install.
 image  = sparse   # Specify sparse vs. full disk images.
 
 #
@@ -166,7 +166,7 @@
 #
 # Uncomment this if you wish the images to use DHCP
 #
-# dhcp = 1
+dhcp = 1
 
 
 ##
@@ -184,7 +184,7 @@
 # Uncomment the following line if you wish to interactively setup
 # a new root password for images.
 #
-# passwd = 1
+passwd = 1
 
 #
 # If you'd like all accounts on your host system which are not present
@@ -254,10 +254,10 @@
 #
 #  You may specify the things to use here:
 #
-# serial_device = hvc0 #default
+serial_device = hvc0 #default
 # serial_device = tty1
 #
-# disk_device = xvda #default
+disk_device = xvda #default
 # disk_device = sda
 #

XEN quickref

Xen Tools, Xen Shell

Sagemath guest

xen-create-image --hostname=sagemath --memory=12288 --swap=12288 --vcpus=20
WARNING strangely the default root password was disabled with a '*' in /etc/shadow so impossible to log in. I had to mount manually the lv (/dev/vg0/sagemath-disk) to remove the '*'
One may prefer to setup the password interactively when creating the image with xen-create-image ... --passwd (or setup passwd=1 in /etc/xen-tools/xen-tools.conf as I've now shown in the above diff file)
lvm> lvcreate -L 1000G -n sagemath-data vg0
/etc/xen/sagemath.cfg: add data volume:

root        = '/dev/xvda2 ro'
disk        = [
                  'phy:/dev/vg0/sagemath-disk,xvda2,w',
                  'phy:/dev/vg0/sagemath-swap,xvda1,w',
                  'phy:/dev/vg0/sagemath-data,xvda3,w',
              ]

xm create -c /etc/xen/sagemath.cfg
Hook the data drive as /home
- mkfs.ext4 /dev/xvda3
- In /etc/fstab: /dev/xvda3 /home ext4 noatime,nodiratime,errors=remount-ro 0 1
- mount /home
Install sage
- apt-get install aria2
- cd /opt
- aria2c http://www.sagemath.org/mirror/linux/64bit/meta/sage-4.5.2-linux-64bit-ubuntu_10.04_lts-x86_64-Linux.tar.lzma.metalink
- tar x --lzma -f sage-4.5.2-linux-64bit-ubuntu_10.04_lts-x86_64-Linux.tar.lzma
- cd sage-4.5.2-linux-64bit-ubuntu_10.04_lts-x86_64-Linux
- Edit sage and set SAGE_ROOT="/opt/sage-4.5.2-linux-64bit-ubuntu_10.04_lts-x86_64-Linux"
- ./sage
- sage> notebook.setup()
  Provide FQDN for TLS certificate generation

Misc notes

Boot & DHCP

I've still sometimes some issues at boot time when the xenbr0 bridge doesn't catch its IP...
In those cases I find dhclient still hooked to the non-existing bnx1 IF (which was renamed pbnx1 by XEN). I've to kill it and start a similar dhclient on IF xenbr0.
Finally I renamed the default interface as eth0 and called the xen bridge script without parameters: (network-script network-bridge)

Allocating too much memory

I tried to create two guests with each 12Gb of RAM while the host had only 16Gb in total. I would have expected a nice handling of the error but it froze xend and I could even not shutdown properly.
And, oh, BTW, no, xen doesn't support memory overcommitment

Hyperthreading

I'm not sure if in the long run it'll help but given that there are already 12 CPUs, I've disabled the logical CPU support in the BIOS.