Using FAI to set up XEN domains

Introduction

This section provides short guidelines on how to set up XEN base systems (aka domain-0) and virtual machines (dom-U) using FAI. Henning Glawe (http://www.physik.fu-berlin.de/~glaweh/) provided many ideas and some scripts - thanks!

Apart from configuring FAI, the following things need to be considered:

• Which XEN packages should be used?
• XEN kernel images need to be built
• Find a partitioning scheme and figure out whether you want to use LVM, native partitions or files

Preparing FAI:

For the domain-0 only the kernel-image must be set correctly. As I'm using LVM for the unpriveledged domains and FAI can't do that yet, I wrote a little script to set up all of them later (it failed when I tried to do it during the FAI installation, should debug that).

The more interesting part is the preparation of unpriveledged domains. As these require no partitioning during installation a hook must be created:

partition.XENU:

[ -f $LOGDIR/our.skip.partition ] && exit 0

for our_cl in $classes; do
  [ $our_cl = "XENU" ] && continue
  [ -f $FAI/hooks/partition.$our_cl ] && exit 0
done

mke2fs -j /dev/hda1 > /dev/null
mkswap /dev/hda2 > /dev/null
mke2fs -j /dev/hda3 > /dev/null
mke2fs -j /dev/hda4 > /dev/null
mke2fs -j /dev/hda5 > /dev/null

cat > /tmp/fai/fstab <<EOF
/dev/hda1 / ext3 defaults 0 0
/dev/hda2 none swap sw 0 0
/dev/hda3 /tmp ext3 defaults 0 0
/dev/hda4 /usr ext3 defaults 0 0
/dev/hda5 /var ext3 defaults 0 0
EOF

echo "#!" > $diskvar
touch $LOGDIR/skip.partition

Furthermore, a XEN-kernel for the class XENU must be configured - and FAI is prepared.

Booting XEN domains from the network

Starting from the example config provided by xen, xmexample2, I made my scripts support the following:

xm create <domainname>

starts the domain as usual,

xm create <domainname> vminstall=1

starts a FAI installation.

The configuration for my domain named boss is thus (obey spaces, this is python code...)

#============================================================================
# This script sets the parameters used when a domain is created using 'xm create'.
#
# "xm create vminstall=1" will (re)install the domain or
# "xm create" will boot the domain 
#
# vminstall is purely a script variable, and has no effect on the the domain
# id assigned to the new domain.
#============================================================================

# Define script variables here.
# xm_vars is defined automatically, use xm_vars.var() to define a variable.

# This function checks whether 'vminstall' has been given a value.
# It is called automatically by 'xm create'.
def vminstall_check(var, val):
  try:
    val = int(val)
  except: 
    val = 0 
  if val <= 0:
    val = 0 
  return val

# Define the 'vminstall' variable so that 'xm create' knows about it.
xm_vars.var('vminstall',
            use="Reinstall domain if value > 0.",
            check=vminstall_check)

# Check the defined variables have valid values..
xm_vars.check()

kernel = "/boot/xen-linux-2.6.11.12-xen"

memory = 384

name = "boss"

cpu = 0

vif = [ 'mac=aa:00:00:00:00:0d,bridge=xen-br2' ]

disk = [ 'phy:/dev/XENU/boss_,hda1,w',
         'phy:/dev/XENU/bossswap,hda2,w',
         'phy:/dev/XENU/boss_tmp,hda3,w',
         'phy:/dev/XENU/boss_usr,hda4,w',
         'phy:/dev/XENU/boss_var,hda5,w' ]

if vminstall > 0:
  vif = [ 'mac=aa:00:00:aa:aa:aa,bridge=xen-br1' ]
  # Set if you want dhcp to allocate the IP address.
  dhcp = "dhcp"
  # Set netmask.
  #netmask=
  # Set default gateway.
  #gateway=
  # Set the hostname.
  hostname = "boss"

  # Set root device.
  root = "/dev/nfs"
  # provided by dhcp
  # The nfs server.
  #nfs_server = '' 
  # Root directory on the nfs server.
  #nfs_root = '/usr/lib/fai/nfsroot'

  extra = "FAI_ACTION=install FAI_FLAGS=verbose,sshd,createvt,syslogd"
else:
  root = "/dev/hda1 ro"

Well, that's it.

Further hints:

example configspace for xen domU's

• I use this configspace to install Xen DomU's: [Image:Fai-configspace.tar.gz] (it's a mix of simple examples, fai-distributions from people/lazyboy in Subversion, some additions for xen and some personal stuff...

combining FAI with xen-tools

• Here I wrote some things to use xen-tools and FAI together: Xen-tools and FAI softupdates

adjustments needed to the fai nfsroot

I use a script called tunenfsroot to make some adjustments for installing xen DomU's via network:

#!/bin/sh

prefix=/usr/lib/fai/nfsroot

touch /usr/lib/fai/nfsroot/.nocolorlogo


# most of them stolen from xen-tools scripts...

# von 30-fix-inittab
sed -i -e 's/tty[0-9]$/console/g' -e 's/^\([2-6].*:respawn*\)/#\1/' -e 's/^T/#\t/' ${prefix}/etc/inittab

#  von 10-disable-tls
mv ${prefix}/lib/tls ${prefix}/lib/tls.disabled
mkdir ${prefix}/lib/tls

# von 15-disable-hwclock
chmod -x ${prefix}/etc/init.d/hwclock.sh
chmod -x ${prefix}/etc/init.d/hwclockfirst.sh


# von 55-create-dev
cd ${prefix}/dev
./MAKEDEV generic
./MAKEDEV sd

cp -a /lib/modules/* /usr/lib/fai/nfsroot/lib/modules/

initrd for netboot

For netinstalls, when using the default debian kernel, you also need a netboot-capable initrd. To get this, set "MODULES=netboot" in /etc/initramfs-tools/initramfs.conf and create a new initrrd. Use this only for installing the domU, otherwise use the normal initrd.