Using FAI to set up XEN domains

Introduction

This section provides short guidelines on how to set up XEN base systems (aka domain-0) and virtual machines (dom-U) using FAI. Henning Glawe provided many ideas and some scripts - thanks!

Apart from configuring FAI, the following things need to be considered:

• Which XEN packages should be used?
• XEN kernel images need to be built
• Find a partitioning scheme and figure out whether you want to use LVM, native partitions or files

Preparing FAI:

For the domain-0 only the kernel-image must be set correctly. As I'm using LVM for the unpriveledged domains and FAI can't do that yet, I wrote a little script to set up all of them later (it failed when I tried to do it during the FAI installation, should debug that).

The more interesting part is the preparation of unpriveledged domains. As these require no partitioning during installation a hook must be created:

partition.XENU:

[ -f $LOGDIR/our.skip.partition ] && exit 0

for our_cl in $classes; do
  [ $our_cl = "XENU" ] && continue
  [ -f $FAI/hooks/partition.$our_cl ] && exit 0
done

mke2fs -j /dev/hda1 > /dev/null
mkswap /dev/hda2 > /dev/null
mke2fs -j /dev/hda3 > /dev/null
mke2fs -j /dev/hda4 > /dev/null
mke2fs -j /dev/hda5 > /dev/null

cat > /tmp/fai/fstab <<EOF
/dev/hda1 / ext3 defaults 0 0
/dev/hda2 none swap sw 0 0
/dev/hda3 /tmp ext3 defaults 0 0
/dev/hda4 /usr ext3 defaults 0 0
/dev/hda5 /var ext3 defaults 0 0
EOF

echo "#!" > $diskvar
touch $LOGDIR/skip.partition

Additionally, you need

Booting XEN domains from the network

Starting from the example config provided by xen, xmexample2, I made my scripts support the following:

xm create <domainname>

starts the domain as usual,

xm create <domainname> vminstall=1

starts a FAI installation.

The configuration for my domain named boss is thus (obey spaces, this is python code...)

#============================================================================
# This script sets the parameters used when a domain is created using 'xm create'.
#
# "xm create vminstall=1" will (re)install the domain or
# "xm create" will boot the domain 
#
# vminstall is purely a script variable, and has no effect on the the domain
# id assigned to the new domain.
#============================================================================

# Define script variables here.
# xm_vars is defined automatically, use xm_vars.var() to define a variable.

# This function checks whether 'vminstall' has been given a value.
# It is called automatically by 'xm create'.
def vminstall_check(var, val):
  try:
    val = int(val)
  except: 
    val = 0 
  if val <= 0:
    val = 0 
  return val

# Define the 'vminstall' variable so that 'xm create' knows about it.
xm_vars.var('vminstall',
            use="Reinstall domain if value > 0.",
            check=vminstall_check)

# Check the defined variables have valid values..
xm_vars.check()

kernel = "/boot/xen-linux-2.6.11.12-xen"

memory = 384

name = "boss"

cpu = 0

vif = [ 'mac=aa:00:00:00:00:0d,bridge=xen-br2' ]

disk = [ 'phy:/dev/XENU/boss_,hda1,w',
         'phy:/dev/XENU/bossswap,hda2,w',
         'phy:/dev/XENU/boss_tmp,hda3,w',
         'phy:/dev/XENU/boss_usr,hda4,w',
         'phy:/dev/XENU/boss_var,hda5,w' ]

if vminstall > 0:
  vif = [ 'mac=aa:00:00:aa:aa:aa,bridge=xen-br1' ]
  # Set if you want dhcp to allocate the IP address.
  dhcp = "dhcp"
  # Set netmask.
  #netmask=
  # Set default gateway.
  #gateway=
  # Set the hostname.
  hostname = "boss"

  # Set root device.
  root = "/dev/nfs"
  # provided by dhcp
  # The nfs server.
  #nfs_server = '' 
  # Root directory on the nfs server.
  #nfs_root = '/usr/lib/fai/nfsroot'

  extra = "FAI_ACTION=install FAI_FLAGS=verbose,sshd,createvt,syslogd"
else:
  root = "/dev/hda1 ro"

Well, that's it.

Further hints:

example configspace for xen domU's

• I use this configspace to install Xen DomU's: [Image:Fai-configspace.tar.gz] (it's a mix of simple examples, fai-distributions from people/lazyboy in Subversion, some additions for xen and some personal stuff...

combining FAI with xen-tools

• Here I wrote some things to use xen-tools and FAI together: Xen-tools and FAI softupdates

adjustments needed to the fai nfsroot

I use a script called tunenfsroot to make some adjustments for installing xen DomU's via network:

#!/bin/sh

prefix=/usr/lib/fai/nfsroot

touch /usr/lib/fai/nfsroot/.nocolorlogo


# most of them stolen from xen-tools scripts...

# von 30-fix-inittab
sed -i -e 's/tty[0-9]$/console/g' -e 's/^\([2-6].*:respawn*\)/#\1/' -e 's/^T/#\t/' ${prefix}/etc/inittab

#  von 10-disable-tls
mv ${prefix}/lib/tls ${prefix}/lib/tls.disabled
mkdir ${prefix}/lib/tls

# von 15-disable-hwclock
chmod -x ${prefix}/etc/init.d/hwclock.sh
chmod -x ${prefix}/etc/init.d/hwclockfirst.sh


# von 55-create-dev
cd ${prefix}/dev
./MAKEDEV generic
./MAKEDEV sd

# only works, if the right xen kernel you also boot the install client with 
# is on the install server.
cp -a /lib/modules/* /usr/lib/fai/nfsroot/lib/modules/

TODO: some of the above script is old and only useful for a sarge nfsroot. In Etch, you'll want to install the packages libc6-xen in the nfsroot and a Xen kernel package which gives you the modules from the Kernel which you start the domU with. And edit the inittab... The packages to install can be added in the DEBOOTSTRAP_OPTS in make-fai-nfsroot.conf. In FAI 3.2beta4 and above there is the file /etc/fai/NFSROOT (might change in future versions) where packages to be installed can be configured.

initrd for netboot

For netinstalls, when using the default debian kernel, you also need a netboot-capable initrd. To get this, set "MODULES=netboot" in /etc/initramfs-tools/initramfs.conf and create a new initrrd. Use this only for installing the domU, otherwise use the normal initrd.