Xenserver

Citrix XenServer paravirtualized FAI Installation

It is possible to do paravirtualized installations with Citrix XenServer using FAI but you should know the traps...

What you need is the following

• basic knowledge about XenServer and how to use the shell
• your own Debian repository to point at
• the Citrix Linux kernel package provided with the xs-tools

XenServer comes along with several installation templates for the most popular distributions that work paravirtualized. In case of Debian and RedHat based distributions XenServer is aware of the repository structures, I cannot tell about any other distribution as I did not try them yet. This is how you can use this feature to run FAI within XenServer.

The Template

Lets login to a shell of a XenServer installation and have a look at the Debian-template provided by Citrix:

[root@xvm03 ~]# xe template-list name-label=Debian\ Lenny\ 5.0\ \(32-bit\)
uuid ( RO)                : afc6aad7-439d-7967-fc9f-80cdc27f9f70
          name-label ( RW): Debian Lenny 5.0 (32-bit)
    name-description ( RW): Template that allows VM installation from Xen-aware Debian-based distros. \
                            To use this template from the CLI, install your VM using vm-install, then set \
                            other-config-install-repository to the path to your network repository, \
                            e.g. http://<server>/<path>

[root@xvm03 ~]# xe template-param-list uuid=afc6aad7-439d-7967-fc9f-80cdc27f9f70
uuid ( RO)                          : afc6aad7-439d-7967-fc9f-80cdc27f9f70
[... lot of stuff deleted ...]
               HVM-boot-policy ( RW): 
                       PV-args ( RW): -- quiet console=hvc0
                 PV-bootloader ( RW): eliloader
                  other-config (MRW): mac_seed: 34fd2431-757f-f92c-f0b6-42a92e2d4fa9; default_template: true; linux_template: true; install-methods: cdrom,http,ftp; install-arch: i386; debian-release: lenny; disks: <provision><disk device="0" size="8589934592" sr="" bootable="true" type="system"/></provision>; install-distro: debianlike
[... lot of stuff deleted ...]

The first important point is the PV-bootloader eliloader which actually accesses the repository to fetch a kernel and initrd. And second the "other-config" options pointing to the repository to use (not yet), the architecture ("install-arch") and the "debian-release" you want. Third, pay attention to "PV-args" which hold command line arguments for the kernel to be run.

I suggest copying the template to have a custom template without altering the vendor version. In Citrix XenCenter GUI this could be done like this:

1. View -> ServerView -> XenServer Templates
2. rightclick on "Debian Lenny 5.0 (32-bit)" in the tree
3. "Copy Template..."

Fill in some good comments and finish.

Now go back to the shell and look for your template UUID and enjoy the tab-completion for the "xe" command which helps a lot.

 xe template-list name-label=MyOwnDebianTemplate
uuid ( RO)                : 42814a5e-c287-7ff4-4770-def0b690e5e0
          name-label ( RW): MyOwnDebianTemplate
    name-description ( RW): Debian Network Installation

Next configure your template properly to use your own repository:

xe template-param-set uuid=42814a5e-c287-7ff4-4770-def0b690e5e0 other-config:install-repository=http://myrepo.example.com/debian-custom

And set the FAI kernel args you need:

xe template-param-set uuid=42814a5e-c287-7ff4-4770-def0b690e5e0 PV-args="quiet console=hvc0 root=/dev/nfs nfsroot=10.1.1.6:/srv/fai/nfsroot boot=live FAI_FLAGS=verbose,sshd FAI_ACTION=install"

The Repository

The interesting part is what happens when you start a VM installed with this template. XenServer uses eliloader to fetch a file "vmlinuz" and "initrd.gz" from your repository using the following path(s):

dists/lenny/main/installer-i386/current/images/netboot/xen/vmlinuz
dists/lenny/main/installer-i386/current/images/netboot/xen/initrd.gz

Thus you have to create these directories and put your kernel image and initrd there.

Unfortunately the original Debian kernels "linux-image-xen-686" do not run stable as DomU inside Citrix XenServer. So you need to extract the ".deb" package provided by the "xs-tools.iso". You can get it by running a small dummy VM booting your custom live distribution like "grml-small". Then attach the "xs-tools.iso" to the VM and copy the files to your repository:

linux-image-2.6-xen_2.6.32+0.xs226_i386.deb
linux-image-2.6.32-4-686-bigmem_2.6.32-10.xs5.6.0.226_i386.deb
linux-image-2.6-xen_2.6.32+0.xs226_amd64.deb
linux-image-2.6.32-4-amd64_2.6.32-10.xs5.6.0.226_amd64.deb

What you also should copy are the xe-guest-utilities:

xe-guest-utilities_5.6.0-578_*.deb

I will not discuss the details on how to maintain a Debian repository, I use "apt-ftparchive" and "dput" there should be a lot of documentation on how to use that.

FAI setup

• Add your repository to the sources.list file and add the package "linux-image-2.6-xen" to /etc/fai/NFSROOT.
• Create the nfsroot ("make-fai-nfsroot")
• copy the installed "vmlinuz" and the corresponding "initrd*.gz" to "dists/lenny/main/installer-i386/current/images/netboot/xen/{vmlinuz,initrd.gz}" on your repository
• Do everything else you like according to the official FAI documentation

FAI config

class

To detect that the VM is running on a XenServer machine I use the following script:

root@fai:/srv/fai/config# cat class/60-detect-XenServer 
#!/bin/sh

dmesg | egrep -q "(ACPI:.*Xen.*HVM|Booting paravirtualized kernel on Xen)" &&
(
  echo XENSERVER
)

exit 0

This detects a VM running in HVM mode (ACPI:...) and in paravirtualized mode.

grub

XenServer uses "pygrub" to fetch the kernel and initrd for paravirtualized VMs. So we need a grub (legacy!) installation with a working "menu.lst" but don't need to install grub into MBR. I added the following to the FAI grub postinst script:

root@fai:/srv/fai/config# cat files/boot/grub/menu.lst/postinst 
#! /bin/bash

error=0 ; trap "error=$((error|1))" ERR

# don't actually install grub on XenServer guests
if ifclass XENSERVER ; then
  export ROOT_PARTITION="/dev/xvda1"
  export GROOT="(hd0,0)"
  perl -pi -e 's/#(\w+)#/$ENV{$1}/g' $2
  $ROOTCMD /usr/sbin/update-grub
  exit 0
fi

[...]

Installing a VM

Well, now create a new VM within your XenCenter GUI, select your Template but don't boot it yet. XenCenter for Citrix XenServer 5.6.0 has a bug in that it ignores the kernel arguments you set earlier, I know no other way to circumvent it than copying the boot arguments by hand to the newly generated VM.

After copying the arguments manually start your VM and you should see FAI running on the console TAB.

IMPORTANT XenServer only runs the installation once. When anything goes wrong and you force the VM to poweroff, you cannot start the installation process again. You have to delete the VM and create a new one.

You also can manually reactivate the installation procedure when you interrupted the first boot like this:

1. get the VM UUDI: "xe vm-list name-label=my_install_vm"
2. set the bootloader back to eliloader "xe vm-param-set uuid=X...X PV-bootloader=eliloader"
3. set the install-ditro: "xe vm-param-set uuid=X...X other-config:install-distro=debianlike"

When you boot the VM after this command sequence it will try to start the install process as on the first boot.