Using FAI as a vserver build method

The basic steps are outlined in each section of this document. This is based on Debian sarge, augmented with Linux-VServer 2.0.2 and util-vserver 0.30.209.

The example system is assumed to want all its vservers set up on the interface "dummy0" in the IP range "192.168.254.0/24". It is assumed that the /etc/network/interfaces file has already been suitably configured, eg:

auto dummy0
iface dummy0 inet static
        address 192.168.254.1
        netmask 255.255.255.0
        up iptables -t nat -I POSTROUTING -j SNAT -s 192.168.254.0/24 --to-source 1.2.3.4

Note when writing firewall rules the packets between vservers on the same machine actually travel over the loopback interface regardless of the interface you set them up on. Go figure.

Anyway, read on and skip the sections you find irrelevant. Normal sysadmin stuff, like setting up hosts records, etc, is your responsibility - but please do send feedback if you think that there are steps that could be included here that would help someone else following these instructions from not making the same foolish mistakes that you did.

Partitioning

VServers will be set up in the standard location of /var/lib/vservers/. It's probably a good idea to partition this separately. Individual vservers may have their own individual partitions, though for filesystem unification to work (allowing multiple identical libraries and binaries to share the same memory image across vservers), you will need to perform extra customisations later.

When using unification, each vserver's /usr, /lib, /sbin and /bin will be on the same partition using the magic of filesystem namespaces and bind mounts.

Compiling the vserver packages

The base system can (and for best practice, should) be a minimal installation, no compilers etc are necessary.

However we need to build a decent VServer kernel and util-vserver in an environment with compilers, because the versions in Debian are hopelessly out of date to the point of being unsupported. This does not have to be done on the target machine, but often for pragmatic reasons is.

mkdir -p /var/lib/vservers/compileit
debootstrap sarge /var/lib/vservers/compileit http://your.local/debian

This space will become a vserver later, but for now you can just use chroot(8) to get into it, install important packages, and build the kernel.

# copy the running kernel config into the system
cp /boot/config-`uname -r` /var/lib/vservers/compileit/usr/src

chroot /var/lib/vservers/compileit

vi /etc/apt/sources.list   # add deb-src lines for unstable
apt-get install build-essential kernel-package

cd /usr/src

# build dietlibc - dependency for util-vserver
apt-get build-dep dietlibc-dev
apt-get source --build dietlibc-dev
dpkg -i dietlibc*deb

# then build util-vserver
apt-get build-dep util-vserver
apt-get source --build util-vserver

# get a kernel and patch it with recent patch
wget -O - -q ftp://ftp.kernel.org/pub/linux/kernel/v2.6/linux-2.6.16.8.tar.gz |
   tar xf -
cd linux-2.6.16.8
wget -O - -q http://vserver.13thfloor.at/Experimental/patch-2.6.16.8-vs2.0.2-rc17.diff |
   patch -p1

# install the magic config we saved from before
cp ../config-* .config

# or `make menuconfig' if you want dumb kernel config decisions
# to be made for you instead of making dumb decisions yourself.
# Be sure to say "Yes" to Linux-VServer :)
make oldconfig

make-kpkg --revision 1 binary

You will then be left with some nice kernel-image-2.6.16.8-vs2.0.2-rc17-ARCH.deb and util-vserver-0.30.209-2.ARCH.deb (or something like that) which you can install on your host server. Reboot.

Build the FAI vserver

This is a good idea, to keep the base system clean. This server only serves the function of running the 'make-fai-nfsroot' commands, etc. IT IS NOT SECURE. This is because it needs to be able to use mknod(1) for debootstrap.

The examples here should also work if fai is installed in the main system, but has received less testing.

vserver fai build -m debootstrap --force \
     --interface dummy0:192.168.254.2/24 \
     --flags sched,nproc --hostname fai \
     -- -d sarge -m http://your.local/debian

vserver fai start
vserver fai enter

vi /etc/fai/fai.conf
vi /etc/fai/sources.list

make-fai-nfsroot

---

---

Setup your install area

Set up the /usr/share/fai directory within your FAI vserver. You will need to include the changes in User:BrianShowalter/Using FAI to install Linux-vservers.

Customise util-vserver

In /usr/lib/util-vserver/vserver-build.fai, put:

#
# Copyright (C) 2006 Sam Vilain <sam.vilain@catalyst.net.nz>
#  
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
#  
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#  
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA.

tmp=$(getopt -o '+d:+f:+n:' --long debug,pkgmgmt -n "$0" -- "$@") || exit 1
eval set -- "$tmp"

. "$_LIB_VSERVER_BUILD_FUNCTIONS_PKGMGMT"

DISTRIBUTION=:

FAI_VSERVER=
FAI_NFSROOT=/usr/lib/fai/nfsroot
FAI_DIR=/usr/share/fai

use_pkgmgmt=
while true; do
    echo "ARGS: =>$*<="
    case "$1" in
        -f)             FAI_VSERVER=$2; shift; ;;
        -n)             FAI_NFSROOT=$2; shift; ;;
        -d)             FAI_DIR=$2; shift; ;;
        --debug)        set -x;;
        --)             shift; break ;;
        *)              echo "vserver-build.fai: internal error: unrecognized option '$1'" >&2
                        exit 1
                        ;;
    esac
    shift
done

getDistribution '' 1

base.init
base.initFilesystem "$OPTION_FORCE"

setup_writeOption "$VSERVER_NAME"
setup_writeInitialFstab

test -z "$BUILD_INITPRE"  || "$BUILD_INITPRE" "$SETUP_CONFDIR"  "$UTIL_VSERVER_VARS"
test -z "$1" || "$@"
test -z "$BUILD_INITPOST" || "$BUILD_INITPOST" "$SETUP_CONFDIR" "$UTIL_VSERVER_VARS"

# if they specified a vserver where FAI is installed, then prefix its
# root to the FAI_NFSROOT etc (unless they have already specified a
# $ROOTDIR-relative path)
REAL_ROOTDIR=`cd $ROOTDIR; pwd -P`
REAL_VDIR=`cd $VDIR; pwd -P`
echo "ROOTDIR is $REAL_ROOTDIR"
echo "VDIR    is $REAL_VDIR"
if [ -n "$FAI_VSERVER" ]
then
    echo "FAI_VSERVER is $FAI_VSERVER"
    if [ `expr "$FAI_NFSROOT" : "$REAL_ROOTDIR"` -eq 0 ]
    then
        FAI_NFSROOT="$REAL_ROOTDIR/$FAI_VSERVER$FAI_NFSROOT"
    fi
    if [ `expr "$FAI_DIR" : "$REAL_ROOTDIR"` -eq 0 ]
    then
        FAI_DIR="$REAL_ROOTDIR/$FAI_VSERVER$FAI_DIR"
    fi
fi
echo "FAI_NFSROOT is $FAI_NFSROOT"
echo "FAI_DIR     is $FAI_DIR"

export VDIR ROOTDIR FAI_VSERVER FAI_DIR FAI_NFSROOT

# setup a new filesystem namespace
set -e
echo "Setting up tmp mounts"
vnamespace -n -- sh -x -c "
    mount -n -o ro --bind $FAI_NFSROOT /mnt &&
    mount -n -t tmpfs -o size=16m,mode=1777 vm /mnt/tmp &&
    mkdir /mnt/tmp/target &&
    mount -n --bind $VDIR /mnt/tmp/target &&
    mount -n -o ro --bind $FAI_DIR /mnt/fai &&
    mount -n -o ro --bind $FAI_DIR /mnt/usr/share/fai &&
    mount -n -t proc proc /mnt/proc &&
    chcontext --secure --cap mknod chroot /mnt"

base.setSuccess

Then in /usr/lib/util-vserver/vserver-build, change

    (xapt-rpm|xcopy|xskeleton|xdebootstrap|xyum|xrpm)

To include fai:

    (xapt-rpm|xcopy|xskeleton|xdebootstrap|xyum|xrpm|xfai)

Build the vserver

The command would be:

vserver vschild build -m fai --force \
   --interface eth2:192.168.254.4/24 \
   --flags sched,nproc --hostname vschild -- \
   -f fai

The first -m fai tells vserver xxx build to use the FAI build method. The second tells the above script to use the fai server called fai as the vserver where the FAI configuration area is installed.

The vserver-build.fai script should be running sh -c '/usr/sbin/fai vschild_create && /usr/bin/fai vschild_install' rather than just starting a shell in the FAI environment.

It looks like the 'subroutines' files, etc need to be customised heavily to work, at least from the version of FAI in Debian sarge. This work is likely to proceed on an FAI branch.