Testing

introduction

When developing FAI, helping to reproduce and fix bugs, doing further development of features, lots of small things can go wrong. Lots of things you do can brek stuff somewhere. Until we have something like a unit testing framework or automated functional testing for FAI, here we have a list of things you can test in between your development cycles, and, if possible before committing code:

It's also a good idea to run all those tests from a fai package that is installed on a system as clean as possible, I set up an extra chroot for each test I do, to have a clean reference.

I have stored the scripts that help me with those task in the FAI development and testing helpers section of the Helper scripts page. That allows me setting up a test enviroment, bulding and testing FAI code with minimal manual interaction.

Untested trunk contents

• r6340: Check whether softupdate behaves appropriately if FAI_CONFIG_SRC is changed/unchanged with svn and git.

What does need to be tested when developing FAI?

(for some of those tests it's assumed simple examples are used for config, but most things should also with any correctly made FAI config)

• fai building from source
  • does it build in pbuilder?
  • what do linda and lintian say when checking the resulting packages?

• nfsroot creation
  • has NFSROOT a kernel and grub installed?

• fai-cd creation
  • does fai-mirror run successful and download a reasonable amount of files?
  • does mirror has a grub bootloader?
  • is there a cd iso after running fai-cd?
  • does the iso have a "correct" sources.list file?
  • can the iso be installed nicely in a qemu VM?

• Install tests: general checks for success of an install (valid for netork and cd installs):
  • before reboot, check in shell:
    • errors/warnings in logfiles? (should be as empty as possible)
    • is there a kernel on target system (e.g. check /tmp/target/boot/)
    • is there a bootloader (grub for i386 and simple examples) on target system? (/tmp/target/boot/grub)
  • after reboot:
    • does the system boot?
    • do "things and services" intended to be on the system run "correctly"? E.g. x-windows, webserver, mailserver, login with ldap, ...

• fai softupdates - what can and needs to bechecked,how?

• other tests - from the lates Ubuntu FAI Team meeting:
  • pxe booting (and configuring it with fai-chboot)
  • dirinstall

TODO: it would be cool if all that stuff could be checked automatically somehow -let's think about it... See http://grml.org/kantan/