Setup-storage

introduction

As setup_harddisks is not portable due to its dependency on sfdisk and the lack of support for RAID and LVM, we are planning a new implementation of the tool. Until it is finished you might want to have a look at

• Software RAID hook with mdadm
• SWRaid and LVM hook (with mdadm)
• patched version of setup_harddisks: Code refactoring, LVM/RAID support and POD manual page: Copyright (c) 2003, Sam Vilain (not updated with recent changes in FAI)
• LVM & RAID

There exists an experimental implementation of the conecpt below - testers are needed: http://archives.free.net.ph/message/20070531.075054.545f74f7.en.html

Proposal for a new configuration file syntax

In the following, we present a complete EBNF description of a modified configuration file syntax, as well as some examples.

file ::= <lines> EOF 

lines ::= EOL 
          /* empty lines or whitespace only */
          | <comment> EOL 
          | <config> EOL 

comment ::= #.* 

config ::= disk_config lvm 
           | disk_config raid
           | disk_config end 
           | disk_config disk[[:digit:]]+( <option>)*
           | disk_config [^[:space:]]+( <option>)*
           /* fully qualified device-path or short form, like hda, whereby full
            * path is assumed to be /dev/hda */
           | <volume>

option ::= /* empty */
           | preserve:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions */
           | disklabel:(msdos|sun)
           /* write a disklabel - default is msdos */
           | bootable:[[:digit:]]
           /* mark a partition bootable, default is / */
           | virtual
           /* do not assume the disk to be a physical device, use with xen */

volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <fs_options>
           | vg <name> <size>
           /* lvm vg */

type ::= primary
         /* for physical disks only */
         | logical
         /* for physical disks only */
         | raid[015]
         /* raid level */
         | [^/[:space:]]+-[^/[:space:]]+
         /* lvm logical volume: vg name and lv name*/

mountpoint ::= -
               /* do not mount */
               | swap
               /* swap space */
               | /[^[:space:]]*
               /* fully qualified path */

name ::= [^/[:space:]]+
         /* lvm volume group name */

size ::= [[:digit:]]+%?(-[[:digit:]]+%?)?(:resize)?
         /* size in megabytes or %, possibly given as a range; physical
          * partitions or lvm logical volumes only */
         | -[[:digit:]]+%?(:resize)?
         /* size in megabytes or % given as upper limit; physical partitions 
          * or lvm logical volumes only */
         | preserve[[:digit:]]+
         /* do not modify this partition */
         | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
         /* devices and options for a raid or lvm vg */

mount_options ::= [^[:space:]]+

filesystem ::= -
               | swap
               | [^[:space:]]
               /* mkfs.xxx must exist */

fs_options ::= .*
               /* options appended to mkfs.xxx call */

The major differences to the prior format:

• the disk_config ... line allows for the keywords lvm and raid
• options may be appended to the disk_config line as well, but this may be subject to change
• the ";" is not used anymore, the options that were given there have now been split up
  • the filesystem is now an explicit parameter; note, that the order of filesystem/mount-options is the same /etc/fstab, as opposed to the previous format of disk_config
  • any options to mkfs.xxx may be given
  • the "boot"-option is one of the options now given on the disk_config line
• support for LVM and RAID is completely new :-)
• resizing partitions is supported

Some examples

# Configure the device /dev/hda
disk_config hda   preserve:6,7   disklabel:msdos  bootable:3
# preserve the 6th and the 7th partition. Alternatively, 
# one could say preserve7 below. The disklabel is msdos, which is the default
# for x86. Furthermore the 3rd partition is made bootable, which would have
# happened as well by default as it is mounted as /
primary /boot     20-100        ext3            rw
# create a primary partition /dev/hda1 with a size between 20 and 100 MB and mount it
# read-write as /boot; it is formatted using ext3 filesystem
primary swap      1000     swap       sw
# /dev/hda2 will be a swap space of 1000 MB
primary /         12000      ext3           rw        -b 2048
# /dev/hda3 should be formatted using ext3 filesystem; when calling mkfs.ext3
# the option "-b 2048" is appended.
logical /tmp      1000      ext3            rw,nosuid
# create the logical partition /dev/hda5
logical /usr      preserve6      ext3          rw
logical /var      10%-      ext3               rw
# make /dev/hda7 at least 10% of the disk size
logical /nobackup 0-        xfs                rw
# use mkfs.xfs to format the partition

# Configure the virtual device /dev/sda (for, e.g., Xen setups), all
# partitions are primary and sizes are ignored (so one could as well specify
# anything other than 0)
disk_config sda virtual
primary /boot     0            ext3                 rw
primary /         0            ext3                 rw
primary /tmp      0            ext3                 rw
primary /usr      0            ext3                 rw
primary /var      0            ext3                 rw

# resizing partitions is possible by appending :resize to any given size
disk_config /dev/scsi/host0/bus0/target1/lun0 
primary /      3000-6000:resize         ext3         rw
# resize to any value between 3000 and 6000 MB, as limited by the disk size,
# the desired sizes of the other partitions, and the data on the partition
primary /tmp   1000                     ext3         rw

# Create a softRAID
disk_config raid
raid1        /    sda1,sdd1  ext2        rw,errors=remount-ro
# create a RAID-1 on /dev/sda1 and /dev/sdd1, format using mkfs.ext2 and mount
# it as /
raid0        -    disk2.2,sdc1,sde1:spare:missing  ext2       default
# create a RAID-0 on the second partition of the second disk, /dev/sdc1, and
# /dev/sde1 as a spare partition, which (may?) me missing

Implementation

Some ideas for the implementation:

• A single function to read partition table for a given physical device and store it in some internal format

 /sbin/parted -s /dev/hda unit B print

might be helpful

• backend to write things to disk, using e.g. parted (parted -s)
• test mode, regression test suite
• command script:
  • init_script
  • make_cmds
    • make_part_cmds
    • make_raid_cmds
    • make_lvm_cmds
  • make_filesystems
• dependencies between LVM/RAID commands -> should be detected by parser/semantic analysis multiple RAID/LVM stanzas, but deps must be satisfied, global RAID-dev counter
• suggestions for modifications of the above format
  • put options on a separate line
  • size specification via size_sda1
  • size specs by RAM size
  • options for lvcreate (e.g. physical volume)
• internal format, generated by parser:
  • list of records: line_no[int], mode[string], type[string], size[string], fs[string], fs_opts[string]
  • map to store fstab-info: dev(unique), mount_point(unique), fs, mount_opts

A parser for the EBNF above is available in the svn, /people/michael/features/setup_harddisks_2