Setup-storage: Difference between revisions

From FAIWiki
Jump to navigation Jump to search
(one link was dupe (masqueraded as ext link) and setup_harddisks2 is old)
(minor fix)
 
(96 intermediate revisions by 8 users not shown)
Line 1: Line 1:
As setup_harddisks is not portable due to its dependency on sfdisk and the lack of support for RAID and LVM, we are planning a new implementation of the tool. Until it is finished you might want to have a look at
== Introduction ==
As setup_harddisks is not portable due to its dependency on sfdisk and the lack of support for RAID and LVM, it has been re-implemented from scratch. As of FAI 3.2.8 it is integrated with mainline FAI, but not yet used by default.
 
To test the current implementation, keep reading. There are some TODOs left and
probably a lot more will arise as soon as people actually start testing it. Before you
start, just let me clarify that this is a very dangerous piece of software that
has not been sufficiently tested for any cases where you want to retain any of
your data. It will definitely destroy any data on any of your disks in your
system. You have been warned.
 
If you're still interested, the following steps are required next:
 
* get FAI 3.2.8 (or later)
* add USE_SETUP_STORAGE=1 to one of your scripts in class/
* you may want to take a look at the SETUPSTORAGE class used in the simple example in FAI 3.2.8 and/or see [http://fai-project.org/doc/man/setup-storage.html man 8 setup-storage]
* add lvm2 and mdadm to one of your client classes (it must be available on the target system, if you define LVMs/RAIDs in your disk_config)
* create a disk_config for some class MY_TEST_CLASS from the old one apart from (at least) the following changes:
** the filesystem is now given as the 4th column, the mount options are now in column 5
** there is no ;-hack anymore -- anything given after the mount options is passed on to mkfs.<filesystem>
** Partitions are marked as bootable using the bootable:<nr> option in the disk_config line
** The other things should at least work like they did before, but there are some new things as well, like RAID support (which is surely incomplete), LVM support, mounting by label or UUID.
** '''Please take a look at the [[Setup-storage#Some_examples|attached examples]] to get an idea of the new format.'''
** All details of the implemented syntax and examples are given below.
* Give it a go (you have backed up all data, don't you?). There will be lots of debug output; if it finishes with the printout of an fstab file that is to your liking, it should have succeeded.


* [[User:Svamberg/Software RAID|Software RAID hook]] with mdadm
* [[User:Gervai/Raid and LVM hook|SWRaid and LVM hook]] (with mdadm)
* [http://www.informatik.uni-koeln.de/fai/download/developers/setup_harddisks patched version of setup_harddisks]: Code refactoring, LVM/RAID support and POD manual page: Copyright (c) 2003, Sam Vilain ''(not updated with recent changes in FAI)''
* [http://www.linuxma.com/fai_raid/ LVM & RAID]


Some further infos may also be found on [http://wiki.debian.org/DebianEdu/HowTo/FaiInstallDebianEdu wiki.debian.org].


== Proposal for a new configuration file syntax ==
== New configuration file syntax ==


In the following, we present a complete EBNF description of a modified configuration file syntax, as well as some examples.
In the following, we present a complete EBNF description of a modified configuration file syntax, as well as some examples.


<pre>
<pre>
file ::= <lines> EOF  
file ::= <lines> EOF  


Line 22: Line 41:
comment ::= #.*  
comment ::= #.*  


config ::= disk_config lvm  
config ::= disk_config lvm( <lvmoption>)*
           | disk_config raid
           | disk_config raid( <raidoption>)*
           | disk_config end  
           | disk_config end  
           | disk_config disk[[:digit:]]+( <option>)*
           | disk_config disk[[:digit:]]+( <option>)*
Line 30: Line 49:
             * path is assumed to be /dev/hda */
             * path is assumed to be /dev/hda */
           | <volume>
           | <volume>
lvmoption ::= /* empty */
          | preserve_always:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
          /* preserve partitions -- always */
          | preserve_reinstall:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
          /* preserve partitions -- unless the system is installed for the
          first time */
          | resize:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
          /* attempt to resize partitions */
          | fstabkey:(device|label|uuid)
          /* when creating the fstab, the key used for defining the device
          may be the device (/dev/xxx), a label given using -L, or the uuid
          */
raidoption ::= /* empty */
          | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
          /* preserve partitions -- always */
          | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
          /* preserve partitions -- unless the system is installed for the
          first time */
          | fstabkey:(device|label|uuid)
          /* when creating the fstab, the key used for defining the device
          may be the device (/dev/xxx), a label given using -L, or the uuid
          */


option ::= /* empty */
option ::= /* empty */
           | preserve:[[:digit:]]+(,[[:digit:]]+)*
           | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions */
          /* preserve partitions -- always */
           | disklabel:(msdos|sun)
          | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- unless the system is installed for the
          first time */
          | resize:[[:digit:]]+(,[[:digit:]]+)*
          /* attempt to resize partitions */
           | disklabel:(msdos|gpt)
           /* write a disklabel - default is msdos */
           /* write a disklabel - default is msdos */
           | bootable:[[:digit:]]
           | bootable:[[:digit:]]+
           /* mark a partition bootable, default is / */
           /* mark a partition bootable, default is / */
           | virtual
           | virtual
           /* do not assume the disk to be a physical device, use with xen */
           /* do not assume the disk to be a physical device, use with xen */
          | fstabkey:(device|label|uuid)
          /* when creating the fstab, the key used for defining the device
          may be the device (/dev/xxx), a label given using -L, or the uuid
          */ 


volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <fs_options>
volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <fs_options>
Line 49: Line 101:
         | logical
         | logical
         /* for physical disks only */
         /* for physical disks only */
         | raid[015]
         | raid[0156]
         /* raid level */
         /* raid level */
         | [^/[:space:]]+-[^/[:space:]]+
         | [^/[:space:]]+-[^/[:space:]]+
Line 58: Line 110:
               | swap
               | swap
               /* swap space */
               /* swap space */
               | /[^[:space:]]*
               | /[^[:space:]]*(:encrypt)?
               /* fully qualified path */
               /* fully qualified path; if :encrypt is given, the partition
                * will be encrypted, the key is generated automatically */


name ::= [^/[:space:]]+
name ::= [^/[:space:]]+
         /* lvm volume group name */
         /* lvm volume group name */


size ::= [[:digit:]]+%?(-[[:digit:]]+%?)?(:resize)?
size ::= [[:digit:]]+[kMGTP%]?(-([[:digit:]]+[kMGTP%]?)?)?(:resize)?
         /* size in megabytes or %, possibly given as a range; physical
         /* size in kilo, mega (default), giga, tera or petabytes or %,
           * partitions or lvm logical volumes only */
          * possibly given as a range; physical
         | -[[:digit:]]+%?(:resize)?
           * partitions or lvm logical volumes only; */
         /* size in megabytes or % given as upper limit; physical partitions  
         | -[[:digit:]]+[kMGTP%]?(:resize)?
         /* size in kilo, mega (default), giga, tera or petabytes or %,
          * given as upper limit; physical partitions
           * or lvm logical volumes only */
           * or lvm logical volumes only */
        | preserve[[:digit:]]+
        /* do not modify this partition */
         | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
         | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
         /* devices and options for a raid or lvm vg */
         /* devices and options for a raid or lvm vg */
Line 82: Line 135:
               /* mkfs.xxx must exist */
               /* mkfs.xxx must exist */


fs_options ::= .*
fs_options ::= (createopts=".*"|tuneopts=".*")*
               /* options appended to mkfs.xxx call */
               /* options to append to mkfs.xxx and to the filesystem-specific
 
                * tuning tool */
</pre>
</pre>


Line 90: Line 143:


* the disk_config ... line allows for the keywords lvm and raid
* the disk_config ... line allows for the keywords lvm and raid
* options may be appended to the disk_config line as well, but this may be subject to change
* options may need to be appended to the disk_config line
* the ";" is not used anymore, the options that were given there have now been split up
* the ";" is not used anymore, the options that were given there have now been split up
** the filesystem is now an explicit parameter; note, that the order of filesystem/mount-options is the same /etc/fstab, as opposed to the previous format of disk_config
** the filesystem is now an explicit parameter; note, that the order of filesystem/mount-options is the same /etc/fstab, as opposed to the previous format of disk_config
** any options to mkfs.xxx may be given
** any options to mkfs.xxx may be given using createopts=""
** the "boot"-option is one of the options now given on the disk_config line
** The "preserveX" and "boot" options are one of the options now given on the  
disk_config line, using preserve_reinstall or preserve_always and bootable.
preserve_always is equivalent to the previous preserveX option, whereas
preserve_reinstall preserves the partition unless "initial" is given as one of
the FAI_FLAGS.
* support for LVM and RAID is completely new :-)
* support for LVM and RAID is completely new :-)
* resizing partitions is supported
* resizing partitions is supported


== Some examples ==
== Some examples ==
In fai 3.2.19 syntax has changed a little bit
use: <pre> createopts="-O dir_index,resize_inode" instead of -O dir_index,resize_inode </pre>
May be we can update the examples if the new syntax also works in debian stable I have not checked it. --Andreas 11:28, 25 April 2009 (CEST)


<pre>
<pre>
# Configure the device /dev/hda
# Configure the device /dev/hda
disk_config hda  preserve:6,7  disklabel:msdos  bootable:3
disk_config hda  preserve_always:6,7  disklabel:msdos  bootable:3
# preserve the 6th and the 7th partition. Alternatively,
# preserve the 6th and the 7th partition. The disklabel is msdos, which is the default
# one could say preserve7 below. The disklabel is msdos, which is the default
# for x86. Furthermore the 3rd partition is made bootable.
# for x86. Furthermore the 3rd partition is made bootable, which would have
# happened as well by default as it is mounted as /
primary /boot    20-100        ext3            rw
primary /boot    20-100        ext3            rw
# create a primary partition /dev/hda1 with a size between 20 and 100 MB and mount it
# create a primary partition /dev/hda1 with a size between 20 and 100 MB and mount it
Line 113: Line 171:
primary swap      1000    swap      sw
primary swap      1000    swap      sw
# /dev/hda2 will be a swap space of 1000 MB
# /dev/hda2 will be a swap space of 1000 MB
primary /        12000      ext3          rw        -b 2048
primary /        12000      ext3          rw        createopts="-b 2048"
# /dev/hda3 should be formatted using ext3 filesystem; when calling mkfs.ext3
# /dev/hda3 should be formatted using ext3 filesystem; when calling mkfs.ext3
# the option "-b 2048" is appended.
# the option "-b 2048" is appended.
logical /tmp      1000      ext3            rw,nosuid
logical /tmp      1000      ext3            rw,nosuid
# create the logical partition /dev/hda5
# create the logical partition /dev/hda5
logical /usr      preserve6     ext3          rw
logical /usr      10000     ext3          rw
logical /var      10%-      ext3              rw
logical /var      10%-      ext3              rw
# make /dev/hda7 at least 10% of the disk size
# make /dev/hda7 at least 10% of the disk size
Line 161: Line 219:
</pre>
</pre>


<pre>
# Simple LVM example
disk_config sda  bootable:1
primary /boot 500 ext3 rw
primary -      4096-  -      -
disk_config lvm
vg my_pv        sda2
my_pv-_swap    swap    2048    swap    sw
my_pv-_root    /      2048    ext3 rw
</pre>
A pretty complex setup looks as follows:
<pre>
disk_config sda
primary  -    256        -    - 
primary  swap 1024        swap      sw
primary  - 0- -    -
disk_config sdb
primary  - 0- - -
disk_config sdc
primary  - 0- - -
disk_config sdd
primary  -            256        - -
primary  -          1024        -                  -
primary  - 0- - -
disk_config sde
primary  - 0- - -
disk_config sdf
primary  - 0- - -
disk_config raid
raid1        /    sda1,sdd1  ext2    rw,errors=remount-ro 
raid1        swap sda2,sdd2  swap    rw                   
raid1        -    sda3,sdd3  ext2    default               
raid0        -    sdb1,sde1  ext2    default
raid0        -    sdc1,sdf1  ext2    default
# config the LVM
disk_config lvm
vg  my_pv  md2,md3 
my_pv-_usr /usr 2048        ext3  rw -O dir_index,resize_inode
my_pv-_var /var 600        ext3  rw -O dir_index,resize_inode
my_pv-_e_h /export/home 10240      reiserfs          rw,notail
my_pv-_e_s /export/sites 2048        reiserfs          rw,notail
my_pv-_v /vservers 2048        ext3    rw
</pre>


== Implementation ==
== Implementation ==
Some ideas for the implementation:
The current implementation still some TODOs left:
 
=== Missing features and regressions ===
 
* Missing features
** lazyformat -- will probably not be implemented as preserve_reinstall is much more stable.
** some auto mode (something like auto:server, auto:desktop?) might be desirable 
** once everything is done, some checks should be performed, e.g., test that all partitions were created
** Having /boot on a SW-RAID or LVM should be tested, I think it doesn't work - yes, it does indeed fail because $BOOT_* don't get set. No idea what the proper setting is -- use class GRUB_PC as shown in the simple example in experimental
** implement disklabels other than msdos and gpt
 
* Implementation details
** preserve/resize deserve thorough testing, especially in case of LVM/RAID
** The code of create_volume_group for the case of an existing vg should handle existing information and not rely on lvm commands to silently ignore existing volumes
** mdadm --misc --zero-superblock /dev/hdx may be necessary
** preserve must retain the flags (bootable, etc.)
** the RAID commands are surely incomplete and lack any management of unanticipated situations
** evaluate the return code of external commands in exec.pm -- currently only a warning is given
** more error messages must be caught by exec.pm
 
=== Documentation of internal data structures ===
 
The hash of all configurations specified in the disk_config file
<pre>
PHY_<DEVICE>
  virtual (0|1)
  disklabel STRING
  bootable -1..n
  partitions
    <1..n>
      size
        extended (0|1)
        preserve (0|1)
        resize (0|1)
        range
        eff_size
      number 1..n
      maps_to_existing 1..n
      start_byte
      end_byte
      mountpoint
      mount_options
      filesystem
      fs_options
      label
VG_<NAME>
  devices
  estimated_size
  volumes
    <logical-volume-name>
      size
        preserve (0|1)
        resize (0|1)
        range
        eff_size
      mountpoint
      mount_options
      filesystem
      fs_options
      label
RAID
  volumes
    <0..n>
      mode
      devices
        /dev/<device-name>
          options
            spare (0|1)
            missing (0|1)
      mountpoint
      mount_options
      filesystem
      fs_options
      label
</pre>
 
The current disk configuration
<pre>
<DEVICE>
  bios_cylinders
  bios_heads
  bios_sectors_per_track
  sector_size
  disklabel
  begin_byte
  end_byte
  partitions
    <1..n>
      begin_byte
      end_byte
      count_byte
      is_extended
      filesystem
</pre>
 
The current LVM configuration
<pre>
<VG>
  physical_volumes
  size
  volumes
    <lv-name>
      size
</pre>
 
The current RAID configuration
<pre>
<0..n>
  devices
  mode
</pre>
 
== Other implementations for RAID and LVM ==
 
Meanwhile you might also want to look at
 
* [[User:Svamberg/Software RAID|Software RAID hook]] with mdadm
* [[User:Gervai/Raid and LVM hook|SWRaid and LVM hook]] (with mdadm)
* [http://www.linuxma.com/fai_raid/ LVM & RAID]


* A single function to read partition table for a given physical device and store it in some internal format
== See also ==
  /sbin/parted -s /dev/hda unit B print
* [http://fai-project.org/doc/man/setup-storage.html setup-storage manpage]
might be helpful
* backend to write things to disk, using e.g. parted (parted -s)
* test mode, regression test suite
* command script:
** init_script
** make_cmds
*** make_part_cmds
*** make_raid_cmds
*** make_lvm_cmds
** make_filesystems
* dependencies between LVM/RAID commands -> should be detected by parser/semantic analysis multiple RAID/LVM stanzas, but deps must be satisfied, global RAID-dev counter
* suggestions for modifications of the above format
** put options on a separate line
** size specification via size_sda1
** size specs by RAM size
** options for lvcreate (e.g. physical volume)
* internal format, generated by parser:
** list of records: line_no[int], mode[string], type[string], size[string], fs[string], fs_opts[string]
** map to store fstab-info: dev(unique), mount_point(unique), fs, mount_opts


A parser for the EBNF above is available in the svn, /people/michael/features/setup_harddisks_2
[[Category:User Manual]]

Latest revision as of 16:26, 21 July 2011

Introduction

As setup_harddisks is not portable due to its dependency on sfdisk and the lack of support for RAID and LVM, it has been re-implemented from scratch. As of FAI 3.2.8 it is integrated with mainline FAI, but not yet used by default.

To test the current implementation, keep reading. There are some TODOs left and probably a lot more will arise as soon as people actually start testing it. Before you start, just let me clarify that this is a very dangerous piece of software that has not been sufficiently tested for any cases where you want to retain any of your data. It will definitely destroy any data on any of your disks in your system. You have been warned.

If you're still interested, the following steps are required next:

  • get FAI 3.2.8 (or later)
  • add USE_SETUP_STORAGE=1 to one of your scripts in class/
  • you may want to take a look at the SETUPSTORAGE class used in the simple example in FAI 3.2.8 and/or see man 8 setup-storage
  • add lvm2 and mdadm to one of your client classes (it must be available on the target system, if you define LVMs/RAIDs in your disk_config)
  • create a disk_config for some class MY_TEST_CLASS from the old one apart from (at least) the following changes:
    • the filesystem is now given as the 4th column, the mount options are now in column 5
    • there is no ;-hack anymore -- anything given after the mount options is passed on to mkfs.<filesystem>
    • Partitions are marked as bootable using the bootable:<nr> option in the disk_config line
    • The other things should at least work like they did before, but there are some new things as well, like RAID support (which is surely incomplete), LVM support, mounting by label or UUID.
    • Please take a look at the attached examples to get an idea of the new format.
    • All details of the implemented syntax and examples are given below.
  • Give it a go (you have backed up all data, don't you?). There will be lots of debug output; if it finishes with the printout of an fstab file that is to your liking, it should have succeeded.


Some further infos may also be found on wiki.debian.org.

New configuration file syntax

In the following, we present a complete EBNF description of a modified configuration file syntax, as well as some examples.

file ::= <lines> EOF 

lines ::= EOL 
          /* empty lines or whitespace only */
          | <comment> EOL 
          | <config> EOL 

comment ::= #.* 

config ::= disk_config lvm( <lvmoption>)* 
           | disk_config raid( <raidoption>)*
           | disk_config end 
           | disk_config disk[[:digit:]]+( <option>)*
           | disk_config [^[:space:]]+( <option>)*
           /* fully qualified device-path or short form, like hda, whereby full
            * path is assumed to be /dev/hda */
           | <volume>

lvmoption ::= /* empty */
           | preserve_always:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* preserve partitions -- always */
           | preserve_reinstall:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* preserve partitions -- unless the system is installed for the 
           first time */
           | resize:[^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*
           /* attempt to resize partitions */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab, the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */

raidoption ::= /* empty */
           | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- always */
           | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- unless the system is installed for the 
           first time */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab, the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */

option ::= /* empty */
           | preserve_always:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- always */
           | preserve_reinstall:[[:digit:]]+(,[[:digit:]]+)*
           /* preserve partitions -- unless the system is installed for the 
           first time */
           | resize:[[:digit:]]+(,[[:digit:]]+)*
           /* attempt to resize partitions */
           | disklabel:(msdos|gpt)
           /* write a disklabel - default is msdos */
           | bootable:[[:digit:]]+
           /* mark a partition bootable, default is / */
           | virtual
           /* do not assume the disk to be a physical device, use with xen */
           | fstabkey:(device|label|uuid)
           /* when creating the fstab, the key used for defining the device
           may be the device (/dev/xxx), a label given using -L, or the uuid
           */  

volume ::= <type> <mountpoint> <size> <filesystem> <mount_options> <fs_options>
           | vg <name> <size>
           /* lvm vg */

type ::= primary
         /* for physical disks only */
         | logical
         /* for physical disks only */
         | raid[0156]
         /* raid level */
         | [^/[:space:]]+-[^/[:space:]]+
         /* lvm logical volume: vg name and lv name*/

mountpoint ::= -
               /* do not mount */
               | swap
               /* swap space */
               | /[^[:space:]]*(:encrypt)?
               /* fully qualified path; if :encrypt is given, the partition
                * will be encrypted, the key is generated automatically */

name ::= [^/[:space:]]+
         /* lvm volume group name */

size ::= [[:digit:]]+[kMGTP%]?(-([[:digit:]]+[kMGTP%]?)?)?(:resize)?
         /* size in kilo, mega (default), giga, tera or petabytes or %,
          * possibly given as a range; physical
          * partitions or lvm logical volumes only; */
         | -[[:digit:]]+[kMGTP%]?(:resize)?
         /* size in kilo, mega (default), giga, tera or petabytes or %,
          * given as upper limit; physical partitions
          * or lvm logical volumes only */
         | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
         /* devices and options for a raid or lvm vg */

mount_options ::= [^[:space:]]+

filesystem ::= -
               | swap
               | [^[:space:]]
               /* mkfs.xxx must exist */

fs_options ::= (createopts=".*"|tuneopts=".*")*
               /* options to append to mkfs.xxx and to the filesystem-specific
                * tuning tool */

The major differences to the prior format:

  • the disk_config ... line allows for the keywords lvm and raid
  • options may need to be appended to the disk_config line
  • the ";" is not used anymore, the options that were given there have now been split up
    • the filesystem is now an explicit parameter; note, that the order of filesystem/mount-options is the same /etc/fstab, as opposed to the previous format of disk_config
    • any options to mkfs.xxx may be given using createopts=""
    • The "preserveX" and "boot" options are one of the options now given on the

disk_config line, using preserve_reinstall or preserve_always and bootable. preserve_always is equivalent to the previous preserveX option, whereas preserve_reinstall preserves the partition unless "initial" is given as one of the FAI_FLAGS.

  • support for LVM and RAID is completely new :-)
  • resizing partitions is supported

Some examples

In fai 3.2.19 syntax has changed a little bit

use:

 createopts="-O dir_index,resize_inode" instead of -O dir_index,resize_inode 

May be we can update the examples if the new syntax also works in debian stable I have not checked it. --Andreas 11:28, 25 April 2009 (CEST)

# Configure the device /dev/hda
disk_config hda   preserve_always:6,7   disklabel:msdos  bootable:3
# preserve the 6th and the 7th partition. The disklabel is msdos, which is the default
# for x86. Furthermore the 3rd partition is made bootable. 
primary /boot     20-100        ext3            rw
# create a primary partition /dev/hda1 with a size between 20 and 100 MB and mount it
# read-write as /boot; it is formatted using ext3 filesystem
primary swap      1000     swap       sw
# /dev/hda2 will be a swap space of 1000 MB
primary /         12000      ext3           rw        createopts="-b 2048"
# /dev/hda3 should be formatted using ext3 filesystem; when calling mkfs.ext3
# the option "-b 2048" is appended.
logical /tmp      1000      ext3            rw,nosuid
# create the logical partition /dev/hda5
logical /usr      10000      ext3          rw
logical /var      10%-      ext3               rw
# make /dev/hda7 at least 10% of the disk size
logical /nobackup 0-        xfs                rw
# use mkfs.xfs to format the partition


# Configure the virtual device /dev/sda (for, e.g., Xen setups), all
# partitions are primary and sizes are ignored (so one could as well specify
# anything other than 0)
disk_config sda virtual
primary /boot     0            ext3                 rw
primary /         0            ext3                 rw
primary /tmp      0            ext3                 rw
primary /usr      0            ext3                 rw
primary /var      0            ext3                 rw

# resizing partitions is possible by appending :resize to any given size
disk_config /dev/scsi/host0/bus0/target1/lun0 
primary /      3000-6000:resize         ext3         rw
# resize to any value between 3000 and 6000 MB, as limited by the disk size,
# the desired sizes of the other partitions, and the data on the partition
primary /tmp   1000                     ext3         rw


# Create a softRAID
disk_config raid
raid1        /    sda1,sdd1  ext2        rw,errors=remount-ro
# create a RAID-1 on /dev/sda1 and /dev/sdd1, format using mkfs.ext2 and mount
# it as /
raid0        -    disk2.2,sdc1,sde1:spare:missing  ext2       default
# create a RAID-0 on the second partition of the second disk, /dev/sdc1, and
# /dev/sde1 as a spare partition, which (may?) me missing


# Simple LVM example
disk_config sda  bootable:1
primary /boot 500 ext3 rw
primary -       4096-   -       -

disk_config lvm
vg my_pv        sda2
my_pv-_swap     swap    2048    swap    sw
my_pv-_root     /       2048    ext3 rw

A pretty complex setup looks as follows:

disk_config sda

primary  -    256         -    -  
primary  swap 1024        swap       sw
primary  -		0-				 -    -

disk_config sdb
primary  -		0-				 - -

disk_config sdc
primary  -		0-				 - -

disk_config sdd
primary  -             	256         - -
primary  -          	1024        -                   -
primary  -		0-				 - -

disk_config sde
primary  -		0-				 - -

disk_config sdf
primary  -		0-				 - -

disk_config raid

raid1        /    sda1,sdd1  ext2    rw,errors=remount-ro   
raid1        swap sda2,sdd2  swap    rw                     
raid1        -    sda3,sdd3  ext2    default                

raid0        -    sdb1,sde1  ext2    default
raid0        -    sdc1,sdf1  ext2    default

# config the LVM
disk_config lvm
vg  my_pv   md2,md3   
my_pv-_usr	/usr			2048        ext3  rw -O dir_index,resize_inode
my_pv-_var	/var			600         ext3  rw -O dir_index,resize_inode
my_pv-_e_h	/export/home		10240       reiserfs          rw,notail
my_pv-_e_s	/export/sites		2048        reiserfs          rw,notail
my_pv-_v	/vservers		2048        ext3    rw 

Implementation

The current implementation still some TODOs left:

Missing features and regressions

  • Missing features
    • lazyformat -- will probably not be implemented as preserve_reinstall is much more stable.
    • some auto mode (something like auto:server, auto:desktop?) might be desirable
    • once everything is done, some checks should be performed, e.g., test that all partitions were created
    • Having /boot on a SW-RAID or LVM should be tested, I think it doesn't work - yes, it does indeed fail because $BOOT_* don't get set. No idea what the proper setting is -- use class GRUB_PC as shown in the simple example in experimental
    • implement disklabels other than msdos and gpt
  • Implementation details
    • preserve/resize deserve thorough testing, especially in case of LVM/RAID
    • The code of create_volume_group for the case of an existing vg should handle existing information and not rely on lvm commands to silently ignore existing volumes
    • mdadm --misc --zero-superblock /dev/hdx may be necessary
    • preserve must retain the flags (bootable, etc.)
    • the RAID commands are surely incomplete and lack any management of unanticipated situations
    • evaluate the return code of external commands in exec.pm -- currently only a warning is given
    • more error messages must be caught by exec.pm

Documentation of internal data structures

The hash of all configurations specified in the disk_config file

PHY_<DEVICE>
  virtual (0|1)
  disklabel STRING
  bootable -1..n
  partitions
    <1..n>
      size
        extended (0|1)
        preserve (0|1)
        resize (0|1)
        range
        eff_size
      number 1..n
      maps_to_existing 1..n
      start_byte
      end_byte
      mountpoint
      mount_options
      filesystem
      fs_options
      label
VG_<NAME>
  devices
  estimated_size
  volumes
    <logical-volume-name>
      size
        preserve (0|1)
        resize (0|1)
        range
        eff_size
      mountpoint
      mount_options
      filesystem
      fs_options
      label
RAID
  volumes
    <0..n>
      mode
      devices
        /dev/<device-name>
          options
            spare (0|1)
            missing (0|1)
      mountpoint
      mount_options
      filesystem
      fs_options
      label

The current disk configuration

<DEVICE>
  bios_cylinders
  bios_heads
  bios_sectors_per_track
  sector_size
  disklabel
  begin_byte
  end_byte
  partitions
    <1..n>
      begin_byte
      end_byte
      count_byte
      is_extended
      filesystem

The current LVM configuration

<VG>
  physical_volumes
  size
  volumes
    <lv-name>
      size

The current RAID configuration

<0..n>
  devices
  mode

Other implementations for RAID and LVM

Meanwhile you might also want to look at

See also