FaiTemplates: Difference between revisions

From FAIWiki
Jump to navigation Jump to search
No edit summary
m (+ category)
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Abstract: i've coded a proof of concept on how to use Template-Toolkit,
= Abstract =
YAML and Hash::Merge. The code in the attachment should
demonstrate what i mean. [[Image:FaiTS-0.01.tar.gz|Download]] it and type


aptitude install libclone-perl libyaml-perl libtemplate-perl
Template Systems are well known in Webprogramming. But they are useful for sysadmins too. In [[Image:FaiTS-0.01.tar.gz|FaiTS]] i try to combine the class concept of fai with templates.
tar xzf FaiTS-0.01.tar.gz
cd FaiTS-0.01/
perl Makefile.PL
make test


If you get lots of errors and a final
= Example =
"All tests successful"
My goal is to create a config file for a router, ''/etc/network/interfaces'', during a fai installation. Here is, what the target file finally should look like:
then the code should work ...


To see something happen, you can set CLEANUP to 0 in line 27 of the file
== goal ==
''t/FaiTS.t'' and then run
''/target/etc/network/interfaces'':
  make test
# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
again
# The loopback interface
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
        address  172.16.240.1
        netmask  255.255.255.0
        network  172.16.240.0
        broadcast 172.16.240.255
   
auto eth1
iface eth1 inet static
        address  192.168.1.5
        netmask  255.255.255.0
        network  192.168.1.0
        broadcast 192.168.1.255
auto eth2
iface eth2 inet static
        address  212.202.236.176
        netmask  255.255.255.0
        network  212.202.236.0
        broadcast 212.202.236.255
        gateway  212.202.236.177


If you then type
== template ==
Here is how a template for this file could look like. The way i've implemented it now, template-toolkit [http://www.template-toolkit.org] is used as template engine. And the template-files are located in a directory tree ''/fai/templates'', similar to the ''/fai/files'' tree in fai:


find -ls
''/fai/templates/etc/network/interfaces/ROUTER.tt'':
 
and you'll see the ''.yml' (YAML) files with config data, ''.tt'' (
Template-Toolkit) templates and a target directory with a freshly
created etc/network/interfaces file.
 
Long Version in German, sorry ...
 
Ich hab diese Woche mal ein wenig gebastelt und drei Ideen in ein
kleines Programm eingebaut. Ist im Moment noch mehr ein
proof-of-concept, aber ich bin von meiner Idee völlig begeistert  ;-)
 
1. Template-Toolkit:
====================================================================
Templates zu schreiben geht einfacher als Configdateien mit perl & co
nachzubearbeiten. Beispiel ''/etc/network/interfaces'':


  # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
  # /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)
Line 55: Line 59:
  [% END %]
  [% END %]


Everything between the [% %] brackets ist template-toolkit syntax. In this case, a variable named ''interfaces'' contains a list of interface definitions. Each interface-definition is assigned a to a varable named ''i''. Each instance of ''i'' contains the parameters ''address'' and ''netmask'', some instances also contain parameters ''network'', ''broadcast'', ''gateway''.


2. YAML:
== YAML ==
====================================================================
But where to get the data from, to fill in these variables? An nice format for flat files is YAML [ http://www.yaml.org ]:
Über Shell-Variablen kann man nur flache Datenstrukturen transportieren.
Das engt manchmal etwas ein. Z.B. wenn ich die Daten für eine
''/etc/network/interfaces'' eines Routers in ''/class/KLASSE.var'' unterbringen
möchte, muß ich ganz schön Klimmzüge machen. Wahrscheinlich löst man es
besser, in dem man dann doch für jeden Router eine eigene Datei
''/files/etc/network/interfaces/*'' schreibt.
 
Mit YAML gibt es ein Dateiformat, für das es in jeder gängigen
Skriptsprache einen Parser gibt. Die Informationen für die interfaces
Datei könnten dann so aussehen:


''/fai/class/ROUTER.yaml'':
  interfaces:
  interfaces:
   eth0:
   eth0:
Line 80: Line 76:
     network: 192.168.1.0
     network: 192.168.1.0
     broadcast: 192.168.1.255
     broadcast: 192.168.1.255
''/fai/class/EXTERN.yaml'':
interfaces:
  eth0:
    address: 172.16.240.1
   eth2:
   eth2:
     address: 192.168.1.5
     address: 212.202.236.176
     netmask: 255.255.255.0
     netmask: 255.255.255.0
     network: 192.168.1.0
     network: 212.202.236.0
     broadcast: 192.168.1.255
     broadcast: 212.202.236.255
     gateway: 212.202.236.177
     gateway: 212.202.236.177


== class based merge ==
Hey, what did i win so far? Didn't i have to write one config file before ( see man interfaces (5) ), and now i have to write two files ( .yml and .tt )?
Yes, but now you're more flexible: only .yml-files that belog two your install client's class are choosen. Data in these files is aggregated / merged together. Than, again respecting fai classes, template files are choosen, where  this data is filled in.
And .yml files are only the first data-source implemented. Other alternatives are an LDAP-directory, a database or the scripts called in task_class or task_configure. Each backend adding more data.
And of course I'm not limited to only fill one template with this data. I could create a template ''/templates/etc/udev/rules.d/30-net_persistent_names.rules/ROUTER.tt'', and ''/templates/etc/hosts/ROUTER.tt'', and so on.
= Cool. Is this already implemented? =
yes. i've coded a proof of concept using Template-Toolkit,
YAML and Hash::Merge.
== using it ==
[[Image:FaiTS-0.01.tar.gz|Download]] it and type:
aptitude install libclone-perl libyaml-perl libtemplate-perl
tar xzf FaiTS-0.01.tar.gz
cd FaiTS-0.01/
perl Makefile.PL
make test
If you get lots of errors and a final
"All tests successful"
then the code should work ...
To see something happen, you can set CLEANUP to 0 in line 27 of the file
''t/FaiTS.t'' and then run
make test
again
If you then type
find -ls
and you'll see the ''.yml' (YAML) files with config data, ''.tt'' (
Template-Toolkit) templates and a target directory with a freshly
created etc/network/interfaces file.
== development ==
There are two "executables" in this package:
bin/fai-ts.pl
and
t/FaiTS.t
First i started only with ''bin/fai-ts.pl''. But i don't like skripts getting to long, so i tried to
source out as much functions as possible to a perl module ''lib/FaiTS.pm''. For every subroutine in ''lib/FaiTS.pm'' there is a test in ''t/FaiTS.t''. This is to further document the module, and to make sure that future changes to ''lib/FaiTS.pm'' don't break things.


3. Config-Daten klassenbasiert zusammenführen
Dokumentation is contained in the source as POD and accessable via perldoc:
==============================================
perldoc lib/FaiTS.pm
perldoc bin/fai-ts.pl


Bei den Variablen in ''class/*.var'' ist es einfach sie klassenbasiert
zusammenzuführen. Später definierte Klassen überschreiben Variablen von
früher definierten Klassen.


Ich dachte erst, bei tiefer strukturierten Daten sei das kompliziert.
[[Category:Development]]
Isses wahrscheinlich auch, aber das Perl Modul Hash::Merge hat mir die
[[Category:Howto]]
Arbeit abgenommen.

Latest revision as of 10:27, 17 November 2009

Abstract

Template Systems are well known in Webprogramming. But they are useful for sysadmins too. In File:FaiTS-0.01.tar.gz i try to combine the class concept of fai with templates.

Example

My goal is to create a config file for a router, /etc/network/interfaces, during a fai installation. Here is, what the target file finally should look like:

goal

/target/etc/network/interfaces:

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
       address   172.16.240.1
       netmask   255.255.255.0
       network   172.16.240.0
       broadcast 172.16.240.255

auto eth1
iface eth1 inet static
        address   192.168.1.5
        netmask   255.255.255.0
        network   192.168.1.0
        broadcast 192.168.1.255

auto eth2
iface eth2 inet static
        address   212.202.236.176
        netmask   255.255.255.0
        network   212.202.236.0
        broadcast 212.202.236.255
        gateway   212.202.236.177

template

Here is how a template for this file could look like. The way i've implemented it now, template-toolkit [1] is used as template engine. And the template-files are located in a directory tree /fai/templates, similar to the /fai/files tree in fai:

/fai/templates/etc/network/interfaces/ROUTER.tt:

# /etc/network/interfaces -- configuration file for ifup(8), ifdown(8)

# The loopback interface
auto lo
iface lo inet loopback

[% FOREACH name = interfaces.keys.sort %]
[% i = interfaces.$name -%]
auto [% name %]
iface [% name %] inet static
        address   [% i.address %]
        netmask   [% i.netmask %]
       [% IF i.network   %] network   [% i.network   %] [% END %]
       [% IF i.broadcast %] broadcast [% i.broadcast %] [% END %]
       [% IF i.gateway   %] gateway   [% i.gateway   %] [% END %] 

[% END %]

Everything between the [% %] brackets ist template-toolkit syntax. In this case, a variable named interfaces contains a list of interface definitions. Each interface-definition is assigned a to a varable named i. Each instance of i contains the parameters address and netmask, some instances also contain parameters network, broadcast, gateway.

YAML

But where to get the data from, to fill in these variables? An nice format for flat files is YAML [ http://www.yaml.org ]:

/fai/class/ROUTER.yaml:

interfaces:
  eth0:
    address: 172.16.240.5
    netmask: 255.255.255.0
    network: 172.16.240.0
    broadcast: 172.16.240.255
  eth1:
    address: 192.168.1.5
    netmask: 255.255.255.0
    network: 192.168.1.0
    broadcast: 192.168.1.255

/fai/class/EXTERN.yaml:

interfaces:
  eth0:
    address: 172.16.240.1
  eth2:
    address: 212.202.236.176
    netmask: 255.255.255.0
    network: 212.202.236.0
    broadcast: 212.202.236.255
    gateway: 212.202.236.177

class based merge

Hey, what did i win so far? Didn't i have to write one config file before ( see man interfaces (5) ), and now i have to write two files ( .yml and .tt )?

Yes, but now you're more flexible: only .yml-files that belog two your install client's class are choosen. Data in these files is aggregated / merged together. Than, again respecting fai classes, template files are choosen, where this data is filled in.

And .yml files are only the first data-source implemented. Other alternatives are an LDAP-directory, a database or the scripts called in task_class or task_configure. Each backend adding more data.

And of course I'm not limited to only fill one template with this data. I could create a template /templates/etc/udev/rules.d/30-net_persistent_names.rules/ROUTER.tt, and /templates/etc/hosts/ROUTER.tt, and so on.

Cool. Is this already implemented?

yes. i've coded a proof of concept using Template-Toolkit, YAML and Hash::Merge.

using it

File:FaiTS-0.01.tar.gz it and type:

aptitude install libclone-perl libyaml-perl libtemplate-perl
tar xzf FaiTS-0.01.tar.gz
cd FaiTS-0.01/
perl Makefile.PL
make test

If you get lots of errors and a final "All tests successful" then the code should work ...

To see something happen, you can set CLEANUP to 0 in line 27 of the file t/FaiTS.t and then run

make test 

again

If you then type

find -ls

and you'll see the .yml' (YAML) files with config data, .tt ( Template-Toolkit) templates and a target directory with a freshly created etc/network/interfaces file.

development

There are two "executables" in this package:

bin/fai-ts.pl

and

t/FaiTS.t

First i started only with bin/fai-ts.pl. But i don't like skripts getting to long, so i tried to source out as much functions as possible to a perl module lib/FaiTS.pm. For every subroutine in lib/FaiTS.pm there is a test in t/FaiTS.t. This is to further document the module, and to make sure that future changes to lib/FaiTS.pm don't break things.

Dokumentation is contained in the source as POD and accessable via perldoc:

perldoc lib/FaiTS.pm
perldoc bin/fai-ts.pl