SysDirector: An open source configuration management framework

Andrew Anderson and Phil D’Amore

SysDirector.org

What is SysDirector?

SysDirector is a template driven configuration management system that allows administrators to deliver content to machines in a protocol independent fashon.
At the most basic level, SysDirector combines a versioned filesystem and a template processor.
The versioned filesystem is provided by Subversion
The template processor is provided by Cheetah
SysDirector provides structured data, and a set of interfaces to tie
everything together.
All of this is implemented using the Python programming language.

Why should I use SysDirector?

SysDirector is a tool that makes managing groups of systems simpler. Think about how many configuration files on your systems are identical except for a handful of lines. Now imagine being able to customize those lines automatically based on system attributes:

Physical Location
Network Location
Function of the machine

SysDirector allows you to stop focusing on editing file contents and start focusing on the interesting parts of system administration.

Basic Concepts

System Objects

Hosts
Host Groups
Singleton Set
Operating Systems
Services
Processes
Paths

Basic Concepts Continued

Text

Template files to use as a base for the final configuation file

Data

Static variables to be used to fill out the templates

scalar — Last write wins
list — Concatinate
dict — Merge

Template Programming Environment

$data

Access to the static variables

$sysutil

System defined utility functions:
- gethostbyname
- gethostbyaddr
- netmask helper functions
- system-object lookup

$siteutil

Locally defined utility functions

Client Support

Current

HTTP

TFTP

Puppet

Future

Cfengine

Getting Started

Make use of SysDirector's templating capabilities to template /etc/resolv.conf. While on the surface this would seem to be a trivial file, it offers the ability to demonstrate many of SysDirector's features without being too complex to serve as a good example.

Start by creating the objects necessary to describe a small network. We will start with two locations, Phoenix, Arizona, and Denver, Colorado. At each site, we will have two DNS servers and two web servers.

Create an Environment and Working Copy

First create a demo environment:

> cd /env

> mkdir demo

And now checkout a copy of that environment to work with:

> cd /wc

> mkdir leap-cf

> cd /wc/leap-cf/demo

Create an Operating System

> createobj PosixOperatingSystem
Operating Sytem description
description: CentOS 5
Posix Operating System member Posix Hosts
hosts:
Operating System name
name: centos5
Posix Operating System member Posix Paths
paths:
Posix Operating System member Posix Services
services:

Create a Singleton Set

> createobj HostGroupSingletonSet
Host Group Singleton Set description
description: Physical equipment location
Host Group Singleton Set member Host Groups
members:
Host Group Singleton Set name
name: location

Create a Host Group with a Singleton Set

> createobj HostGroup
Host Group description
description: Phoenix, Arizona co-location facility
Host Group member hosts
hosts:
Host Group name
name: phx
Host Group processing order
processingorder:
Host Group Singleton Set
singletonset: `name=location`

Create a Host Group

> createobj HostGroup
Host Group description
description: DNS Servers
Host Group member hosts
hosts:
Host Group name
name: dnsservers
Host Group processing order
processingorder:
Host Group Singleton Set
singletonset:

Create a Host

> createobj PosixHost
System comment
comment: Row 3 Rack 14 Slot 15 (Chasis ID: 9265)
System description
description: Phoenix DNS server 1
System environment
environment: demo
System host groups
hostgroups: `name=phx`,`name=dnsservers`
System hostname
name: ns-01.phx.example.com
System operating system
operatingsystem: `name=centos5`
System services
services:

Create a Service

> createobj PosixService
Posix Service description
description: Berkeley Internet Name Domain Server
Posix Service member Posix Hosts
hosts: `name=ns-01.den.example.com`,`name=ns-01.phx.example.com`,`name=ns-02.den.example.com`,`name=ns-02.phx.example.com`
Posix Service name
name: bind
Posix Service member Operating System
operatingsystem: `name=centos5`
Posix Service member Packages
packages:
Posix Service member Posix Paths
paths:
Posix Service member Posix Processes
processes:

Create a Path

> createobj PosixPath
Posix Path description
description: Resolver configuration file
Posix Path group name
group: root
Posix Path mode
mode: 0644
Posix Path name
name: /etc/resolv.conf
Posix Path member Operating System
operatingsystem: `name=centos5`
Posix Path member Posix Services
services: `name=bind`,`name=httpd`
Posix Path template name
template: /resolv.conf
Posix Path type [file, directory, ...]
type: file
Posix Path user name
user: root

Create a Template

#set $dnssearchpaths = $data.dnssearchpaths
#if $dnssearchpaths
search#slurp
#for $domain in $dnssearchpaths
 ${domain}#slurp
#end for

#end if
#for $nameserver in $data.nameservers
nameserver ${nameserver}
#end for
## See resolv.conf(5) for options documentation
#set global $dnsoptions = $data.dnsoptions
#if $dnsoptions
options#slurp
#if $dnsoptions.has_key('debug')
debug#slurp
#end if
#if $dnsoptions.has_key('ndots')
ndots:${dnsoptions.ndots}#slurp
#end if
#if $dnsoptions.has_key('timeout')
timeout:${dnsoptions.timeout}#slurp
#end if
#if $dnsoptions.has_key('attempts')
attempts:${dnsoptions.attempts}#slurp
#end if
#if $dnsoptions.has_key('rotate')
rotate#slurp
#end if
#if $dnsoptions.has_key('nochecknames')
no-check-names#slurp
#end if
#if $dnsoptions.has_key('inet6')
inet6#slurp
#end if
#if $dnsoptions.has_key('ip6dotint')
ip6-dotint#slurp
#end if
#if $dnsoptions.has_key('ip6bytestring')
ip6-bytestring#slurp
#end if
#end if

Upload the Template

> cd /wc/leap-cf/demo/text
> set —file resolv.conf.template —create resolv.conf

Create the Variable Prototypes

> createvar dnssearchpaths [list,None] “Search list for host name lookup”

> createvar nameservers [list,None] “Nameserver IP addresses”

> createvar dnsoptions [dict,None] “Settings for internal resolver variables”

Create the Variable Default Context

> cd /wc/leap-cf/demo/data/default

> mkdir dnssearchpaths

> cd /wc/leap-cf/demo/data/default/dnssearchpaths

> append example.com

> get /wc/leap-cf/demo/variables/ns-01.phx.example.com/dnssearchpaths

['example.com']

Create the Variable Scoped Context

> cd /wc/leap-cf/demo/data/group

> mkdir name=phx

> mkdir name=den

> cd /wc/leap-cf/demo/data/group/name=phx

> mkdir dnssearchpaths

> mkdir nameservers

Set the Variable Values

> cd /wc/leap-cf/demo/data/group/name=phx/nameservers

> append 192.0.2.1

> append 192.0.2.2

> cd /wc/leap-cf/demo/data/group/name=phx/dnssearchpaths

> append phx.example.com

> get /wc/leap-cf/demo/variables/ns-01.phx.example.com/dnssearchpaths

['example.com', 'phx.example.com']

Preview the Generated Content

> get /wc/leap-cf/demo/files/ns-01.phx.example.com/etc/resolv.conf

search phx.example.com example.com
nameserver 192.0.2.1
nameserver 129.0.2.2

> get /wc/leap-cf/demo/files/ns-01.den.example.com/etc/resolv.conf

search den.example.com example.com
nameserver 192.0.2.129
nameserver 129.0.2.130

Add DNS Options

> cd /wc/leap-cf/demo/data/group

> mkdir name=webservers

> cd /wc/leap-cf/demo/data/group/name=webservers

> mkdir dnsoptions

> cd /wc/leap-cf/demo/data/group/name=webservers/dnsoptions

> mkdir timeout

> set timeout 3

Preview the DNS Options

> get /wc/leap-cf/demo/files/www-01.phx.example.com/etc/resolv.conf

search phx.example.com example.com
nameserver 192.0.2.1
nameserver 129.0.2.2
options timeout:3

Commit the Working Copy

> commit /wc/leap-cf/demo/system-objects /wc/leap-cf/demo/data /wc/leap-cf/demo/text —message “Add a demonstration for creating and populating resolv.conf”

Commit Message:
Add a demonstration for creating and populating resolv.conf
Changed Files:
A /data/group/name=dnsservers/dnsoptions
...
A /text/resolv.conf
A /system-objects/posixpath/name=/etc/resolv.conf;operatingsystem=name=centos5
A /system-objects/posixservice/name=httpd
Commit changes? [yes/No] yes

Dynamic Example: /etc/ntp.conf Template

; do not edit by hand
; $sysutil.filename is dynamically generated by 
; SysDirector for $sysutil.host.name
; Service: $sysutil.host.services("name=ntpd").name
#set $ntpservers = $sysutil.gethostbyname("0.pool.ntp.org")
#for $ipaddr in $ntpservers
server $ipaddr
#end for

Dynamic Example: /etc/ntp.conf Content

> get ntp.conf

; do not edit by hand
; /etc/ntp.conf is dynamically generated by 
; SysDirector for test.example.com
; Service: ntpd
server 8.15.10.42
server 66.187.233.4
server 66.250.45.2
server 69.10.36.3
server 74.53.198.146

View the Commit Log Messages

> log

--------------------------------------------------------------------------------
r1063 | andrew | 2008-07-17 12:10:21 -0400 (Thu, 17 Jul 2008) | 1 line

Add ntp.conf as a dynamic template example
--------------------------------------------------------------------------------
r1062 | andrew | 2008-07-17 11:57:41 -0400 (Thu, 17 Jul 2008) | 1 line

Add a demonstration for creating and populating resolv.conf
--------------------------------------------------------------------------------
r1061 | andrew | 2008-07-17 11:11:51 -0400 (Thu, 17 Jul 2008) | 1 line

Created demo environment
--------------------------------------------------------------------------------

View the Differences Between Revisions

> diff /env/demo@1062/text/ntp.conf /wc/leap-cf/demo/text/ntp.conf

--- /env/demo@1062/text/ntp.conf Wed Dec 31 19:00:00 1969
+++ /wc/leap-cf/demo/text/ntp.conf Thu Jul 17 11:59:22 2008
@@ -1,0 +1,8 @@
+; do not edit by hand
+; $sysutil.filename is dynamically generated by 
+; SysDirector for $sysutil.host.name
+; Service: $sysutil.host.services("name=ntpd").name
+#set $ntpservers = $sysutil.gethostbyname("0.pool.ntp.org")
+#for $ipaddr in $ntpservers
+server $ipaddr
+#end for

LEAP-CF July 17, 2008

Getting Started with SysDirector