Using Puppet modules
In the previous example, I used the simple manifest(puppet/templates/motd.pp) which didn't need any additional modules. But in general, Puppet manifests are associated with modules, self-contained bundles of code and data which can be included in the top level manifest. With Packstack, you can put your favorite modules under the directory "puppet/modules", and facts files under "puppet/facts". The current list of modules are as follows:
# ls puppet/modules/ apache firewall keystone openstack ssh vlan cinder glance memcached packstack stdlib xinetd concat horizon mysql qpid swift create_resources inifile nova rsync sysctl
They are copied to the target server and specified in the modulepath. But the problem is that not all of them are actually copied. The modules to be copied are hard coded in "plugins/puppet_950.py" as below:
plugins/puppet_950.py
def copyPuppetModules(): os_modules = ' '.join(('apache', 'cinder', 'concat', 'create_resources', 'firewall', 'glance', 'horizon', 'inifile', 'keystone', 'memcached', 'mysql', 'nova', 'openstack', 'packstack', 'qpid', 'rsync', 'ssh', 'stdlib', 'swift', 'sysctl', 'vlan', 'xinetd'))
If you want to add your own module, you need to add its directory name to the list above.
Multi target option
In some cases, you may want to apply the same configuration to multiple servers. It's not that hard to modify the existing single target plugin to the multi targert one. Here's the modified version of motd_020.py.
plugin/motd_020.py
""" plugin for configuring /etc/motd """ import os import logging from packstack.installer import validators import packstack.installer.common_utils as utils from packstack.installer.exceptions import ScriptRuntimeError from packstack.modules.ospluginutils import getManifestTemplate, appendManifestFile, manifestfiles # Controller object will be initialized from main flow controller = None PLUGIN_NAME = "OS-MOTD" logging.debug("plugin %s loaded", __name__) def initConfig(controllerObject): global controller controller = controllerObject paramsList = [ {"CMD_OPTION" : "motd-host", "USAGE" : "A comma separated list of IP addresses to configure /etc/motd", "PROMPT" : "Enter a comma separated list of IP addresses to configure /etc/motd", "OPTION_LIST" : [], "VALIDATORS" : [validators.validate_multi_ssh], "DEFAULT_VALUE" : utils.getLocalhostIP(), "MASK_INPUT" : False, "LOOSE_VALIDATION": True, "CONF_NAME" : "CONFIG_MOTD_HOSTS", "USE_DEFAULT" : False, "NEED_CONFIRM" : False, "CONDITION" : False }, {"CMD_OPTION" : "motd-message", "USAGE" : "message text in /etc/motd", "PROMPT" : "Enter the message", "OPTION_LIST" : [], "VALIDATORS" : [validators.validate_not_empty], "DEFAULT_VALUE" : "Hello, World!", "MASK_INPUT" : False, "LOOSE_VALIDATION": True, "CONF_NAME" : "CONFIG_MOTD_MESSAGE", "USE_DEFAULT" : False, "NEED_CONFIRM" : False, "CONDITION" : False }, ] groupDict = { "GROUP_NAME" : "MOTD", "DESCRIPTION" : "MOTD Options", "PRE_CONDITION" : "CONFIG_MOTD", "PRE_CONDITION_MATCH" : "y", "POST_CONDITION" : False, "POST_CONDITION_MATCH" : True} controller.addGroup(groupDict, paramsList) def initSequences(controller): if controller.CONF['CONFIG_MOTD'] != 'y': return configmotdsteps = [ {'title': 'Configuring /etd/motd message', 'functions':[createtestmanifest]}, ] controller.addSequence("Installing config-motd Component", [], [], configmotdsteps) def createtestmanifest(): for host in controller.CONF['CONFIG_MOTD_HOSTS'].split(','): manifestfile = "%s_motd.pp" % host manifestdata = getManifestTemplate("motd.pp") appendManifestFile(manifestfile, manifestdata)
The new option CONFIG_MOTD_HOSTS accepts the comma separated list as target servers. As the convention, the option key should end with "_HOSTS". In createmanifest(), the same manifest file is added for each server in the list.
Here is the result of the test run.
# packstack --debug
Welcome to Installer setup utility
Enter the path to your ssh Public key to install on servers [/root/.ssh/id_rsa.pub] :
Enter a comma separated list of NTP server(s). Leave plain if Packstack should not install ntpd on instances.:
Should Packstack configure motd [y|n] [y] :
Enter a comma separated list of IP addresses to configure /etc/motd [192.168.122.191] : 192.168.122.191,192.168.122.192
Enter the message [Hello, World!] : Hello, Worlds!
To subscribe each server to EPEL enter "y" [y|n] [y] :
Enter a comma separated list of URLs to any additional yum repositories to install:
To subscribe each server to Red Hat enter a username here:
To subscribe each server to Red Hat enter your password here :
To subscribe each server to Red Hat Enterprise Linux 6 Server Beta channel (only needed for Preview versions of RHOS) enter "y" [y|n] [n] :
To subscribe each server with RHN Satellite enter RHN Satellite server URL:
Installer will be installed using the following configuration:
==============================================================
ssh-public-key: /root/.ssh/id_rsa.pub
ntp-severs:
config-motd: y
motd-host: 192.168.122.191,192.168.122.192
motd-message: Hello, Worlds!
use-epel: y
additional-repo:
rh-username:
rh-password:
rh-beta-repo: n
rhn-satellite-server:
Proceed with the configuration listed above? (yes|no): yes
Installing:
Clean Up... [ DONE ]
Setting up ssh keys... [ DONE ]
Adding pre install manifest entries... [ DONE ]
Configuring /etd/motd message... [ DONE ]
Preparing servers... [ DONE ]
Adding post install manifest entries... [ DONE ]
Installing Dependencies... [ DONE ]
Copying Puppet modules and manifests... [ DONE ]
Applying Puppet manifests...
Applying 192.168.122.191_prescript.pp
Applying 192.168.122.192_prescript.pp
Applying 192.168.122.191_motd.pp
Applying 192.168.122.192_motd.pp
192.168.122.192_prescript.pp : [ DONE ]
192.168.122.192_motd.pp : [ DONE ]
192.168.122.191_prescript.pp : [ DONE ]
192.168.122.191_motd.pp : [ DONE ]
Applying 192.168.122.191_postscript.pp
Applying 192.168.122.192_postscript.pp
192.168.122.192_postscript.pp : [ DONE ]
192.168.122.191_postscript.pp : [ DONE ]
[ DONE ]
**** Installation completed successfully ******
Additional information:
* A new answerfile was created in: /root/packstack-answers-20130521-154526.txt
* Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
* The installation log file is available at: /var/tmp/packstack/20130521-154457-z4LFcK/openstack-setup.logOne thing you need to know from this output is the following fact...
At first, the following four manifests are applied in parallel.
Applying 192.168.122.191_prescript.pp Applying 192.168.122.192_prescript.pp Applying 192.168.122.191_motd.pp Applying 192.168.122.192_motd.pp 192.168.122.192_prescript.pp : [ DONE ] 192.168.122.192_motd.pp : [ DONE ] 192.168.122.191_prescript.pp : [ DONE ] 192.168.122.191_motd.pp : [ DONE ]
After waiting until they finish, the following two manifests are applied in parallel.
Applying 192.168.122.191_postscript.pp Applying 192.168.122.192_postscript.pp 192.168.122.192_postscript.pp : [ DONE ] 192.168.122.191_postscript.pp : [ DONE ]
You may suspect that there is an underlying mechanism to control the parallel manifest application. Yep, this is the very next topic.
Parallel Puppet configuration
Puppet manifests are applied when applyPuppetManifest() in the worker queue is executed (See the drawing in the previous part.) It takes manifests from the staging area with FIFO manner and apply it. By default, it proceeds to the next manifest without waiting the previous manifest to finish.
If you want to mark some waiting point as in the case above, you need to attach a "marker" to the manifest. When applyPuppetManifest() find the marked manifest, it waits until all the previous manifests are applied, and proceeds with the marked one. But one exception is that if the manifest has the same marker with the previous one, it doesn't wait.
In the case above, the markers were attached as below:
192.168.122.191_prescript.pp : No marker 192.168.122.192_prescript.pp : No marker 192.168.122.191_motd.pp : No marker 192.168.122.192_motd.pp : No marker ....Wait here.... 192.168.122.191_postscript.pp : Marker "postscript" 192.168.122.192_postscript.pp : Marker "postscript"
The marker can be attached as a third argument of appendManifestFile().
plugins/postscript_949.py
def createmanifest(): for hostname in gethostlist(controller.CONF): manifestfile = "%s_postscript.pp" % hostname manifestdata = getManifestTemplate("postscript.pp") appendManifestFile(manifestfile, manifestdata, 'postscript')
The general rule for attaching markers is:
1) If you are sure that the manifest can be applied in parallel with any previous manifests, you don't attach a marker.
2) If you are not sure that the manifest can be applied in parallel with previous manifests, you should attach a unique marker to it.
3) If you are sure that the manifest can be applied in parallel with the previous one, you would attach the same marker with the previous one. This is generally the case when you apply the same manifest to multiple servers.
You can see how this marker mechanism is realised from the source code of applyPuppetManifest().
plugins/puppet_950.py
def applyPuppetManifest(): print currently_running = [] lastmarker = None for manifest, marker in manifestfiles.getFiles(): # if the marker has changed then we don't want to proceed until # all of the previous puppet runs have finished if lastmarker != None and lastmarker != marker: waitforpuppet(currently_running) lastmarker = marker ...
Continue to the next part.
