Documentation and examples related to the puppi actions: deploy, rollback and init
Shell command to launch a deploy:
puppi deploy <project_name>
puppi deploy <project_name> [-f] [-i] [-t] [-d yes|full] [-r yes|no|fail] [-p "parameter=value parameter2=value2"]
Shell command to launch a rollback:
puppi rollback <project_name>
Shell command to launch the first deploy:
puppi init <project_name>
Deploy myapp with the standard logic/parameters defined in Puppet:
puppi deploy myapp
Deploy myapp and doesn't stop in case of Critical errors:
puppi deploy myapp -f
Deploy myapp in interactive mode. Confirmation is asked for each step
puppi deploy myapp -i
Test mode. Just show the commands that would be executed
puppi deploy myapp -t
Deploy myapp with full debugging output
puppi deploy myapp -d full
Deploy myapp in interactive mode and sets some custom options that override the standard Puppet params. Note that these parameters change according to the script you use (and the scripts must honour this override in order to make this option work).
puppi deploy myapp -i -o "version=1.1 source_url=http://dev.example42.com/code/my_app/"
Make the first deploy of "myapp". Can be optional and may require a subsequent puppi deploy myapp
puppi init myapp
Rollback myapp to a previous archived state. User is asked to choose which deploy to override. Note that by default you have 5 backups to rollback from. Older backups are automatically removed.
puppi rollback myapp
Automatically rollback to the latest saved state (unattended).
puppi rollback myapp latest
Here follow some sample defines you can use out of the box in your manifests once you include puppi. Get a war from $source and deploy it in $deploy_root:
puppi::project::war { 'myapp':
source => 'http://repo.example42.com/deploy/prod/myapp.war',
deploy_root => '/store/tomcat/myapp/webapps',
}
Get a tarball from $source, unpack it in $deploy_root, restart service called apache and send a mail to $report_mail (you can have a comma separated list of destination addresses):
puppi::project::tar { 'mysite':
source => 'rsync://repo.example42.com/deploy/prod/release.tgz',
init_script => 'apache',
deploy_root => '/var/www/html/',
report_email => '[email protected]',
enable => 'true', # Redundant
}
Get a tarfile with a .sql query file and apply to to you local Mysql with the credentials provided:
puppi::project::mysql { 'myweb_sql':
source => 'http://repo.example42.com/deploy/prod/database.sql.gz',
mysql_user => 'myweb',
mysql_password => $secret::mysql_myweb_pw,
mysql_host => 'localhost',
mysql_database => 'myweb',
report_email => '[email protected],[email protected]',
enable => 'true',
}
Get a list of files from $source, retrieve the actual files from $source_baseurl and place them in $deploy_root:
puppi::project::files { 'gfxupdates':
source => 'http://deploy.example42.com/prod/website2/list.txt',
source_baseurl => 'http://design.example42.com/website2/gfx/',
deploy_root => '/var/www/html/gfx',
report_email => '[email protected],[email protected]',
enable => 'true',
}
Deploy from a Nexus repository (retrieve maven-metadata.xml from dir specified in $source), get the war (version is achieved from the "release" tag in the xml) and deploy it in $deploy_root and then restart tomcat.
puppi::project::maven { 'supersite':
source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/',
deploy_root => '/usr/local/tomcat/supersite/webapps',
init_script => 'tomcat',
report_email => '[email protected]',
enable => 'true',
}
Get the maven-metadata.xml from a Nexus repository and deploy:
-
The release war in $deploy_root
-
A configurations tarball tagged with the Maven qualifier $config_suffix in $config_root
-
A static files tarball tagged with the Maven qualifier $document_suffix in $document_root
puppi::project::maven { 'supersite': source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/', deploy_root => '/usr/local/tomcat/supersite/webapps', config_suffix => 'cfg', config_root => '/srv/htdocs/supersite', document_suffix => 'css', document_root => '/srv/htdocs/supersite', init_script => 'tomcat', report_email => '[email protected]', enable => 'true', }
The same deploy Nexus repository with some more options:
-
A Source dir to be used to deploy static files when issuing "puppi init supersite"
-
A block from a loadbalancer IP (managing different sites addresses)
-
Some more elaborate rsync exclusion rules
-
A backup retention of 3 archives (instead of the default 5)
puppi::project::maven { 'supersite': source => 'http://nexus.example42.com/nexus/content/repositories/releases/it/example42/supersite/', deploy_root => '/usr/local/tomcat/supersite/webapps', config_suffix => 'cfg', config_root => '/srv/htdocs/supersite', document_suffix => 'css', document_root => '/srv/htdocs/supersite', document_init_source => 'rsync://backup.example42.com/initdir/supersite/', firewall_src_ip => $site ? { dr => '192.168.101.1/30', main => '192.168.1.1/30', }, backup_rsync_options => '--exclude .snapshot --exclude /doc_root/autojs/*** --exclude /doc_root/autocss/*** --exclude /doc_root/xsl', backup_retention => '3', init_script => 'tomcat', report_email => '[email protected]', enable => 'true', }
An elaborated war deploy:
-
get from $source,
-
execute /usr/local/bin/mychecks.sh as root before the deploy (or better with sequence number 39)
-
deploy to /data/tomcat/myapp/webapps as user pippo
-
stop and start tomcat-myapp but also monit and puppet
-
backup passing $backup_rsync_options to rsync:
puppi::project::war { 'myapp': source => 'http://repo.example42.com/deploy/prod/myapp.war', deploy_root => '/store/tomcat/myapp/webapps', predeploy_customcommand => '/usr/local/bin/mychecks.sh', predeploy_priority => '39', predeploy_user => 'root', backup_rsync_options => '--exclude logs/', user => 'pippo', init_script => 'tomcat-myapp', deploy_root => '/data/tomcat/myapp/webapps', report_email => '[email protected]', disable_services => 'monit puppet', }
An example of usage of the generic builder define to deploy a zip file, with an example custom post deploy command executed as root (as all puppi commands, if not specified otherwise)
puppi::project::builder { 'cms':
source => 'http://repo.example42.com/deploy/cms/cms.zip',
source_type => 'zip',
user => 'root',
deploy_root => '/var/www',
postdeploy_customcommand => 'chown -R www-data /var/www/files',
postdeploy_user => 'root',
postdeploy_priority => '41',
report_email => '[email protected]',
enable => 'true',
}
These are just examples, possibilities are many, refer to the docs in puppi/manifests/project/*.pp to have a full list of the available options.
The above puppi::projects defines manage more or less complex deployments procedures for different kind of projects. They are just samples of possible procedures and you may create your one ones, if the ones provided there don't fit your needs. They will have to contain one or more of these basic puppi defines.
Create the main project structure. One or more different deployment projects can exist on a node.
puppi::project
Create a single command to be placed in the init sequence. It's not required for every project.
puppi::initialize
Create a single command to be placed in the deploy sequence. More than one is generally needed for each project.
puppi::deploy
Create a single command to be placed in the rollback sequence. More than one is generally needed for each project.
puppi::rollback
These defines have generally a standard structure and similar arguments. Every one is reversible (enable => false) but you can wipe out the whole /etc/puppi directory to have it rebuilt from scratch. Here is an example for a single deploy command:
puppi::deploy { 'Retrieve files': # The $name of the define is used in the file name
command => 'get_curl.sh', # The name of the general-use script to use
argument => 'file:///storage/file', # The argument(s) passed to the above script
priority => '10', # Lower priority scripts are executed first
user => 'root', # As what user we run the script
project => 'my-web.app', # The name of the project to use
}
This define creates a file named:
/etc/puppi/projects/${project}/deploy/${priority}-${name}
Its content is, simply:
su - ${user} -c "export project=${project} && /etc/puppi/scripts/${command} ${arguments}"
The puppi module provides some examples of deploy workflows they are in the puppi/manifests/project directory and contain simple to use defines that contain one puppi::project and several puppi::deploy and puppi::rollback defines to design a specific workflow using the builtin commands present in puppi/files/scripts. Note that if you need to design your own deploy procedure you have different options:
- Verify if you can reuse the existing ones, using optional arguments as pre/postdeploy_commands
- Use the existing ones as a base to make you own custom defines, reusing parts of their logic and the builtin commands (puppi/files/scripts/*) they use
- Write your own commands (in whatever language) and integrate them in your own procedure.
Here follow the main and most reusable deploy workflows defines available in puppi/manifests/project/. The have generally a set of common arguments that make you manage if to stop and restart specific services, if you want to isolate your server from a loadbalancer during the deploy procedure, what to backup, how many backup copies to maintain, if to send a report mail to specific addresses and if you need to run custom commands during the standard procedure. For all of the you have to specify a source from where to get the source files (http/ftp/rsync/file..) a directory where to place them and the user that has to own the deploy files. Full documentation is available in the relevant .pp files
- puppi::project::tar - Use this to retrieve and deploy a simple tar file
- puppi::project::war - Use this to retrieve and deploy a war
- puppi::project::files - Use this to deploy one or more files based on a provided list
- puppi::project::dir - Use this to synchronize a remote directory to a local deploy one
- puppi::project::mysql - Use this to retrieve and apply a .sql file for mysql schema updates
- puppi::project::maven - Use this to deploy war and tar files generated via Maven released on Nexus or similar. A good source of Open Source Java artifacts is http://www.artifact-repository.org/
- puppi::project::builder - This is a general purpose define that incorporates most the of cases provided by the above procedures
The puppi/files/scripts directory in the module contains some general usage "native" bash scripts that can be used in custom deployments. They are generally made to work together according to a specific logic, which is at the base of the deploy procedures defines in puppi/manifests/project/ but you're free to write your own scripts, in whatever language, according to your needs.
The default scripts are engineered to follow these steps for a deployment:
- Remote files are downloaded in /tmp/puppi/$project/store or directly in the predeploy directory: /tmp/puppi/$project/deploy
- If necessary the downloaded files are expanded in one or more predeploy directories (default:/tmp/puppi/$project/deploy).
- Runtime configuration entries might be saved in /tmp/puppi/$project/config
- Files are eventually backed from the deploy directory (Apache' docroot, Tomcat webapps or whatever) to the archive directory (/var/lib/puppi/archive/$project). Older backups are deleted (by default there's a retention of 5 backups).
- Files are copied from the predeploy directory to the deploy dir.
- Relevant services are eventually stopped and started
The most used common scripts are (they might have different arguments, some of them are quite simple):
- get_file.sh - Retrieves a file via ssh/http/rsync/svn and places it in a temp dir (store or predeploy)
- deploy.sh - Copies the files in the predeploy dir to deploy dir
- archive.sh - Backups and restores files in deploy dir
- service.sh - Stops or starts one or more services
- wait.sh - Waits for the presence or absence of a file, for the presence of a string in a file or a defined number or seconds.
- get_metadata.sh - Extracts metadata from various sources in order to provide info to other scripts. These info are save in the Runtime configuration file (/tmp/puppi/$project/config)
- report_mail.sh - Sends a mail with the report of the operations done
General functions, used by both the puppi command and these scripts, are in puppi/files/scripts/functions