To develop for Magic Castle you will need:
- Terraform (>= 1.4.0)
- git
- Access to a Cloud (e.g.: Compute Canada Arbutus)
- Ability to communicate with the cloud provider API from your computer
- A cloud project with enough room for the resource described in section Magic Caslte Doc 1.1.
- [optional] Puppet Development Kit (PDK)
The Magic Castle project is defined by Terraform infrastructure-as-code component that is responsible of generating a cluster architecture in a cloud and a Puppet environment component that configures the cluster instances based on their role.
If you wish to add device, an instance, add a new networking interface or a filesystem, you will most likely need to develop some Terraform code. The project structure for Terraform code is described in the reference design document. The document also describes how one could work with current Magic Castle code to add support for another cloud provider.
If you wish to add a service to one of the Puppet environments, install a new software, modify an instance configuration or role, you will most likely need to develop some Puppet code. The following section provides more details on the Puppet environments available and how to develop them.
Magic Castle Terraform code initialized every instances to be a Puppet agent and an instance
with the tag puppet as the Puppet main server. On the Puppet main server, there is a folder
containing the configuration code for the instances of the cluster, this folder is called a
Puppet environment and it is pulled from GitHub during the initial configuration of the Puppet
main server.
The source of that environment is provided to Terraform using the variable config_git_url.
A repository describing a Magic Castle Puppet environment must contain at the least the following files and folders:
config_git_repo
┣ Puppetfile
┣ environment.conf
┣ hiera.yaml
┗ data
┗ common.yaml
┗ manifests/
┗ site.pp
Puppetfilespecifies the Puppet modules that need to be installed in the environment.environment.confoverrides the primary server default settings for the environment.hiera.yamlconfigures an ordered list of YAML file data sources.data/common.yamlis common data source for the instances part of hierarchy defined byhiera.yaml.manifests/site.ppdefines how each instance will be configured based on their hostname and/or tags.
An example of a bare-bone Magic Castle Puppet environment is available on GitHub: MagicCastle/puppet-environment, while the Puppet environment that replicates a Compute Canada HPC cluster is named ComputeCanada/puppet-magic_castle.
To provide information on the deployed resources and the value of the input parameters,
Magic Castle Terraform code uploads to the Puppet main server a file named terraform_data.yaml,
in the folder /etc/puppetlabs/data/. There is also a symlink created in
/etc/puppetlabs/code/environment/production/data/ to ease its usage inside the Puppet
environment.
When included in the data hierarchy (hiera.yaml), terraform_data.yaml can provide
information about the instances, the volumes and the variables set by the user
through the main.tf file. The file has the following structure:
---
terraform:
data:
cluster_name: ""
domain_name: ""
guest_passwd: ""
nb_users: ""
public_keys: []
sudoer_username: ""
instances:
host1:
hostkeys:
rsa: ""
ed25519: ""
local_ip: "x.x.x.x"
prefix: "host"
public_ip: ""
specs:
"cpus": 0
"gpus": 0
"ram": 0
tags:
- "tag_1"
- "tag_2"
tag_ip:
tag_1:
- x.x.x.x
tag_2:
- x.x.x.x
volumes:
volume_tag1:
volume_1:
- "/dev/disk/by-id/123-*"
volume_2:
- "/dev/disk/by-id/123-abc-*"The values provided by terraform_data.yaml can be accessed in Puppet by using the
lookup() function. For example, to access an instance's list of tags:
lookup("terraform.instances.${::hostname}.tags")The data source can also be used to define a key in another data source YAML file by using the
alias() function. For example, to define the number of guest accounts using the value of nb_users,
we could add this to common.yaml
profile::accounts::guests::nb_accounts: "%{alias('terraform.data.nb_users')}"The configuration of each instance is defined in manifests/site.pp file of the Puppet environment.
In this file, it is possible to define a configuration based on an instance hostname
node "mgmt1" { }
or using the instance tags by defining the configuration for the default node :
node default {
$instance_tags = lookup("terraform.instances.${::hostname}.tags")
if 'tag_1' in $instances_tags { }
}
It is possible to define Puppet resource directly
in site.pp. However, above a certain level of complexity, which can be reach fairly quickly, it
is preferable to define classes and include these classes in site.pp based on the node hostname
or tags.
Classes can be defined in the Puppet environment under the following path:
site/profile/manifests. These classes are named profile classes and the philosophy
behind it is explained in Puppet documentation. Because these classes are defined in site/profile,
their name has to start with the prefix profile::.
It is also possible to include classes defined externally and installed using the Puppetfile.
These classes installed by r10k can be found in the modules folder of the
Puppet environment.
To test new additions to puppet.yaml, it is possible to
execute cloud-init phases manually. There are four steps that can be executed sequentially: init local, init
modules config and modules final. Here are the corresponding commands to execute each step:
cloud-init init --local
cloud-init init
cloud-init modules --mode=config
cloud-init modules --mode=final
It is also possible to clean a cloud-init execution and have it execute again at next reboot. To do so, enter the following command:
cloud-init clean
Add -r to the previous command to reboot the instance once cloud-init has finishing cleaning.
SELinux is enabled on every instances of a Magic Castle cluster. Some applications do not provide
SELinux policies which can lead to their malfunctionning when SELinux is enabled. It is possible
to track down the reasons why SELinux is preventing an application to work properly using
the command-line tool ausearch.
If you suspect application app-a to be denied by SELinux to work properly, run the following
command as root:
ausearch -c app-a --raw | grep denied
To see all requests denied by SELinux:
ausearch --raw | grep denied
Sometime, the denials are hidden from regular logging. To display all denials, run the following command as root:
semodule --disable_dontaudit --build
then re-execute the application that is not working properly.
Once you have found the denials that are the cause of the problem, you can create a new policy to allow the requests that were previously denied with the following command:
ausearch -c app-a --raw | grep denied | audit2allow -a -M app-a
Finally, you can install the generated policy using the command provided by auditallow.
If you need to tweak an existing enforcement file and you want to recompile the policy package, you can with the following commands:
checkmodule -M -m -o my_policy.mod my_policy.te
semodule_package -o my_policy.pp -m my_policy.mod
- https://wiki.gentoo.org/wiki/SELinux
- https://wiki.gentoo.org/wiki/SELinux/Tutorials/Where_to_find_SELinux_permission_denial_details
To build a release, use the script release.sh located at the root of Magic Castle git repo.
Usage: release.sh VERSION [provider ...]
The script creates a folder named releases where it was called.
The VERSION argument is expected to correspond to git tag in the puppet-magic_castle repo.
It could also be a branch name or a commit. If the provider optional argument is left blank,
release files will be built for all providers currently supported by Magic Castle.
Examples:
- Building a release for OpenStack with the puppet repo main branch:
$ ./release.sh main openstack - Building a release for GCP with the latest Terraform and cloud-init, and version 5.8 of puppet
Magic Castle:
$ ./release.sh 5.8 gcp - Building a release for Azure and OVH with the latest Terraform and cloud-init, and version 5.7 of puppet
Magic Castle:
$ ./release.sh 5.7 azure ovh