-
Notifications
You must be signed in to change notification settings - Fork 55
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
(PE-37233-35) Official support of Add compiler plan (#485)
* (PE-37236) Remove experimental status of add_compiler (#428) * (PE-37233) Adding add_compiler to test matrix (#434) Updated add_compiler spec to check compiler has been added and puppet run before running add compiler Added test matrix for add compiler * (PE-38814) add_compiler - Making primary_postgresql_host and avail_group_letter optional (#468) * (PE-38814) add_compiler - Making primary_postgresql_host and avail_group_letter optional primary_postgresql_host, if not provided will be determined through get_peadm_config avail_group_letter, is defaulting to A * Updating spec tests for add compiler Removing optional from avail_group_letter as not required with enum default value * Updating reference.md * Fixing linting issues --------- Co-authored-by: Neil Anderson <[email protected]> * (PE-37235) Verify and update documentation on add_compiler (#429) * PE-37235 Verify and update documentation on add_compiler if needed and its experimental status * @cathal41 PE-37235 Verify and update documentation on add_compiler - wording updates * Update plans/add_compiler.pp Co-authored-by: Dimitri Tischenko <[email protected]> * Update documentation/add_compiler.md Co-authored-by: Dimitri Tischenko <[email protected]> * Update documentation/add_compiler.md Co-authored-by: Dimitri Tischenko <[email protected]> * Update documentation/add_compiler.md Co-authored-by: Dimitri Tischenko <[email protected]> * Update documentation/add_compiler.md Co-authored-by: Dimitri Tischenko <[email protected]> * Update documentation/add_compiler.md Co-authored-by: Dimitri Tischenko <[email protected]> * Updating add compiler docs to show optional params * Updating docs with text corrections --------- Co-authored-by: Dimitri Tischenko <[email protected]> Co-authored-by: Neil Anderson <[email protected]> * chore(workflows): simplify YAML syntax and improve readability - Removed unnecessary quotes around strings in YAML files. - Consolidated list items to single-line format where applicable. - Ensured consistent naming conventions for job steps. - Improved readability by removing extra blank lines. * fix(workflow): add console_password to test-add-compiler workflows - Added `console_password` parameter to the `test-add-compiler-matrix.yml` and `test-add-compiler.yaml` workflows. - This ensures the console password is passed correctly during the workflow execution. * chore: update test-add-compiler-matrix workflow - Remove centos-7 from the image matrix in test-add-compiler-matrix.yml - Retain only almalinux-cloud/almalinux-8 in the image matrix * feat(workflow): add --stream option to bolt plan run command Added the --stream option to the bolt plan run command in the test-add-compiler GitHub workflow. This change ensures that the output is streamed in real-time. * chore(workflow): update compiler versions in test-add-compiler-matrix.yml Updated the compiler versions in the test-add-compiler-matrix.yml GitHub workflow from 2021.7.7 and 2023.6.0 to 2021.7.9 and 2023.8.0. --------- Co-authored-by: Jo Lillie <[email protected]> Co-authored-by: Neil Anderson <[email protected]> Co-authored-by: Neil Anderson <[email protected]> Co-authored-by: cathal41 <[email protected]> Co-authored-by: Dimitri Tischenko <[email protected]>
- Loading branch information
6 people
authored
Sep 12, 2024
1 parent
33b5dac
commit 68329f0
Showing
9 changed files
with
394 additions
and
65 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| --- | ||
| name: Add compiler matrix | ||
| on: | ||
| pull_request: | ||
| paths: | ||
| - .github/workflows/**/* | ||
| - spec/**/* | ||
| - lib/**/* | ||
| - tasks/**/* | ||
| - functions/**/* | ||
| - types/**/* | ||
| - plans/**/* | ||
| - hiera/**/* | ||
| - manifests/**/* | ||
| - templates/**/* | ||
| - files/**/* | ||
| - metadata.json | ||
| - Rakefile | ||
| - Gemfile | ||
| - provision.yaml | ||
| - .rspec | ||
| - .rubocop.yml | ||
| - .puppet-lint.rc | ||
| - .fixtures.yml | ||
| branches: [main] | ||
| workflow_dispatch: {} | ||
| jobs: | ||
| test-add-compiler: | ||
| name: PE ${{ matrix.version }} ${{ matrix.architecture }} on ${{ matrix.image }} | ||
| runs-on: ubuntu-20.04 | ||
| env: | ||
| BOLT_GEM: true | ||
| BOLT_DISABLE_ANALYTICS: true | ||
| LANG: en_US.UTF-8 | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| architecture: [standard, large, extra-large] | ||
| version: [2021.7.9, 2023.8.0] | ||
| image: [almalinux-cloud/almalinux-8] | ||
| steps: | ||
| - name: Checkout Source | ||
| uses: actions/checkout@v2 | ||
| - name: Activate Ruby 2.7 | ||
| uses: ruby/setup-ruby@v1 | ||
| with: | ||
| ruby-version: '2.7' | ||
| bundler-cache: true | ||
| - name: Print bundle environment | ||
| if: ${{ github.repository_owner == 'puppetlabs' }} | ||
| run: | | ||
| echo ::group::info:bundler | ||
| bundle env | ||
| echo ::endgroup:: | ||
| - name: Provision test cluster | ||
| timeout-minutes: 15 | ||
| run: | | ||
| echo ::group::prepare | ||
| mkdir -p $HOME/.ssh | ||
| echo 'Host *' > $HOME/.ssh/config | ||
| echo ' ServerAliveInterval 150' >> $HOME/.ssh/config | ||
| echo ' ServerAliveCountMax 2' >> $HOME/.ssh/config | ||
| bundle exec rake spec_prep | ||
| echo ::endgroup:: | ||
| echo ::group::provision | ||
| bundle exec bolt plan run peadm_spec::provision_test_cluster \ | ||
| --modulepath spec/fixtures/modules \ | ||
| provider=provision_service \ | ||
| image=${{ matrix.image }} \ | ||
| architecture=${{ matrix.architecture }}-with-extra-compiler | ||
| echo ::endgroup:: | ||
| echo ::group::info:request | ||
| cat request.json || true; echo | ||
| echo ::endgroup:: | ||
| echo ::group::info:inventory | ||
| sed -e 's/password: .*/password: "[redacted]"/' < spec/fixtures/litmus_inventory.yaml || true | ||
| echo ::endgroup:: | ||
| - name: Install PE on test cluster | ||
| timeout-minutes: 120 | ||
| run: | | ||
| bundle exec bolt plan run peadm_spec::install_test_cluster \ | ||
| --inventoryfile spec/fixtures/litmus_inventory.yaml \ | ||
| --modulepath spec/fixtures/modules \ | ||
| architecture=${{ matrix.architecture }} \ | ||
| console_password=${{ secrets.CONSOLE_PASSWORD }} \ | ||
| version=${{ matrix.version }} | ||
| - name: Run add_compiler plan | ||
| timeout-minutes: 50 | ||
| run: | | ||
| bundle exec bolt plan run peadm_spec::add_compiler -v \ | ||
| --inventoryfile spec/fixtures/litmus_inventory.yaml \ | ||
| --modulepath spec/fixtures/modules \ | ||
| architecture=${{ matrix.architecture }} | ||
| - name: Tear down test cluster | ||
| if: ${{ always() }} | ||
| continue-on-error: true | ||
| run: |- | ||
| if [ -f spec/fixtures/litmus_inventory.yaml ]; then | ||
| echo ::group::tear_down | ||
| bundle exec rake 'litmus:tear_down' | ||
| echo ::endgroup:: | ||
| echo ::group::info:request | ||
| cat request.json || true; echo | ||
| echo ::endgroup:: | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| # Add Compiler | ||
|
|
||
| - [Add Compiler](#Add-Compiler) | ||
| - [Introduction](#Introduction) | ||
| - [Add a compiler to an existing PE cluster](#Add-a-compiler-to-an-existing-PE-cluster) | ||
| - [Running the `add_compiler` plan](#running-the-add_compiler-plan) | ||
| - [Optional Parameters](#optional-parameters) | ||
|
|
||
|
|
||
| ## Introduction | ||
|
|
||
| The `peadm::add_compiler` plan can be used to add a new compiler to a PE cluster or replace an existing one with new configuration. | ||
|
|
||
|
|
||
| ## Add a compiler to an existing PE cluster | ||
|
|
||
| As seen in the example below, this is the minimal parameters required to add a compiler to an existing PE cluster. These can be passed as command line variables to the plan, or in this case added to a params.json file. | ||
|
|
||
| ```json | ||
| { | ||
| "compiler_host": "pe-xl-compiler-0.lab1.puppet.vm", | ||
| "primary_host": "pe-xl-core-0.lab1.puppet.vm" | ||
| } | ||
| ``` | ||
|
|
||
| ## Running the `add_compiler` plan | ||
| ``` | ||
| bolt plan run peadm::add_compiler --params @params.json | ||
| ``` | ||
|
|
||
| This command will retrieve the current PEADM configuration to determine the setup rules needed for a compiler's secondary PuppetDB instances. The plan will configure the primary with appropriate rules for allowing access from the new compiler. On the primary, the `puppet` service is stopped and the `pe-postgresql` service is reloaded. If required, a puppet agent will be installed on the new compiler host. The compiler agent's certificate is regenerated to include data required by the `peadm::subplans::component_install` plan. A subsequent Puppet agent run will happen on the following components. | ||
| * _\<compiler-host\>_ | ||
| * _\<primary_postgresql_host\>_ | ||
| * _\<replica host\>_ | ||
| * _\<primary_postgresql_host\>_ | ||
|
|
||
| The `puppet` service is then restarted. | ||
|
|
||
|
|
||
| ## Optional Parameters | ||
|
|
||
| As well as `compiler_host` and `primary_host`, the `add_compiler` plan has a number of optional parameters. These can be viewed in the following params example. | ||
|
|
||
| ```json | ||
| { | ||
| "avail_group_letter": "B", | ||
| "compiler_host": "pe-xl-compiler-0.lab1.puppet.vm", | ||
| "dns_alt_names": [ "puppet", "puppet.lab1.puppet.vm" ], | ||
| "primary_host": "pe-xl-core-0.lab1.puppet.vm", | ||
| "primary_postgresql_host": "pe-psql-6251cd-0.us-west1-a.c.slice-cody.internal" | ||
| } | ||
| ``` | ||
|
|
||
| Please note, the optional parameters and values of the plan are as follows. | ||
|
|
||
| <!-- table --> | ||
| | Parameter | Default value | Description | | ||
| |--------------------------|---------------|--------------------------------------------------| | ||
| | `avail_group_letter` | `A` | By default, each compiler will be added to the primary group A. | | ||
| | `dns_alt_names` | `undef` | | | ||
| | `primary_postgresql_host`| `undef` | By default, this will pre-populate to the required value depending if your architecture contains HA and or external databases. | | ||
|
|
||
| For more information around adding compilers to your infrastructure [Expanding Your Deployment](expanding.md#adding-compilers-with-peadmadd_compiler) | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.