Skip to content

Commit

Permalink
Make $pe_installer_source more flexible
Browse files Browse the repository at this point in the history
This makes the peadm::install plan more flexible. Previously the
variable $pe_installer_source could point to an absolute URL directly to
the desired .tar.gz. Now it can also point to a web directory. Then
peadm will calculate the .tar.gz name and fetch it from the web
directory.

This applies for the peadm::upgrade and peadm::install plan.
  • Loading branch information
bastelfreak committed Aug 5, 2024
1 parent 9a6d500 commit 9f5b4ab
Show file tree
Hide file tree
Showing 7 changed files with 168 additions and 42 deletions.
60 changes: 49 additions & 11 deletions REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
* [`peadm::migration_opts_default`](#peadm--migration_opts_default)
* [`peadm::node_manager_yaml_location`](#peadm--node_manager_yaml_location)
* [`peadm::oid`](#peadm--oid)
* [`peadm::pe_installer_source`](#peadm--pe_installer_source): calculates the PE installer URL and archive name
* [`peadm::plan_step`](#peadm--plan_step)
* [`peadm::recovery_opts_all`](#peadm--recovery_opts_all)
* [`peadm::recovery_opts_default`](#peadm--recovery_opts_default)
Expand Down Expand Up @@ -805,6 +806,40 @@ Data type: `String`



### <a name="peadm--pe_installer_source"></a>`peadm::pe_installer_source`

Type: Puppet Language

calculates the PE installer URL and archive name

#### `peadm::pe_installer_source(Optional[Stdlib::HTTPSUrl] $pe_installer_source = undef, Optional[Peadm::Pe_version] $version = undef, Optional[String[1]] $platform = undef)`

The peadm::pe_installer_source function.

Returns: `Hash[String[1],String[1]]`

##### `pe_installer_source`

Data type: `Optional[Stdlib::HTTPSUrl]`

The URL to download the Puppet Enterprise installer media from. If not
specified, PEAdm will attempt to download PE installation media from its
standard public source. When specified, PEAdm will download directly from the
URL given. Can be an URL, that ends with a /, to a web directory that
contains the original archives or an absolute URL to the .tar.gz archive.

##### `version`

Data type: `Optional[Peadm::Pe_version]`

The desired version for PE. This is optional for custom provided absolute URLs.

##### `platform`

Data type: `Optional[String[1]]`

The platform we're on, for example el-9-x86_64 (osfamily short name - version - arch)

### <a name="peadm--plan_step"></a>`peadm::plan_step`

Type: Ruby 4.x API
Expand Down Expand Up @@ -1862,7 +1897,8 @@ Data type: `Optional[Stdlib::HTTPSUrl]`
The URL to download the Puppet Enterprise installer media from. If not
specified, PEAdm will attempt to download PE installation media from its
standard public source. When specified, PEAdm will download directly from the
URL given.
URL given. Can be an URL, that ends with a /, to a web directory that
contains the original archives or an absolute URL to the .tar.gz archive.

Default value: `undef`

Expand Down Expand Up @@ -2277,12 +2313,12 @@ The following parameters are available in the `peadm::upgrade` plan:
* [`r10k_known_hosts`](#-peadm--upgrade--r10k_known_hosts)
* [`stagingdir`](#-peadm--upgrade--stagingdir)
* [`uploaddir`](#-peadm--upgrade--uploaddir)
* [`version`](#-peadm--upgrade--version)
* [`primary_host`](#-peadm--upgrade--primary_host)
* [`replica_host`](#-peadm--upgrade--replica_host)
* [`compiler_hosts`](#-peadm--upgrade--compiler_hosts)
* [`primary_postgresql_host`](#-peadm--upgrade--primary_postgresql_host)
* [`replica_postgresql_host`](#-peadm--upgrade--replica_postgresql_host)
* [`version`](#-peadm--upgrade--version)
* [`token_file`](#-peadm--upgrade--token_file)
* [`download_mode`](#-peadm--upgrade--download_mode)
* [`permit_unsafe_versions`](#-peadm--upgrade--permit_unsafe_versions)
Expand Down Expand Up @@ -2324,7 +2360,9 @@ Data type: `Optional[Stdlib::HTTPSUrl]`
The URL to download the Puppet Enterprise installer media from. If not
specified, PEAdm will attempt to download PE installation media from its
standard public source. When specified, PEAdm will download directly from the
URL given.
URL given. Can be an URL, that ends with a /, to a web directory that
contains the original archives or an absolute URL to the .tar.gz archive.
If it's an URL ending with the archive name, you don't need to set $version.

Default value: `undef`

Expand Down Expand Up @@ -2366,6 +2404,14 @@ for offline usage.

Default value: `'/tmp'`

##### <a name="-peadm--upgrade--version"></a>`version`

Data type: `Optional[Peadm::Pe_version]`

The desired version for PE. This is optional for custom provided absolute URLs.

Default value: `undef`

##### <a name="-peadm--upgrade--primary_host"></a>`primary_host`

Data type: `Peadm::SingleTargetSpec`
Expand Down Expand Up @@ -2402,14 +2448,6 @@ Data type: `Optional[Peadm::SingleTargetSpec]`



Default value: `undef`

##### <a name="-peadm--upgrade--version"></a>`version`

Data type: `Optional[Peadm::Pe_version]`



Default value: `undef`

##### <a name="-peadm--upgrade--token_file"></a>`token_file`
Expand Down
46 changes: 46 additions & 0 deletions functions/pe_installer_source.pp
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#
# @summary calculates the PE installer URL and archive name
#
# @param pe_installer_source
# The URL to download the Puppet Enterprise installer media from. If not
# specified, PEAdm will attempt to download PE installation media from its
# standard public source. When specified, PEAdm will download directly from the
# URL given. Can be an URL, that ends with a /, to a web directory that
# contains the original archives or an absolute URL to the .tar.gz archive.
#
# @param version
# The desired version for PE. This is optional for custom provided absolute URLs.
#
# @param platform
# The platform we're on, for example el-9-x86_64 (osfamily short name - version - arch)
#
# @author Tim Meusel <[email protected]>
#
function peadm::pe_installer_source (
Optional[Stdlib::HTTPSUrl] $pe_installer_source = undef,
Optional[Peadm::Pe_version] $version = undef,
Optional[String[1]] $platform = undef,
) >> Hash[String[1],String[1]] {
if $pe_installer_source {
# custom URL ends with /, so we assume it's a webdir with the original installer
if $pe_installer_source[-1] == '/' {
assert_type(Peadm::Pe_version, $version)
assert_type(String[1], $platform)
$_version = $version
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
$pe_tarball_source = "${pe_installer_source}${pe_tarball_name}"
} else {
$pe_tarball_name = $pe_installer_source.split('/')[-1]
$pe_tarball_source = $pe_installer_source
$_version = $pe_tarball_name.split('-')[2]
}
$data = { 'url' => $pe_tarball_source, 'filename' => $pe_tarball_name, 'version' => pick($_version,$version), }
} else {
assert_type(Peadm::Pe_version, $version)
assert_type(String[1], $platform)
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
$data = { 'url' => $pe_tarball_source, 'filename' => $pe_tarball_name, 'version' => $version }
}
$data
}
3 changes: 2 additions & 1 deletion plans/install.pp
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
# The URL to download the Puppet Enterprise installer media from. If not
# specified, PEAdm will attempt to download PE installation media from its
# standard public source. When specified, PEAdm will download directly from the
# URL given.
# URL given. Can be an URL, that ends with a /, to a web directory that
# contains the original archives or an absolute URL to the .tar.gz archive.
# @param ldap_config
# If specified, configures PE RBAC DS with the supplied configuration hash.
# The parameter should be set to a valid set of connection settings as
Expand Down
19 changes: 7 additions & 12 deletions plans/subplans/install.pp
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,8 @@
# The URL to download the Puppet Enterprise installer media from. If not
# specified, PEAdm will attempt to download PE installation media from its
# standard public source. When specified, PEAdm will download directly from the
# URL given.
# URL given. Can be an URL, that ends with a /, to a web directory that
# contains the original archives or an absolute URL to the .tar.gz archive.
#
plan peadm::subplans::install (
# Standard
Expand Down Expand Up @@ -228,27 +229,21 @@
)
}

if $pe_installer_source {
$pe_tarball_name = $pe_installer_source.split('/')[-1]
$pe_tarball_source = $pe_installer_source
} else {
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
}
$pe_installer = peadm::pe_installer_source($pe_installer_source, $version, $platform)

$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
$upload_tarball_path = "${uploaddir}/${pe_installer['filename']}"

if $download_mode == 'bolthost' {
# Download the PE tarball and send it to the nodes that need it
run_plan('peadm::util::retrieve_and_upload', $pe_installer_targets,
source => $pe_tarball_source,
local_path => "${stagingdir}/${pe_tarball_name}",
source => $pe_installer['url'],
local_path => "${stagingdir}/${pe_installer['filename']}",
upload_path => $upload_tarball_path,
)
} else {
# Download PE tarballs directly to nodes that need it
run_task('peadm::download', $pe_installer_targets,
source => $pe_tarball_source,
source => $pe_installer['url'],
path => $upload_tarball_path,
)
}
Expand Down
30 changes: 13 additions & 17 deletions plans/upgrade.pp
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,9 @@
# The URL to download the Puppet Enterprise installer media from. If not
# specified, PEAdm will attempt to download PE installation media from its
# standard public source. When specified, PEAdm will download directly from the
# URL given.
# URL given. Can be an URL, that ends with a /, to a web directory that
# contains the original archives or an absolute URL to the .tar.gz archive.
# If it's an URL ending with the archive name, you don't need to set $version.
# @param final_agent_state
# Configures the state the puppet agent should be in on infrastructure nodes
# after PE is upgraded successfully.
Expand All @@ -30,6 +32,8 @@
# @param uploaddir
# Directory the installer tarball will be uploaded to or expected to be in
# for offline usage.
# @param version
# The desired version for PE. This is optional for custom provided absolute URLs.
#
plan peadm::upgrade (
# Standard
Expand Down Expand Up @@ -117,21 +121,13 @@

$platform = run_task('peadm::precheck', $primary_target).first['platform']

if $pe_installer_source {
$pe_tarball_name = $pe_installer_source.split('/')[-1]
$pe_tarball_source = $pe_installer_source
$_version = $pe_tarball_name.split('-')[2]
} else {
$_version = $version
$pe_tarball_name = "puppet-enterprise-${_version}-${platform}.tar.gz"
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${_version}/${pe_tarball_name}"
}
$pe_installer = peadm::pe_installer_source($pe_installer_source, $version, $platform)

$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
$upload_tarball_path = "${uploaddir}/${pe_installer['filename']}"

peadm::assert_supported_bolt_version()

peadm::assert_supported_pe_version($_version, $permit_unsafe_versions)
peadm::assert_supported_pe_version($pe_installer['version'], $permit_unsafe_versions)

# Gather certificate extension information from all systems
$cert_extensions = run_task('peadm::cert_data', $all_targets).reduce({}) |$memo,$result| {
Expand Down Expand Up @@ -176,14 +172,14 @@
if $download_mode == 'bolthost' {
# Download the PE tarball on the nodes that need it
run_plan('peadm::util::retrieve_and_upload', $pe_installer_targets,
source => $pe_tarball_source,
local_path => "${stagingdir}/${pe_tarball_name}",
source => $pe_installer['url'],
local_path => "${stagingdir}/${pe_installer['filename']}",
upload_path => $upload_tarball_path,
)
} else {
# Download PE tarballs directly to nodes that need it
run_task('peadm::download', $pe_installer_targets,
source => $pe_tarball_source,
source => $pe_installer['url'],
path => $upload_tarball_path,
)
}
Expand Down Expand Up @@ -366,7 +362,7 @@
# doesn't deal well with the PuppetDB database being on a separate node.
# So, move it aside before running the upgrade.
$pdbapps = '/opt/puppetlabs/server/apps/puppetdb/cli/apps'
$workaround_delete_reports = $arch['disaster-recovery'] and $_version =~ SemVerRange('>= 2019.8')
$workaround_delete_reports = $arch['disaster-recovery'] and $pe_installer['version'] =~ SemVerRange('>= 2019.8')
if $workaround_delete_reports {
# lint:ignore:strict_indent
run_command(@("COMMAND"/$), $replica_target)
Expand Down Expand Up @@ -418,7 +414,7 @@
)
}

peadm::check_version_and_known_hosts($current_pe_version, $_version, $r10k_known_hosts)
peadm::check_version_and_known_hosts($current_pe_version, $pe_installer['version'], $r10k_known_hosts)

return("Upgrade of Puppet Enterprise ${arch['architecture']} completed.")
}
47 changes: 47 additions & 0 deletions spec/functions/pe_installer_source_spec.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# frozen_string_literal: true

require 'spec_helper'

describe 'peadm::pe_installer_source' do
it 'exists' do
is_expected.not_to be_nil
end

context 'when called with no parameters' do
it { is_expected.to run.with_params.and_raise_error(Puppet::PreformattedError) }
end
context 'when called with absolute url and version' do
result = {
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'version' => '2019.8.12'
}
it { is_expected.to run.with_params('https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz', '2019.8.12').and_return(result) }
end
context 'when called with absolute url' do
result = {
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'version' => '2019.8.12'
}
it { is_expected.to run.with_params('https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz').and_return(result) }
end
context 'when called with url and version and platform' do
result = {
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'version' => '2019.8.12'
}
it { is_expected.to run.with_params('https://url/', '2019.8.12', 'el-8-x86_64').and_return(result) }
end
context 'when called without url and with version and platform' do
result = {
'url' => 'https://s3.amazonaws.com/pe-builds/released/2019.8.12/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
'version' => '2019.8.12'
}
it do
is_expected.to run.with_params(nil, '2019.8.12', 'el-8-x86_64').and_return(result)
end
end
end
5 changes: 4 additions & 1 deletion spec/plans/upgrade_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def allow_standard_non_returning_calls
.always_return({ 'content' => '2021.7.3' })

expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary)

allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })
expect(run_plan('peadm::upgrade',
'primary_host' => 'primary',
'version' => '2021.7.8')).to be_ok
Expand All @@ -43,6 +43,7 @@ def allow_standard_non_returning_calls

expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary,
'compiler' => trusted_compiler)
allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })

expect(run_plan('peadm::upgrade',
'primary_host' => 'primary',
Expand Down Expand Up @@ -92,6 +93,8 @@ def allow_standard_non_returning_calls
.always_return({ 'content' => installed_version })

expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary)

allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })
end

it 'updates pe.conf if r10k_known_hosts is set' do
Expand Down

0 comments on commit 9f5b4ab

Please sign in to comment.