diff --git a/README b/README index 3302579b..b9b5c910 100644 --- a/README +++ b/README @@ -1,77 +1,71 @@ - -GETSSL - +# GETSSL [Run all tests] [shellcheck] Obtain SSL certificates from the letsencrypt.org ACME server. Suitable for automating the process on remote servers. - -Table of Contents - -- Upgrade broken in v2.43 -- Features -- Overview -- Quick Start Guide -- Manual Installation -- Getting started -- Detailed guide to getting started with more examples -- Wildcard certificates -- ISPConfig -- Automating updates -- Structure -- Server-Types -- Revoke a certificate -- Elliptic curve keys -- Preferred Chain -- Include Root certificate in full chain -- Windows Server and IIS Support -- Building getssl as an RPM Package (Redhat/CentOS/SuSe/Oracle/AWS) -- Building getssl as a Debian Package (Debian/Ubuntu) -- Issues / problems / help - +Table of Contents + +- Upgrade broken in v2.43 +- Features +- Overview +- Quick Start Guide +- Manual Installation +- Getting started +- Detailed guide to getting started with more examples +- Wildcard certificates +- ISPConfig +- Automating updates +- Structure +- Server-Types +- Revoke a certificate +- Elliptic curve keys +- Preferred Chain +- Include Root certificate in full chain +- Windows Server and IIS Support +- Building getssl as an RPM Package (Redhat/CentOS/SuSe/Oracle/AWS) +- Building getssl as a Debian Package (Debian/Ubuntu) +- Issues / problems / help Upgrade broken in v2.43 The automatic upgrade in v2.43 is broken as the url is incorrect. If you have this version installed you’ll need to manually upgrade using: -curl --silent --user-agent getssl/manual https://raw.githubusercontent.com/srvrco/getssl/latest/getssl --output getssl - +curl --silent --user-agent getssl/manual --output getssl Features -- BASH - It runs on virtually all unix machines, including BSD, most +- BASH - It runs on virtually all unix machines, including BSD, most Linux distributions, macOS. -- GET CERTIFICATES FOR REMOTE SERVERS - The tokens used to provide +- GET CERTIFICATES FOR REMOTE SERVERS - The tokens used to provide validation of domain ownership, and the certificates themselves can be automatically copied to remote servers (via ssh, sftp or ftp for tokens). The script doesn’t need to run on the server itself. This can be useful if you don’t have access to run such scripts on the server itself, e.g. if it’s a shared server. -- RUNS AS A DAILY CRON - so certificates will be automatically renewed +- RUNS AS A DAILY CRON - so certificates will be automatically renewed when required. -- AUTOMATIC CERTIFICATE RENEWALS -- CHECKS CERTIFICATES ARE CORRECTLY LOADED - After installation of a +- AUTOMATIC CERTIFICATE RENEWALS +- CHECKS CERTIFICATES ARE CORRECTLY LOADED - After installation of a new certificate it will test the port specified ( see Server-Types for options ) that the certificate is actually being used correctly. -- AUTOMATICALLY UPDATES - The script can automatically update itself +- AUTOMATICALLY UPDATES - The script can automatically update itself with bug fixes etc if required. -- EXTENSIVELY CONFIGURABLE - With a simple configuration file for each +- EXTENSIVELY CONFIGURABLE - With a simple configuration file for each certificate it is possible to configure it exactly for your needs, whether a simple single domain or multiple domains across multiple servers on the same certificate. -- SUPPORTS HTTP AND DNS CHALLENGES - Full ACME implementation -- SIMPLE AND EASY TO USE -- DETAILED DEBUG INFO - Whilst it shouldn’t be needed, detailed debug +- SUPPORTS HTTP AND DNS CHALLENGES - Full ACME implementation +- SIMPLE AND EASY TO USE +- DETAILED DEBUG INFO - Whilst it shouldn’t be needed, detailed debug information is available. -- RELOAD SERVICES - After a new certificate is obtained then the +- RELOAD SERVICES - After a new certificate is obtained then the relevant services (e.g. apache/nginx/postfix) can be reloaded. -- ACME V1 AND V2 - Supports both ACME versions 1 and 2 (note ACMEv1 is +- ACME V1 AND V2 - Supports both ACME versions 1 and 2 (note ACMEv1 is deprecated and clients will automatically use v2) - Overview GetSSL was written in standard bash ( so it can be run on a server, a @@ -163,7 +157,7 @@ INSTALLING SOURCE PACKAGES To install the source package with the rpm package manager for RedHat, CentOS, SuSe, Oracle Linux, or AWS Linux distributions: - rpm -i getssl-2.47-1.src.rpm + rpm -i getssl-2.47-1.src.rpm _(Note: rpm installs the source code files in /root/rpmbuild/ as top directory for RedHat, CentOS, Oracle Linux, and AWS Linux platforms. @@ -183,12 +177,12 @@ SPECS and SOURCES directory tree structure. Subsequently, an SDEB can also be extracted and installed with the TAR -XVF COMMAND or the files listed with the TAR -TVF COMMAND: - [root@localhost getssl]$ tar -tvf /root/debbuild/SDEBS/getssl-2.47-1.sdeb + [root@localhost getssl]$ tar -tvf /root/debbuild/SDEBS/getssl-2.47-1.sdeb -rw-r--r-- root/root 1772110 2022-10-12 20:42 SOURCES/getssl-2.47.tar.gz -rw-r--r-- root/root 192 2022-08-02 15:02 SOURCES/getssl.crontab -rw-r--r-- root/root 126 2022-08-02 15:02 SOURCES/getssl.logrotate -rw-r--r-- root/root 1537 2022-08-02 15:02 SPECS/getssl.spec - [root@localhost getssl]$ + [root@localhost getssl]$ For building or rebuilding RPMS or DEB Packages after you have installed the associated source packages on your platform, refer to the following: @@ -473,21 +467,21 @@ certificate is installed correctly Server-Type Port Extra ------------------ ------ -------------- - https 443 + https 443 ftp 21 FTP Explicit ftpi 990 FTP Implicit imap 143 StartTLS - imaps 993 + imaps 993 pop3 110 StartTLS - pop3s 995 + pop3s 995 smtp 25 StartTLS - smtps_deprecated 465 + smtps_deprecated 465 smtps 587 StartTLS smtp_submission 587 StartTLS xmpp 5222 StartTLS - xmpps 5269 - ldaps 636 - port number + xmpps 5269 + ldaps 636 + port number Revoke a certificate diff --git a/README.md b/README.md index 87fbd50a..bbcbd555 100644 --- a/README.md +++ b/README.md @@ -120,7 +120,7 @@ For example, the release v2.49 contains the following packages in the release se ### **Debian Based Packages (Debian, Ubuntu)** -- [getssl_2.49-1_all.deb](https://github.com/srvrco/getssl/releases/download/2.49/getssl_2.49-1_all.deb) (binary) +- [getssl_2.49-1_all.deb](https://github.com/srvrco/getssl/releases/download/v2.49/getssl_2.49-1_all.deb) (binary) ### **Installing Binary Packages** diff --git a/dns_scripts/GoDaddy-README.txt b/dns_scripts/GoDaddy-README.txt index 54db3b8e..9670cc02 100644 --- a/dns_scripts/GoDaddy-README.txt +++ b/dns_scripts/GoDaddy-README.txt @@ -1,3 +1,13 @@ +--------------------------------------------------------------------------- + NOTE: GoDaddy updated their domain API requirements in 2024! + + It's now required to have at least 10-50 domains in your account. + Another option is to purchase the "Discount Domain Club" (Premium!) + to gain access to the API. + + Source: https://www.reddit.com/r/godaddy/comments/1bl0f5r/ +--------------------------------------------------------------------------- + Using GoDaddy DNS for LetsEncrypt domain validation. Quick guide to setting up getssl for domain validation of @@ -22,7 +32,7 @@ There are two prerequisites to using getssl with GoDaddy DNS: With those in hand, the installation procedure is: -1) Put JSON.sh in the getssl DNS scripts directory +1) Put JSON.sh in the getssl DNS scripts directory Default: /usr/share/getssl/dns_scripts 2) Open your config file (the global file in ~/.getssl/getssl.cfg diff --git a/dns_scripts/INWX-README.md b/dns_scripts/INWX-README.md new file mode 100644 index 00000000..19ab059b --- /dev/null +++ b/dns_scripts/INWX-README.md @@ -0,0 +1,62 @@ +## Using INWX DNS for LetsEncrypt domain validation + +### Install Requirements + +The INWX API Python3 script requires two Python packages: + +```bash +pip3 install INWX.Domrobot tldextract +``` + +You could install it for the user running getssl, or you could create a python3 venv. + +```bash +# install python3 venv apt packages +sudo apt install python3 python3-venv + +# Create venv +python3 -m venv venv + +# activate venv +source venv/bin/activate + +# install requirements +pip3 install INWX.Domrobot tldextract +``` + +If you are installing the Python packages in venv, you should make sure that you either +you either enable the venv before running getssl, or you +add the venv to the ``DNS_ADD_COMMAND'' and ``DNS_DEL_COMMAND'' commands. +See example below. + +### Enabling the scripts + +Set the following options in `getssl.cfg` (either global or domain-specific): + +``` +VALIDATE_VIA_DNS="true" +DNS_ADD_COMMAND="/usr/share/getssl/dns_scripts/dns_add_inwx.py" +DNS_DEL_COMMAND="/usr/share/getssl/dns_scripts/dns_del_inwx.py" +``` + +If you are using a python3 venv as described above, this is an example of how to include it: + +``` +VALIDATE_VIA_DNS="true" +DNS_ADD_COMMAND="/path/to/venv/bin/python3 /usr/share/getssl/dns_scripts/dns_add_inwx.py" +DNS_DEL_COMMAND="/path/to/venv/bin/python3 /usr/share/getssl/dns_scripts/dns_del_inwx.py" +``` + +*Obviously the "/path/to/venv" needs to be replaced with the actual path to your venv, e.g. "/home/getssl/venv".* + +### Authentication + +Your INWX credentials will be used to authenticate to INWX. +If you are using a second factor, please have a look at the [INWX Domrobot Pthon3 Client](https://github.com/inwx/python-client) as it is currently not implemented in the inwx api script. + +Set the following options in the domain-specific `getssl.cfg` or make sure these enviroment variables are present. + +``` +export INWX_USERNAME="your_inwx_username" +export INWX_PASSWORD="..." +``` diff --git a/dns_scripts/Route53-README.md b/dns_scripts/Route53-README.md new file mode 100644 index 00000000..b078a782 --- /dev/null +++ b/dns_scripts/Route53-README.md @@ -0,0 +1,37 @@ +# Using Route53 BASH scripts for LetsEncrypt domain validation. + +## Quick guide to setting up getssl for domain validation of Route53 DNS domains. + +There a few prerequisites to using getssl with Route53 DNS: + +1. You will need to set up an IAM user with the necessary permissions to modify resource records in the hosted zone. + + - route53:ListHostedZones + - route53:ChangeResourceRecordSets + +1. You will need the AWS CLI Client installed on your machine. + +1. You will need to configure the client for the IAM user that has permission to modify the resource records. + +With those in hand, the installation procedure is: + +1. Open your config file (the global file in ~/.getssl/getssl.cfg + or the per-account file in ~/.getssl/example.net/getssl.cfg) + +1. Set the following options: + + - VALIDATE_VIA_DNS="true" + - DNS_ADD_COMMAND="/usr/share/getssl/dns_scripts/dns_add_route53" + - DNS_DEL_COMMAND="/usr/share/getssl/dns_scripts/dns_del_route53" + + The AWS CLI profile to use (will use _default_ if not specified) + + - export AWS*CLI_PROFILE="\_profile name*" + +1. Set any other options that you wish (per the standard + directions.) Use the test CA to make sure that + everything is setup correctly. + +That's it. getssl example.net will now validate with DNS. + +There are additional options, which are documented in `dns_route53 -h` diff --git a/dns_scripts/dns_add_inwx.py b/dns_scripts/dns_add_inwx.py new file mode 100755 index 00000000..cfa2aaee --- /dev/null +++ b/dns_scripts/dns_add_inwx.py @@ -0,0 +1,93 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Script to Set TXT Record at INWX using the API +This script requires the pip packages INWX.Domrobot and tldextract +This script is using enviroment variables to get inwx credentials +""" + +import sys +import os +import argparse +from INWX.Domrobot import ApiClient +import tldextract + +# Create Parser-Objekt +parser = argparse.ArgumentParser( + description='Using the INWX API to change DNS TXT Records for the ACME DNS-01 Challange', + epilog= "The environment variables 'INWX_USERNAME' and 'INWX_PASSWORD' are required too") + +# Adding Args +parser.add_argument('fulldomain', type=str, help='The full domain to add TXT Record.') +parser.add_argument('token', type=str, help='The ACME DNS-01 token.') +parser.add_argument('--debug', action='store_true', help='Enable debug mode.') + +# Parsing Args +args = parser.parse_args() +INWX_FULLDOMAIN = args.fulldomain +ACME_TOKEN = args.token +DEBUG = args.debug + +# Parsing ENV +INWX_USERNAME = os.getenv('INWX_USERNAME', '') +INWX_PASSWORD = os.getenv('INWX_PASSWORD', '') + +# Splitting Domain +domain = tldextract.extract(INWX_FULLDOMAIN) +INWX_SUBDOMAIN = domain.subdomain +INWX_DOMAIN = f"{domain.domain}.{domain.suffix}" + +# Check if either environment variable is empty and handle the error +if not INWX_USERNAME or not INWX_PASSWORD: + print("Error: The following environment variables are required and cannot be empty:") + if not INWX_USERNAME: + print(" - INWX_USERNAME: Your INWX account username.") + if not INWX_PASSWORD: + print(" - INWX_PASSWORD: Your INWX account password.") + sys.exit(1) + +if DEBUG: + print(f'FQDN: {INWX_FULLDOMAIN}') + print(f'Domain: {INWX_DOMAIN}') + print(f'Subdomain: {INWX_SUBDOMAIN}') + print(f'Token: {ACME_TOKEN}') + print(f'User: {INWX_USERNAME}') + print(f'Password: {INWX_PASSWORD}') + +# By default the ApiClient uses the test api (OT&E). +# If you want to use the production/live api we have a +# constant named API_LIVE_URL in the ApiClient class. +# Just set api_url=ApiClient.API_LIVE_URL and you're good. +# api_client = ApiClient(api_url=ApiClient.API_OTE_URL, debug_mode=DEBUG) +api_client = ApiClient(api_url=ApiClient.API_LIVE_URL, debug_mode=DEBUG) + +# If you have 2fa enabled, take a look at the documentation of the ApiClient#login method +# to get further information about the login, especially the shared_secret parameter. +login_result = api_client.login(INWX_USERNAME, INWX_PASSWORD) + +# login was successful +if login_result['code'] == 1000: + + # Make an api call and save the result in a variable. + # We want to create a new nameserver record, so we call the api method nameserver.createRecord. + # See https://www.inwx.de/en/help/apidoc/f/ch02s15.html#nameserver.createRecord for parameters + # ApiClient#call_api returns the api response as a dict. + if INWX_SUBDOMAIN == '': + domain_entry_result = api_client.call_api(api_method='nameserver.createRecord', method_params={'domain': INWX_DOMAIN, 'name': '_acme-challenge', 'type': 'TXT', 'content': ACME_TOKEN}) # pylint: disable=C0301 + else: + domain_entry_result = api_client.call_api(api_method='nameserver.createRecord', method_params={'domain': INWX_DOMAIN, 'name': f'_acme-challenge.{INWX_SUBDOMAIN}', 'type': 'TXT', 'content': ACME_TOKEN}) # pylint: disable=C0301 + + # With or without successful check, we perform a logout. + api_client.logout() + + # validating return code + if domain_entry_result['code'] == 2302: + sys.exit(f"{domain_entry_result['msg']}.\nTry nameserver.updateRecord or nameserver.deleteRecord instead") # pylint: disable=C0301 + elif domain_entry_result['code'] == 1000: + if DEBUG: + print(domain_entry_result['msg']) + sys.exit() + else: + sys.exit(domain_entry_result) +else: + sys.exit('Api login error. Code: ' + str(login_result['code']) + ' Message: ' + login_result['msg']) # pylint: disable=C0301 diff --git a/dns_scripts/dns_add_route53 b/dns_scripts/dns_add_route53 new file mode 100755 index 00000000..88e3f0b5 --- /dev/null +++ b/dns_scripts/dns_add_route53 @@ -0,0 +1,18 @@ +#!/bin/bash + +# Add token to Route53 dns using dns_route53 bash version + +fulldomain="$1" +token="$2" + +[ -z "$ROUTE53_SCRIPT" ] && ROUTE53_SCRIPT="/usr/share/getssl/dns_scripts/dns_route53" +[[ "$ROUTE53_SCRIPT" =~ ^~ ]] && \ + eval 'ROUTE53_SCRIPT=`readlink -nf ' $ROUTE53_SCRIPT '`' + +if [ ! -x "$ROUTE53_SCRIPT" ]; then + echo "$ROUTE53_SCRIPT: not found. Please install, softlink or set ROUTE53_SCRIPT to its full path" + echo "See ROUTE53-README.txt for complete instructions." + exit 3 +fi + +$ROUTE53_SCRIPT -q add "${fulldomain}." "${token}" diff --git a/dns_scripts/dns_del_inwx.py b/dns_scripts/dns_del_inwx.py new file mode 100755 index 00000000..b7d57ff8 --- /dev/null +++ b/dns_scripts/dns_del_inwx.py @@ -0,0 +1,106 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- +""" +Script to remove TXT Record at INWX using the API +This script requires the pip packages INWX.Domrobot and tldextract +This script is using enviroment variables to get inwx credentials +""" + +import sys +import os +import argparse +from INWX.Domrobot import ApiClient +import tldextract + +# Create Parser-Objekt +parser = argparse.ArgumentParser( + description='Using the INWX API to remove DNS TXT Records for the ACME DNS-01 Challange', + epilog= "The environment variables 'INWX_USERNAME' and 'INWX_PASSWORD' are required too") + +# Adding Args +parser.add_argument('fulldomain', type=str, help='The full domain to add TXT Record.') +parser.add_argument('token', type=str, help='The ACME DNS-01 token.') +parser.add_argument('--debug', action='store_true', help='Enable debug mode.') + +# Parsing Args +args = parser.parse_args() +INWX_FULLDOMAIN = args.fulldomain +ACME_TOKEN = args.token +DEBUG = args.debug + +# Parsing ENV +INWX_USERNAME = os.getenv('INWX_USERNAME', '') +INWX_PASSWORD = os.getenv('INWX_PASSWORD', '') + +# Splitting Domain +domain = tldextract.extract(INWX_FULLDOMAIN) +INWX_SUBDOMAIN = domain.subdomain +INWX_DOMAIN = f"{domain.domain}.{domain.suffix}" + +# Check if either environment variable is empty and handle the error +if not INWX_USERNAME or not INWX_PASSWORD: + print("Error: The following environment variables are required and cannot be empty:") + if not INWX_USERNAME: + print(" - INWX_USERNAME: Your INWX account username.") + if not INWX_PASSWORD: + print(" - INWX_PASSWORD: Your INWX account password.") + sys.exit(1) + +if DEBUG: + print(f'FQDN: {INWX_FULLDOMAIN}') + print(f'Domain: {INWX_DOMAIN}') + print(f'Subdomain: {INWX_SUBDOMAIN}') + print(f'Token: {ACME_TOKEN}') + print(f'User: {INWX_USERNAME}') + print(f'Password: {INWX_PASSWORD}') + +# By default the ApiClient uses the test api (OT&E). +# If you want to use the production/live api we have a +# constant named API_LIVE_URL in the ApiClient class. +# Just set api_url=ApiClient.API_LIVE_URL and you're good. +# api_client = ApiClient(api_url=ApiClient.API_OTE_URL, debug_mode=DEBUG) +api_client = ApiClient(api_url=ApiClient.API_LIVE_URL, debug_mode=DEBUG) + +# If you have 2fa enabled, take a look at the documentation of the ApiClient#login method +# to get further information about the login, especially the shared_secret parameter. +login_result = api_client.login(INWX_USERNAME, INWX_PASSWORD) + +# login was successful +if login_result['code'] == 1000: + + # Make an api call and save the result in a variable. + # We want to get a the id of the _acme-challange TXT record, + # so we call the api method nameserver.info. + # See https://www.inwx.de/en/help/apidoc/f/ch02s15.html#nameserver.info for parameters + # ApiClient#call_api returns the api response as a dict. + if INWX_SUBDOMAIN == '': + domain_info_result = api_client.call_api(api_method='nameserver.info', method_params={'domain': INWX_DOMAIN, 'name': '_acme-challenge', 'type': 'TXT', 'content': ACME_TOKEN}) # pylint: disable=C0301 + else: + domain_info_result = api_client.call_api(api_method='nameserver.info', method_params={'domain': INWX_DOMAIN, 'name': f'_acme-challenge.{INWX_SUBDOMAIN}', 'type': 'TXT', 'content': ACME_TOKEN}) # pylint: disable=C0301 + if 'record' not in domain_info_result['resData']: + api_client.logout() + sys.exit(f'No DNS TXT Entry found for _acme-challenge.{INWX_FULLDOMAIN}.') + else: + for row in domain_info_result['resData']['record']: + domain_delete_result = api_client.call_api(api_method='nameserver.deleteRecord', method_params={'id': row['id']}) # pylint: disable=C0301 + if domain_delete_result['code'] == 1000: + if DEBUG: + print(domain_delete_result['msg']) + else: + api_client.logout() + sys.exit(domain_delete_result) + + # With or without successful check, we perform a logout. + api_client.logout() + + # validating return code + if domain_info_result['code'] == 2302: + sys.exit(f"{domain_info_result['msg']}.\nTry nameserver.updateRecord or nameserver.deleteRecord instead") # pylint: disable=C0301 + elif domain_info_result['code'] == 1000: + if DEBUG: + print(domain_info_result['msg']) + sys.exit() + else: + sys.exit(domain_info_result) +else: + sys.exit('Api login error. Code: ' + str(login_result['code']) + ' Message: ' + login_result['msg']) # pylint: disable=C0301 diff --git a/dns_scripts/dns_del_route53 b/dns_scripts/dns_del_route53 new file mode 100755 index 00000000..79a05700 --- /dev/null +++ b/dns_scripts/dns_del_route53 @@ -0,0 +1,18 @@ +#!/bin/bash + +# Delete token from Route53 dns using dns_route53 bash version + +fulldomain="$1" +token="$2" + +[ -z "$ROUTE53_SCRIPT" ] && ROUTE53_SCRIPT="/usr/share/getssl/dns_scripts/dns_route53" +[[ "$ROUTE53_SCRIPT" =~ ^~ ]] && \ + eval 'ROUTE53_SCRIPT=`readlink -nf ' $ROUTE53_SCRIPT '`' + +if [ ! -x "$ROUTE53_SCRIPT" ]; then + echo "$ROUTE53_SCRIPT: not found. Please install, softlink or set ROUTE53_SCRIPT to its full path" + echo "See ROUTE53-README.txt for complete instructions." + exit 3 +fi + +$ROUTE53_SCRIPT -q del "${fulldomain}." "${token}" diff --git a/dns_scripts/dns_route53 b/dns_scripts/dns_route53 new file mode 100644 index 00000000..5f1aaab7 --- /dev/null +++ b/dns_scripts/dns_route53 @@ -0,0 +1,199 @@ +#!/usr/bin/env bash +VERSION="1.0" +PROG="$(basename "$0")" + +QUIET=n + +while getopts 'dhp:t:z:i:qv' opt; do + case $opt in + d) DEBUG="Y" ;; + p) AWS_CLI_PROFILE="$OPTARG" ;; + q) QUIET= ;; + v) echo "dns_route53 version $VERSION"; exit 0 ;; + z) ROUTE53_HOSTED_ZONE_NAME="$OPTARG" ;; + i) ROUTE53_HOSTED_ZONE_ID="$OPTARG" ;; + *) + cat <&2 + AWS_CLI_PROFILE=default +fi + +op="$1" +if ! [[ "$op" =~ ^(add|del)$ ]]; then + echo "Operation must be \"add\" or \"del\"" >&2 + exit 3 +fi +name="$2" +if [ -z "$name" ]; then + echo "'name' parameter is required, see -h" >&2 + exit 3 +fi +data="$3" +if [ -z "$data" ]; then + echo "'data' parameter is required, see -h" >&2 + exit 3 +fi + +if [ "$op" = 'del' ]; then + ttl=120 +elif [ -z "$5" ]; then + ttl="120" +elif ! [[ "$5" =~ ^[0-9]+$ ]]; then + echo "TTL $5 is not numeric" >&2 + exit 3 +elif [ "$5" -lt 120 ]; then + [ -n "$VERB" ] && \ + echo "$5 is too small. Using TTL of 120 instead" >&2 + ttl="120" +else + ttl="$5" +fi + +# end processing parameters + +[ -n "$DEBUG" ] && \ + echo "$PROG: $op $name \"$data\" $ttl" >&2 + +# Determine what actual hosted zone to use. + +HOSTED_ZONE_NAME=$ROUTE53_HOSTED_ZONE_NAME +HOSTED_ZONE_ID=$ROUTE53_HOSTED_ZONE_ID +RR_NAME="_acme-challenge.${name}" +RR_VALUE="${data}" + +# Function to parse through the segments in the supplied name +# to determine the zone and its id +function determine_hosted_zone_name_and_id() { + TMP_NAME=$name + TMP_RR_NAME= + while [[ "$TMP_NAME" =~ ^([^.]+)\.([^.]+.*) ]]; do + if [ -n "${TMP_RR_NAME}" ]; then + TMP_RR_NAME="${TMP_RR_NAME}."; + fi + TMP_RR_NAME="${TMP_RR_NAME}${BASH_REMATCH[1]}" + testdomain="${BASH_REMATCH[2]}" + [ -n "$DEBUG" ] && echo "Testing hosted zone ${testdomain}" + TMP_NAME=$testdomain + if [[ ! "$TMP_NAME" =~ [^.]+\.[^.]+ ]]; then + [ -n "$DEBUG" ] && echo "No segments left" + exit 1 + fi + + TMP_ZONE_ID=$(aws --profile=${AWS_CLI_PROFILE} route53 list-hosted-zones --query "HostedZones[?Name=='${testdomain}'].Id | [0]" | sed -e 's/^"//' -e 's/"$//') + + + if [ "${TMP_ZONE_ID}" != "null" ]; then + [ -n "$DEBUG" ] && echo "Found hosted zone ${testdomain}" + HOSTED_ZONE_NAME=${testdomain} + HOSTED_ZONE_ID=$TMP_ZONE_ID + break + fi + done +} + +# If zone ID is specified, then use it to determine the hosted zone name +if [ -n "${HOSTED_ZONE_ID}" ]; then + HOSTED_ZONE_NAME=$(aws --profile=${AWS_CLI_PROFILE} route53 list-hosted-zones --query "HostedZones[?Id=='${ZONE_ID}'].Name | [0]" | sed -e 's/^"//' -e 's/"$//') +# If zone name is specified, then use it to get the zone id +elif [ -n "${HOSTED_ZONE_NAME}" ]; then + HOSTED_ZONE_ID=$(aws --profile=${AWS_CLI_PROFILE} route53 list-hosted-zones --query "HostedZones[?Name=='${HOSTED_ZONE_NAME}'].Id | [0]" | sed -e 's/^"//' -e 's/"$//') +else + determine_hosted_zone_name_and_id +fi + + + if [ -z "${HOSTED_ZONE_ID}" ]; then + echo "Hosted zone id not specified or determined" >&2 + exit 3 +fi + +if [ "$op" = "add" ]; then + ACTION="UPSERT" +elif [ "$op" = "del" ]; then + ACTION="DELETE" +else + echo "Unsupported Operation: $op" >&2 +fi + +CHANGE_BATCH=' +{ + "Comment": "GetSSL LetsEncrypt DNS Challenge", + "Changes": [{ + "Action" : "'"$ACTION"'", + "ResourceRecordSet" : { + "Name" : "'"$RR_NAME"'", + "Type" : "TXT", + "TTL" : '${ttl}', + "ResourceRecords" : [{ + "Value" : "\"'$RR_VALUE'\"" + }] + } + }] + } +' + + +[ -n "$DEBUG" ] && echo "${CHANGE_BATCH}" >&2 + +aws \ + --profile=${AWS_CLI_PROFILE} \ + route53 \ + change-resource-record-sets \ + --hosted-zone-id=${HOSTED_ZONE_ID} \ + --change-batch "${CHANGE_BATCH}" +exit $? \ No newline at end of file diff --git a/docker-compose.yml b/docker-compose.yml index f2b14892..a201b17f 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,9 +1,8 @@ -version: '3' services: pebble: - image: letsencrypt/pebble:latest + image: ghcr.io/letsencrypt/pebble:latest # TODO enable -strict - command: pebble -config /test/config/pebble-config.json -dnsserver 10.30.50.3:53 + command: -dnsserver 10.30.50.3:53 environment: # with Go 1.13.x which defaults TLS 1.3 to on GODEBUG: "tls13=1" @@ -15,8 +14,8 @@ services: acmenet: ipv4_address: 10.30.50.2 challtestsrv: - image: letsencrypt/pebble-challtestsrv:latest - command: pebble-challtestsrv -defaultIPv6 "" -defaultIPv4 10.30.50.3 -dns01 ":53" + image: ghcr.io/letsencrypt/pebble-challtestsrv:latest + command: -defaultIPv6 "" -defaultIPv4 10.30.50.3 -dns01 ":53" ports: - 8055:8055 # HTTP Management API networks: diff --git a/getssl b/getssl index 98e664fa..a7837dbe 100755 --- a/getssl +++ b/getssl @@ -548,7 +548,7 @@ check_challenge_completion() { # checks with the ACME server if our challenge is # if ACME response is pending (they haven't completed checks yet) # or valid (completed checks but not created certificate) then wait and try again. - if [[ "$status" == "pending" ]] || [[ "$status" == "valid" ]]; then + if [[ "$status" == "pending" ]] || [[ "$status" == "valid" ]] || [[ "$status" == "processing" ]]; then info "Pending" else err_detail=$(echo "$response" | grep "detail") diff --git a/test/34-ftp-passive.bats b/test/34-ftp-passive.bats index 3eae3874..3c685dc4 100644 --- a/test/34-ftp-passive.bats +++ b/test/34-ftp-passive.bats @@ -212,7 +212,16 @@ EOF # assert_line --partial "SSL connection using TLSv1.3" assert_line --partial "200 PROT now Private" - check_output_for_errors + # 22-May-2024 tweak assert_success on ubuntu16 as ftp output contains the + # message "error fetching CN from cert:The requested data were not available." + if [[ $GETSSL_OS == ubuntu16 ]]; then + refute_output --regexp '[Ff][Aa][Ii][Ll][Ee][Dd]' + refute_output --regexp '[^_][Ee][Rr][Rr][Oo][Rr][^:badNonce|^ fetching CN from cert]' + refute_output --regexp '[^_][Ww][Aa][Rr][Nn][Ii][Nn][Gg]' + refute_line --partial 'command not found' + else + check_output_for_errors + fi } @@ -275,5 +284,14 @@ EOF create_certificate assert_success assert_line --partial "200 PROT now Private" - check_output_for_errors + # 22-May-2024 skip assert_success on ubuntu16 as ftp output contains the + # message "error fetching CN from cert:The requested data were not available." + if [[ $GETSSL_OS == ubuntu16 ]]; then + refute_output --regexp '[Ff][Aa][Ii][Ll][Ee][Dd]' + refute_output --regexp '[^_][Ee][Rr][Rr][Oo][Rr][^:badNonce|^ fetching CN from cert]' + refute_output --regexp '[^_][Ww][Aa][Rr][Nn][Ii][Nn][Gg]' + refute_line --partial 'command not found' + else + check_output_for_errors + fi } diff --git a/test/34-ftp-ports.bats b/test/34-ftp-ports.bats index f377f232..d679f8cb 100644 --- a/test/34-ftp-ports.bats +++ b/test/34-ftp-ports.bats @@ -98,7 +98,16 @@ EOF # assert_line --partial "SSL connection using TLSv1.3" assert_line --partial "200 PROT now Private" - check_output_for_errors + # 22-May-2024 skip assert_success on ubuntu16 as ftp output contains the + # message "error fetching CN from cert:The requested data were not available." + if [[ $GETSSL_OS == ubuntu16 ]]; then + refute_output --regexp '[Ff][Aa][Ii][Ll][Ee][Dd]' + refute_output --regexp '[^_][Ee][Rr][Rr][Oo][Rr][^:badNonce|^ fetching CN from cert]' + refute_output --regexp '[^_][Ww][Aa][Rr][Nn][Ii][Nn][Gg]' + refute_line --partial 'command not found' + else + check_output_for_errors + fi } @@ -163,5 +172,14 @@ EOF create_certificate assert_success assert_line --partial "200 PROT now Private" - check_output_for_errors + # 22-May-2024 skip assert_success on ubuntu16 as ftp output contains the + # message "error fetching CN from cert:The requested data were not available." + if [[ $GETSSL_OS == ubuntu16 ]]; then + refute_output --regexp '[Ff][Aa][Ii][Ll][Ee][Dd]' + refute_output --regexp '[^_][Ee][Rr][Rr][Oo][Rr][^:badNonce|^ fetching CN from cert]' + refute_output --regexp '[^_][Ww][Aa][Rr][Nn][Ii][Nn][Gg]' + refute_line --partial 'command not found' + else + check_output_for_errors + fi } diff --git a/test/35-preferred-chain.bats b/test/35-preferred-chain.bats index ad94cafb..267c53cf 100644 --- a/test/35-preferred-chain.bats +++ b/test/35-preferred-chain.bats @@ -53,8 +53,8 @@ EOF @test "Use PREFERRED_CHAIN to select the default root" { if [ -n "$STAGING" ]; then - PREFERRED_CHAIN="\(STAGING\) Doctored Durian Root CA X3" - CHECK_CHAIN="(STAGING) Doctored Durian Root CA X3" + PREFERRED_CHAIN="\(STAGING\) Pretend Pear X1" + CHECK_CHAIN="(STAGING) Pretend Pear X1" else PREFERRED_CHAIN=$(curl --silent https://pebble:15000/roots/0 | openssl x509 -text -noout | grep Issuer: | awk -F"CN *= *" '{ print $2 }') PREFERRED_CHAIN="${PREFERRED_CHAIN# }" # remove leading whitespace diff --git a/test/Dockerfile-centos7 b/test/Dockerfile-centos7 index 127e2d42..5bb32297 100644 --- a/test/Dockerfile-centos7 +++ b/test/Dockerfile-centos7 @@ -1,5 +1,10 @@ FROM centos:centos7 +# Centos 7 is EOL and is no longer available from the usual mirrors, so switch to https://vault.centos.org +RUN sed -i 's/enabled=1/enabled=0/g' /etc/yum/pluginconf.d/fastestmirror.conf && \ + sed -i 's/^mirrorlist/#mirrorlist/g' /etc/yum.repos.d/*.repo && \ + sed -i 's;^#baseurl=http://mirror;baseurl=https://vault;g' /etc/yum.repos.d/*.repo + # Update and install required software RUN yum -y update RUN yum -y install epel-release diff --git a/test/README-Testing.md b/test/README-Testing.md index 10786bc9..5dd18640 100644 --- a/test/README-Testing.md +++ b/test/README-Testing.md @@ -6,7 +6,7 @@ For continuous integration testing we have the following: `gitactions` script which runs whenever a PR is pushed: -1. Uses `docker-compose` to start `pebble` (letsencrypt test server) and `challtestsrv` (minimal dns client for pebble) +1. Uses `docker compose` to start `pebble` (letsencrypt test server) and `challtestsrv` (minimal dns client for pebble) 2. Then runs the `bats` test scripts (all the files with a ".bats" extension) for each OS (alpine, centos6, debian, ubuntu) 3. Runs the `bats` test script against the staging server (using ubuntu docker image and duckdns.org) @@ -30,20 +30,20 @@ For individual accounts, \ is your github account name. ## To run all the tests on a single OS -1. Start `pebble` and `challtestsrv` using ```docker-compose up -d --build``` -2. Run the test suite ```test/run-test.sh [\]``` +1. Start `pebble` and `challtestsrv` using ```docker compose up -d --build``` +2. Run the test suite ```test/run-test.sh []``` 3. eg. `test/run-test.sh ubuntu16` ## To run a single bats test on a single OS -1. Start `pebble` and `challtestsrv` using ```docker-compose up -d --build``` -2. ```test/run-test.sh \ bats \``` +1. Start `pebble` and `challtestsrv` using ```docker compose up -d --build``` +2. ```test/run-test.sh bats ``` 3. e.g. `test/run-test.sh ubuntu bats /getssl/test/1-simple-http01.bats` ## To debug a test -1. Start `pebble` and `challtestsrv` using ```docker-compose up -d --build``` -2. ```run-test.sh \ /getssl/test/debug-test.sh \``` +1. Start `pebble` and `challtestsrv` using ```docker compose up -d --build``` +2. ```run-test.sh /getssl/test/debug-test.sh ``` 3. e.g. `test/run-test.sh ubuntu /getssl/test/debug-test.sh -d /getssl/test/test-config/getssl-http01-cfg` ## TODO diff --git a/test/debug-test.sh b/test/debug-test.sh index 07e6cd6b..78ad17af 100755 --- a/test/debug-test.sh +++ b/test/debug-test.sh @@ -28,3 +28,4 @@ fi cp "${CONFIG_FILE}" "${INSTALL_DIR}/.getssl/${GETSSL_HOST}/getssl.cfg" # shellcheck disable=SC2086 "${CODE_DIR}/getssl" -U ${DEBUG} -f "$GETSSL_HOST" 3>&1 +#bash diff --git a/test/test_helper.bash b/test/test_helper.bash index 4a02872b..7b6932a5 100644 --- a/test/test_helper.bash +++ b/test/test_helper.bash @@ -16,9 +16,11 @@ check_github_quota() { need="$1" echo "# Checking github limits" while true ; do + # shellcheck disable=SC2086 limits="$(curl ${_NOMETER:---silent} --user-agent "srvrco/getssl/github-actions" -H 'Accept: application/vnd.github.v3+json' "$LIMIT_API")" - echo "# limits = $limits" + # save error code before calling echo errcode=$? + echo "# limits = $limits" if [[ $errcode -eq 60 ]]; then echo "curl needs updating, your version does not support SNI (multiple SSL domains on a single IP)" exit 1 @@ -41,7 +43,7 @@ check_github_quota() { echo "# sleeping $(( reset - now )) seconds for GitHub quota" sleep "$(( reset - now ))" now="$(date +%s)" - done + done done } diff --git a/test/u2-test-get_auth_dns-drill.bats b/test/u2-test-get_auth_dns-drill.bats index 72fa6dc6..1baaa178 100644 --- a/test/u2-test-get_auth_dns-drill.bats +++ b/test/u2-test-get_auth_dns-drill.bats @@ -136,6 +136,7 @@ teardown() { CHECK_PUBLIC_DNS_SERVER=false CHECK_ALL_AUTH_DNS=false + echo "# Checking we can find the primary_ns server" run get_auth_dns www.duckdns.org # Assert that we've found the primary_ns server @@ -146,11 +147,13 @@ teardown() { assert_line --regexp 'Using drill.* NS' # Check all Authoritive DNS servers are returned if requested + echo "# Checking all authoritive DNS servers are returned if requested" CHECK_ALL_AUTH_DNS=true run get_auth_dns www.duckdns.org assert_output --regexp 'set primary_ns = ns.*\.awsdns.*\.net' # Check that we also check the public DNS server if requested + echo "# Checking we use the public DNS server if requested" CHECK_PUBLIC_DNS_SERVER=true run get_auth_dns www.duckdns.org assert_output --regexp 'set primary_ns = ns.*\.awsdns.*\.net 1\.0\.0\.1' diff --git a/test/u8-test-get_auth_dns-cname-nslookup.bats b/test/u8-test-get_auth_dns-cname-nslookup.bats index 9a25503e..6751bb9d 100644 --- a/test/u8-test-get_auth_dns-cname-nslookup.bats +++ b/test/u8-test-get_auth_dns-cname-nslookup.bats @@ -22,9 +22,9 @@ setup() { NSLOOKUP_VERSION=$(echo "" | nslookup -version 2>/dev/null | awk -F"[ -]" '{ print $2 }') # Version 9.11.3 on Ubuntu -debug doesn't work inside docker in my test env, version 9.16.1 does if [[ "${NSLOOKUP_VERSION}" != "Invalid" ]] && check_version "${NSLOOKUP_VERSION}" "9.11.4" ; then - DNS_CHECK_OPTIONS="$DNS_CHECK_OPTIONS -debug" + DNS_CHECK_OPTIONS="$DNS_CHECK_OPTIONS -debug" else - skip "This version of nslookup either doesn't support -debug or it doesn't work in local docker" + skip "This version of nslookup either doesn't support -debug or it doesn't work in local docker" fi }