Skip to content

Latest commit

 

History

History
1011 lines (729 loc) · 44.5 KB

README-GitHub.md

File metadata and controls

1011 lines (729 loc) · 44.5 KB
Raw

Deploying a GitHub actions compatible environment

The objective of the instructions below is to configure the infrastructure that allows you to run CI/CD deployments using GitHub Actions for the Terraform Example Foundation stages (0-bootstrap, 1-org, 2-environments, 3-networks, 4-projects). The infrastructure consists in two Google Cloud Platform projects (prj-b-seed and prj-b-cicd-wif-gh).

It is a best practice to have two separate projects here (prj-b-seed and prj-b-cicd-wif-gh) for separation of concerns. On one hand, prj-b-seed stores terraform state and has the Service Accounts able to create / modify infrastructure. On the other hand, the authentication infrastructure using Workload identity federation is implemented in prj-b-cicd-wif-gh.

Requirements

To run the instructions described in this document, install the following:

For the manual steps described in this document, you need to use the same Terraform version used on the build pipeline. Otherwise, you might experience Terraform state snapshot lock errors.

Version 1.5.7 is the last version before the license model change. To use a later version of Terraform, ensure that the Terraform version used in the Operational System to manually execute part of the steps in 3-networks and 4-projects is the same version configured in the following code

  • 0-bootstrap/modules/jenkins-agent/variables.tf

    default     = "1.5.7"

  • 0-bootstrap/cb.tf

    terraform_version = "1.5.7"

  • scripts/validate-requirements.sh

    TF_VERSION="1.5.7"

  • build/github-tf-apply.yaml

    terraform_version: '1.5.7'

  • github-tf-pull-request.yaml

    terraform_version: "1.5.7"

  • 0-bootstrap/Dockerfile

    ARG TERRAFORM_VERSION=1.5.7

Also make sure that you have the following:

  • A GitHub account for your User or Organization.
  • A private GitHub repository for each one of the stages of Foundation:
    • Bootstrap
    • Organization
    • Environments
    • Networks
    • Projects
  • A Fine grained personal access token configured with:
    • Repository access to Only select repositories, including all the repositories in the previous item.
    • Permissions:
      • Actions: Read and Write
      • Metadata: Read-only
      • Secrets: Read and Write
      • Variables: Read and Write
      • Workflows: Read and Write
  • A Google Cloud organization.
  • A Google Cloud billing account.
  • Cloud Identity or Google Workspace groups for organization and billing admins.
  • For the user who will run the procedures in this document, grant the following roles:
    • The roles/resourcemanager.organizationAdmin role on the Google Cloud organization.
    • The roles/orgpolicy.policyAdmin role on the Google Cloud organization.
    • The roles/resourcemanager.projectCreator role on the Google Cloud organization.
    • The roles/billing.admin role on the billing account.
    • The roles/resourcemanager.folderCreator role.

Instructions

Due to issue First commit pushed triggers 2 workflow runs, The repositories created must be initialized with an initial commit, so that the initial push for the plan branch does not trigger two workflow runs. The instructions in the following steps will ensure the creation of an initial commit for each one of the repositories.

Deploying step 0-bootstrap

  1. Clone terraform-example-foundation into your local environment.

    git clone https://github.com/terraform-google-modules/terraform-example-foundation.git

  2. Clone the private repository you created to host the 0-bootstrap terraform configuration at the same level of the terraform-example-foundation folder. You must be authenticated to GitHub.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-BOOTSTRAP-REPO>.git gcp-bootstrap

  3. The layout should be:

    gcp-bootstrap/
terraform-example-foundation/

  4. Navigate into the repo. All subsequent steps assume you are running them from the gcp-bootstrap directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-bootstrap

  5. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

  6. change to a nonproduction branch.

    git checkout -b plan

  7. Copy contents of foundation to new repo (modify accordingly based on your current directory).

    mkdir -p envs/shared

cp -RT ../terraform-example-foundation/0-bootstrap/ ./envs/shared
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh
cd ./envs/shared

  8. In the versions file ./versions.tf un-comment the github required provider

  9. In the variables file ./variables.tf un-comment variables in the section Specific to github_bootstrap

  10. In the outputs file ./outputs.tf Comment-out outputs in the section Specific to cloudbuild_module

  11. In the outputs file ./outputs.tf un-comment outputs in the section Specific to github_bootstrap

  12. Rename file ./cb.tf to ./cb.tf.example

    mv ./cb.tf ./cb.tf.example

  13. Rename file ./github.tf.example to ./github.tf

    mv ./github.tf.example ./github.tf

  14. Rename file terraform.example.tfvars to terraform.tfvars

    mv ./terraform.example.tfvars ./terraform.tfvars

  15. Update the file terraform.tfvars with values from your Google Cloud environment

  16. Update the file terraform.tfvars with values from your GitHub repositories

  17. To prevent saving the gh_token in plain text in the terraform.tfvars file, export the GitHub fine grained access token as an environment variable:

    export TF_VAR_gh_token="YOUR-FINE-GRAINED-ACCESS-TOKEN"

  18. Use the helper script validate-requirements.sh to validate your environment:

    ../../../terraform-example-foundation/scripts/validate-requirements.sh  -o <ORGANIZATION_ID> -b <BILLING_ACCOUNT_ID> -u <END_USER_EMAIL> -e

    Note: The script is not able to validate if the user is in a Cloud Identity or Google Workspace group with the required roles.

  19. Run terraform init and terraform plan and review the output.

    terraform init
terraform plan -input=false -out bootstrap.tfplan

  20. To validate your policies, run gcloud beta terraform vet. For installation instructions, see Validate policies instructions for the Google Cloud CLI.

  21. Run the following commands and check for violations:

    export VET_PROJECT_ID=A-VALID-PROJECT-ID

terraform show -json bootstrap.tfplan > bootstrap.json
gcloud beta terraform vet bootstrap.json --policy-library="../../policy-library" --project ${VET_PROJECT_ID}

    A-VALID-PROJECT-ID must be an existing project you have access to. This is necessary because Terraform-validator needs to link resources to a valid Google Cloud Platform project.

  22. No violations and an output with done means the validation was successful.

  23. Run terraform apply.

    terraform apply bootstrap.tfplan

  24. Run terraform output to get the email address of the terraform service accounts that will be used to run manual steps for shared environments in steps 3-networks-dual-svpc, 3-networks-hub-and-spoke, and 4-projects.

    export network_step_sa=$(terraform output -raw networks_step_terraform_service_account_email)
export projects_step_sa=$(terraform output -raw projects_step_terraform_service_account_email)

echo "network step service account = ${network_step_sa}"
echo "projects step service account = ${projects_step_sa}"

  25. Run terraform output to get the ID of your CI/CD project:

    export cicd_project_id=$(terraform output -raw cicd_project_id)
echo "CI/CD Project ID = ${cicd_project_id}"

  26. Copy the backend and update backend.tf with the name of your Google Cloud Storage bucket for Terraform's state. Also update the backend.tf of all steps.

    export backend_bucket=$(terraform output -raw gcs_bucket_tfstate)
echo "backend_bucket = ${backend_bucket}"

cp backend.tf.example backend.tf
cd ../../../

for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_ME/${backend_bucket}/" $i; done
for i in `find . -name 'backend.tf'`; do sed -i'' -e "s/UPDATE_PROJECTS_BACKEND/${backend_bucket}/" $i; done

cd gcp-bootstrap/envs/shared

  27. Re-run terraform init. When you're prompted, agree to copy Terraform state to Cloud Storage.

    terraform init

  28. (Optional) Run terraform plan to verify that state is configured correctly. You should see no changes from the previous state.

  29. Save the Terraform configuration to gcp-bootstrap github repository:

    cd ../..
git add .
git commit -m 'Initialize bootstrap repo'
git push --set-upstream origin plan

  30. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-BOOTSTRAP-REPO/pull/new/plan from the plan branch to the production branch and review the output.

  31. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  32. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-BOOTSTRAP-REPO/actions under tf-pull-request.

  33. If the GitHub action is successful, merge the pull request in to the production branch.

  34. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  35. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-BOOTSTRAP-REPO/actions under tf-apply.

  36. Before moving to the next step, go back to the parent directory.

    cd ..

Note 1: The stages after 0-bootstrap use terraform_remote_state data source to read common configuration like the organization ID from the output of the 0-bootstrap stage. They will fail if the state is not copied to the Cloud Storage bucket.

Note 2: After the deploy, to prevent the project quota error described in the Troubleshooting guide, we recommend that you request 50 additional projects for the projects step service account created in this step.

Deploying step 1-org

  1. Clone the repository you created to host the 1-org terraform configuration at the same level of the terraform-example-foundation folder.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-ORGANIZATION-REPO>.git gcp-org

  2. Navigate into the repo. All subsequent steps assume you are running them from the gcp-org directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-org

  3. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

  4. change to a nonproduction branch.

    git checkout -b plan

  5. Copy contents of foundation to new repo.

    cp -RT ../terraform-example-foundation/1-org/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh

  6. Rename ./envs/shared/terraform.example.tfvars to ./envs/shared/terraform.tfvars

    mv ./envs/shared/terraform.example.tfvars ./envs/shared/terraform.tfvars

  7. Update the file envs/shared/terraform.tfvars with values from your GCP environment. See the shared folder README.md for additional information on the values in the terraform.tfvars file.

  8. Un-comment the variable create_access_context_manager_access_policy = false if your organization already has an Access Context Manager Policy.

    export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -json common_config | jq '.org_id' --raw-output)

export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)")

echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}"

if [ ! -z "${ACCESS_CONTEXT_MANAGER_ID}" ]; then sed -i'' -e "s=//create_access_context_manager_access_policy=create_access_context_manager_access_policy=" ./envs/shared/terraform.tfvars; fi

  9. Update the remote_state_bucket variable with the backend bucket from step Bootstrap.

    export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate)

echo "remote_state_bucket = ${backend_bucket}"

sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./envs/shared/terraform.tfvars

  10. Check if a Security Command Center Notification with the default name, scc-notify, already exists in your organization.

    export ORG_STEP_SA=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw organization_step_terraform_service_account_email)

gcloud scc notifications describe "scc-notify" --format="value(name)" --organization=${ORGANIZATION_ID} --impersonate-service-account=${ORG_STEP_SA}

  11. If the notification exists the output will be:

    organizations/ORGANIZATION_ID/notificationConfigs/scc-notify

  12. If the notification does not exist the output will be:

    ERROR: (gcloud.scc.notifications.describe) NOT_FOUND: Requested entity was not found.

  13. If the notification exists, choose a different value for the scc_notification_name variable in the ./envs/shared/terraform.tfvars file.

  14. Commit changes.

    git add .
git commit -m 'Initialize org repo'

  15. Push your plan branch.

    git push --set-upstream origin plan

  16. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-ORGANIZATION-REPO/pull/new/plan from the plan branch to the production branch and review the output.

  17. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  18. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ORGANIZATION-REPO/actions under tf-pull-request.

  19. If the GitHub action is successful, merge the pull request in to the production branch.

  20. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  21. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ORGANIZATION-REPO/actions under tf-apply.

  22. Before moving to the next step, go back to the parent directory.

    cd ..

Deploying step 2-environments

  1. Clone the repository you created to host the 2-environments terraform configuration at the same level of the terraform-example-foundation folder.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-ENVIRONMENTS-REPO>.git gcp-environments

  2. Navigate into the repo. All subsequent steps assume you are running them from the gcp-environments directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-environments

  3. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

git checkout -b nonproduction
git push --set-upstream origin nonproduction

git checkout -b development
git push --set-upstream origin development

  4. change to a nonproduction branch.

    git checkout -b plan

  5. Copy contents of foundation to new repo.

    cp -RT ../terraform-example-foundation/2-environments/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh

  6. Rename terraform.example.tfvars to terraform.tfvars.

    mv terraform.example.tfvars terraform.tfvars

  7. Update the file with values from your GCP environment. See any of the envs folder README.md files for additional information on the values in the terraform.tfvars file.

  8. Update the remote_state_bucket variable with the backend bucket from step Bootstrap.

    export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared" output -raw gcs_bucket_tfstate)
echo "remote_state_bucket = ${backend_bucket}"

sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" terraform.tfvars

  9. Commit changes.

    git add .
git commit -m 'Initialize environments repo'

  10. Push your plan branch.

    git push --set-upstream origin plan

  11. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/pull/new/plan from the plan branch to the development branch and review the output.

  12. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the development environment.

  13. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-pull-request.

  14. If the GitHub action is successful, merge the pull request in to the development branch.

  15. The merge will trigger a GitHub Action that will apply the terraform configuration for the development environment.

  16. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-apply.

  17. If the GitHub action is successful, apply the next environment.

  18. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/pull/new/development from the development branch to the nonproduction branch and review the output.

  19. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the nonproduction environment.

  20. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-pull-request.

  21. If the GitHub action is successful, merge the pull request in to the nonproduction branch.

  22. The merge will trigger a GitHub Action that will apply the terraform configuration for the nonproduction environment.

  23. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-apply.

  24. If the GitHub action is successful, apply the next environment.

  25. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/pull/new/nonproduction from the nonproduction branch to the production branch and review the output.

  26. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  27. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-pull-request.

  28. If the GitHub action is successful, merge the pull request in to the production branch.

  29. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  30. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-ENVIRONMENTS-REPO/actions under tf-apply.

  31. You can now move to the instructions in the network stage. To use the Dual Shared VPC network mode go to Deploying step 3-networks-dual-svpc, or go to Deploying step 3-networks-hub-and-spoke to use the Hub and Spoke network mode.

  32. Before moving to the next step, go back to the parent directory.

    cd ..

Deploying step 3-networks-dual-svpc

  1. Clone the repository you created to host the 3-networks-dual-svpc terraform configuration at the same level of the terraform-example-foundation folder.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-NETWORKS-REPO>.git gcp-networks

  2. Navigate into the repo. All subsequent steps assume you are running them from the gcp-networks directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-networks

  3. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

git checkout -b nonproduction
git push --set-upstream origin nonproduction

git checkout -b development
git push --set-upstream origin development

  4. change to a nonproduction branch.

    git checkout -b plan

  5. Copy contents of foundation to new repo.

    cp -RT ../terraform-example-foundation/3-networks-dual-svpc/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh

  6. Rename common.auto.example.tfvars to common.auto.tfvars, rename shared.auto.example.tfvars to shared.auto.tfvars and rename access_context.auto.example.tfvars to access_context.auto.tfvars.

    mv common.auto.example.tfvars common.auto.tfvars
mv shared.auto.example.tfvars shared.auto.tfvars
mv access_context.auto.example.tfvars access_context.auto.tfvars

  7. Update the file shared.auto.tfvars with the values for the target_name_server_addresses.

  8. Update the file access_context.auto.tfvars with the organization's access_context_manager_policy_id.

    export ORGANIZATION_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -json common_config | jq '.org_id' --raw-output)

export ACCESS_CONTEXT_MANAGER_ID=$(gcloud access-context-manager policies list --organization ${ORGANIZATION_ID} --format="value(name)")

echo "access_context_manager_policy_id = ${ACCESS_CONTEXT_MANAGER_ID}"

sed -i'' -e "s/ACCESS_CONTEXT_MANAGER_ID/${ACCESS_CONTEXT_MANAGER_ID}/" ./access_context.auto.tfvars

  9. Update common.auto.tfvars file with values from your GCP environment. See any of the envs folder README.md files for additional information on the values in the common.auto.tfvars file.

  10. You must add your user email in the variable perimeter_additional_members to be able to see the resources created in the restricted project.

  11. Update the remote_state_bucket variable with the backend bucket from step Bootstrap in the common.auto.tfvars file.

    export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate)

echo "remote_state_bucket = ${backend_bucket}"

sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars

  12. Commit changes

    git add .
git commit -m 'Initialize networks repo'

  13. You must manually plan and apply the shared environment (only once) since the development, nonproduction and production environments depend on it.

  14. Use terraform output to get the CI/CD project ID and the networks step Terraform Service Account from gcp-bootstrap output.

  15. The CI/CD project ID will be used in the validation of the Terraform configuration

    export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id)
echo ${CICD_PROJECT_ID}

  16. The networks step Terraform Service Account will be used for Service Account impersonation in the following steps. An environment variable GOOGLE_IMPERSONATE_SERVICE_ACCOUNT will be set with the Terraform Service Account to enable impersonation.

    export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw networks_step_terraform_service_account_email)
echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}

  17. Run init and plan and review output for environment shared.

    ./tf-wrapper.sh init shared
./tf-wrapper.sh plan shared

  18. To use the validate option of the tf-wrapper.sh script, please follow the instructions to install the terraform-tools component.

  19. Run validate and check for violations.

    ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}

  20. Run apply shared.

    ./tf-wrapper.sh apply shared

  21. Push your plan branch.

    git push --set-upstream origin plan

  22. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the plan branch to the development branch and review the output.

  23. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the development environment.

  24. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  25. If the GitHub action is successful, merge the pull request in to the development branch.

  26. The merge will trigger a GitHub Action that will apply the terraform configuration for the development environment.

  27. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  28. If the GitHub action is successful, apply the next environment.

  29. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/development from the development branch to the nonproduction branch and review the output.

  30. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the nonproduction environment.

  31. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  32. If the GitHub action is successful, merge the pull request in to the nonproduction branch.

  33. The merge will trigger a GitHub Action that will apply the terraform configuration for the nonproduction environment.

  34. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  35. If the GitHub action is successful, apply the next environment.

  36. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/nonproduction from the nonproduction branch to the production branch and review the output.

  37. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  38. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  39. If the GitHub action is successful, merge the pull request in to the production branch.

  40. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  41. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  42. Before executing the next steps, unset the GOOGLE_IMPERSONATE_SERVICE_ACCOUNT environment variable.

    unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT

  43. Before moving to the next step, go back to the parent directory.

    cd ..

  44. You can now move to the instructions in the 4-projects stage.

Deploying step 3-networks-hub-and-spoke

  1. Clone the repository you created to host the 3-networks-hub-and-spoke terraform configuration at the same level of the terraform-example-foundation folder.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-NETWORKS-REPO>.git gcp-networks

  2. Navigate into the repo. All subsequent steps assume you are running them from the gcp-networks directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-networks

  3. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

git checkout -b nonproduction
git push --set-upstream origin nonproduction

git checkout -b development
git push --set-upstream origin development

  4. change to a nonproduction branch.

    git checkout -b plan

  5. Copy contents of foundation to new repo.

    cp -RT ../terraform-example-foundation/3-networks-hub-and-spoke/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh

  6. Rename common.auto.example.tfvars to common.auto.tfvars, rename shared.auto.example.tfvars to shared.auto.tfvars and rename access_context.auto.example.tfvars to access_context.auto.tfvars.

    mv common.auto.example.tfvars common.auto.tfvars
mv shared.auto.example.tfvars shared.auto.tfvars
mv access_context.auto.example.tfvars access_context.auto.tfvars

  7. Update common.auto.tfvars file with values from your GCP environment. See any of the envs folder README.md files for additional information on the values in the common.auto.tfvars file.

  8. You must add your user email in the variable perimeter_additional_members to be able to see the resources created in the restricted project.

  9. Update the remote_state_bucket variable with the backend bucket from step Bootstrap in the common.auto.tfvars file.

    export backend_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate)

echo "remote_state_bucket = ${backend_bucket}"

sed -i'' -e "s/REMOTE_STATE_BUCKET/${backend_bucket}/" ./common.auto.tfvars

  10. Commit changes

    git add .
git commit -m 'Initialize networks repo'

  11. You must manually plan and apply the shared environment (only once) since the development, nonproduction and production environments depend on it.

  12. Use terraform output to get the CI/CD project ID and the networks step Terraform Service Account from gcp-bootstrap output.

  13. The CI/CD project ID will be used in the validation of the Terraform configuration

    export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id)
echo ${CICD_PROJECT_ID}

  14. The networks step Terraform Service Account will be used for Service Account impersonation in the following steps. An environment variable GOOGLE_IMPERSONATE_SERVICE_ACCOUNT will be set with the Terraform Service Account to enable impersonation.

    export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw networks_step_terraform_service_account_email)
echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}

  15. Run init and plan and review output for environment shared.

    ./tf-wrapper.sh init shared
./tf-wrapper.sh plan shared

  16. To use the validate option of the tf-wrapper.sh script, please follow the instructions to install the terraform-tools component.

  17. Run validate and check for violations.

    ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}

  18. Run apply shared.

    ./tf-wrapper.sh apply shared

  19. Push your plan branch.

    git push --set-upstream origin plan

  20. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/plan from the plan branch to the development branch and review the output.

  21. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the development environment.

  22. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  23. If the GitHub action is successful, merge the pull request in to the development branch.

  24. The merge will trigger a GitHub Action that will apply the terraform configuration for the development environment.

  25. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  26. If the GitHub action is successful, apply the next environment.

  27. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/development from the development branch to the nonproduction branch and review the output.

  28. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the nonproduction environment.

  29. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  30. If the GitHub action is successful, merge the pull request in to the nonproduction branch.

  31. The merge will trigger a GitHub Action that will apply the terraform configuration for the nonproduction environment.

  32. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  33. If the GitHub action is successful, apply the next environment.

  34. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/pull/new/nonproduction from the nonproduction branch to the production branch and review the output.

  35. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  36. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-pull-request.

  37. If the GitHub action is successful, merge the pull request in to the production branch.

  38. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  39. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-NETWORKS-REPO/actions under tf-apply.

  40. Before executing the next steps, unset the GOOGLE_IMPERSONATE_SERVICE_ACCOUNT environment variable.

    unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT

  41. Before moving to the next step, go back to the parent directory.

    cd ..

  42. You can now move to the instructions in the 4-projects stage.

Deploying step 4-projects

  1. Clone the repository you created to host the 4-projects terraform configuration at the same level of the terraform-example-foundation folder.

    git clone [email protected]:<GITHUB-OWNER>/<GITHUB-PROJECTS-REPO>.git gcp-projects

  2. Navigate into the repo. All subsequent steps assume you are running them from the gcp-projects directory. If you run them from another directory, adjust your copy paths accordingly.

    cd gcp-projects

  3. Seed the repository if it has not been initialized yet.

    git commit --allow-empty -m 'repository seed'
git push --set-upstream origin main

git checkout -b production
git push --set-upstream origin production

git checkout -b nonproduction
git push --set-upstream origin nonproduction

git checkout -b development
git push --set-upstream origin development

  4. change to a nonproduction branch.

    git checkout -b plan

  5. Copy contents of foundation to new repo.

    cp -RT ../terraform-example-foundation/4-projects/ .
cp -RT ../terraform-example-foundation/policy-library/ ./policy-library
mkdir -p .github/workflows
cp ../terraform-example-foundation/build/github-tf-* ./.github/workflows/
cp ../terraform-example-foundation/build/tf-wrapper.sh .
chmod 755 ./tf-wrapper.sh

  6. Rename auto.example.tfvars files to auto.tfvars.

    mv common.auto.example.tfvars common.auto.tfvars
mv shared.auto.example.tfvars shared.auto.tfvars
mv development.auto.example.tfvars development.auto.tfvars
mv nonproduction.auto.example.tfvars nonproduction.auto.tfvars
mv production.auto.example.tfvars production.auto.tfvars

  7. See any of the envs folder README.md files for additional information on the values in the common.auto.tfvars, development.auto.tfvars, nonproduction.auto.tfvars, and production.auto.tfvars files.

  8. See any of the shared folder README.md files for additional information on the values in the shared.auto.tfvars file.

  9. Use terraform output to get the backend bucket value from bootstrap output.

    export remote_state_bucket=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw gcs_bucket_tfstate)
echo "remote_state_bucket = ${remote_state_bucket}"

sed -i'' -e "s/REMOTE_STATE_BUCKET/${remote_state_bucket}/" ./common.auto.tfvars

  10. (Optional) If you want additional subfolders for separate business units or entities, make additional copies of the folder business_unit_1 and modify any values that vary across business unit like business_code, business_unit, or subnet_ip_range.

For example, to create a new business unit similar to business_unit_1, run the following:

#copy the business_unit_1 folder and it's contents to a new folder business_unit_2
cp -r  business_unit_1 business_unit_2

# search all files under the folder `business_unit_2` and replace strings for business_unit_1 with strings for business_unit_2
grep -rl bu1 business_unit_2/ | xargs sed -i 's/bu1/bu2/g'
grep -rl business_unit_1 business_unit_2/ | xargs sed -i 's/business_unit_1/business_unit_2/g'

  1. Commit changes.

    git add .
git commit -m 'Initialize projects repo'

  2. You need to manually plan and apply only once the business_unit_1/shared and business_unit_2/shared environments since development, nonproduction, and production depend on them.

  3. Use terraform output to get the CI/CD project ID and the projects step Terraform Service Account from gcp-bootstrap output.

  4. The CI/CD project ID will be used in the validation of the Terraform configuration

    export CICD_PROJECT_ID=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw cicd_project_id)
echo ${CICD_PROJECT_ID}

  5. The projects step Terraform Service Account will be used for Service Account impersonation in the following steps. An environment variable GOOGLE_IMPERSONATE_SERVICE_ACCOUNT will be set with the Terraform Service Account to enable impersonation.

    export GOOGLE_IMPERSONATE_SERVICE_ACCOUNT=$(terraform -chdir="../gcp-bootstrap/envs/shared/" output -raw projects_step_terraform_service_account_email)
echo ${GOOGLE_IMPERSONATE_SERVICE_ACCOUNT}

  6. Run init and plan and review output for environment shared.

    ./tf-wrapper.sh init shared
./tf-wrapper.sh plan shared

  7. To use the validate option of the tf-wrapper.sh script, please follow the instructions to install the terraform-tools component.

  8. Run validate and check for violations.

    ./tf-wrapper.sh validate shared $(pwd)/policy-library ${CICD_PROJECT_ID}

  9. Run apply shared.

    ./tf-wrapper.sh apply shared

  10. Push your plan branch.

    git push --set-upstream origin plan

  11. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/pull/new/plan from the plan branch to the development branch and review the output.

  12. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the development environment.

  13. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-pull-request.

  14. If the GitHub action is successful, merge the pull request in to the development branch.

  15. The merge will trigger a GitHub Action that will apply the terraform configuration for the development environment.

  16. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-apply.

  17. If the GitHub action is successful, apply the next environment.

  18. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/pull/new/development from the development branch to the nonproduction branch and review the output.

  19. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the nonproduction environment.

  20. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-pull-request.

  21. If the GitHub action is successful, merge the pull request in to the nonproduction branch.

  22. The merge will trigger a GitHub Action that will apply the terraform configuration for the nonproduction environment.

  23. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-apply.

  24. If the GitHub action is successful, apply the next environment.

  25. Open a pull request in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/pull/new/nonproduction from the nonproduction branch to the production branch and review the output.

  26. The Pull request will trigger a GitHub Action that will run Terraform init/plan/validate in the production environment.

  27. Review the GitHub Action output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-pull-request.

  28. If the GitHub action is successful, merge the pull request in to the production branch.

  29. The merge will trigger a GitHub Action that will apply the terraform configuration for the production environment.

  30. Review merge output in GitHub https://github.com/GITHUB-OWNER/GITHUB-PROJECTS-REPO/actions under tf-apply.

  31. Unset the GOOGLE_IMPERSONATE_SERVICE_ACCOUNT environment variable.

    unset GOOGLE_IMPERSONATE_SERVICE_ACCOUNT