Skip to content

Commit

Permalink
feat: dynamically configurable post deployment stages (#100)
Browse files Browse the repository at this point in the history
Co-authored-by: Moritz Zimmer <[email protected]>
  • Loading branch information
saefty and moritzzimmer committed Sep 28, 2023
1 parent 8fd2cf7 commit 1280a19
Show file tree
Hide file tree
Showing 13 changed files with 278 additions and 12 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/static-analysis.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
terraform: [ ~1.0 ]
terraform: [ ~1.3 ]
steps:
- uses: actions/checkout@v3

Expand All @@ -31,7 +31,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
terraform: [ ~1.0 ]
terraform: [ ~1.3 ]
steps:
- uses: actions/checkout@v3

Expand All @@ -45,7 +45,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
terraform: [ ~1.0 ]
terraform: [ ~1.3 ]
steps:
- uses: actions/checkout@v3

Expand All @@ -68,7 +68,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
terraform: [ ~1.0 ]
terraform: [ ~1.3 ]
steps:
- uses: actions/checkout@v3

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,7 @@ should migrate to this module as a drop-in replacement to benefit from new featu

| Name | Version |
|------|---------|
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.0 |
| <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.3 |
| <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.0 |

## Providers
Expand Down
10 changes: 10 additions & 0 deletions examples/deployment/complete/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ available features:
* custom [deployment configurations](https://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-configurations.html) to shift traffic to the new version and executes traffic [hooks](https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html#reference-appspec-file-structure-hooks-section-structure-ecs-sample-function)
* before and after traffic hooks
* rollback and CloudWatch alarms configuration
* additional custom CodePipeline step executed after the deployment

## usage

Expand Down Expand Up @@ -53,9 +54,16 @@ aws s3api put-object --bucket example-ci-{account_id}-{region} --key deployment-

| Name | Type |
|------|------|
| [aws_cloudwatch_log_group.custom_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
| [aws_cloudwatch_metric_alarm.error_rate](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_metric_alarm) | resource |
| [aws_codebuild_project.custom_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codebuild_project) | resource |
| [aws_codedeploy_deployment_config.canary](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/codedeploy_deployment_config) | resource |
| [aws_iam_policy.codepipeline_execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.custom_codepipeline_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.traffic_hook](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy_attachment.codepipeline_execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy_attachment) | resource |
| [aws_iam_role.custom_codepipeline_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.custom_codepipeline_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.traffic_hook](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_lambda_alias.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/lambda_alias) | resource |
| [aws_s3_bucket.source](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_bucket) | resource |
Expand All @@ -65,6 +73,8 @@ aws s3api put-object --bucket example-ci-{account_id}-{region} --key deployment-
| [aws_s3_object.initial](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/s3_object) | resource |
| [archive_file.traffic_hook](https://registry.terraform.io/providers/hashicorp/archive/latest/docs/data-sources/file) | data source |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_iam_policy_document.codepipeline_execution](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.custom_codepipeline_step](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.traffic_hook](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

Expand Down
120 changes: 120 additions & 0 deletions examples/deployment/complete/codepipeline_step.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
locals {
codebuild_name = "custom-pipeline-step"
}

resource "aws_cloudwatch_log_group" "custom_step" {
name = "/aws/codebuild/${local.codebuild_name}"
retention_in_days = 1
}

resource "aws_codebuild_project" "custom_step" {
name = local.codebuild_name
service_role = aws_iam_role.custom_codepipeline_step.arn

artifacts {
type = "CODEPIPELINE"
}

logs_config {
cloudwatch_logs {
group_name = aws_cloudwatch_log_group.custom_step.name
status = "ENABLED"
}

s3_logs {
status = "DISABLED"
}
}

environment {
compute_type = "BUILD_GENERAL1_SMALL"
image = "aws/codebuild/amazonlinux2-x86_64-standard:5.0"
type = "LINUX_CONTAINER"
}

source {
type = "CODEPIPELINE"
buildspec = templatefile("${path.module}/codepipeline_step/buildspec.yml", { script = file("${path.module}/codepipeline_step/step.py") })
}
}

data "aws_iam_policy_document" "custom_codepipeline_step" {
statement {
sid = "Logging"

actions = [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
]

resources = ["arn:aws:logs:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:log-group:/aws/codebuild/*"]
}

# Required for code build to access the code pipeline artifact bucket
# to be modified depending on input artifacts
statement {
sid = "S3BuildArtifactAccess"

actions = [
"s3:GetObject",
"s3:GetObjectVersion"
]

#tfsec:ignore:aws-iam-no-policy-wildcards
resources = ["${module.deployment.codepipeline_artifact_storage_arn}/deploy/*"]
}
}

resource "aws_iam_policy" "custom_codepipeline_step" {
name = "${local.codebuild_name}-${data.aws_region.current.name}"
policy = data.aws_iam_policy_document.custom_codepipeline_step.json
}

resource "aws_iam_role_policy_attachment" "custom_codepipeline_step" {
role = aws_iam_role.custom_codepipeline_step.name
policy_arn = aws_iam_policy.custom_codepipeline_step.arn
}

resource "aws_iam_role" "custom_codepipeline_step" {
name = "${local.codebuild_name}-${data.aws_region.current.name}"

assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Action = "sts:AssumeRole"
Effect = "Allow"
Sid = ""
Principal = {
Service = "codebuild.amazonaws.com"
}
},
]
})
}

# Required for code pipeline to execute custom code build steps
data "aws_iam_policy_document" "codepipeline_execution" {
statement {
sid = "CustomCodeBuild"

actions = [
"codebuild:BatchGetBuilds",
"codebuild:StartBuild",
]

resources = [aws_codebuild_project.custom_step.arn]
}
}

resource "aws_iam_policy" "codepipeline_execution" {
name = "allow-${local.codebuild_name}-${data.aws_region.current.name}"
policy = data.aws_iam_policy_document.codepipeline_execution.json
}

resource "aws_iam_policy_attachment" "codepipeline_execution" {
name = "allow-${local.codebuild_name}-${data.aws_region.current.name}"
policy_arn = aws_iam_policy.codepipeline_execution.arn
roles = [module.deployment.codepipeline_role_name]
}
13 changes: 13 additions & 0 deletions examples/deployment/complete/codepipeline_step/buildspec.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
version: 0.2
phases:
install:
on-failure: ABORT
runtime-versions:
python: 3.11
build:
commands:
- |
cat << BUILD > function.py
${indent(8, script)}
BUILD
- python function.py
3 changes: 3 additions & 0 deletions examples/deployment/complete/codepipeline_step/step.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
import os

print("Hello world from CodePipeline stage: " + os.environ.get("FOO", "default value"))
File renamed without changes.
35 changes: 32 additions & 3 deletions examples/deployment/complete/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,35 @@ module "deployment" {
function_name = local.function_name
s3_bucket = aws_s3_bucket.source.bucket
s3_key = local.s3_key

codepipeline_post_deployment_stages = [
{
name = "Custom"

actions = [
{
name = "CustomCodeBuildStep"
category = "Build"
owner = "AWS"
provider = "CodeBuild"
version = "1"
input_artifacts = ["deploy"]

configuration = {
ProjectName : aws_codebuild_project.custom_step.name

EnvironmentVariables = jsonencode([
{
name = "FOO"
value = "bar"
type = "PLAINTEXT"
}
])
}
}
]
}
]
}

resource "aws_codedeploy_deployment_config" "canary" {
Expand All @@ -113,7 +142,7 @@ resource "aws_codedeploy_deployment_config" "canary" {
type = "TimeBasedCanary"

time_based_canary {
interval = 5
interval = 2
percentage = 50
}
}
Expand All @@ -137,9 +166,9 @@ module "traffic_hook" {
}

data "archive_file" "traffic_hook" {
output_path = "${path.module}/function/traffic_hook.zip"
output_path = "${path.module}/hook/traffic_hook.zip"
output_file_mode = "0666"
source_file = "${path.module}/function/hook.py"
source_file = "${path.module}/hook/hook.py"
type = "zip"
}

Expand Down
37 changes: 37 additions & 0 deletions modules/deployment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ to update the function code and CodeDeploy to deploy the new function version.
- `BeforeAllowTraffic` and `AfterAllowTraffic` [hooks](https://docs.aws.amazon.com/codedeploy/latest/userguide/reference-appspec-file-structure-hooks.html#appspec-hooks-lambda) for CodeDeploy
- AWS predefined and custom [deployment configurations](https://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-configurations.html) for CodeDeploy
- automatic [rollbacks](https://docs.aws.amazon.com/codedeploy/latest/userguide/deployments-rollback-and-redeploy.html#deployments-rollback-and-redeploy-automatic-rollbacks) and support of [CloudWatch alarms](https://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-groups-configure-advanced-options.html) to stop deployments
- additional custom CodePipeline steps executed after the deployment

## How do I use this module?

Expand Down Expand Up @@ -314,6 +315,39 @@ resource "aws_codedeploy_deployment_config" "canary" {
}
```

### with custom CodePipeline steps

see [complete example](../../examples/deployment/complete) for details:

```terraform
// see above and make sure to add required IAM permissions
module "deployment" {
source = "moritzzimmer/lambda/aws//modules/deployment"
// see above
codepipeline_post_deployment_stages = [
{
name = "Custom"
actions = [
{
name = "CustomCodeBuildStep"
category = "Build"
owner = "AWS"
provider = "CodeBuild"
version = "1"
input_artifacts = ["deploy"]
configuration = {
ProjectName : aws_codebuild_project.custom_step.name
}
}
]
}
]
}
```
### Examples

- [complete](../../examples/deployment/complete)
Expand Down Expand Up @@ -387,6 +421,7 @@ No modules.
| <a name="input_codedeploy_deployment_group_auto_rollback_configuration_events"></a> [codedeploy\_deployment\_group\_auto\_rollback\_configuration\_events](#input\_codedeploy\_deployment\_group\_auto\_rollback\_configuration\_events) | The event type or types that trigger a rollback. Supported types are `DEPLOYMENT_FAILURE` and `DEPLOYMENT_STOP_ON_ALARM` | `list(string)` | `[]` | no |
| <a name="input_codepipeline_artifact_store_bucket"></a> [codepipeline\_artifact\_store\_bucket](#input\_codepipeline\_artifact\_store\_bucket) | Name of an existing S3 bucket used by AWS CodePipeline to store pipeline artifacts. Use the same bucket name as in `s3_bucket` to store deployment packages and pipeline artifacts in one bucket for `package_type=Zip` functions. If empty, a dedicated S3 bucket for your Lambda function will be created. | `string` | `""` | no |
| <a name="input_codepipeline_artifact_store_encryption_key_id"></a> [codepipeline\_artifact\_store\_encryption\_key\_id](#input\_codepipeline\_artifact\_store\_encryption\_key\_id) | The KMS key ARN or ID of a key block AWS CodePipeline uses to encrypt the data in the artifact store, such as an AWS Key Management Service (AWS KMS) key. If you don't specify a key, AWS CodePipeline uses the default key for Amazon Simple Storage Service (Amazon S3). | `string` | `""` | no |
| <a name="input_codepipeline_post_deployment_stages"></a> [codepipeline\_post\_deployment\_stages](#input\_codepipeline\_post\_deployment\_stages) | A map of post deployment stages to execute after the Lambda function has been deployed. The following stages are supported: `CodeBuild`, `CodeDeploy`, `CodePipeline`, `CodeStarNotifications`. | <pre>list(object({<br> name = string<br> actions = list(object({<br> name = string<br> category = string<br> owner = string<br> provider = string<br> version = string<br> input_artifacts = optional(list(any))<br> output_artifacts = optional(list(any))<br> configuration = optional(map(string))<br> }))<br> }))</pre> | `[]` | no |
| <a name="input_codepipeline_role_arn"></a> [codepipeline\_role\_arn](#input\_codepipeline\_role\_arn) | ARN of an existing IAM role for CodePipeline execution. If empty, a dedicated role for your Lambda function with minimal required permissions will be created. | `string` | `""` | no |
| <a name="input_codestar_notifications_detail_type"></a> [codestar\_notifications\_detail\_type](#input\_codestar\_notifications\_detail\_type) | The level of detail to include in the notifications for this resource. Possible values are BASIC and FULL. | `string` | `"BASIC"` | no |
| <a name="input_codestar_notifications_enabled"></a> [codestar\_notifications\_enabled](#input\_codestar\_notifications\_enabled) | Enable CodeStar notifications for your pipeline. | `bool` | `true` | no |
Expand All @@ -412,5 +447,7 @@ No modules.
| <a name="output_codedeploy_deployment_group_deployment_group_id"></a> [codedeploy\_deployment\_group\_deployment\_group\_id](#output\_codedeploy\_deployment\_group\_deployment\_group\_id) | The ID of the CodeDeploy deployment group. |
| <a name="output_codedeploy_deployment_group_id"></a> [codedeploy\_deployment\_group\_id](#output\_codedeploy\_deployment\_group\_id) | Application name and deployment group name. |
| <a name="output_codepipeline_arn"></a> [codepipeline\_arn](#output\_codepipeline\_arn) | The Amazon Resource Name (ARN) of the CodePipeline. |
| <a name="output_codepipeline_artifact_storage_arn"></a> [codepipeline\_artifact\_storage\_arn](#output\_codepipeline\_artifact\_storage\_arn) | The Amazon Resource Name (ARN) of the CodePipeline artifact store. |
| <a name="output_codepipeline_id"></a> [codepipeline\_id](#output\_codepipeline\_id) | The ID of the CodePipeline. |
| <a name="output_codepipeline_role_name"></a> [codepipeline\_role\_name](#output\_codepipeline\_role\_name) | The name of the IAM role used for the CodePipeline. |
<!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
24 changes: 24 additions & 0 deletions modules/deployment/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,30 @@ resource "aws_codepipeline" "this" {
}
}
}

# add arbitrary post deployment stages like, e.g. a manual approval stage
dynamic "stage" {
for_each = var.codepipeline_post_deployment_stages
content {
name = stage.value.name

dynamic "action" {
for_each = stage.value.actions
content {
name = action.value.name
category = action.value.category

owner = action.value.owner
provider = action.value.provider
version = action.value.version
input_artifacts = action.value.input_artifacts
output_artifacts = action.value.output_artifacts

configuration = action.value.configuration
}
}
}
}
}

resource "aws_s3_bucket" "pipeline" {
Expand Down
12 changes: 11 additions & 1 deletion modules/deployment/outputs.tf
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,17 @@ output "codepipeline_arn" {
value = aws_codepipeline.this.arn
}

output "codepipeline_artifact_storage_arn" {
description = "The Amazon Resource Name (ARN) of the CodePipeline artifact store."
value = "${local.artifact_store_bucket_arn}/${local.pipeline_name}"
}

output "codepipeline_id" {
description = "The ID of the CodePipeline."
value = aws_codepipeline.this.id
}
}

output "codepipeline_role_name" {
description = "The name of the IAM role used for the CodePipeline."
value = try(aws_iam_role.codepipeline_role[0].name, "")
}
Loading

0 comments on commit 1280a19

Please sign in to comment.