Update module to use the new container insights with CW for EKS (#252)

* Adding Module and Example for ECS cluster monitoring with ecs_observer * Adding Module and Example for ECS cluster monitoring with ecs_observer * Incorporating PR comments * Restructuring Examples and modules folder for ECS, Added content in main Readme * Fixing path as per PR comments * Parameterzing the config files, incorporated PR review comments * Adding condition for AMP WS and fixing AMP endpoint * Adding Document for ECS Monitoring and parameterized some variables * Added sample dashboard * Adding Document for ECS Monitoring and parameterized some variables * Fixing failures detected by pre-commit * Fixing failures detected by pre-commit * Fixing failures detected by pre-commit * Pre-commit fixes * Fixing failures detected by pre-commit * Fixing failures detected by pre-commit * Pre-commit * Fixing HIGH security alerts detected by pre-commit * Fixing HIGH security alerts detected by pre-commit * Fixing HIGH security alerts detected by pre-commit, 31stOct * Add links after merge * 2ndNov - Added condiotnal creation for Grafana WS and module versions for AMG, AMP * Module to use the new container insights with CW for EKS * Module to use the new container insights with CW for EKS * Fixing precommit error * Updating EKS container insights to replace deamonset with tf resource * Updating EKS container insights to replace deamonset with tf resource * Updating EKS container insights to replace deamonset with tf resource * Updating EKS container insights to replace deamonset with tf resource * Updating EKS container insights- Adding doc and fixing pre-commit errors * Update Images in the doc * Update data.tf pre-commit error * Apply suggestions from code review * fixing broken hyperlink * EKS Container Insights - updating docs and few varibaleable names, definitions
aws-observability · Jan 19, 2024 · 1f91205 · 1f91205
1 parent 2ecdcd1
commit 1f91205
Show file tree

Hide file tree

Showing 14 changed files with 190 additions and 255 deletions.
diff --git a/docs/container-insights/eks.md b/docs/container-insights/eks.md
@@ -1,8 +1,9 @@
 # Setting Up Container Insights for your EKS Cluster
 
-This example deploys AWS Distro of OpenTelemetry on your EKS cluster as a Daemonset which will enable
-Container Insights metrics Dashboard on Amazon CloudWatch.
+This example deploys CloudWatch Observability EKS add-on on an exisiting Amazon EKS cluster, which enables Container Insights enhanced observability for Amazon EKS and CloudWatch Application Signals by default.
 
+1. Enables the CloudWatch Observability Add-on on EKS using the IAM service account role
+2. Creates an IAM Service Linked role for enabling Application Signals
 
 ## Prerequisites
 
@@ -34,6 +35,16 @@ Specify the EKS Cluster Name where the resources will be deployed:
 export TF_VAR_eks_cluster_id=xxx
 ```
 
+### 3. Disable creation of `Cloudwatch Application Signals Service-linked Role`
+If you already have Application Signals deployed in your AWS account, please set the value of this variable to `false`
+```
+variable "create_cloudwatch_application_signals_role" {
+  type        = bool
+  default     = true
+  description = "Create a Cloudwatch Application Signals service-linked role"
+}
+```
+
 ## Deploy
 
 Simply run this command to deploy the example
@@ -42,12 +53,40 @@ Simply run this command to deploy the example
 terraform apply
 ```
 
-## Visualization
+## Enabling Application Signals (preview) for your services
+CloudWatch Application Signals (preview) is currenlty supported for **Java** applications running on your Amazon EKS cluster.
+
+Next, you have to update your Application to `Configure application metrics and trace sampling`. For this, you must add an annotation to a manifest YAML in your cluster. Adding this annotation auto-instruments the application to send metrics, traces, and logs to Application Signals. You have two options for the annotation:
+
+1. **Annotate Workload** auto-instruments a single workload in the cluster.
+    - Paste the below line into the PodTemplate section of the workload manifest.
+    ```
+    annotations: instrumentation.opentelemetry.io/inject-java: "true"
+    ```
+    - In your terminal, enter `kubectl apply -f your_deployment_yaml` to apply the change.
+
+2. **Annotate Namespace** auto-instruments all workloads deployed in the selected namespace.
+    - Paste the below line into the metadata section of the namespace manifest.
+    ```
+    annotations: instrumentation.opentelemetry.io/inject-java: "true"
+    ```
+    - In your terminal, enter `kubectl apply -f your_namespace_yaml` to apply the change.
+    - In your terminal, enter a command to restart all pods in the namespace. An example command to restart deployment workloads is `kubectl rollout restart deployment -n namespace_name`
+
+## Visualization of Container Insights data
+
+After `terraform apply` is successful, open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Insights -> Container Insights`, there choose the `EKS` from the drop down and you will see the metrics shown on the dashboard:
+
+<img width="1345" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/31686b29-8ec2-46ff-a266-ebfa1de9768a">
+
+
+## Visualization of CloudWatch Application Signals (preview) data
 
-After apply, open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Insights -> Container Insights`, there choose the `Performance montoring` from the drop down, choose the `cluster name` and you will see the metrics shown on the dashboard:
+After enabling your Application to pass metrics and traces by following [the steps provided above](#enabling-application-signals-preview-for-your-services), open your Amazon CloudWatch console in the same region as your EKS cluster, then from the left hand side choose `Application Signals -> Services` and you will see the metrics shown on the sample dashboard below:
 
-![image](https://github.com/aws-observability/terraform-aws-observability-accelerator/assets/10175027/c5b9b685-5894-4350-b68a-ca86d1128f6f)
+<img width="1321" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/2fccf784-6560-45a9-8be0-4e843c9653f1">
 
+<img width="1300" alt="image" src="https://github.com/ruchimo/terraform-aws-observability-accelerator/assets/106240341/69749d78-7c9a-4019-a023-f5412ab3b2de">
 
 ## Cleanup
 

diff --git a/examples/eks-container-insights/README.md b/examples/eks-container-insights/README.md
@@ -1,10 +1,8 @@
 # Enable Container Insights for EKS cluster
 
-This example deploys ADOT as a daemonset on your EKS cluster which enables Container Insights metrics on CloudWatch.
-
-Step-by-step instructions available on our [docs site](https://aws-observability.github.io/terraform-aws-observability-accelerator/)
-under **Amazon CloudWatch Container Insights**
+This example enables enhanced CloudWatch Container Insights for EKS and CloudWatch Application Signals (preview) through our CloudWatch EKS add-ons, providing comprehensive metrics, logs, and insights for cluster and application monitoring.
 
+Step-by-step instructions available on our [docs site](https://aws-observability.github.io/terraform-aws-observability-accelerator/container-insights/eks/)
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
 ## Requirements
@@ -13,9 +11,6 @@ under **Amazon CloudWatch Container Insights**
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.0.0 |
-| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.4.1 |
-| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 2.0.3 |
-| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
 
 ## Providers
 
@@ -42,10 +37,11 @@ under **Amazon CloudWatch Container Insights**
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_aws_region"></a> [aws\_region](#input\_aws\_region) | EKS cluster region | `string` | n/a | yes |
+| <a name="input_eks_cluster_domain"></a> [eks\_cluster\_domain](#input\_eks\_cluster\_domain) | The domain for the EKS cluster | `string` | `""` | no |
 | <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS cluster name | `string` | n/a | yes |
-| <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
-| <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
+| <a name="input_eks_cluster_version"></a> [eks\_cluster\_version](#input\_eks\_cluster\_version) | The Kubernetes version for the cluster | `string` | `"1.28"` | no |
+| <a name="input_eks_oidc_provider"></a> [eks\_oidc\_provider](#input\_eks\_oidc\_provider) | The OpenID Connect identity provider (issuer URL without leading `https://`) | `string` | `null` | no |
+| <a name="input_eks_oidc_provider_arn"></a> [eks\_oidc\_provider\_arn](#input\_eks\_oidc\_provider\_arn) | The OpenID Connect identity provider ARN | `string` | `null` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
 
 ## Outputs

diff --git a/examples/eks-container-insights/data.tf b/examples/eks-container-insights/data.tf
@@ -0,0 +1,7 @@
+data "aws_partition" "current" {}
+data "aws_caller_identity" "current" {}
+data "aws_region" "current" {}
+
+data "aws_eks_cluster" "eks_cluster" {
+  name = var.eks_cluster_id
+}
diff --git a/examples/eks-container-insights/locals.tf b/examples/eks-container-insights/locals.tf
@@ -1,9 +1,19 @@
-data "aws_partition" "current" {}
+locals {
+  name                = "amazon-cloudwatch-observability"
+  eks_oidc_issuer_url = replace(data.aws_eks_cluster.eks_cluster.identity[0].oidc[0].issuer, "https://", "")
 
-data "aws_caller_identity" "current" {}
+  addon_context = {
+    aws_caller_identity_account_id = data.aws_caller_identity.current.account_id
+    aws_caller_identity_arn        = data.aws_caller_identity.current.arn
+    aws_partition_id               = data.aws_partition.current.partition
+    aws_region_name                = data.aws_region.current.name
+    eks_oidc_provider_arn          = "arn:${data.aws_partition.current.partition}:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${local.eks_oidc_issuer_url}"
+    eks_cluster_id                 = data.aws_eks_cluster.eks_cluster.id
+    tags                           = var.tags
+  }
 
-data "aws_region" "current" {}
-
-data "aws_eks_cluster" "eks_cluster" {
-  name = var.eks_cluster_id
+  addon_config = {
+    kubernetes_version = var.eks_cluster_version
+    most_recent        = true
+  }
 }
diff --git a/examples/eks-container-insights/main.tf b/examples/eks-container-insights/main.tf
@@ -1,34 +1,8 @@
-provider "aws" {
-  region = var.aws_region
-}
-
-provider "kubernetes" {
-  host                   = local.eks_cluster_endpoint
-  cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks_cluster.certificate_authority[0].data)
-  exec {
-    api_version = "client.authentication.k8s.io/v1beta1"
-    args        = ["eks", "get-token", "--cluster-name", var.eks_cluster_id]
-    command     = "aws"
-  }
-}
-
-provider "helm" {
-  kubernetes {
-    host                   = local.eks_cluster_endpoint
-    cluster_ca_certificate = base64decode(data.aws_eks_cluster.eks_cluster.certificate_authority[0].data)
-    exec {
-      api_version = "client.authentication.k8s.io/v1beta1"
-      args        = ["eks", "get-token", "--cluster-name", var.eks_cluster_id]
-      command     = "aws"
-    }
-  }
-}
-
-
-# Deploy the ADOT Container Insights
-
 module "eks_container_insights" {
-  source = "../../modules/eks-container-insights"
-  # source = "github.com/aws-observability/terraform-aws-observability-accelerator//modules/eks-container-insights?ref=v2.5.4"
-  eks_cluster_id = var.eks_cluster_id
+  source                                     = "../../modules/eks-container-insights"
+  eks_cluster_id                             = var.eks_cluster_id
+  enable_amazon_eks_cw_observability         = true
+  create_cloudwatch_observability_irsa_role  = true
+  eks_oidc_provider_arn                      = local.addon_context.eks_oidc_provider_arn
+  create_cloudwatch_application_signals_role = true
 }
diff --git a/examples/eks-container-insights/variables.tf b/examples/eks-container-insights/variables.tf
@@ -3,23 +3,30 @@ variable "eks_cluster_id" {
   type        = string
 }
 
-variable "aws_region" {
-  description = "EKS cluster region"
+variable "eks_cluster_domain" {
+  description = "The domain for the EKS cluster"
   type        = string
+  default     = ""
 }
 
-variable "irsa_iam_role_path" {
-  description = "IAM role path for IRSA roles"
+variable "eks_oidc_provider" {
+  description = "The OpenID Connect identity provider (issuer URL without leading `https://`)"
   type        = string
-  default     = "/"
+  default     = null
 }
 
-variable "irsa_iam_permissions_boundary" {
-  description = "IAM permissions boundary for IRSA roles"
+variable "eks_oidc_provider_arn" {
+  description = "The OpenID Connect identity provider ARN"
   type        = string
   default     = null
 }
 
+variable "eks_cluster_version" {
+  description = "The Kubernetes version for the cluster"
+  type        = string
+  default     = "1.28"
+}
+
 variable "tags" {
   description = "Additional tags (e.g. `map('BusinessUnit`,`XYZ`)"
   type        = map(string)

diff --git a/examples/eks-container-insights/versions.tf b/examples/eks-container-insights/versions.tf
@@ -6,25 +6,5 @@ terraform {
       source  = "hashicorp/aws"
       version = ">= 5.0.0"
     }
-    kubernetes = {
-      source  = "hashicorp/kubernetes"
-      version = ">= 2.10"
-    }
-    kubectl = {
-      source  = "alekc/kubectl"
-      version = ">= 2.0.3"
-    }
-    helm = {
-      source  = "hashicorp/helm"
-      version = ">= 2.4.1"
-    }
   }
-
-  # ##  Used for end-to-end testing on project; update to suit your needs
-  # backend "s3" {
-  #   bucket = "aws-observability-accelerator-terraform-states"
-  #   region = "us-west-2"
-  #   key    = "e2e/eks_container_insights/terraform.tfstate"
-  # }
-
 }
diff --git a/modules/eks-container-insights/README.md b/modules/eks-container-insights/README.md
@@ -1,10 +1,10 @@
-# Container Insights ADOT implementation for EKS Cluster Observability
+# Container Insights CloudWatch implementation for EKS Cluster Observability
 
-This module provides an automated experience around enabling Container Insights for your EKS cluster using ADOT (AWS Distro for OpenTelemetry).
-It provides the following resources:
+This module configures AWS CloudWatch Agent used for CloudWatch Application signals and Container Insights.
 
-- ADOT Collector Deployment to your EKS cluster
-- Enabling Container Insights on CloudWatch
+Use CloudWatch Application Signals to automatically instrument your applications on AWS so that you can monitor current application health and track long-term application performance against your business objectives. Application Signals provides you with a unified, application-centric view of your applications, services, and dependencies, and helps you monitor and triage application health.
+
+Use CloudWatch Container Insights to collect, aggregate, and summarize metrics and logs from your containerized applications and microservices. CloudWatch automatically collects metrics for many resources, such as CPU, memory, disk, and network. Container Insights also provides diagnostic information, such as container restart failures, to help you isolate issues and resolve them quickly. You can also set CloudWatch alarms on metrics that Container Insights collects.
 
 
 <!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK -->
@@ -14,9 +14,6 @@ It provides the following resources:
 |------|---------|
 | <a name="requirement_terraform"></a> [terraform](#requirement\_terraform) | >= 1.1.0 |
 | <a name="requirement_aws"></a> [aws](#requirement\_aws) | >= 5.0.0 |
-| <a name="requirement_helm"></a> [helm](#requirement\_helm) | >= 2.4.1 |
-| <a name="requirement_kubectl"></a> [kubectl](#requirement\_kubectl) | >= 2.0.3 |
-| <a name="requirement_kubernetes"></a> [kubernetes](#requirement\_kubernetes) | >= 2.10 |
 
 ## Providers
 
@@ -28,31 +25,32 @@ It provides the following resources:
 
 | Name | Source | Version |
 |------|--------|---------|
-| <a name="module_helm_addon"></a> [helm\_addon](#module\_helm\_addon) | github.com/aws-ia/terraform-aws-eks-blueprints//modules/kubernetes-addons/helm-addon | v4.32.1 |
+| <a name="module_cloudwatch_observability_irsa_role"></a> [cloudwatch\_observability\_irsa\_role](#module\_cloudwatch\_observability\_irsa\_role) | terraform-aws-modules/iam/aws//modules/iam-role-for-service-accounts-eks | v5.33.0 |
 
 ## Resources
 
 | Name | Type |
 |------|------|
+| [aws_eks_addon.amazon_cloudwatch_observability](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/eks_addon) | resource |
+| [aws_iam_service_linked_role.application_signals_cw](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
 | [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
+| [aws_eks_addon_version.eks_addon_version](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_addon_version) | data source |
 | [aws_eks_cluster.eks_cluster](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/eks_cluster) | data source |
-| [aws_iam_policy.irsa](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy) | data source |
 | [aws_partition.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/partition) | data source |
 | [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
 
 ## Inputs
 
 | Name | Description | Type | Default | Required |
 |------|-------------|------|---------|:--------:|
-| <a name="input_adot_otel_helm_chart_verison"></a> [adot\_otel\_helm\_chart\_verison](#input\_adot\_otel\_helm\_chart\_verison) | ADOT collector helm chart version | `string` | `"0.17.0"` | no |
-| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | EKS Cluster Id | `string` | n/a | yes |
-| <a name="input_helm_config"></a> [helm\_config](#input\_helm\_config) | Helm provider config for adot-exporter-for-eks-on-ec2 | `any` | `{}` | no |
-| <a name="input_irsa_iam_permissions_boundary"></a> [irsa\_iam\_permissions\_boundary](#input\_irsa\_iam\_permissions\_boundary) | IAM permissions boundary for IRSA roles | `string` | `null` | no |
-| <a name="input_irsa_iam_role_path"></a> [irsa\_iam\_role\_path](#input\_irsa\_iam\_role\_path) | IAM role path for IRSA roles | `string` | `"/"` | no |
-| <a name="input_irsa_policies"></a> [irsa\_policies](#input\_irsa\_policies) | Additional IAM policies for a IAM role for service accounts | `list(string)` | `[]` | no |
-| <a name="input_manage_via_gitops"></a> [manage\_via\_gitops](#input\_manage\_via\_gitops) | Determines if the add-on should be managed via GitOps. | `bool` | `false` | no |
-| <a name="input_service_exporters"></a> [service\_exporters](#input\_service\_exporters) | exporter for adot-ci setup | `string` | `"awsemf"` | no |
-| <a name="input_service_receivers"></a> [service\_receivers](#input\_service\_receivers) | receiver for adot-ci setup | `string` | `"awscontainerinsightreceiver"` | no |
+| <a name="input_addon_config"></a> [addon\_config](#input\_addon\_config) | Amazon EKS Managed CloudWatch Observability Add-on config | `any` | `{}` | no |
+| <a name="input_create_cloudwatch_application_signals_role"></a> [create\_cloudwatch\_application\_signals\_role](#input\_create\_cloudwatch\_application\_signals\_role) | Create a Cloudwatch Application Signals service-linked role | `bool` | `true` | no |
+| <a name="input_create_cloudwatch_observability_irsa_role"></a> [create\_cloudwatch\_observability\_irsa\_role](#input\_create\_cloudwatch\_observability\_irsa\_role) | Create a Cloudwatch Observability IRSA | `bool` | `true` | no |
+| <a name="input_eks_cluster_id"></a> [eks\_cluster\_id](#input\_eks\_cluster\_id) | Name of the EKS cluster | `string` | `"eks-cw"` | no |
+| <a name="input_eks_oidc_provider_arn"></a> [eks\_oidc\_provider\_arn](#input\_eks\_oidc\_provider\_arn) | The OIDC Provider ARN of AWS EKS cluster | `string` | `""` | no |
+| <a name="input_enable_amazon_eks_cw_observability"></a> [enable\_amazon\_eks\_cw\_observability](#input\_enable\_amazon\_eks\_cw\_observability) | Enable Amazon EKS CloudWatch Observability add-on | `bool` | `true` | no |
+| <a name="input_kubernetes_version"></a> [kubernetes\_version](#input\_kubernetes\_version) | Kubernetes version | `string` | `"1.28"` | no |
+| <a name="input_most_recent"></a> [most\_recent](#input\_most\_recent) | Determines if the most recent or default version of the addon should be returned. | `bool` | `false` | no |
 | <a name="input_tags"></a> [tags](#input\_tags) | Additional tags (e.g. `map('BusinessUnit`,`XYZ`) | `map(string)` | `{}` | no |
 
 ## Outputs

diff --git a/modules/eks-container-insights/data.tf b/modules/eks-container-insights/data.tf
@@ -0,0 +1,7 @@
+data "aws_partition" "current" {}
+data "aws_caller_identity" "current" {}
+data "aws_region" "current" {}
+
+data "aws_eks_cluster" "eks_cluster" {
+  name = var.eks_cluster_id
+}