Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding codeowners file creation script #32

Merged
merged 8 commits into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/workflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
---
name: Howard integration workflow

on:

Check warning on line 4 in .github/workflows/workflow.yml

View workflow job for this annotation

GitHub Actions / yaml-check / yaml-lint-check

4:1 [truthy] truthy value should be one of [false, true]
pull_request:
types:
- opened
- closed
- synchronize

jobs:
markdown-check:
uses:
ai-cfia/github-workflows/.github/workflows/workflow-markdown-check.yml@main

yaml-check:
uses:
ai-cfia/github-workflows/.github/workflows/workflow-yaml-check.yml@main

repo-standard:
uses:
ai-cfia/github-workflows/.github/workflows/workflow-repo-standards-validation.yml@main
secrets: inherit
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,19 @@
# DevOps Repository for AI-CFIA

This repository contains scripts, configurations, and documentation related to the DevOps practices at AI-CFIA. It's designed to help team members set up, deploy, and manage applications on various platforms.
This repository contains scripts, configurations, and documentation related to
the DevOps practices at AI-CFIA. It's designed to help team members set up,
deploy, and manage applications on various platforms.

## Contents

- **GCP Setup**: This section houses automation scripts crafted to simplify the GCP setup. It encompasses tasks such as initiating a new project, associating a billing account, orchestrating an artifact repository, and establishing a service account. Comprehensive documentation is included, ensuring you have all the guidance needed to execute the script.
- **GCP Setup**: This section houses automation scripts crafted to simplify the
GCP setup. It encompasses tasks such as initiating a new project, associating
a billing account, orchestrating an artifact repository, and establishing a
service account. Comprehensive documentation is included, ensuring you have
all the guidance needed to execute the script.

- **GitHub Repository Creation Guide**: This guide will show you how to create a new repository to conform to the CFIA's standards.
- **GitHub Repository Creation Guide**: This guide will show you how to create a
new repository to conform to the CFIA's standards.

## Getting Started

Expand All @@ -16,4 +23,5 @@ This repository contains scripts, configurations, and documentation related to t

## Related Repositories

- [Dev-Rel-Docs](https://github.com/ai-cfia/dev-rel-docs): Contains introduction files and documentation related to developer relations at AI-CFIA.
- [Dev-Rel-Docs](https://github.com/ai-cfia/dev-rel-docs): Contains introduction
files and documentation related to developer relations at AI-CFIA.
Empty file added TESTING.md
Empty file.
43 changes: 30 additions & 13 deletions gcp-setup-script/gcp-project-setup-guide.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,47 @@
# GCP Project Setup Guide

This guide provides a step-by-step process to set up a Google Cloud Platform (GCP) project using the provided automation script. The script will help you create a new GCP project, link a billing account, set up an artifact repository, and configure a service account.
This guide provides a step-by-step process to set up a Google Cloud Platform
(GCP) project using the provided automation script. The script will help you
create a new GCP project, link a billing account, set up an artifact repository,
and configure a service account.

## Environment Variables Explanation

Before running the script, it's essential to understand the variables used:

- `PROJECT_ID` : The Google Cloud Platform project ID. This ID will be used to uniquely identify your project on GCP.
- `PROJECT_ID` : The Google Cloud Platform project ID. This ID will be used to
uniquely identify your project on GCP.

- `BILLING_ACCOUNT_ID` : Your GCP Billing Account ID. You can find this on the GCP Console under "Billing".
- `BILLING_ACCOUNT_ID` : Your GCP Billing Account ID. You can find this on the
GCP Console under "Billing".

- `REPO_NAME` : Choose a name for your Docker repository.

- `REGION` : Specify the Google Cloud region where you want to set up your repository.
- `REGION` : Specify the Google Cloud region where you want to set up your
repository.

- `DESCRIPTION` (Optional) : Provide a description of the Docker repository for reference.
- `DESCRIPTION` (Optional) : Provide a description of the Docker repository for
reference.

- `SA_NAME` : Set a name for your service account.

- `SA_DISPLAY_NAME` : Set a display name for the service account.

- `FILE_NAME.json` : Choose a name for the JSON key file. This will be used to store the credentials for the service account.
- `FILE_NAME.json` : Choose a name for the JSON key file. This will be used to
store the credentials for the service account.

- `SA_NAME@$PROJECT_ID.iam.gserviceaccount.com` : The email address format of your service account. This is constructed using the `$SA_NAME` and `$PROJECT_ID` variables.
- `SA_NAME@$PROJECT_ID.iam.gserviceaccount.com` : The email address format of
your service account. This is constructed using the `$SA_NAME` and
`$PROJECT_ID` variables.

- `ROLE` : The role you want to assign to the service account. The script will iterate over a predefined set of roles and assign them to the service account.
- `ROLE` : The role you want to assign to the service account. The script will
iterate over a predefined set of roles and assign them to the service account.

**Note** : The roles set by the script are the minimum required to build and deploy the application. These roles have been chosen to ensure the least privilege principle while allowing the necessary operations. If you need additional permissions or roles, you can add them to the service account after running the script.
**Note** : The roles set by the script are the minimum required to build and
deploy the application. These roles have been chosen to ensure the least
privilege principle while allowing the necessary operations. If you need
additional permissions or roles, you can add them to the service account after
running the script.

## Deployment Steps

Expand All @@ -35,13 +50,15 @@ Before running the script, it's essential to understand the variables used:
3. Make the script executable with the command `chmod +x gcp_project_setup.sh`.
4. Run the script using `./gcp_project_setup.sh`.

**Note** : The script name `gcp_project_setup.sh` is just an example, you can decide to call the script whatever fits you best.
**Note** : The script name `gcp_project_setup.sh` is just an example, you can
decide to call the script whatever fits you best.

## Adding A Role

If you wish to add a role to your service account, you will need to find the exact role of that name which you can find [here](https://cloud.google.com/iam/docs/understanding-roles),
If you wish to add a role to your service account, you will need to find the
exact role of that name which you can find
[here](https://cloud.google.com/iam/docs/understanding-roles),

1. Set the variable as your role ROLE="ROLE_NAME"
2. Run the command
`gcloud projects add-iam-policy-binding $PROJECT_ID \
2. Run the command `gcloud projects add-iam-policy-binding $PROJECT_ID \
--member="serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com" \--role=$ROLE`
43 changes: 26 additions & 17 deletions gcp-setup-script/gcp-project-setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -7,36 +7,45 @@ prompt_until_input() {
local user_input

while true; do
echo -n "$prompt_message"
read user_input
if [[ -n "$user_input" ]]; then
echo -n "${prompt_message}"
read -r user_input
if [[ -n "${user_input}" ]]; then
break
fi
done
eval "$return_var='$user_input'"
eval "${return_var}='${user_input}'"
}

PROJECT_ID=""
BILLING_ACCOUNT_ID=""

# Prompt the user for required variables
prompt_until_input "Enter desired PROJECT_ID (e.g. cfia-ai-lab): " PROJECT_ID
prompt_until_input "Enter your BILLING_ACCOUNT_ID (You can find this on the GCP Console under Billing): " BILLING_ACCOUNT_ID

# Create a new project
gcloud projects create $PROJECT_ID
gcloud projects create "${PROJECT_ID}"

# Set the project as the active project
gcloud config set project $PROJECT_ID
gcloud config set project "${PROJECT_ID}"

# Link the billing account to the project
gcloud beta billing projects link $PROJECT_ID --billing-account=$BILLING_ACCOUNT_ID
gcloud beta billing projects link "${PROJECT_ID}" --billing-account="${BILLING_ACCOUNT_ID}"

# Retrieve and display a list of Google Cloud regions
echo "Available Google Cloud regions"
gcloud compute regions list --format=value(name)
gcloud compute regions list --format="value(name)"
echo

REGION=""
REPO_NAME=""
DESCRIPTION=""
SA_NAME=""
SA_DISPLAY_NAME=""
FILE_NAME=""

# Prompt user for necessary variables
prompt_until_input "Enter a Google Cloud region from the above list: " REGION
prompt_until_input "Enter a name for your Google Cloud project: " PROJECT_NAME
prompt_until_input "Enter the Docker repository name: " REPO_NAME
prompt_until_input "Enter a description for the Docker repository [Optional]: " DESCRIPTION
prompt_until_input "Enter a name for your service account: " SA_NAME
Expand All @@ -46,16 +55,16 @@ prompt_until_input "Choose a name for the JSON key file (without .json): " FILE_
# Execute commands

# Create an artifact repository
gcloud artifacts repositories create $REPO_NAME \
gcloud artifacts repositories create "${REPO_NAME}" \
--repository-format=docker \
--location=$REGION \
--description="$DESCRIPTION"
--location="${REGION}" \
--description="${DESCRIPTION}"

# Create a service account (SA)
gcloud iam service-accounts create $SA_NAME --display-name "$SA_DISPLAY_NAME"
gcloud iam service-accounts create "${SA_NAME}" --display-name "${SA_DISPLAY_NAME}"

# Create the key for the service account (SA)
gcloud iam service-accounts keys create "$FILE_NAME.json" --iam-account=$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com
gcloud iam service-accounts keys create "${FILE_NAME}.json" --iam-account="${SA_NAME}"@"${PROJECT_ID}".iam.gserviceaccount.com

# Automatically apply the roles to the service account
ROLES=(
Expand All @@ -65,9 +74,9 @@ ROLES=(
)

for ROLE in "${ROLES[@]}"; do
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member=serviceAccount:$SA_NAME@$PROJECT_ID.iam.gserviceaccount.com \
--role=$ROLE
gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
--member=serviceAccount:"${SA_NAME}"@"${PROJECT_ID}".iam.gserviceaccount.com \
--role="${ROLE}"
done

echo "All commands executed successfully!"
17 changes: 14 additions & 3 deletions github-management-script/branch-protection-ruleset.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,29 @@
# GitHub Branch Protection Automation Script
This script adds a branch protection requiring lint-test to pass in order to be able to merge the changes to the main branch.

This script adds a branch protection requiring lint-test to pass in order to be
able to merge the changes to the main branch.

## Description
This script is designed to automate the process of setting up branch protection rules on the main branch of each public repository within a specified GitHub organization. It specifically adds a branch protection rule that requires the lint-test / build status check to pass before changes can be merged into the main branch.

This script is designed to automate the process of setting up branch protection
rules on the main branch of each public repository within a specified GitHub
organization. It specifically adds a branch protection rule that requires the
lint-test / build status check to pass before changes can be merged into the
main branch.

## Permission required

Administration - Access: Read and write

## Usage
Run the script in a Bash-compatible shell. Ensure you have the necessary permissions on the GitHub token for the organization.

Run the script in a Bash-compatible shell. Ensure you have the necessary
permissions on the GitHub token for the organization.

## Breakdown

The script performs three actions :

- Prompt user for token.
- Get all public repositories from the organisation.
- Apply the branch protection rule to the repository.
13 changes: 8 additions & 5 deletions github-management-script/branch-protection-ruleset.sh
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,18 @@

# prompt for GitHub token
echo "Please enter your GitHub token:"
read GITHUB_TOKEN
read -r GITHUB_TOKEN

ORG_NAME="ai-cfia"

API_URL="https://api.github.com/orgs/${ORG_NAME}/repos?type=public"

# get list of all public repos
REPOS=$(curl -s -H "Accept: application/vnd.github.v3+json" \
RESPONSE=$(curl -s -H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
"${API_URL}" | jq -r '.[].full_name')
"${API_URL}")

REPOS=$(echo "${RESPONSE}" | jq -r '.[].full_name')

set_branch_protection() {
REPO_NAME=$1
Expand All @@ -27,6 +29,7 @@ set_branch_protection() {
"enforce_admins": true,
"required_pull_request_reviews": {
"required_approving_review_count": 1
"require_code_owner_reviews": true
},
"restrictions": null
}'
Expand All @@ -47,8 +50,8 @@ for REPO in ${REPOS}; do
"${WORKFLOWS_URL}")

# if the http response code is 200, the directory exists
if [ "${RESPONSE}" -eq 200 ]; then
if [[ "${RESPONSE}" -eq 200 ]]; then
echo "Setting branch protection rules for ${REPO}"
set_branch_protection ${REPO}
set_branch_protection "${REPO}"
fi
done
28 changes: 28 additions & 0 deletions github-management-script/codeowners-file-creation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# CODEOWNERS Automation Script

This script automates the creation of CODEOWNERS files within repositories of
the CFIA organization and applies tag rules based on repository names.

## Functionality

* **Creates CODEOWNERS Files:** The script generates CODEOWNERS files in target
repositories, defining code ownership rules to streamline the review process.
* **Customizable Team Tagging:** It tags relevant teams (`backend`, `frontend`,
`data`, `devops`) based on the repository name.
* **DevOps Ownership:** The script assigns specific ownership to the DevOps team
for files within the `.github` directory, Dockerfile, and docker-compose
configurations.

## Requirements

* **GitHub Personal Access Token (PAT):** A PAT with the `repo` scope.

## Usage

1. **Set Environment Variables:**
1. `GITHUB_TOKEN`: Store your GitHub PAT in this environment variable.
2. `ORG_NAME`: Set this to the name of your target GitHub organization.
2. **Execute the Script:** Run the script. It will:
1. Prompt for your GitHub token (if not set).
2. Retrieve a list of repositories within the organization.
3. Process each repository, generating and adding the CODEOWNERS file.
55 changes: 55 additions & 0 deletions github-management-script/codeowners-file-creation.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
#!/bin/bash
generate_codeowners() {
repo_name=$1

echo "# This CODEOWNERS file is auto-generated. See the script for modification details." > .github/CODEOWNERS

# Default rules for AI-CFIA ownership for repositories which name ends with "backend", "frontend" or "db"
if [[ ${repo_name} == *"backend" ]]; then
echo "* @ai-cfia/backend" >> .github/CODEOWNERS
elif [[ ${repo_name} == *"frontend" ]]; then
echo "* @ai-cfia/frontend" >> .github/CODEOWNERS
elif [[ ${repo_name} == *"db" ]]; then
echo "* @ai-cfia/data" >> .github/CODEOWNERS
fi

{
echo "/.github/ @ai-cfia/devops"
echo "Dockerfile @ai-cfia/devops"
echo "docker-compose.yml @ai-cfia/devops"
echo "docker-compose.*.yml @ai-cfia/devops"
} >> .github/CODEOWNERS
}

create_codeowners() {
org_name=$1
repo_name=$2
codeowners_content=$(generate_codeowners "${repo_name}")

encoded_content=$(echo "${codeowners_content}" | base64 -w 0)

API_URL="https://api.github.com/repos/${org_name}/${repo_name}/contents/.github/CODEOWNERS"

curl -s -X PUT \
-H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
-d "{\"message\": \"Add CODEOWNERS file\", \"content\": \"${encoded_content}\"}" \
"${API_URL}"
}

echo "Please enter your GitHub token:"
read -r GITHUB_TOKEN

ORG_NAME="ai-cfia"
API_URL="https://api.github.com/orgs/${ORG_NAME}/repos?type=public"
RESPONSE=$(curl -s -H "Accept: application/vnd.github.v3+json" \
-H "Authorization: Bearer ${GITHUB_TOKEN}" \
"${API_URL}")
REPOS=$(echo "${RESPONSE}" | jq -r '.[].full_name')

for REPO in ${REPOS}; do
echo "Processing repository: ${REPO}"

create_codeowners "$(dirname "${REPO}") $(basename "${REPO}")"

done
rngadam marked this conversation as resolved.
Show resolved Hide resolved
Loading
Loading