Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an optional Cloud Ops Agent install script #4

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

killthekitten
Copy link

@killthekitten killthekitten commented Aug 20, 2024

What changed

This PR adds the cloud_ops_agent_enabled flag that fetches and installs the latest cloud ops agent during the startup. The actual script is taken from the official documentation.

This is set to false by default to avoid unexpected behavior for existing deployments. If this gets merged, we would need to update all relevant official snowplow modules, i.e. enrich, collector etc to support passing this flag from the user land – happy to do this work!

An alternative implementation that I would be equally happy with would be to expose the user_supplied_script argument in the official modules to let the users append arbitrary setup scripts such as this.

Why is this needed

We lack the basic monitoring capabilities that are only available when the Cloud Ops Agent is installed (disk, memory and CPU monitoring). We tried to install this without messing with the terraform configuration by using Ops Agent Policies, but apparently they only work when the Ops Config Agents are installed, which can be limiting for some teams.

@snowplowcla
Copy link

Thanks for your pull request. Is this your first contribution to a Snowplow open source project? Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://docs.snowplowanalytics.com/docs/contributing/contributor-license-agreement/ to learn more and sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.

@killthekitten
Copy link
Author

I signed it 👌

@snowplowcla
Copy link

Confirmed! @killthekitten has signed the Contributor License Agreement. Thanks so much.

@killthekitten
Copy link
Author

To make sure this works, however, a bunch of extra permissions are required on the service account (logs writer, metrics writer etc). Currently, terraform-google-service-ce doesn't manage the service account access, so may be this PR would be a better fit in the other modules, such as enrich-pubsub-ce.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants