Utility to autogenerate a terraform module

[sed-i]

What needs to get done

Terraform modules for charms seem to be quite deterministic and repeatable, for example:

• https://github.com/Gmerold/terraform-juju-cos-lite-k8s/tree/cos-lite-module/modules
• https://github.com/canonical/data-platform-benchmark-env/tree/main/stacks/cos
• feat: Adding Terraform module prometheus-k8s-operator#577
• feat: Adding the Terraform module grafana-agent-k8s-operator#274

If indeed possible, it would be great to enable this for the entire ecosystem with something like

• charmcraft terraform
• tox -e terraform

And have it evolve organically with each charm.

Why it needs to get done

A uniform solution for the entire ecosystem, simplifying maintenance overhead for multiple teams.

[syncronize-issues-to-jira]

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/CRAFT-2492.

This message was autogenerated

[gatici]

Thanks @sed-i for opening the issue. That's great idea to generate Terraform modules of each charm by the help of charmcraft. As a Telco team, we manually produced the Terraform modules for each SD-Core charms as we are aiming to replace the bundles with high level Terraform modules.

We tried to keep the things as much as generic to reduce the maintenance cost. For example, used monolithic configuration instead of expanding each config parameters. However, we still need to maintain the relation endpoints manually.

The charms' Terraform modules could be used in the integration tests. Setting a an integration environment and testing the scenarios using Terraform will be quite straightforward.

The Terraform module's structure seems explicit. I hope it will be feasible to generate them automatically.

[Gmerold]

I think it would be really cool to have it. This would significantly minimize the effort of creating modules.
And it should also have a corresponding GitHub action too ;)

Having that said, I see two things worth mentioning:

1. Creating Terraform modules for bundles - While we could have them generated based on the bundle.yaml, it doesn't make any sense to create a bundle just to replace with a Terraform module :) This part would probably have to stay manual.

2. Publishing our modules to the public Terraform registry - Generating Terraform modules based on metadata.yaml of the charm means that the Terraform module will live in the charm repo. This means we won't be able to publish our modules to the public Terraform registry (it has specific requirements for the repository name, which we don't meet). To reuse the module, we'll have to source it from Git directly. Probably not a big deal, but worth noting. Perhaps, at some point, Canonical could host a private registry?

[lengau]

I've never actually created a terraform module so I'm not sure how feasible this is, but looking at the source docs I don't see why not. My main concern is that Charmcraft will probably need to generate pretty opinionated versions of these, and if I used my own novice opinions it would probably produce some bad ones, so I'd like to get stakeholder feedback on it.

Would it be reasonable to use jinja templates to generate the Terraform modules? We're already using them for init, and if can use that we could have relevant user stakeholders (you, @ghislainbourgeois, etc.) take ownership of them and iterate them (with the command being marked as experimental) until you're comfortable that they're stable.