GUIDE_COMPLETE.md

Terraform Guide

This repo has code for creating a complete implementation of both layers described in Workflow. This can be done on any host with NodeJS, Yarn and Git installed.

Prerequisites

Before using polkadot-secure-validator you need to have installed:

• NodeJS v14 or above (we recommend using nvm)

• Yarn

• Terraform (the snap package available via your package manager will not work)

• Ansible (v2.8+, available through pip)

You will need credentials as environment variables for all the infrastructure providers used in the platform creation phase. The tool now supports AWS, Azure, GCP and packet, these are the required variables:

• AWS: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY of an IAM account with EC2 and VPC write access.
• Azure: ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID, ARM_TENANT_ID, TF_VAR_client_id (same as ARM_CLIENT_ID), TF_VAR_client_secret (same as ARM_CLIENT_SECRET). All these credentials should correspond to a service principal with at least a Contributor role, see here for details or create an issue for finer grained access control.
• GCP: GOOGLE_APPLICATION_CREDENTIALS (path to json file with credentials of the service account you want to use; this service account needs to have write access to compute and network resources).
• PACKET: TF_VAR_auth_token.
• DigitalOcean: TF_VAR_do_token.
• Hetzner: TF_VAR_hcloud_token.

The tool allows you to specify which providers to use, so you don't need to have accounts in all of them, see here for an example of how to define the providers. You could use, for instance, packet for the validators and GCP for the public nodes. Keep in mind that, the more distributed your public nodes, the fewer opportunities to be affected by potential incidents in the respective cloud providers.

You need two additional environment variables to allow ansible to connect to the created machines:

• SSH_ID_RSA_PUBLIC: path to private SSH key you want to use for the public nodes.

• SSH_ID_RSA_VALIDATOR: path to private SSH key you want to use for the validators.

You can easily create and add them to your ssh-agent as follows:

$ ssh-keygen -m PEM -f <path>
$ ssh-add <path>

Note: The key paths MUST be added to ssh-add.

Provider Account Setup & Configuration

• Create a project in Google Cloud Provider (GCP). This is currently the only supported backend for Terrafrom state changes. Copy the project ID for the next step.
• Create a project in the cloud Provider account of your choice with the name of the ID that you copied in the GCP project. e.g. My-Project-1234567
  • Note: Under the hood Terraform uses the projectId field in your config/main.json to store the state using this format.
• The ssh username in config.main.json only works with root with some providers such as digitalocean.
• Inside your config/main.json the machineType field must have a slug value. e.g. for DigitalOcean. s-4vcpu-8gb-amd.

Synchronization

$ git clone https://github.com/w3f/secure-validator
$ cd secure-validator
$ yarn
$ cp config/main.template.json config/main.json
# now you should complete and customize config/main.json, using main.sample.json as a reference
$ yarn sync -c config/main.json

You can also just provision a set of previously created machines with the ansible code. We have provided an example inventory that you can customize. See the Ansible Guide for more.

The sync command is idempotent, unless there are errors it will always have the same results. You can execute it as much as you want, it will only make changes when the actual infrastructure state doesn't match the desired state.

Restore db

$ yarn restore-db -c config/main.json

A possibile configuration can be:

...
"validators": {
    "additionalFlags": "--unsafe-pruning --pruning 1000",
    "dbSnapshot": {
      "url": "https://ksm-rocksdb.polkashots.io/kusama-6658753.RocksDb.7z",
      "checksum": "sha256:4f61a99e4b00acb335aff52f2383880d53b30617c0ae67ac47c611e7bf6971ff"
    },
    ...
}

Update Binary

$ yarn update-binary -c config/main.json

Rotate Keys

$ yarn rotate-keys -c config/main.json

Cleaning up

You can remove all the created infrastructure with:

$ yarn clean -c config/main.json