Secretly

Secretly was created to allow Go applications to easily interface with secret management services and reduce secret ingestion boiler-plate. In-memory secret caching is included to reduce the number of operations against the secret management service, when dealing with secrets that store map data in the form of JSON and YAML.

Usage

type Secrets struct {
    DatabaseUsername string `type:"yaml" name:"My-DB-Credentials" key:"username" split_words:"true"`

    DatabasePassword string `type:"yaml" name:"My-DB-Credentials" key:"password"`
}

func getSecret (ctx context.Context, name, version string) ([]byte, error) {
    // Use your secret manager of choice's client library here to retrieve secrets.
    return []byte(""), nil
}

func example(ctx context.Context) Secrets {
    var s Secrets
    
    err := secretly.Process(ctx, &s, getSecret)
    if err != nil {
        log.Fatal(err)
    }
    
    return s
}

Overview

Tag Support

Available Tags

• type - The secret content's structure.
  • Valid Values: "text", "json", "yaml"
  • Default: "text"
• name - The secret's name
• key - The specific field to extract from the secret's content. Note: Requires type "json" or "yaml".
  • Default: The struct field name (split if split_words is true).
• version - The version of the secret to retrieve.
  • Default: 0 (translates to the latest version within the client wrappers, e.g. with GCP Secret Manager, 0 -> "latest")
• split_words - If the field name is used as the secret name and/or key, split it with underscores. If set to true and a process option is provided that combines name and key, the name and key will be separated with an underscore.
  • Default: false

Below is an example structure definition detailing default behavior, and the available tags:

type Specification struct {
    // The latest version of a secret named "TextSecret" that stores text data.
    TextSecret string `type:"text"`

    // The first version of a secret named "TextSecretVersion" that stores text data.
    // Rather than retrieving the latest version, retrieve version 1.
    TextSecretWithVersion1 string `type:"text" version:"1"`

    // The latest version of a secret named "Split_Text_Secret" that stores text data.
    SplitTextSecret string `type:"text" split_words:"true"`

    // The latest version of a secret named "Json_Secret" that stores json data
    // including a key "Json_Secret_Key".
    JsonSecretKey int `type:"json" name:"Json_Secret" split_words:"true"`

    // The latest version of a secret named "Yaml_Secret" that stores yaml data
    // including a key "Yaml_Secret_Key".
    YamlSecretExplicitKey float64 `type:"json" name:"Yaml_Secret" key:"Yaml_Secret_Key"`

    // Ignored.
    IgnoredField string `ignored:"true"`

    // Also ignored.
    ignoredField string

    // The fields from the nested struct are also processed.
    SubSpecification SubSpecification
}

type SubSpecification struct {
    // The latest version of a secret named "SubTextSecret" that stores text data.
    // Since the type is not specified, the default type, text, is used.
    SubTextSecret string

    // The latest version of a secret named "SubJsonSecretAndKey" that stores yaml data
    // including a key "SubJsonSecretAndKey".
    SubJsonSecretAndKey string `type:"json"`
}

Supported Field types

• text - Plain text. Any secret value can be read as plain text.

  Example secrets that stores text data:

  sensitive data
  

• json - JSON map. The secret stores JSON data; read a specific field from the JSON map. Note: If you want to read the entire json object, use the text type.

  Example secret that stores a JSON map:

  {
      "sensitive-field-1": "sensitive data"
  }

• yaml - YAML map. The secret stores YAML data; read a specific field from the YAML map. Note: If you want to read the entire yaml mapping, use the text type.

  Example secret that stores a YAML map:

  sensitive-field-1: sensitive data

Secret Versioning

Secretly provides two options for specifying secret versions other than the version tag:

1. Read secret versions (and all other field values) from a patch file:

   • Supported patch file types:
     • JSON (ext: .json)
     • YAML (ext: .yaml OR .yml)

   Example of reading secret versions from a JSON patch file:

   • versions.json

     {
         "My-DB-Credentials_username": {
             "version": "latest"
         },
         "My-DB-Credentialspassword": {
             "version": "5"
         }
     }

   • example.go

     type Secrets struct {
         DatabaseUsername string `type:"yaml" name:"My-DB-Credentials" key:"username" split_words:"true"`
     
         DatabasePassword string `type:"yaml" name:"My-DB-Credentials" key:"password"`
     }
     
     func getSecret (ctx context.Context, name, version string) ([]byte, error) {
         // Use your secret manager of choice's client library here to retrieve secrets.
         return []byte(""), nil
     }
     
     func example(ctx context.Context) Secrets {
         var s Secrets
         
         err := secretly.Process(ctx, &s, getSecret, secretly.WithPatchFile("versions.json"))
         if err != nil {
             log.Fatal(err)
         }
         
         return s
     }

2. Read secret versions from environment variables:

   Example of reading secret versions from environment variables:

   • Export environment variables:

     export EXAMPLE_MY_DB_CREDENTIALS_USERNAME_VERSION=latest
     export EXAMPLE_MYDBCREDENTIALSPASSWORD_VERSION=5

   • example.go

     type Secrets struct {
         DatabaseUsername string `type:"yaml" name:"My-DB-Credentials" key:"username" split_words:"true"`
     
         DatabasePassword string `type:"yaml" name:"My-DB-Credentials" key:"password"`
     }
     
     func getSecret (ctx context.Context, name, version string) ([]byte, error) {
         // Use your secret manager of choice's client library here to retrieve secrets.
         return []byte(""), nil
     }
     
     func example(ctx context.Context) Secrets {
         var s Secrets
         
         err := secretly.Process(ctx, &s, getSecret, secretly.WithVersionsFromEnv("EXAMPLE"))
         if err != nil {
             log.Fatal(err)
         }
         
         return s
     }

References

• envconfig

  The API for secretly was inspired by and borrows from envconfig. A simple yet powerful package for managing configuration data from environment variables.