Feature: SecretSource API support (#27)

jenkinsci · Jun 20, 2020 · 3fed37d · 3fed37d
1 parent c1c44ce
commit 3fed37d
Show file tree

Hide file tree

Showing 48 changed files with 1,674 additions and 735 deletions.
diff --git a/docs/README.md b/docs/README.md
@@ -8,6 +8,7 @@ Access credentials from AWS Secrets Manager in your Jenkins jobs.
 ## Contents
 
 - [Caching](caching/index.md)
+- [Filters](filters/index.md)
 - [Networking](networking/index.md)
 - [Screenshots](screenshots/index.md)
 - Project
@@ -18,8 +19,8 @@ Access credentials from AWS Secrets Manager in your Jenkins jobs.
 ## Features
 
 - Read-only view of Secrets Manager.
+- `CredentialsProvider` and `SecretSource` API support.
 - Credential metadata caching (duration: 5 minutes).
-- Jenkins [Configuration As Code](https://github.com/jenkinsci/configuration-as-code-plugin) support.
 - [Cross-account](http://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html) Secrets Manager support with IAM roles.
 
 ## Setup 
@@ -43,23 +44,30 @@ Optional permissions:
 
 ## Usage
 
-1. **Upload the secret** to Secrets Manager as shown below (see also the [AWS documentation](https://docs.aws.amazon.com/cli/latest/reference/secretsmanager/create-secret.html)).
-2. **Reference the secret** by name in your Jenkins job.
+The plugin supports the following secrets resolution APIs:
 
-A Secrets Manager secret acts as one of the following Jenkins credential types, depending on the `jenkins:credentials:type` tag that you add to it. The tag's value must be the relevant Jenkinsfile credentials binding [type name](https://jenkins.io/doc/pipeline/steps/credentials-binding/), e.g. `string` for Secret Text.
+- [CredentialsProvider](#CredentialsProvider) (high-level API)
+- [SecretSource](#SecretSource) (low-level API)
 
-### Secret Text
+Note: Any string secret is accessible through SecretSource, but only a secret with the `jenkins:credentials:type` tag is accessible through CredentialsProvider. This distinction allows you to share tagged secrets between both APIs, while untagged secrets are only accessible through SecretSource.
+
+### CredentialsProvider
+
+The plugin allows secrets from Secrets Manager to be used as Jenkins credentials.
+
+A secret will act as one of the following Jenkins [credential types](https://jenkins.io/doc/pipeline/steps/credentials-binding/), based on the `jenkins:credentials:type` tag that you add to it.
+
+#### Secret Text
 
 A simple text *secret*.
 
 - Value: *secret*
 - Tags:
   - `jenkins:credentials:type` = `string`
 
-:white_check_mark: Use this credential type whenever it is practical. It is the simplest and most widely compatible type.
+##### Example
 
-
-#### Example
+AWS CLI:
 
 ```bash
 aws secretsmanager create-secret --name 'newrelic-api-key' --secret-string 'abc123' --tags 'Key=jenkins:credentials:type,Value=string' --description 'Acme Corp Newrelic API key'
@@ -93,7 +101,7 @@ node {
 }
 ```
 
-### Username with Password
+#### Username with Password
 
 A *username* and *password* pair.
 
@@ -102,7 +110,9 @@ A *username* and *password* pair.
   - `jenkins:credentials:type` = `usernamePassword`
   - `jenkins:credentials:username` = *username*
 
-#### Example
+##### Example
+
+AWS CLI:
 
 ```bash
 aws secretsmanager create-secret --name 'artifactory' --secret-string 'supersecret' --tags 'Key=jenkins:credentials:type,Value=usernamePassword' 'Key=jenkins:credentials:username,Value=joe' --description 'Acme Corp Artifactory login'
@@ -137,7 +147,7 @@ node {
 }
 ```
 
-### SSH User Private Key
+#### SSH User Private Key
 
 An SSH *private key*, with a *username*.
 
@@ -148,7 +158,9 @@ An SSH *private key*, with a *username*.
 
 Common private key formats include PKCS#1 (starts with `-----BEGIN [ALGORITHM] PRIVATE KEY-----`) and PKCS#8 (starts with `-----BEGIN PRIVATE KEY-----`).
 
-#### Example
+##### Example
+
+AWS CLI:
 
 ```bash
 ssh-keygen -t rsa -b 4096 -C '[email protected]' -f id_rsa
@@ -184,15 +196,17 @@ node {
 }
 ```
 
-### Certificate
+#### Certificate
 
 A client certificate *keystore* in PKCS#12 format, encrypted with a zero-length password.
 
 - Value: *keystore*
 - Tags:
   - `jenkins:credentials:type` = `certificate`
 
-#### Example
+##### Example
+
+AWS CLI:
 
 ```bash
 openssl pkcs12 -export -in /path/to/cert.pem -inkey /path/to/key.pem -out certificate.p12 -passout pass:
@@ -209,7 +223,7 @@ node {
 }
 ```
 
-### Secret File
+#### Secret File
 
 A secret file with binary *content* and an optional *filename*.
 
@@ -220,7 +234,9 @@ A secret file with binary *content* and an optional *filename*.
 
 The credential ID is used as the filename by default. In the rare cases when you need to override this (for example, if the credential ID would be an invalid filename on your filesystem), you can set the `jenkins:credentials:filename` tag.
 
-#### Example
+##### Example
+
+AWS CLI:
 
 ```bash
 echo -n $'\x01\x02\x03' > license.bin
@@ -255,6 +271,30 @@ node {
 }
 ```
 
+### SecretSource
+
+The plugin allows JCasC to interpolate string secrets from Secrets Manager.
+
+#### Example
+
+AWS CLI:
+
+```bash
+aws secretsmanager create-secret --name 'my-password' --secret-string 'abc123' --description 'Jenkins user password'
+```
+
+JCasC:
+
+```yaml
+jenkins:
+  securityRealm:
+    local:
+      allowsSignup: false
+      users:
+      - id: "foo"
+        password: "${my-password}"
+```
+
 ## Configuration
 
 Available settings:
@@ -304,14 +344,15 @@ All secrets must be uploaded via the AWS CLI or API. This is because the AWS Web
 
 In Maven:
 
-```bash
-mvn verify
+```shell script
+mvn clean verify
 ```
 
 In your IDE:
 
-1. Generate translations: `mvn localizer:generate`. (This is a one-off task. You only need to re-run this if you change the translations, or if you clean the Maven target directory.)
+1. Generate translations: `mvn localizer:generate`. (This is a one-off task. You only need to re-run this if you change the translations, or if you clean the Maven `target` directory.)
 2. Compile.
 3. Start Moto: `mvn docker:build docker:start`.
 4. Run tests.
 5. Stop Moto: `mvn docker:stop`.
+
diff --git a/docs/filters/index.md b/docs/filters/index.md
@@ -0,0 +1,25 @@
+# Filters
+
+The CredentialsProvider implementation in this plugin calls `secretsmanager:ListSecrets` to cache the secrets' metadata. At this time, Secrets Manager does not support server-side restrictions on this list, so it returns all secrets in the AWS account, whether you have given Jenkins the `secretsmanager:GetSecretValue` permission to actually resolve those secrets or not. This can result in unwanted entries appearing in the credentials UI, which users will mistake for resolvable credentials. 
+
+To improve the user experience of this aspect of Jenkins, you can specify optional filters in the plugin configuration. Only the secrets that match the filter criteria will be presented through the CredentialsProvider. This hides unwanted entries from the credentials UI. 
+
+Notes:
+
+- These are client-side filters. As such they only provide usability benefits. They have no security benefits, as Jenkins still fetches the full secret list from AWS.
+- The SecretSource implementation does not use the filters, as they are not relevant to it.
+
+## Tag Filter
+
+You can choose to only show credentials that have a tag with a particular key and value.
+
+Example: `product` = `foo`.
+
+```yaml
+unclassified:
+  awsCredentialsProvider:
+    filters:
+      tag:
+        key: product
+        value: foo
+```
diff --git a/pom.xml b/pom.xml
@@ -75,6 +75,12 @@
             <artifactId>aws-java-sdk</artifactId>
             <version>1.11.636</version>
         </dependency>
+        <dependency>
+            <groupId>io.jenkins</groupId>
+            <artifactId>configuration-as-code</artifactId>
+            <version>${jcasc.version}</version>
+            <optional>true</optional>
+        </dependency>
         <dependency>
             <groupId>org.jenkins-ci.plugins</groupId>
             <artifactId>bouncycastle-api</artifactId>
@@ -90,7 +96,7 @@
         <dependency>
             <groupId>org.assertj</groupId>
             <artifactId>assertj-core</artifactId>
-            <version>3.11.1</version>
+            <version>3.16.1</version>
             <scope>test</scope>
         </dependency>
         <dependency>
@@ -99,12 +105,6 @@
             <version>3.10.0</version>
             <scope>test</scope>
         </dependency>
-        <dependency>
-            <groupId>io.jenkins</groupId>
-            <artifactId>configuration-as-code</artifactId>
-            <version>${jcasc.version}</version>
-            <scope>test</scope>
-        </dependency>
         <dependency>
             <groupId>io.jenkins.configuration-as-code</groupId>
             <artifactId>test-harness</artifactId>
@@ -187,18 +187,6 @@
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-dependency-plugin</artifactId>
             </plugin>
-            <plugin>
-                <groupId>org.jacoco</groupId>
-                <artifactId>jacoco-maven-plugin</artifactId>
-                <executions>
-                    <execution>
-                        <goals>
-                            <goal>prepare-agent</goal>
-                            <goal>report</goal>
-                        </goals>
-                    </execution>
-                </executions>
-            </plugin>
             <plugin>
                 <groupId>org.apache.maven.plugins</groupId>
                 <artifactId>maven-release-plugin</artifactId>

diff --git a/src/main/java/io/jenkins/plugins/credentials/secretsmanager/AwsSecretSource.java b/src/main/java/io/jenkins/plugins/credentials/secretsmanager/AwsSecretSource.java
@@ -0,0 +1,98 @@
+package io.jenkins.plugins.credentials.secretsmanager;
+
+import com.amazonaws.SdkClientException;
+import com.amazonaws.client.builder.AwsClientBuilder;
+import com.amazonaws.services.secretsmanager.AWSSecretsManager;
+import com.amazonaws.services.secretsmanager.AWSSecretsManagerClient;
+import com.amazonaws.services.secretsmanager.AWSSecretsManagerClientBuilder;
+import com.amazonaws.services.secretsmanager.model.AWSSecretsManagerException;
+import com.amazonaws.services.secretsmanager.model.GetSecretValueRequest;
+import com.amazonaws.services.secretsmanager.model.GetSecretValueResult;
+import com.amazonaws.services.secretsmanager.model.ResourceNotFoundException;
+import hudson.Extension;
+import io.jenkins.plugins.casc.SecretSource;
+import io.jenkins.plugins.credentials.secretsmanager.config.EndpointConfiguration;
+import io.jenkins.plugins.credentials.secretsmanager.config.PluginConfiguration;
+
+import java.io.IOException;
+import java.util.Optional;
+import java.util.logging.Level;
+import java.util.logging.Logger;
+
+@Extension(optional = true)
+public class AwsSecretSource extends SecretSource {
+
+    private static final Logger LOG = Logger.getLogger(AwsSecretSource.class.getName());
+
+    private static final String AWS_SERVICE_ENDPOINT = "AWS_SERVICE_ENDPOINT";
+    private static final String AWS_SIGNING_REGION = "AWS_SIGNING_REGION";
+
+    private transient AWSSecretsManager client = null;
+
+    @Override
+    public Optional<String> reveal(String id) throws IOException {
+        try {
+            final GetSecretValueResult result = client.getSecretValue(new GetSecretValueRequest().withSecretId(id));
+
+            if (result.getSecretBinary() != null) {
+                throw new IOException(String.format("The binary secret '%s' is not supported. Please change its value to a string, or alternatively delete it.", result.getName()));
+            }
+
+            return Optional.ofNullable(result.getSecretString());
+        } catch (ResourceNotFoundException e) {
+            // Recoverable errors
+            LOG.info(e.getMessage());
+            return Optional.empty();
+        } catch (AWSSecretsManagerException e) {
+            // Unrecoverable errors
+            throw new IOException(e);
+        }
+    }
+
+    @Override
+    public void init() {
+        try {
+            client = createClient();
+        } catch (SdkClientException e) {
+            LOG.log(Level.WARNING, "Could not set up AWS Secrets Manager client. Reason: {0}", e.getMessage());
+        }
+    }
+
+    private static AWSSecretsManager createClient() throws SdkClientException {
+        final PluginConfiguration config = PluginConfiguration.getInstance();
+        final EndpointConfiguration ec = config.getEndpointConfiguration();
+
+        final AWSSecretsManagerClientBuilder builder = AWSSecretsManagerClient.builder();
+
+        final Optional<String> maybeServiceEndpoint = getServiceEndpoint(ec);
+        final Optional<String> maybeSigningRegion = getSigningRegion(ec);
+
+        if (maybeServiceEndpoint.isPresent() && maybeSigningRegion.isPresent()) {
+            LOG.log(Level.CONFIG, "Custom Endpoint Configuration: {0}", ec);
+
+            final AwsClientBuilder.EndpointConfiguration endpointConfiguration =
+                    new AwsClientBuilder.EndpointConfiguration(maybeServiceEndpoint.get(), maybeSigningRegion.get());
+            builder.setEndpointConfiguration(endpointConfiguration);
+        } else {
+            LOG.log(Level.CONFIG, "Default Endpoint Configuration");
+        }
+
+        return builder.build();
+    }
+
+    private static Optional<String> getServiceEndpoint(EndpointConfiguration ec) {
+        if ((ec != null) && (ec.getServiceEndpoint() != null)) {
+            return Optional.of(ec.getServiceEndpoint());
+        } else {
+            return Optional.ofNullable(System.getenv(AWS_SERVICE_ENDPOINT));
+        }
+    }
+
+    private static Optional<String> getSigningRegion(EndpointConfiguration ec) {
+        if ((ec != null) && (ec.getSigningRegion() != null)) {
+            return Optional.of(ec.getSigningRegion());
+        } else {
+            return Optional.ofNullable(System.getenv(AWS_SIGNING_REGION));
+        }
+    }
+}