Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Security analytics plugin - added more details for S3 connection setup #8374

Merged
merged 14 commits into from
Sep 26, 2024
+52 −3
Merged

Security analytics plugin - added more details for S3 connection setup #8374

Changes from 7 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
4eb6380
Added more details to the s3 connection setup.
AWSHurneyt Sep 24, 2024
2155560
Adjusted wording for cross-account bucket download.
AWSHurneyt Sep 25, 2024
550f47f
Merge branch 'main' into 3.0-threat-intel-s3-setup
AWSHurneyt Sep 25, 2024
78110e0
Created subsection for cross-account bucket download.
AWSHurneyt Sep 25, 2024
558da56
Merge remote-tracking branch 'origin/3.0-threat-intel-s3-setup' into …
AWSHurneyt Sep 25, 2024
532a3c6
Adjusted wording based on suggestions.
AWSHurneyt Sep 26, 2024
17e978d
Update getting-started.md
Naarcha-AWS Sep 26, 2024
6322957
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
28cdca2
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
b360086
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
7da4df2
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
5a26892
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
a2c2f90
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
c923933
Apply suggestions from code review
Naarcha-AWS Sep 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 52 additions & 3 deletions _security-analytics/threat-intelligence/getting-started.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,64 @@

When using the `S3_SOURCE` as a remote store, the following connection information must be provided:

- **IAM Role ARN**: The Amazon Resource Name (ARN) for an AWS Identity and Access Management (IAM) role.
- **S3 bucket directory**: The name of the Amazon Simple Storage Service (Amazon S3) bucket in which the `STIX2` file is stored.
- **Specify a directory or file**: The object key or directory path for the `STIX2` file in the S3 bucket.
- **IAM Role ARN**: The Amazon Resource Name (ARN) for an AWS Identity and Access Management (IAM) role. When using the AWS OpenSearch Service, the role ARN needs to be in the same account as the OpenSearch domain. For more information on adding a new role for AWS OpenSearch Service, see [Add service ARN](#add-service-arn).

Check failure on line 53 in _security-analytics/threat-intelligence/getting-started.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SubstitutionsError] Use 'for more information about' instead of 'For more information on'.

Raw Output:
{"message": "[OpenSearch.SubstitutionsError] Use 'for more information about' instead of 'For more information on'.", "location": {"path": "_security-analytics/threat-intelligence/getting-started.md", "range": {"start": {"line": 53, "column": 216}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- **S3 bucket directory**: The name of the Amazon Simple Storage Service (Amazon S3) bucket in which the `STIX2` file is stored. To access an S3 bucket in a different AWS account, see the [Cross-account S3 bucket connection](#cross-account-s3-bucket-connection) section for more details.
- **Specify a file**: The object key for the `STIX2` file in the S3 bucket.
- **Region**: The AWS Region for the S3 bucket.

You can also set the **Download schedule**, which determines to where OpenSearch downloads an updated `STIX2` file from the connected S3 bucket. The default interval is once a day. Only daily intervals are supported.

Alternatively, you can check the **Download on demand** option, which prevents new data from the bucket from being automatically downloaded.

#### Add service ARN
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

When using the AWS OpenSearch Service, using the following steps to add a new ARN role:
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

1. Create a new role with a custom trust policy. The following example trust policy adds OpenSearch as a trusted entity:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AWSHurneyt: Where does the procedure occur? The IAM console? If so, we might need to add steps similar to this section in the AWS documentation (https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-service.html#roles-creatingrole-service-console).

Copy link
Contributor Author

@AWSHurneyt AWSHurneyt Sep 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS Yes, it would take place in the IAM console.

This step doesn't actually take place as part of the plugin experience; and isn't entirely unique to this plugin. Users were just having trouble creating a role that could be used for this feature, so we wanted to add some tips to our documentation.

We could link to the AWS documentation, or perhaps extract these steps into a sub-section. Whichever you think makes more sense. We do want to include the tips in our documentation though since this has been a pain point for a lot of users.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets link to reduce our general maintenance, while keeping the example policy below. Have a preference on where we should link to?

Copy link
Contributor Author

@AWSHurneyt AWSHurneyt Sep 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS would it make sense to reword the steps to something like the following? It calls out the configurations that are necessary for this feature while leaving explanation of the specific role creation steps to the AWS documentation.

  • IAM Role ARN: The Amazon Resource Name (ARN) for an AWS Identity and Access Management (IAM) role. Note that, when using AWS OpenSearch Service, this role ARN needs to be in the same account as the OpenSearch domain. E.g.,
    1. Create a new role with a custom trust policy. The following example trust policy adds OpenSearch as a trusted entity. 
      { 
   "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Service": [
                    "opensearchservice.amazonaws.com"
                ]
            },
            "Action": "sts:AssumeRole"
        }
    ]
}
    2. Add the AmazonS3ReadOnlyAccess permission to the role.
    3. For more information on creating a role, see the AWS documentation.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would make sense. I'm going to keep the example as a separate section though, since the formatting can get temperamental with the code block underneath a subitem.

Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```bash
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"opensearchservice.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}
```

2. Select **Next**.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
3. On the Permissions policies page, add the `AmazonS3ReadOnlyAccess` permission. Then, select **Next**.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
4. Add a name and a description for the new role, then select **Create role**.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved


#### Cross-account S3 bucket connection

Because the role ARN needs to be in the same account as the OpenSearch domain, a trust policy needs to be configured that allowsthe OpenSearch domain to download from S3 buckets from the same account.

Check failure on line 92 in _security-analytics/threat-intelligence/getting-started.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.Spelling] Error: allowsthe. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.

Raw Output:
{"message": "[OpenSearch.Spelling] Error: allowsthe. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_security-analytics/threat-intelligence/getting-started.md", "range": {"start": {"line": 92, "column": 123}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

To download from an S3 bucket in another account, the trust policy for that bucket needs to give the role ARN permission to read from the object, as shown in the following example:

```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::123456789012:role/account-1-threat-intel-role"
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::account-2-threat-intel-bucket/*"
}
]
}
```

## Step 2: Set up scanning for your log sources

Expand Down
Loading