Skip to content

Commit

Permalink
Merge branch 'main' of github.com:trufflesecurity/trufflehog into fix…
Browse files Browse the repository at this point in the history
…-aws-session
  • Loading branch information
zricethezav committed Nov 3, 2023
2 parents 41cad7f + 7a15633 commit 04f4268
Show file tree
Hide file tree
Showing 121 changed files with 8,147 additions and 4,378 deletions.
15 changes: 0 additions & 15 deletions .github/dependabot.yml

This file was deleted.

8 changes: 8 additions & 0 deletions .github/renovate.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": [
"config:base"
],
"prConcurrentLimit": 3,
"prHourlyLimit": 2
}
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ RUN --mount=type=cache,target=/go/pkg/mod \
--mount=type=cache,target=/root/.cache/go-build \
GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build -o trufflehog .

FROM alpine:3.15
RUN apk add --no-cache bash git openssh-client ca-certificates \
&& update-ca-certificates
FROM alpine:3.18
RUN apk add --no-cache bash git openssh-client ca-certificates rpm2cpio binutils cpio \
&& rm -rf /var/cache/apk/* && update-ca-certificates
COPY --from=builder /build/trufflehog /usr/bin/trufflehog
COPY entrypoint.sh /etc/entrypoint.sh
RUN chmod +x /etc/entrypoint.sh
Expand Down
7 changes: 3 additions & 4 deletions Dockerfile.goreleaser
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
FROM alpine:3.15
FROM alpine:3.18

RUN apk add --no-cache bash git openssh-client ca-certificates \
&& rm -rf /var/cache/apk/* && \
update-ca-certificates
RUN apk add --no-cache bash git openssh-client ca-certificates rpm2cpio binutils cpio \
&& rm -rf /var/cache/apk/* && update-ca-certificates
WORKDIR /usr/bin/
COPY trufflehog .
COPY entrypoint.sh /etc/entrypoint.sh
Expand Down
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -52,10 +52,10 @@ run-debug:
CGO_ENABLED=0 go run . git file://. --json --debug

protos:
docker run -u "$(shell id -u)" -v "$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))":/pwd "${PROTOS_IMAGE}" bash -c "cd /pwd; /pwd/scripts/gen_proto.sh"
docker run --rm -u "$(shell id -u)" -v "$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))":/pwd "${PROTOS_IMAGE}" bash -c "cd /pwd; /pwd/scripts/gen_proto.sh"

protos-windows:
docker run -v "$(shell cygpath -w $(shell pwd))":/pwd "${PROTOS_IMAGE}" bash -c "cd /pwd; ./scripts/gen_proto.sh"
docker run --rm -v "$(shell cygpath -w $(shell pwd))":/pwd "${PROTOS_IMAGE}" bash -c "cd /pwd; ./scripts/gen_proto.sh"

release-protos-image:
docker buildx build --push --platform=linux/amd64,linux/arm64 \
Expand Down
91 changes: 45 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

---

# :mag_right: _Now Scanning_
# :mag*right: \_Now Scanning*

<div align="center">

Expand Down Expand Up @@ -45,6 +45,7 @@ docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --or
# :floppy_disk: Installation

Several options available for you:

```bash
# MacOS users
brew install trufflesecurity/trufflehog/trufflehog
Expand Down Expand Up @@ -79,6 +80,7 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified
```

Expected output:

```
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷
Expand Down Expand Up @@ -110,14 +112,14 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified --js
```

Expected output:

```
{"SourceMetadata":{"Data":{"Git":{"commit":"fbc14303ffbf8fb1c2c1914e8dda7d0121633aca","file":"keys","email":"counter \[email protected]\u003e","repository":"https://github.com/trufflesecurity/test_keys","timestamp":"2022-06-16 10:17:40 -0700 PDT","line":4}}},"SourceID":0,"SourceType":16,"SourceName":"trufflehog - git","DetectorType":2,"DetectorName":"AWS","DecoderName":"PLAIN","Verified":true,"Raw":"AKIAYVP4CIPPERUVIFXG","Redacted":"AKIAYVP4CIPPERUVIFXG","ExtraData":{"account":"595918472158","arn":"arn:aws:iam::595918472158:user/canarytokens.com@@mirux23ppyky6hx3l6vclmhnj","user_id":"AIDAYVP4CIPPJ5M54LRCY"},"StructuredData":null}
...
```

## 4: Scan a GitHub Repo + its Issues and Pull Requests.


```bash
trufflehog github --repo=https://github.com/trufflesecurity/test_keys --issue-comments --pr-comments
```
Expand Down Expand Up @@ -162,28 +164,27 @@ trufflehog docker --image trufflesecurity/secrets --only-verified

# :question: FAQ

+ All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives?
+ That means no secrets were detected
+ Why is the scan taking a long time when I scan a GitHub org
+ Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token
+ It says a private key was verified, what does that mean?
+ Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
+ Is there an easy way to ignore specific secrets?
+ If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets.

- All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives?
- That means no secrets were detected
- Why is the scan taking a long time when I scan a GitHub org
- Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token
- It says a private key was verified, what does that mean?
- Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
- Is there an easy way to ignore specific secrets?
- If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets.

# :newspaper: What's new in v3?

TruffleHog v3 is a complete rewrite in Go with many new powerful features.

- We've **added over 700 credential detectors that support active verification against their respective APIs**.
- We've also added native **support for scanning GitHub, GitLab, filesystems, S3, GCS and Circle CI**.
- We've also added native **support for scanning GitHub, GitLab, Docker, filesystems, S3, GCS, Circle CI and Travis CI**.
- **Instantly verify private keys** against millions of github users and **billions** of TLS certificates using our [Driftwood](https://trufflesecurity.com/blog/driftwood) technology.
- Scan binaries and other file formats
- Scan binaries, documents, and other file formats
- Available as a GitHub Action and a pre-commit hook


## What is credential verification?

For every potential credential that is detected, we've painstakingly implemented programmatic verification against the API that we think it belongs to. Verification eliminates false positives. For example, the [AWS credential detector](pkg/detectors/aws/aws.go) performs a `GetCallerIdentity` API call against the AWS API to verify if an AWS credential is active.

# :memo: Usage
Expand All @@ -198,8 +199,8 @@ TruffleHog has a sub-command for each source of data that you may want to scan:
- filesystem (files and directories)
- syslog
- circleci
- travisci
- GCS (Google Cloud Storage)
- stdin (coming soon)

Each subcommand can have options that you can see with the `--help` flag provided to the sub command:

Expand Down Expand Up @@ -230,13 +231,13 @@ Args:
<uri> Git repository URL. https://, file://, or ssh:// schema expected.
```

For example, to scan a `git` repository, start with
For example, to scan a `git` repository, start with

```
$ trufflehog git https://github.com/trufflesecurity/trufflehog.git
```

## S3
## S3

The S3 source supports assuming IAM roles for scanning in addition to IAM users. This makes it easier for users to scan multiple AWS accounts without needing to rely on hardcoded credentials for each account.

Expand All @@ -261,11 +262,12 @@ trufflehog s3 --role-arn=<iam-role-arn-1> --role-arn=<iam-role-arn-2>
```

Exit Codes:

- 0: No errors and no results were found.
- 1: An error was encountered. Sources may not have completed scans.
- 183: No errors were encountered, but results were found. Will only be returned if `--fail` flag is used.

# :octocat: TruffleHog Github Action
## :octocat: TruffleHog Github Action

```yaml
- name: TruffleHog
Expand All @@ -285,6 +287,7 @@ The TruffleHog OSS Github Action can be used to scan a range of commits for leak
any results are found.
For example, to scan the contents of pull requests you could use the following workflow:
```yaml
name: TruffleHog Secrets Scan
on: [pull_request]
Expand All @@ -305,26 +308,26 @@ jobs:
extra_args: --debug --only-verified
```
# Pre-commit Hook
## Pre-commit Hook
Trufflehog can be used in a pre-commit hook to prevent credentials from leaking before they ever leave your computer.
An example `.pre-commit-config.yaml` is provided (see [pre-commit.com](https://pre-commit.com/) for installation).

```yaml
repos:
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
language: system
stages: ["commit", "push"]
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
language: system
stages: ["commit", "push"]
```

# Regex Detector (alpha)
## Regex Detector (alpha)

Trufflehog supports detection and verification of custom regular expressions.
For detection, at least one **regular expression** and **keyword** is required.
Expand All @@ -343,17 +346,17 @@ status code, the secret is considered verified.
```yaml
# config.yaml
detectors:
- name: hog detector
keywords:
- hog
regex:
adjective: hogs are (\S+)
verify:
- endpoint: http://localhost:8000/
# unsafe must be set if the endpoint is HTTP
unsafe: true
headers:
- 'Authorization: super secret authorization header'
- name: hog detector
keywords:
- hog
regex:
adjective: hogs are (\S+)
verify:
- endpoint: http://localhost:8000/
# unsafe must be set if the endpoint is HTTP
unsafe: true
headers:
- "Authorization: super secret authorization header"
```

```
Expand Down Expand Up @@ -421,33 +424,29 @@ with HTTPServer(('', 8000), Verifier) as server:

This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)].


<a href="https://github.com/trufflesecurity/trufflehog/graphs/contributors">
<img src="https://contrib.rocks/image?repo=trufflesecurity/trufflehog" />
</a>


# :computer: Contributing

Contributions are very welcome! Please see our [contribution guidelines first](CONTRIBUTING.md).

We no longer accept contributions to TruffleHog v2, but that code is available in the `v2` branch.


## Adding new secret detectors

We have published some [documentation and tooling to get started on adding new secret detectors](hack/docs/Adding_Detectors_external.md). Let's improve detection together!


# Use as a library

Currently, trufflehog is in heavy development and no guarantees can be made on
the stability of the public APIs at this time.

# License Change

Since v3.0, TruffleHog is released under a AGPL 3 license, included in [`LICENSE`](LICENSE). TruffleHog v3.0 uses none of the previous codebase, but care was taken to preserve backwards compatibility on the command line interface. The work previous to this release is still available licensed under GPL 2.0 in the history of this repository and the previous package releases and tags. A completed CLA is required for us to accept contributions going forward.


# :money_with_wings: Enterprise product

Are you interested in continuously monitoring your Git, Jira, Slack, Confluence, etc.. for credentials? We have an enterprise product that can help. Reach out here to learn more https://trufflesecurity.com/contact/
Expand Down
41 changes: 16 additions & 25 deletions assets/scanning_logos.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 04f4268

Please sign in to comment.