-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch 'main' of github.com:trufflesecurity/trufflehog into fix…
…-aws-session
- Loading branch information
Showing
121 changed files
with
8,147 additions
and
4,378 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
{ | ||
"$schema": "https://docs.renovatebot.com/renovate-schema.json", | ||
"extends": [ | ||
"config:base" | ||
], | ||
"prConcurrentLimit": 3, | ||
"prHourlyLimit": 2 | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -16,7 +16,7 @@ | |
|
||
--- | ||
|
||
# :mag_right: _Now Scanning_ | ||
# :mag*right: \_Now Scanning* | ||
|
||
<div align="center"> | ||
|
||
|
@@ -45,6 +45,7 @@ docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --or | |
# :floppy_disk: Installation | ||
|
||
Several options available for you: | ||
|
||
```bash | ||
# MacOS users | ||
brew install trufflesecurity/trufflehog/trufflehog | ||
|
@@ -79,6 +80,7 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified | |
``` | ||
|
||
Expected output: | ||
|
||
``` | ||
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷 | ||
|
@@ -110,14 +112,14 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified --js | |
``` | ||
|
||
Expected output: | ||
|
||
``` | ||
{"SourceMetadata":{"Data":{"Git":{"commit":"fbc14303ffbf8fb1c2c1914e8dda7d0121633aca","file":"keys","email":"counter \[email protected]\u003e","repository":"https://github.com/trufflesecurity/test_keys","timestamp":"2022-06-16 10:17:40 -0700 PDT","line":4}}},"SourceID":0,"SourceType":16,"SourceName":"trufflehog - git","DetectorType":2,"DetectorName":"AWS","DecoderName":"PLAIN","Verified":true,"Raw":"AKIAYVP4CIPPERUVIFXG","Redacted":"AKIAYVP4CIPPERUVIFXG","ExtraData":{"account":"595918472158","arn":"arn:aws:iam::595918472158:user/canarytokens.com@@mirux23ppyky6hx3l6vclmhnj","user_id":"AIDAYVP4CIPPJ5M54LRCY"},"StructuredData":null} | ||
... | ||
``` | ||
|
||
## 4: Scan a GitHub Repo + its Issues and Pull Requests. | ||
|
||
|
||
```bash | ||
trufflehog github --repo=https://github.com/trufflesecurity/test_keys --issue-comments --pr-comments | ||
``` | ||
|
@@ -162,28 +164,27 @@ trufflehog docker --image trufflesecurity/secrets --only-verified | |
|
||
# :question: FAQ | ||
|
||
+ All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives? | ||
+ That means no secrets were detected | ||
+ Why is the scan taking a long time when I scan a GitHub org | ||
+ Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token | ||
+ It says a private key was verified, what does that mean? | ||
+ Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/) | ||
+ Is there an easy way to ignore specific secrets? | ||
+ If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets. | ||
|
||
- All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives? | ||
- That means no secrets were detected | ||
- Why is the scan taking a long time when I scan a GitHub org | ||
- Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token | ||
- It says a private key was verified, what does that mean? | ||
- Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/) | ||
- Is there an easy way to ignore specific secrets? | ||
- If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets. | ||
|
||
# :newspaper: What's new in v3? | ||
|
||
TruffleHog v3 is a complete rewrite in Go with many new powerful features. | ||
|
||
- We've **added over 700 credential detectors that support active verification against their respective APIs**. | ||
- We've also added native **support for scanning GitHub, GitLab, filesystems, S3, GCS and Circle CI**. | ||
- We've also added native **support for scanning GitHub, GitLab, Docker, filesystems, S3, GCS, Circle CI and Travis CI**. | ||
- **Instantly verify private keys** against millions of github users and **billions** of TLS certificates using our [Driftwood](https://trufflesecurity.com/blog/driftwood) technology. | ||
- Scan binaries and other file formats | ||
- Scan binaries, documents, and other file formats | ||
- Available as a GitHub Action and a pre-commit hook | ||
|
||
|
||
## What is credential verification? | ||
|
||
For every potential credential that is detected, we've painstakingly implemented programmatic verification against the API that we think it belongs to. Verification eliminates false positives. For example, the [AWS credential detector](pkg/detectors/aws/aws.go) performs a `GetCallerIdentity` API call against the AWS API to verify if an AWS credential is active. | ||
|
||
# :memo: Usage | ||
|
@@ -198,8 +199,8 @@ TruffleHog has a sub-command for each source of data that you may want to scan: | |
- filesystem (files and directories) | ||
- syslog | ||
- circleci | ||
- travisci | ||
- GCS (Google Cloud Storage) | ||
- stdin (coming soon) | ||
|
||
Each subcommand can have options that you can see with the `--help` flag provided to the sub command: | ||
|
||
|
@@ -230,13 +231,13 @@ Args: | |
<uri> Git repository URL. https://, file://, or ssh:// schema expected. | ||
``` | ||
|
||
For example, to scan a `git` repository, start with | ||
For example, to scan a `git` repository, start with | ||
|
||
``` | ||
$ trufflehog git https://github.com/trufflesecurity/trufflehog.git | ||
``` | ||
|
||
## S3 | ||
## S3 | ||
|
||
The S3 source supports assuming IAM roles for scanning in addition to IAM users. This makes it easier for users to scan multiple AWS accounts without needing to rely on hardcoded credentials for each account. | ||
|
||
|
@@ -261,11 +262,12 @@ trufflehog s3 --role-arn=<iam-role-arn-1> --role-arn=<iam-role-arn-2> | |
``` | ||
|
||
Exit Codes: | ||
|
||
- 0: No errors and no results were found. | ||
- 1: An error was encountered. Sources may not have completed scans. | ||
- 183: No errors were encountered, but results were found. Will only be returned if `--fail` flag is used. | ||
|
||
# :octocat: TruffleHog Github Action | ||
## :octocat: TruffleHog Github Action | ||
|
||
```yaml | ||
- name: TruffleHog | ||
|
@@ -285,6 +287,7 @@ The TruffleHog OSS Github Action can be used to scan a range of commits for leak | |
any results are found. | ||
For example, to scan the contents of pull requests you could use the following workflow: | ||
```yaml | ||
name: TruffleHog Secrets Scan | ||
on: [pull_request] | ||
|
@@ -305,26 +308,26 @@ jobs: | |
extra_args: --debug --only-verified | ||
``` | ||
# Pre-commit Hook | ||
## Pre-commit Hook | ||
Trufflehog can be used in a pre-commit hook to prevent credentials from leaking before they ever leave your computer. | ||
An example `.pre-commit-config.yaml` is provided (see [pre-commit.com](https://pre-commit.com/) for installation). | ||
|
||
```yaml | ||
repos: | ||
- repo: local | ||
hooks: | ||
- id: trufflehog | ||
name: TruffleHog | ||
description: Detect secrets in your data. | ||
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail' | ||
# For running trufflehog in docker, use the following entry instead: | ||
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail' | ||
language: system | ||
stages: ["commit", "push"] | ||
- repo: local | ||
hooks: | ||
- id: trufflehog | ||
name: TruffleHog | ||
description: Detect secrets in your data. | ||
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail' | ||
# For running trufflehog in docker, use the following entry instead: | ||
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail' | ||
language: system | ||
stages: ["commit", "push"] | ||
``` | ||
|
||
# Regex Detector (alpha) | ||
## Regex Detector (alpha) | ||
|
||
Trufflehog supports detection and verification of custom regular expressions. | ||
For detection, at least one **regular expression** and **keyword** is required. | ||
|
@@ -343,17 +346,17 @@ status code, the secret is considered verified. | |
```yaml | ||
# config.yaml | ||
detectors: | ||
- name: hog detector | ||
keywords: | ||
- hog | ||
regex: | ||
adjective: hogs are (\S+) | ||
verify: | ||
- endpoint: http://localhost:8000/ | ||
# unsafe must be set if the endpoint is HTTP | ||
unsafe: true | ||
headers: | ||
- 'Authorization: super secret authorization header' | ||
- name: hog detector | ||
keywords: | ||
- hog | ||
regex: | ||
adjective: hogs are (\S+) | ||
verify: | ||
- endpoint: http://localhost:8000/ | ||
# unsafe must be set if the endpoint is HTTP | ||
unsafe: true | ||
headers: | ||
- "Authorization: super secret authorization header" | ||
``` | ||
|
||
``` | ||
|
@@ -421,33 +424,29 @@ with HTTPServer(('', 8000), Verifier) as server: | |
|
||
This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)]. | ||
|
||
|
||
<a href="https://github.com/trufflesecurity/trufflehog/graphs/contributors"> | ||
<img src="https://contrib.rocks/image?repo=trufflesecurity/trufflehog" /> | ||
</a> | ||
|
||
|
||
# :computer: Contributing | ||
|
||
Contributions are very welcome! Please see our [contribution guidelines first](CONTRIBUTING.md). | ||
|
||
We no longer accept contributions to TruffleHog v2, but that code is available in the `v2` branch. | ||
|
||
|
||
## Adding new secret detectors | ||
|
||
We have published some [documentation and tooling to get started on adding new secret detectors](hack/docs/Adding_Detectors_external.md). Let's improve detection together! | ||
|
||
|
||
# Use as a library | ||
|
||
Currently, trufflehog is in heavy development and no guarantees can be made on | ||
the stability of the public APIs at this time. | ||
|
||
# License Change | ||
|
||
Since v3.0, TruffleHog is released under a AGPL 3 license, included in [`LICENSE`](LICENSE). TruffleHog v3.0 uses none of the previous codebase, but care was taken to preserve backwards compatibility on the command line interface. The work previous to this release is still available licensed under GPL 2.0 in the history of this repository and the previous package releases and tags. A completed CLA is required for us to accept contributions going forward. | ||
|
||
|
||
# :money_with_wings: Enterprise product | ||
|
||
Are you interested in continuously monitoring your Git, Jira, Slack, Confluence, etc.. for credentials? We have an enterprise product that can help. Reach out here to learn more https://trufflesecurity.com/contact/ | ||
|
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Oops, something went wrong.