Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add TravisCI source #1877

Merged
merged 8 commits into from
Oct 30, 2023
Merged

Add TravisCI source #1877

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
b960231
Add TravisCI source
dustin-decker Oct 9, 2023
c99e0b9
Merge branch 'main' of github.com:trufflesecurity/trufflehog into tra…
dustin-decker Oct 16, 2023
ec3c59d
update test to use sourcestest
dustin-decker Oct 16, 2023
e5a34a9
Remove jobPage loop
mcastorina Oct 16, 2023
fe90eff
Continue chunking on error
mcastorina Oct 16, 2023
065984e
review updates
dustin-decker Oct 18, 2023
628a55d
update readme
dustin-decker Oct 30, 2023
d707ef5
Merge branch 'main' of github.com:trufflesecurity/trufflehog into tra…
dustin-decker Oct 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 45 additions & 46 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

---

# :mag_right: _Now Scanning_
# :mag*right: \_Now Scanning*

<div align="center">

Expand Down Expand Up @@ -45,6 +45,7 @@ docker run --rm -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --or
# :floppy_disk: Installation

Several options available for you:

```bash
# MacOS users
brew install trufflesecurity/trufflehog/trufflehog
Expand Down Expand Up @@ -79,6 +80,7 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified
```

Expected output:

```
🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷

Expand Down Expand Up @@ -110,14 +112,14 @@ trufflehog git https://github.com/trufflesecurity/test_keys --only-verified --js
```

Expected output:

```
{"SourceMetadata":{"Data":{"Git":{"commit":"fbc14303ffbf8fb1c2c1914e8dda7d0121633aca","file":"keys","email":"counter \[email protected]\u003e","repository":"https://github.com/trufflesecurity/test_keys","timestamp":"2022-06-16 10:17:40 -0700 PDT","line":4}}},"SourceID":0,"SourceType":16,"SourceName":"trufflehog - git","DetectorType":2,"DetectorName":"AWS","DecoderName":"PLAIN","Verified":true,"Raw":"AKIAYVP4CIPPERUVIFXG","Redacted":"AKIAYVP4CIPPERUVIFXG","ExtraData":{"account":"595918472158","arn":"arn:aws:iam::595918472158:user/canarytokens.com@@mirux23ppyky6hx3l6vclmhnj","user_id":"AIDAYVP4CIPPJ5M54LRCY"},"StructuredData":null}
...
```

## 4: Scan a GitHub Repo + its Issues and Pull Requests.


```bash
trufflehog github --repo=https://github.com/trufflesecurity/test_keys --issue-comments --pr-comments
```
Expand Down Expand Up @@ -162,28 +164,27 @@ trufflehog docker --image trufflesecurity/secrets --only-verified

# :question: FAQ

+ All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives?
+ That means no secrets were detected
+ Why is the scan taking a long time when I scan a GitHub org
+ Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token
+ It says a private key was verified, what does that mean?
+ Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
+ Is there an easy way to ignore specific secrets?
+ If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets.

- All I see is `🐷🔑🐷 TruffleHog. Unearth your secrets. 🐷🔑🐷` and the program exits, what gives?
- That means no secrets were detected
- Why is the scan taking a long time when I scan a GitHub org
- Unauthenticated GitHub scans have rate limits. To improve your rate limits, include the `--token` flag with a personal access token
- It says a private key was verified, what does that mean?
- Check out our Driftwood blog post to learn how to do this, in short we've confirmed the key can be used live for SSH or SSL [Blog post](https://trufflesecurity.com/blog/driftwood-know-if-private-keys-are-sensitive/)
- Is there an easy way to ignore specific secrets?
- If the scanned source [supports line numbers](https://github.com/trufflesecurity/trufflehog/blob/d6375ba92172fd830abb4247cca15e3176448c5d/pkg/engine/engine.go#L358-L365), then you can add a `trufflehog:ignore` comment on the line containing the secret to ignore that secrets.

# :newspaper: What's new in v3?

TruffleHog v3 is a complete rewrite in Go with many new powerful features.

- We've **added over 700 credential detectors that support active verification against their respective APIs**.
- We've also added native **support for scanning GitHub, GitLab, filesystems, S3, GCS and Circle CI**.
- We've also added native **support for scanning GitHub, GitLab, Docker, filesystems, S3, GCS, Circle CI and Travis CI**.
- **Instantly verify private keys** against millions of github users and **billions** of TLS certificates using our [Driftwood](https://trufflesecurity.com/blog/driftwood) technology.
- Scan binaries and other file formats
- Scan binaries, documents, and other file formats
- Available as a GitHub Action and a pre-commit hook


## What is credential verification?

For every potential credential that is detected, we've painstakingly implemented programmatic verification against the API that we think it belongs to. Verification eliminates false positives. For example, the [AWS credential detector](pkg/detectors/aws/aws.go) performs a `GetCallerIdentity` API call against the AWS API to verify if an AWS credential is active.

# :memo: Usage
Expand All @@ -198,8 +199,8 @@ TruffleHog has a sub-command for each source of data that you may want to scan:
- filesystem (files and directories)
- syslog
- circleci
- travisci
- GCS (Google Cloud Storage)
- stdin (coming soon)

Each subcommand can have options that you can see with the `--help` flag provided to the sub command:

Expand Down Expand Up @@ -230,13 +231,13 @@ Args:
<uri> Git repository URL. https://, file://, or ssh:// schema expected.
```

For example, to scan a `git` repository, start with
For example, to scan a `git` repository, start with

```
$ trufflehog git https://github.com/trufflesecurity/trufflehog.git
```

## S3
## S3

The S3 source supports assuming IAM roles for scanning in addition to IAM users. This makes it easier for users to scan multiple AWS accounts without needing to rely on hardcoded credentials for each account.

Expand All @@ -261,11 +262,12 @@ trufflehog s3 --role-arn=<iam-role-arn-1> --role-arn=<iam-role-arn-2>
```

Exit Codes:

- 0: No errors and no results were found.
- 1: An error was encountered. Sources may not have completed scans.
- 183: No errors were encountered, but results were found. Will only be returned if `--fail` flag is used.

# :octocat: TruffleHog Github Action
## :octocat: TruffleHog Github Action

```yaml
- name: TruffleHog
Expand All @@ -285,6 +287,7 @@ The TruffleHog OSS Github Action can be used to scan a range of commits for leak
any results are found.

For example, to scan the contents of pull requests you could use the following workflow:

```yaml
name: TruffleHog Secrets Scan
on: [pull_request]
Expand All @@ -305,26 +308,26 @@ jobs:
extra_args: --debug --only-verified
```

# Pre-commit Hook
## Pre-commit Hook

Trufflehog can be used in a pre-commit hook to prevent credentials from leaking before they ever leave your computer.
An example `.pre-commit-config.yaml` is provided (see [pre-commit.com](https://pre-commit.com/) for installation).

```yaml
repos:
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
language: system
stages: ["commit", "push"]
- repo: local
hooks:
- id: trufflehog
name: TruffleHog
description: Detect secrets in your data.
entry: bash -c 'trufflehog git file://. --since-commit HEAD --only-verified --fail'
# For running trufflehog in docker, use the following entry instead:
# entry: bash -c 'docker run --rm -v "$(pwd):/workdir" -i --rm trufflesecurity/trufflehog:latest git file:///workdir --since-commit HEAD --only-verified --fail'
language: system
stages: ["commit", "push"]
```

# Regex Detector (alpha)
## Regex Detector (alpha)

Trufflehog supports detection and verification of custom regular expressions.
For detection, at least one **regular expression** and **keyword** is required.
Expand All @@ -343,17 +346,17 @@ status code, the secret is considered verified.
```yaml
# config.yaml
detectors:
- name: hog detector
keywords:
- hog
regex:
adjective: hogs are (\S+)
verify:
- endpoint: http://localhost:8000/
# unsafe must be set if the endpoint is HTTP
unsafe: true
headers:
- 'Authorization: super secret authorization header'
- name: hog detector
keywords:
- hog
regex:
adjective: hogs are (\S+)
verify:
- endpoint: http://localhost:8000/
# unsafe must be set if the endpoint is HTTP
unsafe: true
headers:
- "Authorization: super secret authorization header"
```

```
Expand Down Expand Up @@ -421,33 +424,29 @@ with HTTPServer(('', 8000), Verifier) as server:

This project exists thanks to all the people who contribute. [[Contribute](CONTRIBUTING.md)].


<a href="https://github.com/trufflesecurity/trufflehog/graphs/contributors">
<img src="https://contrib.rocks/image?repo=trufflesecurity/trufflehog" />
</a>


# :computer: Contributing

Contributions are very welcome! Please see our [contribution guidelines first](CONTRIBUTING.md).

We no longer accept contributions to TruffleHog v2, but that code is available in the `v2` branch.


## Adding new secret detectors

We have published some [documentation and tooling to get started on adding new secret detectors](hack/docs/Adding_Detectors_external.md). Let's improve detection together!


# Use as a library

Currently, trufflehog is in heavy development and no guarantees can be made on
the stability of the public APIs at this time.

# License Change

Since v3.0, TruffleHog is released under a AGPL 3 license, included in [`LICENSE`](LICENSE). TruffleHog v3.0 uses none of the previous codebase, but care was taken to preserve backwards compatibility on the command line interface. The work previous to this release is still available licensed under GPL 2.0 in the history of this repository and the previous package releases and tags. A completed CLA is required for us to accept contributions going forward.


# :money_with_wings: Enterprise product

Are you interested in continuously monitoring your Git, Jira, Slack, Confluence, etc.. for credentials? We have an enterprise product that can help. Reach out here to learn more https://trufflesecurity.com/contact/
Expand Down
41 changes: 16 additions & 25 deletions assets/scanning_logos.svg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 1 addition & 8 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ require (
github.com/prometheus/client_golang v1.17.0
github.com/rabbitmq/amqp091-go v1.9.0
github.com/sergi/go-diff v1.3.1
github.com/shuheiktgw/go-travis v0.3.1
github.com/snowflakedb/gosnowflake v1.6.25
github.com/stretchr/testify v1.8.4
github.com/tailscale/depaware v0.0.0-20210622194025-720c4b409502
Expand Down Expand Up @@ -132,14 +133,12 @@ require (
github.com/aws/smithy-go v1.13.5 // indirect
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
github.com/aymerick/douceur v0.2.0 // indirect
github.com/benbjohnson/clock v1.1.0 // indirect
github.com/beorn7/perks v1.0.1 // indirect
github.com/bodgit/plumbing v1.3.0 // indirect
github.com/bodgit/sevenzip v1.4.3 // indirect
github.com/bodgit/windows v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cloudflare/circl v1.3.3 // indirect
github.com/connesc/cipherio v0.2.1 // indirect
github.com/containerd/console v1.0.4-0.20230313162750-1ae8d489ac81 // indirect
github.com/containerd/stargz-snapshotter/estargz v0.14.3 // indirect
github.com/couchbase/gocbcore/v10 v10.2.9 // indirect
Expand Down Expand Up @@ -170,7 +169,6 @@ require (
github.com/golang/protobuf v1.5.3 // indirect
github.com/golang/snappy v0.0.4 // indirect
github.com/google/flatbuffers v23.1.21+incompatible // indirect
github.com/google/go-github/v55 v55.0.0 // indirect
github.com/google/go-github/v56 v56.0.0 // indirect
github.com/google/go-querystring v1.1.0 // indirect
github.com/google/pprof v0.0.0-20211214055906-6f57359322fd // indirect
Expand All @@ -184,7 +182,6 @@ require (
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/imdario/mergo v0.3.15 // indirect
github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/josharian/intern v1.0.0 // indirect
Expand Down Expand Up @@ -219,7 +216,6 @@ require (
github.com/nwaples/rardecode/v2 v2.0.0-beta.2 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/onsi/ginkgo v1.16.5 // indirect
github.com/onsi/gomega v1.27.10 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0-rc3 // indirect
github.com/pierrec/lz4/v4 v4.1.18 // indirect
Expand All @@ -231,7 +227,6 @@ require (
github.com/prometheus/common v0.44.0 // indirect
github.com/prometheus/procfs v0.11.1 // indirect
github.com/rivo/uniseg v0.4.2 // indirect
github.com/rogpeppe/go-internal v1.11.0 // indirect
github.com/sahilm/fuzzy v0.1.0 // indirect
github.com/sirupsen/logrus v1.9.1 // indirect
github.com/skeema/knownhosts v1.2.0 // indirect
Expand All @@ -249,7 +244,6 @@ require (
github.com/zeebo/xxh3 v1.0.2 // indirect
go.einride.tech/aip v0.60.0 // indirect
go.opencensus.io v0.24.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.10.0 // indirect
go4.org v0.0.0-20200411211856-f5505b9728dd // indirect
golang.org/x/mod v0.13.0 // indirect
Expand All @@ -266,6 +260,5 @@ require (
google.golang.org/grpc v1.59.0 // indirect
gopkg.in/square/go-jose.v2 v2.6.0 // indirect
gopkg.in/warnings.v0 v0.1.2 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading
Loading