Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Github Oauth2 verification #1584

Merged
merged 6 commits into from
Aug 2, 2023
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions pkg/detectors/github_oauth2/github_oauth2.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
package github_oauth2

import (
"context"
"regexp"
"strings"

"github.com/trufflesecurity/trufflehog/v3/pkg/detectors"
"github.com/trufflesecurity/trufflehog/v3/pkg/pb/detectorspb"
"golang.org/x/oauth2/clientcredentials"
"golang.org/x/oauth2/github"
)

type Scanner struct{ detectors.EndpointSetter }

// Ensure the Scanner satisfies the interfaces at compile time.
var _ detectors.Detector = (*Scanner)(nil)
var _ detectors.EndpointCustomizer = (*Scanner)(nil)

func (Scanner) DefaultEndpoint() string { return "https://api.github.com" }
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit (non-blocking): It doesn't look like we're using this endpoint customizer either. I think that can be added in another PR though.


var (
// Oauth2 client ID and secret
oauth2ClientIDPat = regexp.MustCompile(detectors.PrefixRegex([]string{"github"}) + `\b([a-f0-9]{20})\b`)
oauth2ClientSecretPat = regexp.MustCompile(detectors.PrefixRegex([]string{"github"}) + `\b([a-f0-9]{40})\b`)
)

const (
githubBadVerificationCodeError = "bad_verification_code"
)

// Keywords are used for efficiently pre-filtering chunks.
// Use identifiers in the secret preferably, or the provider name.
func (s Scanner) Keywords() []string {
return []string{"github"}
}

// FromData will find and optionally verify GitHub secrets in a given set of bytes.
func (s Scanner) FromData(ctx context.Context, verify bool, data []byte) (results []detectors.Result, err error) {
dataStr := string(data)

// Oauth2 client ID and secret
oauth2ClientIDMatches := oauth2ClientIDPat.FindAllStringSubmatch(dataStr, -1)
oauth2ClientSecretMatches := oauth2ClientSecretPat.FindAllStringSubmatch(dataStr, -1)

for _, idMatch := range oauth2ClientIDMatches {
if len(idMatch) != 2 {
continue
}
for _, secretMatch := range oauth2ClientSecretMatches {
if len(secretMatch) != 2 {
continue
}

s1 := detectors.Result{
DetectorType: detectorspb.DetectorType_GitHubOauth2,
Raw: []byte(idMatch[1]),
RawV2: []byte(idMatch[1] + secretMatch[1]),
Comment on lines +54 to +55
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can make Raw: []byte(idMatch[1] + secretMatch[1]) and filling omit RawV2
The only reason why RawV2 exists is because we did not capture all of the components of some credentials the first time in the Raw field.

cc @mcastorina for your understanding as well

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't that cause confusion in OSS where the raw value is shown?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, you're right, I didn't consider that aspect. Keep it how it is then.

}

config := &clientcredentials.Config{
ClientID: idMatch[1],
ClientSecret: secretMatch[1],
TokenURL: github.Endpoint.TokenURL,
}
_, err := config.Token(ctx)
if err != nil && strings.Contains(err.Error(), githubBadVerificationCodeError) {
s1.Verified = true
}

if !s1.Verified && detectors.IsKnownFalsePositive(string(s1.Raw), detectors.DefaultFalsePositives, true) {
continue
}

results = append(results, s1)
}
}

return
}

func (s Scanner) Type() detectorspb.DetectorType {
return detectorspb.DetectorType_GitHubOauth2
}
2 changes: 2 additions & 0 deletions pkg/engine/defaults.go
Original file line number Diff line number Diff line change
Expand Up @@ -556,6 +556,7 @@ import (
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/satismeterwritekey"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/saucelabs"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/scalewaykey"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/github_oauth2"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/scalr"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/scrapeowl"
"github.com/trufflesecurity/trufflehog/v3/pkg/detectors/scraperapi"
Expand Down Expand Up @@ -1530,6 +1531,7 @@ func DefaultDetectors() []detectors.Detector {
dockerhub.Scanner{},
couchbase.Scanner{},
envoyapikey.Scanner{},
github_oauth2.Scanner{},
}

}
Expand Down
16 changes: 10 additions & 6 deletions pkg/pb/detectorspb/detectors.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions proto/detectors.proto
Original file line number Diff line number Diff line change
Expand Up @@ -932,6 +932,7 @@ enum DetectorType {
Dockerhub = 921;
TrufflehogEnterprise = 922;
EnvoyApiKey = 923;
GitHubOauth2 = 924;
}

message Result {
Expand Down
Loading