Gist Scanning Errors

[joeleonjr]

TruffleHog Version

trufflehog 3.71.1

Trace Output

Trace back

Expected Behavior

Gists should be scanned just like any other repository on GitHub.

Actual Behavior

Depending on the Gist URL structure, one of two different errors spits out:

URLs with the username (https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git)

Failed to fetch repository	{"source_manager_worker_id": "Xwjd7", "repo": "https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git", "error": "GET https://api.github.com/repos/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb: 404 Not Found []"}

URLs without the username (https://gist.github.com/274463.git)

Unable to cache repository info	{"source_manager_worker_id": "ATqG1", "repo": "https://gist.github.com/274463.git", "error": "missing cached info for gist: https://gist.github.com/274463.git"}

Steps to Reproduce

Run the following commands:

trufflehog github --repo https://gist.github.com/274463.git
trufflehog github --repo https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git

Environment

• OS: OSX
• Version 14.2.1

Additional Context

I believe this was introduced in PR #2379 .

[rgmz]

This is caused by two faulty assumptions:

1. that Gist URLs only contain one path segment
2. that Gists wouldn't be scanned directly via the --repo flag
   

   trufflehog/pkg/sources/github/github.go

   Lines 429 to 437 in 55b3c1c

   	// Ignore any gists in |s.filteredRepoCache|.
   	// (Repos have three parts: [github.com, owner, name], gists have two.)
   	if len(urlParts) != 3 {
   	// Gists _should_ be cached elsewhere.
   	err = fmt.Errorf("missing cached info for gist: %s", r)
   	repoCtx.Logger().Error(err, "Unable to cache repository info")
   	continue RepoLoop
   	}

An obvious hot-fix would be to check whether the host is "gist.github.com", similar to what the existing code does elsewhere (prior code to #2379). However, this is a bad long-term solution as it won't work on GitHub Enterprise Server.
https://github.com/rgmz/trufflehog/blob/283d83f113ff8762c8c301d1f3de7767c67f02e2/pkg/sources/github/github.go#L1074-L1075

[joeleonjr]

Would it be reasonable to add a CLI flag just for gists?

[rgmz]

I think they are different enough that it would make sense. There's a lot of awkward code around mingling repositories and gists.

[JefriReynaldi]

TruffleHog Version

trufflehog 3.71.1

Trace Output

Trace back

Expected Behavior

Gists should be scanned just like any other repository on GitHub.

Actual Behavior

Depending on the Gist URL structure, one of two different errors spits out:

URLs with the username (https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git)

Failed to fetch repository	{"source_manager_worker_id": "Xwjd7", "repo": "https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git", "error": "GET https://api.github.com/repos/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb: 404 Not Found []"}

URLs without the username (https://gist.github.com/274463.git)

Unable to cache repository info	{"source_manager_worker_id": "ATqG1", "repo": "https://gist.github.com/274463.git", "error": "missing cached info for gist: https://gist.github.com/274463.git"}

Steps to Reproduce

Run the following commands:

trufflehog github --repo https://gist.github.com/274463.git trufflehog github --repo https://gist.github.com/raccoons-bot/627e15a45a596068ce8bfef3fd05ccdb.git

Environment

• OS: OSX
• Version 14.2.1

Additional Context

I believe this was introduced in PR #2379 .