Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #1227 - Get licenses for NuGet packages #3329

Open
wants to merge 47 commits into
base: main
Choose a base branch
from

Conversation

HeyeOpenSource
Copy link

@HeyeOpenSource HeyeOpenSource commented Oct 14, 2024

Description

Type of change

  • New feature (non-breaking change which adds functionality)

Checklist:

  • I have added unit tests that cover changed behavior
  • I have tested my code in common scenarios and confirmed there are no regressions
  • I have added comments to my code, particularly in hard-to-understand sections

@spiffcs spiffcs self-requested a review October 14, 2024 15:41
@spiffcs spiffcs self-assigned this Oct 14, 2024
HeyeOpenSource and others added 14 commits October 15, 2024 09:03
Signed-off-by: HeyeOpenSource <[email protected]>
Improves the go cataloger semver extraction logic to include getting the
release version of traefik.  This is based off of the regex pattern that
already existed in the traefik binary classifier.

Signed-off-by: Weston Steimel <[email protected]>
Signed-off-by: HeyeOpenSource <[email protected]>
Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: wagoodman <[email protected]>
Signed-off-by: HeyeOpenSource <[email protected]>
…options dotnetConfig struct.

Signed-off-by: HeyeOpenSource <[email protected]>
…re#3327)

Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.26.12 to 3.26.13.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](github/codeql-action@c36620d...f779452)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: HeyeOpenSource <[email protected]>
…3326)

Bumps [anchore/sbom-action](https://github.com/anchore/sbom-action) from 0.17.2 to 0.17.3.
- [Release notes](https://github.com/anchore/sbom-action/releases)
- [Changelog](https://github.com/anchore/sbom-action/blob/main/RELEASE.md)
- [Commits](anchore/sbom-action@61119d4...f5e124a)

---
updated-dependencies:
- dependency-name: anchore/sbom-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: HeyeOpenSource <[email protected]>
…09e9e5 (anchore#3331)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: willmurphyscode <[email protected]>
Signed-off-by: HeyeOpenSource <[email protected]>
…870434 (anchore#3332)

Signed-off-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: willmurphyscode <[email protected]>
Signed-off-by: HeyeOpenSource <[email protected]>
…rs() function in syft/pkg/cataloger/dotnet.

Signed-off-by: HeyeOpenSource <[email protected]>
@HeyeOpenSource
Copy link
Author

HeyeOpenSource commented Oct 16, 2024

Just for the record:
I oriented myself at the golang cataloger.

The configuration can also be influenced by the following four environment variables:

  • NUGET_SEARCH_LOCAL_LICENSES or SYFT_DOTNET_SEARCH_LOCAL_LICENSES
    • 'true' | 'false'
    • Search for NuGet packages in all known local cache directories.
  • NUGET_SEARCH_REMOTE_LICENSES or SYFT_DOTNET_SEARCH_REMOTE_LICENSES
    • 'true' | 'false'
    • Search for NuGet packages in all enabled known remote NuGet package repositories eg. https://api.nuget.org/v3-flatcontainer/ as retrieved from https://api.nuget.org/v3/index.json (unless the NuGet package providers setting is overridden)
  • NUGET_PACKAGE_PROVIDERS or SYFT_DOTNET_PACKAGE_PROVIDERS
    • Expects a comma-separated lists of the URLs of remote NuGet package repositories to use when searching for remote NuGet packages / licenses.
    • Defaults to all enabled known remote NuGet repositories if undefined.
  • NUGET_PACKAGE_PROVIDER_CREDENTIALS or SYFT_DOTNET_PACKAGE_PROVIDER_CREDENTIALS
    • Expects a comma-separated lists of the credentials (<username>:<password>) for remote NuGet package repositories to use when searching for remote NuGet packages / licenses in a package repository requiring authentication.

…age provider URLs terminated by '/'.

Signed-off-by: HeyeOpenSource <[email protected]>
…hen accessing remote NuGet package repositories.

Signed-off-by: HeyeOpenSource <[email protected]>
  - Fix remote NuGet license retrieval.
  - Allow for NuGet package retrieval from package repositories requiring authentication.

Signed-off-by: HeyeOpenSource <[email protected]>
@HeyeOpenSource
Copy link
Author

FYI:
The feature for defining credentials should mostly be applicable to commercial CI/CD environments,
where access restriction is required by appropriate data security requirements.

Gitea for example allows to create such code- and NuGet package repositories.

@HeyeOpenSource HeyeOpenSource marked this pull request as ready for review October 18, 2024 10:02
@HeyeOpenSource HeyeOpenSource marked this pull request as draft October 22, 2024 11:20
@HeyeOpenSource
Copy link
Author

Switched back to draft due to commit by @wagoodman
Create single license scanner for all catalogers (#3348)

@HeyeOpenSource
Copy link
Author

Since make test did not throw any errors, I feel free to return this pull request back to Ready for review. 😄

@HeyeOpenSource HeyeOpenSource marked this pull request as ready for review October 22, 2024 12:19
@HeyeOpenSource
Copy link
Author

The Validations workflow finally throws no more errors at me. 👍
cf. Validations #17

Copy link
Contributor

@kzantow kzantow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @HeyeOpenSource, thanks very much for this PR! Generally speaking, it looks like you've followed the same patterns used elsewhere and this change looks really good. There are a couple specific things I'd like to note:

  • instead of adding a bunch of binary files for the tests (including unneeded images and such), could we build these to a test docker image? for example Java runs some builds for test artifacts
  • it looks as though there are a number of things reliant on the dotnet executable; can we get this information without it? if we could, this will definitely help move this PR forward more quickly
  • environment variable usage should be cleaned up as noted in more specific comments

syft/pkg/cataloger/dotnet/config.go Outdated Show resolved Hide resolved
cmd/syft/internal/options/dotnet.go Outdated Show resolved Hide resolved
syft/pkg/cataloger/dotnet/config.go Show resolved Hide resolved
cmd/syft/internal/options/catalog.go Outdated Show resolved Hide resolved
syft/pkg/cataloger/dotnet/config.go Show resolved Hide resolved
syft/pkg/cataloger/dotnet/config.go Show resolved Hide resolved
syft/pkg/cataloger/dotnet/licenses.go Outdated Show resolved Hide resolved
syft/pkg/cataloger/dotnet/parse_dotnet_deps_test.go Outdated Show resolved Hide resolved
@HeyeOpenSource
Copy link
Author

After tackling all review conversations, the Validations action is all green once more:
Validations #27

I have removed any artifacts, which are not strictly neccessary.
As you correctly stated it does not really make sense to have obsolete images and binaries clogging up the repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: In Review
Development

Successfully merging this pull request may close these issues.

Get licenses for NuGet packages
4 participants