Decouple dependencies of tools/ and the rest of the codebase

[phoracek]

What this PR does / why we need it:

This PR introduces a dedicated subpackage for tools used in the project. This allows us to shed much of the vendored dependencies that are tied to the production code.

This is important, because it allows us to quickly evaluate whether any reported CVE/CWE affects the production code. Decoupling the dependencies also makes is easier to keep them up to date. Finally, making it clear which dependencies are used by the production code enables us to audit the list and decide whether we want to remove any of the dependencies.

Number of go lines vendored by the production code was cut by 96 %, from 384015 to 14460. Checked with cd vendor && find . -name '*.go' | xargs wc -l.

Number of dependencies (including indirect) of the production code was cut by 30 %, from 806 to 566. Checked with cat go.sum | awk '{print $1}' | sort | uniq | wc -l.

The list of dependencies we were able to shed off is here: https://gist.github.com/phoracek/aafc6cae0275291117b0c13e94c48e66

Special notes for your reviewer:

Review the PR commit by commit. The last commit contains all the vendoring and go.sum updates.

Release note:

NONE

[kubevirt-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from phoracek. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[RamLavi]

This is a preparation step before making tooks/ its own module.

typo in commit msg: tooks/ --> tools

[phoracek]

@oshoval don't bother reviewing it, it's WIP

[sonarcloud]

Quality Gate failed

Failed conditions
7 Security Hotspots

See analysis details on SonarCloud

[oshoval]

IPAM lane started to fail on random tests, not related to this PR i believe,
always at "Waiting for readiness at virtual machine alpine..." stage,
git actions is low on resources, so it might give best effort resources,
for some reason it doesnt collect the artifacts that should also be solved and would able to hint about the problem

The teardown from glance looks fine after each test.

[oshoval]

nit: in order to exclude /vendor/ folders that might exists in the future
git ls-files | grep -vE '^vendor/|/vendor/'
or maybe also
git ls-files -- ':!vendor' ':!**/vendor/**'
but just nice to have please, it can wait for once we have new vendor folders

[oshoval]

Very nice, thank you
/lgtm

We can just ignore failures on the IPAM for this PR imo, and fix it soon
checking here main branch state #1880
(fails there as well)

also sonar can be ignored please

[RamLavi]

/lgtm

[oshoval]

Can you please rebase this PR ?
(maybe once 1882 is merged so ipam flakes will be solved as well)

Git actions doesn't auto rebase, it is always good please to rebase manually just before merge
(we added a github config that should block merging when it is not the case, but it seems it doesn't work perfectly, need to revisit it on relevant repos)
We can also make git actions rebase once running (but it is not a bullet proof, because it is only when running, not also when ref HEAD was changed)
Btw this is one of the things prow is better than git actions imho,
moreover all those flakes, i wish we would move to prow on CNAO IPAM,
the drawback is that we can't use kind 0.20+ / k8s 1.29+ yet on prow, so it can be considered blocker

[oshoval]

#1882
EDIT - merged

should fix the ipam flakiness, at least some of them are due to that (will run few times)
ipam artifacts collecting stopped working because of this i believe
https://github.blog/changelog/2024-08-19-notice-of-upcoming-deprecations-and-breaking-changes-in-github-actions-runners/
(the folder we are on is .output which is hidden)
Will fix it soon

[kubevirt-bot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[RamLavi]

@phoracek sorry for conflicting with your PR, can you re-copy the conflicted files to the new location?

[kubevirt-bot]

Pull requests that are marked with lgtm should receive a review
from an approver within 1 week.

After that period the bot marks them with the label needs-approver-review.

/label needs-approver-review