feat(kustomize): Add origin annotations to calculate bases of kustomize checks #5298
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…g us to copy overlays to wrong directory
bo156
force-pushed
the
feature/kustomize-origin-annotations
branch
from
10e9411
to
c572d37
Compare
gruebel
reviewed
Jul 6, 2023
checkov/kustomize/runner.py
Outdated
Comment on lines
521
to
527
| proc = subprocess.Popen(full_command.split(' '), cwd=filePath, stdout=subprocess.PIPE, stderr=subprocess.PIPE) # nosec | ||
| output, _ = proc.communicate() | ||
|
|
||
| if env_vars_config.ALLOW_KUSTOMIZE_FILE_EDITS and add_origin_annotations_return_code == 0: | ||
| # If the return code is not 0, we didn't add the new buildmetadata field, so we shouldn't remove it | ||
| remove_origin_annotaions = 'kustomize edit remove buildmetadata originAnnotations' | ||
| subprocess.Popen(remove_origin_annotaions.split(' '), cwd=filePath).wait() # nosec |
There was a problem hiding this comment.
not sure, why we use Popen better to use run instead as recommend by Python, at least for the second call we can switch to run
There was a problem hiding this comment.
tbh I just used popen as that's what the function already used. will do it now
Co-authored-by: Anton Grübel <[email protected]>
Co-authored-by: Anton Grübel <[email protected]>
checkov/kustomize/runner.py
Outdated
| resource=kustomizeResourceID, evaluations=variable_evaluations, | ||
| check_class=check.__class__.__module__, file_abs_path=absolute_file_path, severity=check.severity) | ||
| record.set_guideline(check.guideline) | ||
| report.add_record(record=record) | ||
|
|
||
| return report | ||
|
|
||
| def _get_caller_file_info(self, entity_context: _EntityContext, k8_file: str, k8_file_path: str, resource_id: str, | ||
| root_folder: str | None) -> tuple[tuple[int, int] | None, str | None]: | ||
| origin_relative_path = entity_context['origin_relative_path'] |
There was a problem hiding this comment.
Suggested change
| origin_relative_path = entity_context['origin_relative_path'] | |
| origin_relative_path = entity_context.get('origin_relative_path') |
lirshindalman
approved these changes
Jul 10, 2023
bo156
force-pushed
the
feature/kustomize-origin-annotations
branch
from
b9c8999
to
e5c71a5
Compare
gruebel
added a commit
that referenced
this pull request
Jul 10, 2023
…uild github action to use github runners (#5316) * feat(kustomize): Add origin annotations to calculate bases of kustomize checks (#5298) * Added basic implementation of adding and removing buildmetadata originAnnotations * Only remove the new buildmetadata if it wasn't added successfully by checkov * middle of debugging * more improvements * Successfully updated the file path * middle of adding correct base code lines * Fixed calculation of caller file path * another debug way * Another addition * Fixed bug where we calculate the base for overlay incorrectly, causing us to copy overlays to wrong directory * Calculated relative directory to new base * Allowed empty caller file lines and fixed all kustomize tests * Added tests for caller file paths * Added support for caller file path in graph reports and updated tests accordingly * Fixed parsing of kustomization files with resources which actually represent overlays * Added env variable to allow kustomize file edits * Removed comment * Updated _get_caller_file_path signature * Linters * nosec on subprocess.Popen as we are incharge of the command * Added condition that annotations are not None * CR * find correct parent without loop Co-authored-by: Anton Grübel <[email protected]> * don't check len Co-authored-by: Anton Grübel <[email protected]> * Used subprocess.run * moved env var to another place in the instance * Fixed all tests: * Check for origin_relative_path before concat * Added documentation --------- Co-authored-by: Anton Grübel <[email protected]> * Moved build job unit tests to use public images instead of self-hosted, as a bug exist in the hosted version --------- Co-authored-by: Anton Grübel <[email protected]>
Teko012
pushed a commit
to Teko012/checkov_fork_bridgecrewio
that referenced
this pull request
Jul 11, 2023
…ases of… (bridgecrewio#5312) Revert "feat(kustomize): Add origin annotations to calculate bases of kustomize checks (bridgecrewio#5298)" This reverts commit 40e3626.
Teko012
pushed a commit
to Teko012/checkov_fork_bridgecrewio
that referenced
this pull request
Jul 11, 2023
…uild github action to use github runners (bridgecrewio#5316) * feat(kustomize): Add origin annotations to calculate bases of kustomize checks (bridgecrewio#5298) * Added basic implementation of adding and removing buildmetadata originAnnotations * Only remove the new buildmetadata if it wasn't added successfully by checkov * middle of debugging * more improvements * Successfully updated the file path * middle of adding correct base code lines * Fixed calculation of caller file path * another debug way * Another addition * Fixed bug where we calculate the base for overlay incorrectly, causing us to copy overlays to wrong directory * Calculated relative directory to new base * Allowed empty caller file lines and fixed all kustomize tests * Added tests for caller file paths * Added support for caller file path in graph reports and updated tests accordingly * Fixed parsing of kustomization files with resources which actually represent overlays * Added env variable to allow kustomize file edits * Removed comment * Updated _get_caller_file_path signature * Linters * nosec on subprocess.Popen as we are incharge of the command * Added condition that annotations are not None * CR * find correct parent without loop Co-authored-by: Anton Grübel <[email protected]> * don't check len Co-authored-by: Anton Grübel <[email protected]> * Used subprocess.run * moved env var to another place in the instance * Fixed all tests: * Check for origin_relative_path before concat * Added documentation --------- Co-authored-by: Anton Grübel <[email protected]> * Moved build job unit tests to use public images instead of self-hosted, as a bug exist in the hosted version --------- Co-authored-by: Anton Grübel <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
Description
This PR allows adding
buildMetadata: [originAnnotations]tokustomization.yamlfiles, which enables calculating the origin base which created the resource when running kustomize checks.This feature edits the kustomization.yaml files for the user, so it is hidden behind the env variable
ALLOW_KUSTOMIZE_FILE_EDITSChecklist: