SARIF support for trivy (#1071)

* [automation] Auto-update linters version, help and documentation (#1038) * Implement new linter PHPLint (#1037) * add new phplint linter (see #1031) * use composer global install for overture/phplint * update changelog * fix feature #1043 (#1044) * Add @babel/eslint-parser as dependency for eslint (#1045) * [automation] Auto-update linters version, help and documentation (#1046) * [automation] Auto-update linters version, help and documentation + @babel-core * Add @babel-core dependency * Use node.js LTS and not nodejs-current * Downgrade v8r version * Fix arm test case * Fix again arm test case * [automation] Auto-update linters version, help and documentation * Build * [automation] Auto-update linters version, help and documentation (#1050) * [automation] Auto-update linters version, help and documentation (#1052) * [automation] Auto-update linters version, help and documentation * Update doc for gitlab MegaLinter config * Bandit default config file (#1051) * remove default parameters * add default bandit configfile built with bandit-config-generator command * update changelog * update bandit doc page Co-authored-by: Nicolas Vuillamy <[email protected]> * Try again ktlint auto-upgrade (#976) * [automation] Auto-update linters version, help and documentation (#1055) * [automation] Auto-update linters version, help and documentation (#1056) * Docs & links (#1058) * Update stale config * Fix getting linter version of npm plugin. (#804) * [automation] Auto-update linters version, help and documentation (#1059) * Fix doc * [automation] Auto-update linters version, help and documentation (#1065) * [automation] Auto-update linters version, help and documentation (#1067) * SARIF for gitleaks * Sarif python test class v0 * Lint fix + changelog * SARIF support for trivy * Accelerate and fix internal tests * Fix SARIF files * Fix trivy sarif args * [automation] Auto-update linters version, help and documentation (#1072) * Fix SarifReporter * Better handling of SARIF error management * Fix SARIF default file name use quick build * Fix sarif reporter exception management * Variabilize default release * Improve runtime perfs when FLAVOR_SUGGESTIONS: false (#1073) * Improve runtime perfs when FLAVOR_SUGGESTIONS: false * [MegaLinter] Apply linters fixes * Fix how to get MEGALINTER_FLAVOR * Fix ignore flavor suggestion test class Co-authored-by: nvuillam <[email protected]> * Build Co-authored-by: Laurent Laville <[email protected]> Co-authored-by: nvuillam <[email protected]>
oxsecurity · Dec 5, 2021 · f38b978 · f38b978
1 parent da32f53
commit f38b978
Show file tree

Hide file tree

Showing 43 changed files with 263 additions and 55 deletions.
diff --git a/.automation/build.py b/.automation/build.py
@@ -25,6 +25,7 @@
 from bs4 import BeautifulSoup
 from giturlparse import parse
 from megalinter.constants import (
+    DEFAULT_RELEASE,
     DEFAULT_REPORT_FOLDER_NAME,
     ML_DOC_URL,
     ML_DOCKER_IMAGE,
@@ -191,7 +192,7 @@ def generate_flavor(flavor, flavor_info):
         if RELEASE is True:
             image_release = RELEASE_TAG
         else:
-            image_release = "v5"
+            image_release = DEFAULT_RELEASE
         flavor_x = f"[{flavor} flavor]"
         action_yml = f""" # Automatically {'@'}generated by build.py
 name: "MegaLinter"
@@ -544,11 +545,11 @@ def generate_descriptor_documentation(descriptor):
 
 
 def generate_flavor_documentation(flavor_id, flavor, linters_tables_md):
-    flavor_github_action = f"{ML_REPO}/flavors/{flavor_id}@v5"
-    flavor_docker_image = f"{ML_DOCKER_IMAGE}-{flavor_id}:v5"
+    flavor_github_action = f"{ML_REPO}/flavors/{flavor_id}@{DEFAULT_RELEASE}"
+    flavor_docker_image = f"{ML_DOCKER_IMAGE}-{flavor_id}:{DEFAULT_RELEASE}"
     docker_image_badge = (
         f"![Docker Image Size (tag)]({BASE_SHIELD_IMAGE_LINK}/"
-        f"{ML_DOCKER_IMAGE}-{flavor_id}/v5)"
+        f"{ML_DOCKER_IMAGE}-{flavor_id}/{DEFAULT_RELEASE})"
     )
     docker_pulls_badge = (
         f"![Docker Pulls]({BASE_SHIELD_COUNT_LINK}/" f"{ML_DOCKER_IMAGE}-{flavor_id})"
@@ -1224,7 +1225,7 @@ def build_flavors_md_table(filter_linter_name=None, replace_link=False):
         + +len(linters_by_type["other"])
     )
     docker_image_badge = (
-        f"![Docker Image Size (tag)]({BASE_SHIELD_IMAGE_LINK}/{ML_DOCKER_IMAGE}/v5)"
+        f"![Docker Image Size (tag)]({BASE_SHIELD_IMAGE_LINK}/{ML_DOCKER_IMAGE}/{DEFAULT_RELEASE})"
     )
     docker_pulls_badge = (
         f"![Docker Pulls]({BASE_SHIELD_COUNT_LINK}/" f"{ML_DOCKER_IMAGE})"
@@ -1252,7 +1253,7 @@ def build_flavors_md_table(filter_linter_name=None, replace_link=False):
         flavor_doc_url = f"{DOCS_URL_FLAVORS_ROOT}/{flavor_id}.md"
         docker_image_badge = (
             f"![Docker Image Size (tag)]({BASE_SHIELD_IMAGE_LINK}/"
-            f"{ML_DOCKER_IMAGE}-{flavor_id}/v5)"
+            f"{ML_DOCKER_IMAGE}-{flavor_id}/{DEFAULT_RELEASE})"
         )
         docker_pulls_badge = (
             f"![Docker Pulls]({BASE_SHIELD_COUNT_LINK}/"

diff --git a/.automation/generated/flavors-stats.json b/.automation/generated/flavors-stats.json
@@ -383,6 +383,10 @@
         [
             "2021-12-04T10:20:40",
             1949864
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1981415
         ]
     ],
     "ci_light": [
@@ -769,6 +773,10 @@
         [
             "2021-12-04T10:20:40",
             5468
+        ],
+        [
+            "2021-12-05T01:37:50",
+            5473
         ]
     ],
     "dart": [
@@ -1155,6 +1163,10 @@
         [
             "2021-12-04T10:20:40",
             1630
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1634
         ]
     ],
     "documentation": [
@@ -1541,6 +1553,10 @@
         [
             "2021-12-04T10:20:40",
             23337
+        ],
+        [
+            "2021-12-05T01:37:50",
+            23344
         ]
     ],
     "dotnet": [
@@ -1927,6 +1943,10 @@
         [
             "2021-12-04T10:20:40",
             256159
+        ],
+        [
+            "2021-12-05T01:37:50",
+            256346
         ]
     ],
     "go": [
@@ -2313,6 +2333,10 @@
         [
             "2021-12-04T10:20:40",
             5717
+        ],
+        [
+            "2021-12-05T01:37:50",
+            5729
         ]
     ],
     "java": [
@@ -2699,6 +2723,10 @@
         [
             "2021-12-04T10:20:40",
             42313
+        ],
+        [
+            "2021-12-05T01:37:50",
+            42323
         ]
     ],
     "javascript": [
@@ -3085,6 +3113,10 @@
         [
             "2021-12-04T10:20:40",
             59476
+        ],
+        [
+            "2021-12-05T01:37:50",
+            59584
         ]
     ],
     "php": [
@@ -3471,6 +3503,10 @@
         [
             "2021-12-04T10:20:40",
             2675
+        ],
+        [
+            "2021-12-05T01:37:50",
+            2699
         ]
     ],
     "python": [
@@ -3857,6 +3893,10 @@
         [
             "2021-12-04T10:20:40",
             32672
+        ],
+        [
+            "2021-12-05T01:37:50",
+            32768
         ]
     ],
     "ruby": [
@@ -4239,6 +4279,10 @@
         [
             "2021-12-04T10:20:40",
             1562
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1566
         ]
     ],
     "rust": [
@@ -4621,6 +4665,10 @@
         [
             "2021-12-04T10:20:40",
             1305
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1309
         ]
     ],
     "salesforce": [
@@ -5007,6 +5055,10 @@
         [
             "2021-12-04T10:20:40",
             8673
+        ],
+        [
+            "2021-12-05T01:37:50",
+            8685
         ]
     ],
     "scala": [
@@ -5389,6 +5441,10 @@
         [
             "2021-12-04T10:20:40",
             1158
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1162
         ]
     ],
     "swift": [
@@ -5771,6 +5827,10 @@
         [
             "2021-12-04T10:20:40",
             1058
+        ],
+        [
+            "2021-12-05T01:37:50",
+            1062
         ]
     ],
     "terraform": [
@@ -6157,6 +6217,10 @@
         [
             "2021-12-04T10:20:40",
             24900
+        ],
+        [
+            "2021-12-05T01:37:50",
+            24950
         ]
     ]
 }
diff --git a/.automation/generated/linter-helps.json b/.automation/generated/linter-helps.json
@@ -1379,8 +1379,8 @@
         "                        of opening bracket's line.",
         "  --ignore errors       Comma-separated list of errors and warnings to ignore",
         "                        (or skip). For example, ``--ignore=E4,E51,W234``.",
-        "                        (Default: ['E704', 'E24', 'E126', 'E123', 'W503',",
-        "                        'W504', 'E121', 'E226'])",
+        "                        (Default: ['E126', 'E24', 'E704', 'W503', 'E123',",
+        "                        'W504', 'E226', 'E121'])",
         "  --extend-ignore errors",
         "                        Comma-separated list of errors and warnings to add to",
         "                        the list of ignored ones. For example, ``--extend-",
@@ -3480,6 +3480,9 @@
         "    --no-diff",
         "        Turns off Psalm\u2019s diff mode, checks all files regardless of whether they\u2019ve changed.",
         "",
+        "    --php-version=PHP_VERSION",
+        "        Explicitly set PHP version to analyse code against.",
+        "",
         "Surfacing issues:",
         "    --show-info[=BOOLEAN]",
         "        Show non-exception parser findings (defaults to false).",
@@ -5798,7 +5801,7 @@
         "                 [--shadow-prefix DIR] [--scheduler [{ilp,greedy}]]",
         "                 [--wms-monitor [WMS_MONITOR]]",
         "                 [--wms-monitor-arg [NAME=VALUE ...]]",
-        "                 [--scheduler-ilp-solver {PULP_CBC_CMD,PULP_CHOCO_CMD}]",
+        "                 [--scheduler-ilp-solver {PULP_CBC_CMD}]",
         "                 [--scheduler-solver-path SCHEDULER_SOLVER_PATH]",
         "                 [--conda-base-path CONDA_BASE_PATH] [--no-subworkflows]",
         "                 [--groups GROUPS [GROUPS ...]]",
@@ -6099,7 +6102,7 @@
         "                        id=12345 and the arguments will be provided to the",
         "                        endpoint to first interact with the workflow (default:",
         "                        None)",
-        "  --scheduler-ilp-solver {PULP_CBC_CMD,PULP_CHOCO_CMD}",
+        "  --scheduler-ilp-solver {PULP_CBC_CMD}",
         "                        Specifies solver to be utilized when selecting ilp-",
         "                        scheduler. (default: COIN_CMD)",
         "  --scheduler-solver-path SCHEDULER_SOLVER_PATH",

diff --git a/.automation/generated/linter-versions.json b/.automation/generated/linter-versions.json
@@ -50,7 +50,7 @@
     "phplint": "3.0",
     "phpstan": "1.2.0",
     "powershell": "7.2.0",
-    "prettier": "2.5.0",
+    "prettier": "2.5.1",
     "protolint": "0.35.2",
     "psalm": "Psalm.4.x-dev@",
     "puppet-lint": "2.5.2",

diff --git a/.vscode/launch.json b/.vscode/launch.json
@@ -0,0 +1,15 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "name": "Python : CurrentFile",
+            "type": "python",
+            "request": "test",
+            "program": "${file}",
+            "console": "integratedTerminal"
+        }
+    ]
+}
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -12,9 +12,11 @@ Note: Can be used with `megalinter/megalinter@beta` in your GitHub Action mega-l
 
 - Core architecture
   - New reporter **SARIF_REPORTER** that aggregates all SARIF output files into a single one
+    - Correct SARIF files for known format errors
   - Manage offline run of `bash build.sh` for those who want to code in planes :)
   - Automate update of CHANGELOG.md after release (pilot)
   - Rename default report folder from `report` to `megalinter-reports`
+  - Accelerate internal CI testing performances
 
 - Linters:
   - Add [PMD](https://pmd.github.io/) to lint java files (disabled for now)
@@ -26,6 +28,7 @@ Note: Can be used with `megalinter/megalinter@beta` in your GitHub Action mega-l
     - checkov
     - eslint
     - gitleaks
+    - trivy
 
 - Descriptors:
   - New flavor **Security**
@@ -41,6 +44,7 @@ Note: Can be used with `megalinter/megalinter@beta` in your GitHub Action mega-l
 - Docker run -- clean-up containers when exits (#1033)
 - Add missing Bandit config file and rules path options (#679)
 - Fix getting linter version of npm plugin. (#845)
+- Improve runtime performances when using a flavor and defining `FLAVORS_SUGGESTION: false`
 
 - Linters
   - New linter `phplint` to speed-up linting of php files (#1031)
@@ -92,6 +96,7 @@ Note: Can be used with `megalinter/megalinter@beta` in your GitHub Action mega-l
   - [pylint](https://www.pylint.org) from 2.12.1 to **2.12.2** on 2021-12-04
   - [checkov](https://www.checkov.io/) from 2.0.625 to **2.0.626** on 2021-12-04
   - [eslint](https://eslint.org) from 8.3.0 to **8.4.0** on 2021-12-04
+  - [prettier](https://prettier.io/) from 2.5.0 to **2.5.1** on 2021-12-05
 <!-- linter-versions-end -->
 
 ## [v5.2.0] - 2021-11-18

diff --git a/README.md b/README.md
@@ -14,7 +14,7 @@
 <!-- mega-linter-title-end -->
 
 ![GitHub release](https://img.shields.io/github/v/release/megalinter/megalinter?sort=semver)
-[![Docker Pulls](https://img.shields.io/badge/docker%20pulls-2.4M-blue)](https://megalinter.github.io/flavors/)
+[![Docker Pulls](https://img.shields.io/badge/docker%20pulls-2.5M-blue)](https://megalinter.github.io/flavors/)
 [![Downloads/week](https://img.shields.io/npm/dw/mega-linter-runner.svg)](https://npmjs.org/package/mega-linter-runner)
 [![GitHub stars](https://img.shields.io/github/stars/megalinter/megalinter?cacheSeconds=3600)](https://github.com/megalinter/megalinter/stargazers/)
 [![MegaLinter](https://github.com/megalinter/megalinter/workflows/MegaLinter/badge.svg?branch=main)](https://github.com/megalinter/megalinter/actions?query=workflow%3AMegaLinter+branch%3Amain)

diff --git a/docs/all_linters.md b/docs/all_linters.md
@@ -56,7 +56,7 @@
 | [**phplint**](https://github.com/overtrue/phplint){target=_blank}                                       |      3.0       |   <!-- -->   | [PHP](descriptors/php_phplint.md)                                                                                                                                                          |   :white_circle:    |                                           [Web Site](https://github.com/overtrue/phplint){target=_blank}                                            |
 | [**phpstan**](https://github.com/phpstan/phpstan){target=_blank}                                        |     1.2.0      |     MIT      | [PHP](descriptors/php_phpstan.md)                                                                                                                                                          |   :white_circle:    |                                           [Repository](https://github.com/phpstan/phpstan){target=_blank}                                           |
 | [**powershell**](https://github.com/PowerShell/PSScriptAnalyzer){target=_blank}                         |     7.2.0      |   <!-- -->   | [POWERSHELL](descriptors/powershell_powershell.md)                                                                                                                                         |   :white_circle:    |                                      [Web Site](https://github.com/PowerShell/PSScriptAnalyzer){target=_blank}                                      |
-| [**prettier**](https://github.com/prettier/prettier){target=_blank}                                     |     2.5.0      |     MIT      | [JAVASCRIPT](descriptors/javascript_prettier.md)<br/> [JSON](descriptors/json_prettier.md)<br/> [TYPESCRIPT](descriptors/typescript_prettier.md)<br/> [YAML](descriptors/yaml_prettier.md) |   :white_circle:    |                                          [Repository](https://github.com/prettier/prettier){target=_blank}                                          |
+| [**prettier**](https://github.com/prettier/prettier){target=_blank}                                     |     2.5.1      |     MIT      | [JAVASCRIPT](descriptors/javascript_prettier.md)<br/> [JSON](descriptors/json_prettier.md)<br/> [TYPESCRIPT](descriptors/typescript_prettier.md)<br/> [YAML](descriptors/yaml_prettier.md) |   :white_circle:    |                                          [Repository](https://github.com/prettier/prettier){target=_blank}                                          |
 | [**protolint**](https://github.com/yoheimuta/protolint){target=_blank}                                  |     0.35.2     |   <!-- -->   | [PROTOBUF](descriptors/protobuf_protolint.md)                                                                                                                                              |   :white_circle:    |                                          [Web Site](https://github.com/yoheimuta/protolint){target=_blank}                                          |
 | [**psalm**](https://github.com/vimeo/psalm){target=_blank}                                              | Psalm.4.x-dev@ |     MIT      | [PHP](descriptors/php_psalm.md)                                                                                                                                                            |   :white_circle:    |                                             [Repository](https://github.com/vimeo/psalm){target=_blank}                                             |
 | [**puppet-lint**](https://github.com/rodjek/puppet-lint){target=_blank}                                 |     2.5.2      |     MIT      | [PUPPET](descriptors/puppet_puppet_lint.md)                                                                                                                                                |   :white_circle:    |                                         [Repository](https://github.com/rodjek/puppet-lint){target=_blank}                                          |

diff --git a/docs/descriptors/javascript_prettier.md b/docs/descriptors/javascript_prettier.md
@@ -9,7 +9,7 @@
 
 ## prettier documentation
 
-- Version in MegaLinter: **2.5.0**
+- Version in MegaLinter: **2.5.1**
 - Visit [Official Web Site](https://prettier.io/){target=_blank}
 - See [How to configure prettier rules](https://prettier.io/docs/en/configuration.html){target=_blank}
 - See [How to disable prettier rules in files](https://prettier.io/docs/en/ignore.html#javascript){target=_blank}

diff --git a/docs/descriptors/json_prettier.md b/docs/descriptors/json_prettier.md
@@ -9,7 +9,7 @@
 
 ## prettier documentation
 
-- Version in MegaLinter: **2.5.0**
+- Version in MegaLinter: **2.5.1**
 - Visit [Official Web Site](https://prettier.io/){target=_blank}
 - See [How to configure prettier rules](https://prettier.io/docs/en/configuration.html){target=_blank}
 - See [How to disable prettier rules in files](https://prettier.io/docs/en/ignore.html#javascript){target=_blank}

diff --git a/docs/descriptors/php_psalm.md b/docs/descriptors/php_psalm.md
@@ -112,6 +112,9 @@ Basic configuration:
     --no-diff
         Turns off Psalm’s diff mode, checks all files regardless of whether they’ve changed.
 
+    --php-version=PHP_VERSION
+        Explicitly set PHP version to analyse code against.
+
 Surfacing issues:
     --show-info[=BOOLEAN]
         Show non-exception parser findings (defaults to false).

diff --git a/docs/descriptors/python_flake8.md b/docs/descriptors/python_flake8.md
@@ -124,8 +124,8 @@ optional arguments:
                         of opening bracket's line.
   --ignore errors       Comma-separated list of errors and warnings to ignore
                         (or skip). For example, ``--ignore=E4,E51,W234``.
-                        (Default: ['E704', 'E24', 'E126', 'E123', 'W503',
-                        'W504', 'E121', 'E226'])
+                        (Default: ['E126', 'E24', 'E704', 'W503', 'E123',
+                        'W504', 'E226', 'E121'])
   --extend-ignore errors
                         Comma-separated list of errors and warnings to add to
                         the list of ignored ones. For example, ``--extend-