feat(general): Add an option to pick beetwen native OpenAI and Azure OpenAI #5569

lif2 · 2023-09-14T19:50:25Z

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Description

added --openai-api-type parameter with allowed default and azure values, to allow user, to use Azure OpenAI resource.

default is of course default option, and it does not change current OpenAPI workflow
azure is an additional option to choose, to make checkov "talk to" Azure OpenAI. Additional Environment Variables have to be defined to work with this type:
- CKV_AZURE_OPENAI_API_ENDPOINT
- CKV_AZURE_OPENAI_DEPLOYMENT_NAME
- CKV_AZURE_OPENAI_API_VERSION (default: 2023-05-15).
  These were included in documentation and cli help.

Azure OpenAI integration gives more control over the AI findings. Companies can built their own, private Knowlege Base/Suggestions Database with Azure OpenAI for resolving policies violations.

Calling checkov with command checkov -d . --compact --openai-api-type azure --openai-api-key 1********************************0, without CKV_AZURE_OPENAI_API_ENDPOINT and/or CKV_AZURE_OPENAI_DEPLOYMENT_NAME set, will result in Error messages; however checks will be performed anyway:


       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.4.39

ERROR: Configuration for Azure OpenAI is missing: Please specify CKV_AZURE_OPENAI_API_ENDPOINT environment variable for --openai-api-type 'azure' type.       
ERROR: Configuration for Azure OpenAI is missing: Please specify CKV_AZURE_OPENAI_DEPLOYMENT_NAME environment variable for --openai-api-type 'azure' type.    
ansible scan results:

Passed checks: 4, Failed checks: 1, Skipped checks: 0
(...)

Same checkov command, with CKV_AZURE_OPENAI_API_ENDPOINT and CKV_AZURE_OPENAI_DEPLOYMENT_NAME set, will result in Azure OpenAI call and AI suggestions:


       _               _
   ___| |__   ___  ___| | _______   __
  / __| '_ \ / _ \/ __| |/ / _ \ \ / /
 | (__| | | |  __/ (__|   < (_) \ V /
  \___|_| |_|\___|\___|_|\_\___/ \_/

By bridgecrew.io | version: 2.4.39

WARNING: About to request 1 enhanced guidelines and it may take up to 15s.

ansible scan results:

Passed checks: 4, Failed checks: 1, Skipped checks: 0
(...)
Check: CKV2_ANSIBLE_3: "Ensure block is handling task errors properly"
        FAILED for resource: block.Load Terraform output file and make variables for ansible playbook
        Details: The following text is AI generated and should be treated as a suggestion.

                 The code you provided is an Ansible playbook that is not handling task errors properly.
                 The Checkov policy 'Ensure block is handling task errors properly' requires that each task in an Ansible playbook should handle errors properly to prevent unexpected failures.
(...)

Fix

How does someone fix the issue in code and/or in runtime?

Checklist:

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
I have added tests that prove my feature, policy, or fix is effective and works
New and existing tests pass locally with my changes
Any dependent changes have been merged and published in downstream modules

lif2 · 2023-09-15T15:13:00Z

Thank you @gruebel for approving checks on this PR yesterday! Yesterday I didn't have flake8, mypy findings addressed and unittests added. Today branch should be in better shape.

Have a nice weekend!

force _instance to be None (to be sure we are on right configuration).

lif2 · 2023-09-21T19:36:23Z

Hi @gruebel ! Let me know if you think I need to add anything to that branch? Tests are passing - I'm just merging back main branch to my forked repo.
If you could suggest another reviewer - it will be more than appreciated! Thank you!

lif2 · 2023-09-29T20:29:43Z

Hey @gruebel ! Gentle reminder ob this PR 😃 Thanks! 🥩🍺

gruebel

the idea is general great, but the implementation doesn't fit the code style here. Also please test your implementation, because setting the parameter messages=messages[0] when calling the chat completion would have shown you that it doesn't work at all.

gruebel · 2023-10-20T12:59:14Z

checkov/common/output/ai.py

+            cls._api_type = api_type.lower()
+            if (cls._api_type == 'azure'):


no need to create a class attribute for this, when you set it on openai a few lines later

gruebel · 2023-10-20T12:59:58Z

checkov/common/util/ext_argument_parser.py

+        self.add(
+            "--openai-api-type",
+            env_var="CKV_OPENAI_API_TYPE",
+            choices=['azure', 'default'],
+            default='default',
+            help="By switching this flag to 'azure', you are able to send violated policies to Azure OpenAI service. "
+                 "You have to provide Key, either with --openai-api-key or CKV_OPENAI_API_KEY. "
+                 "Before you switch to 'azure' you also have to define CKV_AZURE_OPENAI_API_ENDPOINT and CKV_AZURE_OPENAI_DEPLOYMENT_NAME environment variables. "
+                 "CKV_AZURE_OPENAI_API_ENDPOINT - is the Azure OpenAI service endpoint, eg: 'https://eastus.api.cognitive.microsoft.com/'. "
+                 "CKV_AZURE_OPENAI_DEPLOYMENT_NAME - this is your Deployment Name from https://oai.portal.com portal you wish to use. "
+                 "With CKV_AZURE_OPENAI_API_VERSION environment variable you are also able to control API version - this defaults to '2023-05-15'. ",
+        )


it doesn't make sense to add a flag for it, because you need multiple other settings, which are set via env vars to make it work

gruebel · 2023-10-20T13:03:01Z

checkov/common/output/ai.py

+            if (self._api_type == 'azure'):
+                completion = await openai.ChatCompletion.acreate(  # type:ignore[no-untyped-call]
+                    engine=self.AZURE_OPENAI_DEPLOYMENT_NAME,
+                    messages=messages[0],


Suggested change

messages=messages[0],

messages=messages,

otherwise you only send the first message, same a few lines lower

gruebel · 2023-10-20T13:05:34Z

checkov/common/output/ai.py

+            print(
+                colored(
+                    f"ERROR: Configuration for Azure OpenAI is missing: Please specify {environment_variable_name} environment variable for --openai-api-type '{self._api_type}' type.",
+                    "red",
+                )
+            )


just use a normal log message

gruebel · 2023-10-20T13:06:50Z

checkov/common/output/ai.py

+                cls._should_run = cls._should_run & cls._validate_azure_env(cls._instance, cls.AZURE_OPENAI_API_ENDPOINT, 'CKV_AZURE_OPENAI_API_ENDPOINT')
+                cls._should_run = cls._should_run & cls._validate_azure_env(cls._instance, cls.AZURE_OPENAI_API_VERSION, 'CKV_AZURE_OPENAI_API_VERSION')
+                cls._should_run = cls._should_run & cls._validate_azure_env(cls._instance, cls.AZURE_OPENAI_DEPLOYMENT_NAME, 'CKV_AZURE_OPENAI_DEPLOYMENT_NAME')


instead of validating each one by one, pass all to a validate function and decide, if everything is set as needed

gruebel · 2023-10-20T13:11:30Z

checkov/common/output/ai.py

+                    max_tokens=OPENAI_MAX_TOKENS,
+                )
+
+            logging.info(f"[COMPLETION]{completion}")


Suggested change

logging.info(f"[COMPLETION]{completion}")

logging.debug(f"OpenAI request returned: {completion}")

or something similar to have a direct context. Also debug level is more than enough.

gruebel · 2023-10-20T13:13:05Z

checkov/common/output/report.py

@@ -272,6 +272,7 @@ def print_console(
            use_bc_ids: bool = False,
            summary_position: str = 'top',
            openai_api_key: str | None = None,
+            openai_api_type: str = "default",


remove all of the openai_api_type parameter changes and change it to an env var with a default of default

gruebel · 2023-10-20T13:13:17Z

docs/2.Basics/CLI Command Reference.md

@@ -65,4 +65,5 @@ nav_order: 2
 | `--scan-secrets-history`                                                                                                                                                                                                                                                                                                                                                   | Enable secret scan history of commits                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
 | `--block-list-secret-scan CKV_SECRETS_SCAN_BLOCK_LIST`                                                                                                                                                                                                                                                                                                                     | List of files to filter out in the secret scanner                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
 | `--support`                                                                                                                                                                                                                                                                                                                                                                | Enable debug logs and upload the logs to the server. Requires a Bridgecrew or Prisma Cloud API key.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+| `--openai-api-type`                                                                                                                                                                                                                                                                                                                                                                | By switching this flag to 'azure', you are able to send violated policies to Azure OpenAI service. You have to provide Key, either with --openai-api-key or CKV_OPENAI_API_KEY. Before you switch to 'azure' you also have to define CKV_AZURE_OPENAI_API_ENDPOINT and CKV_AZURE_OPENAI_DEPLOYMENT_NAME environment variables. CKV_AZURE_OPENAI_API_ENDPOINT - is the Azure OpenAI service endpoint, eg: 'https://eastus.api.cognitive.microsoft.com/'. CKV_AZURE_OPENAI_DEPLOYMENT_NAME - this is your Deployment Name from https://oai.portal.com portal you wish to use. With CKV_AZURE_OPENAI_API_VERSION environment variable you are also able to control API version - this defaults to '2023-05-15'.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |


gruebel · 2023-10-20T13:14:29Z

docs/4.Integrations/OpenAI.md

+## OpenAI type 
+
+With `--openai-api-type` flag, you are able to choose OpenAI source. Possible values here are `default` and `azure`. Default valuie is obvoiusly `default`, and this flag will redirect your requests to public-generally available OpenAI service. By switching this flag to `--openai-api-type azure` you can query **Azure OpenAI** resource using Azure OpenAI Key. Please see below [Azure OpenAI] section for more configuration details.
+


move this to the Settings block as an env var

gruebel · 2023-10-20T13:22:05Z

checkov/common/output/ai.py

+                {"role": "user", "content": "Explain"},
+            ],
+            # depends on api_type, call ChatCompletion differently
+            logging.info(f"[_chat_complete]: self._api_type: {self._api_type}")


instead of logging it here, just do it once at the end of the class instance creation, then you can also log other infos like deployment name and endpoint used.

gruebel · 2023-10-20T13:27:00Z

and there is also no need to update the branch all the time, because this code part is rarely changed and we typically trigger it, when needed before merging

stale · 2024-04-18T03:02:11Z

Thanks for contributing to Checkov! We've automatically marked this issue as stale to keep our issues list tidy, because it has not had any activity for 6 months. It will be closed in 14 days if no further activity occurs. Commenting on this issue will remove the stale tag. If you want to talk through the issue or help us understand the priority and context, feel free to add a comment or join us in the Checkov slack channel at codifiedsecurity.slack.com
Thanks!

stale · 2024-05-05T11:45:48Z

Closing issue due to inactivity. If you feel this is in error, please re-open, or reach out to the community via slack: codifiedsecurity.slack.com Thanks!

lif2 added 2 commits September 14, 2023 20:22

Azure OpenAI

6a87a40

CLI Command Reference updated as well

58351d2

lif2 had a problem deploying to scan-security September 14, 2023 19:50 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

f4057ae

lif2 temporarily deployed to scan-security September 14, 2023 19:55 — with GitHub Actions Inactive

Merge branch 'main' into feat/AzureOpenAI

a166987

lif2 had a problem deploying to scan-security September 15, 2023 08:02 — with GitHub Actions Failure

fix linter and static type issues

14a723a

lif2 had a problem deploying to scan-security September 15, 2023 09:28 — with GitHub Actions Failure

unittests added for '_should_run'

02b7bbf

lif2 had a problem deploying to scan-security September 15, 2023 10:01 — with GitHub Actions Failure

testing correct and missing configurations

9b5dbc9

lif2 had a problem deploying to scan-security September 15, 2023 14:34 — with GitHub Actions Failure

line added - to adjust code with 'main' version

e8cdd37

lif2 temporarily deployed to scan-security September 15, 2023 15:04 — with GitHub Actions Inactive

Merge branch 'main' into feat/AzureOpenAI

44be5a8

lif2 had a problem deploying to scan-security September 18, 2023 14:22 — with GitHub Actions Failure

minimize unittest files for Azure OpenAI

e1ae799

force _instance to be None (to be sure we are on right configuration).

lif2 had a problem deploying to scan-security September 18, 2023 15:14 — with GitHub Actions Failure

typos in documentation fixed

d91b94e

lif2 had a problem deploying to scan-security September 18, 2023 18:43 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

c8101c7

lif2 had a problem deploying to scan-security September 19, 2023 07:01 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

71ccce6

lif2 temporarily deployed to scan-security September 19, 2023 09:30 — with GitHub Actions Inactive

Merge branch 'main' into feat/AzureOpenAI

442b9c4

lif2 temporarily deployed to scan-security September 20, 2023 10:46 — with GitHub Actions Inactive

Merge branch 'main' into feat/AzureOpenAI

70d3673

lif2 had a problem deploying to scan-security September 20, 2023 12:50 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

97f5fd3

lif2 had a problem deploying to scan-security September 22, 2023 07:15 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

1e8d3b5

lif2 had a problem deploying to scan-security September 26, 2023 09:17 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

1a9a35a

lif2 had a problem deploying to scan-security September 27, 2023 20:03 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

61e06b5

lif2 had a problem deploying to scan-security September 28, 2023 09:32 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

972dd4c

lif2 had a problem deploying to scan-security October 2, 2023 07:36 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

cc809dd

lif2 had a problem deploying to scan-security October 10, 2023 10:41 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

b7cbeb5

lif2 had a problem deploying to scan-security October 15, 2023 08:19 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

e00412f

lif2 had a problem deploying to scan-security October 15, 2023 08:19 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

5e6c7c9

lif2 had a problem deploying to scan-security October 17, 2023 19:29 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

629d717

lif2 had a problem deploying to scan-security October 18, 2023 20:33 — with GitHub Actions Failure

Merge branch 'main' into feat/AzureOpenAI

c1d5d62

lif2 had a problem deploying to scan-security October 19, 2023 07:53 — with GitHub Actions Failure

gruebel reviewed Oct 20, 2023

View reviewed changes

stale bot added the stale label Apr 18, 2024

stale bot closed this May 5, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(general): Add an option to pick beetwen native OpenAI and Azure OpenAI #5569

feat(general): Add an option to pick beetwen native OpenAI and Azure OpenAI #5569

lif2 commented Sep 14, 2023 •

edited

Loading

lif2 commented Sep 15, 2023

lif2 commented Sep 21, 2023

lif2 commented Sep 29, 2023

gruebel left a comment •

edited

Loading

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel Oct 20, 2023

gruebel commented Oct 20, 2023

stale bot commented Apr 18, 2024

stale bot commented May 5, 2024

		cls._api_type = api_type.lower()
		if (cls._api_type == 'azure'):

	logging.info(f"[COMPLETION]{completion}")
	logging.debug(f"OpenAI request returned: {completion}")

		## OpenAI type

		With `--openai-api-type` flag, you are able to choose OpenAI source. Possible values here are `default` and `azure`. Default valuie is obvoiusly `default`, and this flag will redirect your requests to public-generally available OpenAI service. By switching this flag to `--openai-api-type azure` you can query Azure OpenAI resource using Azure OpenAI Key. Please see below [Azure OpenAI] section for more configuration details.

feat(general): Add an option to pick beetwen native OpenAI and Azure OpenAI #5569

feat(general): Add an option to pick beetwen native OpenAI and Azure OpenAI #5569

Conversation

lif2 commented Sep 14, 2023 • edited Loading

Description

Fix

Checklist:

lif2 commented Sep 15, 2023

lif2 commented Sep 21, 2023

lif2 commented Sep 29, 2023

gruebel left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gruebel commented Oct 20, 2023

stale bot commented Apr 18, 2024

stale bot commented May 5, 2024

lif2 commented Sep 14, 2023 •

edited

Loading

gruebel left a comment •

edited

Loading