Optimize chat prompts (Remove JSON Output:) (#82)

* Remove "JSON Output:" from prompt

* llm_providers: Explicitly append "JSON Output:" to non-chat completion prompts

README.md

@@ -143,8 +143,6 @@ Given below is XML that describes the information to extract from this document
 </output>
 
 ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise.
-
-JSON Output:
 ```
 
 Call the `Guard` object with the LLM API call as the first argument and add any additional arguments to the LLM API call as the remaining arguments.

docs/rail/prompt.md

@@ -41,6 +41,4 @@ Given the following document, answer the following questions. If the answer does
 
 ```
 ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere, enter `None`.
-
-JSON Output:
 ```

guardrails/constants.xml

@@ -13,15 +13,11 @@ Given below is XML that describes the information to extract from this document
 
 <json_suffix_prompt_v2>
 ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise. If you are unsure anywhere, enter "None".
-
-JSON Output:
 </json_suffix_prompt_v2>
 
 
 <json_suffix_prompt_v2_wo_none>
 ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise.
-
-JSON Output:
 </json_suffix_prompt_v2_wo_none>
 
 
@@ -46,8 +42,7 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<![CDATA[<string name='foo' format='two-words lower-case' />`]]> => `{{{{'foo': 'example one'}}}}`
 - `<![CDATA[<list name='bar'><string format='upper-case' /></list>]]>` => `{{{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}}}`
 - `<![CDATA[<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>]]>` => `{{{{'baz': {{{{'foo': 'Some String', 'index': 1}}}}}}}}`
-
-JSON Object:</complete_json_suffix>
+</complete_json_suffix>
 
 <complete_json_suffix_v2>
 Given below is XML that describes the information to extract from this document and the tags to extract it into.
@@ -60,8 +55,7 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<![CDATA[<string name='foo' format='two-words lower-case' />`]]> => `{{{{'foo': 'example one'}}}}`
 - `<![CDATA[<list name='bar'><string format='upper-case' /></list>]]>` => `{{{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}}}`
 - `<![CDATA[<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>]]>` => `{{{{'baz': {{{{'foo': 'Some String', 'index': 1}}}}}}}}`
-
-JSON Object:</complete_json_suffix_v2>
+</complete_json_suffix_v2>
 
 
 </constants>

guardrails/llm_providers.py

@@ -46,7 +46,7 @@ def openai_wrapper(text: str, *args, **kwargs):
     api_key = os.environ.get("OPENAI_API_KEY")
     openai_response = openai.Completion.create(
         api_key=api_key,
-        prompt=text,
+        prompt=text + "\n\nJson Output:\n\n",
         *args,
         **kwargs,
     )

tests/integration_tests/test_cases/entity_extraction/compiled_prompt.txt

@@ -131,5 +131,3 @@ Given below is XML that describes the information to extract from this document
 
 
 ONLY return a valid JSON object (no other text is necessary). The JSON MUST conform to the XML format, including any types and format requests e.g. requests for lists, objects and specific types. Be correct and concise.
-
-JSON Output:

tests/integration_tests/test_cases/entity_extraction/compiled_prompt_reask.txt

@@ -37,5 +37,3 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<string name='foo' format='two-words lower-case' />` => `{{'foo': 'example one'}}`
 - `<list name='bar'><string format='upper-case' /></list>` => `{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}`
 - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>` => `{{'baz': {{'foo': 'Some String', 'index': 1}}}}`
-
-JSON Object:

tests/integration_tests/test_cases/pydantic/compiled_prompt.txt

@@ -36,5 +36,3 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<string name='foo' format='two-words lower-case' />` => `{'foo': 'example one'}`
 - `<list name='bar'><string format='upper-case' /></list>` => `{"bar": ['STRING ONE', 'STRING TWO', etc.]}`
 - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>` => `{'baz': {'foo': 'Some String', 'index': 1}}`
-
-JSON Object:

tests/integration_tests/test_cases/pydantic/compiled_prompt_reask_1.txt

@@ -35,5 +35,3 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<string name='foo' format='two-words lower-case' />` => `{{'foo': 'example one'}}`
 - `<list name='bar'><string format='upper-case' /></list>` => `{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}`
 - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>` => `{{'baz': {{'foo': 'Some String', 'index': 1}}}}`
-
-JSON Object:

tests/integration_tests/test_cases/pydantic/compiled_prompt_reask_2.txt

@@ -35,5 +35,3 @@ Here are examples of simple (XML, JSON) pairs that show the expected behavior:
 - `<string name='foo' format='two-words lower-case' />` => `{{'foo': 'example one'}}`
 - `<list name='bar'><string format='upper-case' /></list>` => `{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}`
 - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>` => `{{'baz': {{'foo': 'Some String', 'index': 1}}}}`
-
-JSON Object:

tests/unit_tests/utils/test_reask_utils.py

@@ -158,8 +158,7 @@ def test_get_reask_prompt(example_rail, reasks, reask_json):
 - `<string name='foo' format='two-words lower-case' />` => `{{'foo': 'example one'}}`
 - `<list name='bar'><string format='upper-case' /></list>` => `{{"bar": ['STRING ONE', 'STRING TWO', etc.]}}`
 - `<object name='baz'><string name="foo" format="capitalize two-words" /><integer name="index" format="1-indexed" /></object>` => `{{'baz': {{'foo': 'Some String', 'index': 1}}}}`
-
-JSON Object:"""  # noqa: E501
+"""  # noqa: E501
 
     result_prompt, _ = reask_utils.get_reask_prompt(
         ET.fromstring(example_rail), reasks, reask_json