feat: introducing configurable retrieval workflows #3227

jacopo-chevallard · 2024-09-18T16:57:30Z

Description

Major PR which, among other things, introduces the possibility of easily customizing the retrieval workflows. Workflows are based on LangGraph, and can be customized using a yaml configuration file, and adding the implementation of the nodes logic into quivr_rag_langgraph.py

This is a first, simple implementation that will significantly evolve in the coming weeks to enable more complex workflows (for instance, with conditional nodes). We also plan to adopt a similar approach for the ingestion part, i.e. to enable user to easily customize the ingestion pipeline.

Closes CORE-195, CORE-203, CORE-204

Checklist before requesting a review

Please delete options that are not relevant.

My code follows the style guidelines of this project
I have performed a self-review of my code
I have commented hard-to-understand areas
I have ideally added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
Any dependent changes have been merged

Screenshots (if appropriate):

…n of rerankers

…, you need to set the env variables COHERE_API_KEY or JINA_API_KEY. If both are present, the Cohere reranker (rerank-multilingual-v3.0) is used.

…iguration classes

…for retrieval, ingestion, parsing, etc.

… and retrieval)

…opment of more advanced ingestion pipelines

…the use of rerankers

…configuration fields of RAGConfig into RetrievalConfig

…laced RAGConfig)

…itioning from QuivrQARAG to QuivrQARAGLangGraph

…valConfig instead of AssistantConfig

…QARAG

…kens and max_input --> max_input_tokens

…ation file

…at-with-llm modalities

…hat-with-llm modalities

… by the front for the chat-with-llm modality

…onfig has been loaded from a yaml file

…r v3 from Mistral

…yaml configuration with the configuration pulled from the front

…erging it with the user-made configuration setup in the front

…or models (e.g. Mistral, Groq) which don't have a specific interface

…ion and merging it with the configuration coming from the front

vercel · 2024-09-18T16:57:40Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
quivrapp	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 19, 2024 8:30am

jacopo-chevallard · 2024-09-18T16:58:55Z

Tomorrow morning I'll merge the latest changes from main to avoid conflicts

… API key env variable

jacopo-chevallard · 2024-09-19T08:29:33Z

@AmineDiro cc @StanGirard I merged main and fixed a bug, I think that the PR is ready for review

linear · 2024-09-19T08:56:47Z

CORE-195 Integrating the Brain class into the RAGService class

CORE-203 Remove duplicated code in chat_llm and chat_llm_service

CORE-204 Allowing configuration of the ingestion and retrieval pipelines via a yaml file

AmineDiro

Great work !

Had some questions on interface design for quivr-core
Comments on Knowledge entities
Comments on the use of pydantic to validate data and the use of env variables

AmineDiro · 2024-09-19T07:50:28Z

backend/core/tests/test_quivr_rag.py

    chat_history = ChatHistory(uuid4(), uuid4())
-    rag_pipeline = QuivrQARAG(
-        rag_config=rag_config, llm=llm, vector_store=mem_vector_store
+    rag_pipeline = QuivrQARAGLangGraph(


not a major point but the QuivrQA interface is a bit weird, we are passing retrieval by config and llm by value. Either construct object before passing them to QuivrQA or pass the config and have a build_llm & build_retriever step. Nothing major 👍

AmineDiro · 2024-09-19T08:27:12Z

backend/core/pyproject.toml

@@ -16,6 +16,8 @@ dependencies = [
    "aiofiles>=23.1.0",
    "langchain-community>=0.2.12",
    "langchain-anthropic>=0.1.23",
+    "types-pyyaml>=6.0.12.20240808",
+    "transformers[sentencepiece]>=4.44.2",


transformers depends on torch if I am not mistaken, this is a heavy dep. Should probably be added to optional dependencies

AmineDiro · 2024-09-19T08:28:10Z

backend/pyproject.toml

@@ -40,7 +41,7 @@ dev-dependencies = [
 ]

 [tool.rye.workspace]
-members = [".", "core", "worker", "api", "docs", "core/examples/chatbot"]
+members = [".", "core", "worker", "api", "docs", "core/examples/chatbot", "core/MegaParse"]


I don't really know if Megaparse should be in core or at the same level as the worker, api, core. 🤔 ? @jacopo-chevallard @StanGirard

AmineDiro · 2024-09-19T08:28:46Z

backend/core/tests/rag_config_workflow.yaml

Are those example configs or are they used in the tests ?

AmineDiro · 2024-09-19T08:29:36Z

backend/core/quivr_core/processor/implementations/megaparse_processor.py

+        megaparse_config: MegaparseConfig = MegaparseConfig(),
    ) -> None:


Much cleaner

AmineDiro · 2024-09-19T09:42:35Z

backend/api/quivr_api/modules/chat/controller/retrieval_config_workflow.yaml

refacto : put yaml configs in config/

AmineDiro · 2024-09-19T12:47:26Z

backend/api/quivr_api/modules/chat/controller/chat_routes.py

@@ -271,6 +282,8 @@ async def create_stream_question_handler(
    for model in models:
        if brain_id == generate_uuid_from_string(model.name):


I don't get this function. This seems complicated for getting model ?

AmineDiro · 2024-09-19T12:49:28Z

backend/api/quivr_api/modules/rag_service/rag_service.py

+
+        return retrieval_config
+
+    async def _build_retrieval_config(self) -> RetrievalConfig:
        model = await self.model_service.get_model(self.model_to_use)  # type: ignore
        api_key = os.getenv(model.env_variable_name, "not-defined")


Pydantic has SecretStr type to encode keys + you can define you model as having parameter that can be loaded from environement variable :

https://docs.pydantic.dev/latest/concepts/pydantic_settings/#environment-variable-names

AmineDiro · 2024-09-19T12:52:06Z

backend/api/quivr_api/modules/rag_service/rag_service.py

-                answer=full_answer,
-                metadata=RAGResponseMetadata.model_validate(
-                    streamed_chat_history.metadata
+        if self.brain_service:


brain_service should probably be required.

AmineDiro · 2024-09-19T12:53:02Z

backend/api/quivr_api/modules/rag_service/rag_service.py


        # Save the answer to db
-        new_chat_entry = self.save_answer(question, parsed_response)
+        if self.brain_service:


Why do we need the brain_service to save chat answer ? This should depend on the chat_service ?

StanGirard · 2024-09-19T13:23:28Z

.env.example

+CHAT_LLM_CONFIG_PATH=chat_llm_config.yaml
+
+# LangSmith
+LANGCHAIN_TRACING_V2=true


Attention when you add that it uses that and can generate an issue

StanGirard · 2024-09-19T13:25:41Z

backend/api/quivr_api/modules/chat/controller/chat_routes.py

-                chat_id,
-                chat_service,
-                model_service,
+            if not os.getenv("CHAT_LLM_CONFIG_PATH"):


Not a big fan of this. I'd use a function instead to return the llm config so that later we can easily change how we load it

StanGirard · 2024-09-19T13:26:19Z

backend/api/quivr_api/modules/chat/controller/chat_routes.py

@@ -283,27 +296,43 @@ async def create_stream_question_handler(
            assert model is not None
            brain.model = model.name
            validate_authorization(user_id=current_user.id, brain_id=brain_id)
+            if not os.getenv("BRAIN_CONFIG_PATH"):


Same comment here. Use of function ?

StanGirard · 2024-09-19T13:26:32Z

backend/api/quivr_api/modules/chat/controller/chat_routes.py

-                chat_id,
-                chat_service,
-                model_service,
+            if not os.getenv("CHAT_LLM_CONFIG_PATH"):


Same comment. Use function ?

jacopo-chevallard added 30 commits September 10, 2024 12:14

add: added Pydantic class 'RerankerConfig' to manage the configuratio…

faff989

…n of rerankers

feat: adding Cohere / Jina (multilingual) rerankers. To use reranking…

b332cfd

…, you need to set the env variables COHERE_API_KEY or JINA_API_KEY. If both are present, the Cohere reranker (rerank-multilingual-v3.0) is used.

add: added QuivrBaseConfig class as the base class for all Quivr conf…

ce71e3c

…iguration classes

add: added configuration for MegaParse

3a0bb05

feat: using QuivrBaseConfig as the base class, adding configurations …

db443d8

…for retrieval, ingestion, parsing, etc.

add: added example yaml file to configure our RAG pipeline (ingestion…

5cfe2c7

… and retrieval)

add: adding default initializations to the different config classes

74f67a4

add: adding MegaParse as an internal dependency to simplify the devel…

47334c4

…opment of more advanced ingestion pipelines

chore: adding MegaParse as an internal package

1111a27

feat: adding configuration to Megaparse

075f8c0

feat: adding logger, using MegaParseConfig to pass options to parsers

9547444

feat: using MegaParseConfig

3cb08a4

feat: checking that the correct API key is provided when configuring …

f9f1619

…the use of rerankers

fix: correctly passing parsers (processor) kwargs

adf7231

chore: adding MegaParse as an internal dependency

35f19cb

feat: improving logging message

c332e81

feat: changing default RAG configuration

09d3a31

feat: adding example usage of yaml configuration

d87a804

refactor: changed RAGConfig --> AssistantConfig

5caae05

feat: renaming RAGConfig into AssistantConfig and moving the current …

2bafc1d

…configuration fields of RAGConfig into RetrievalConfig

feat: using RetrievalConfig instead of AssistantConfig (which has rep…

5b7b1b0

…laced RAGConfig)

feat: update test to use RetrievalConfig

5f4087f

feat: using QuivrQARAGLangGraph instead of QuivrQARAG

3f67398

refactor: renaming build_langgraph_chain to build_chain to ease trans…

3a3afe7

…itioning from QuivrQARAG to QuivrQARAGLangGraph

feat: testing QuivrQARAGLangGraph instead of QuivrQARAG; using Retrie…

061c9c6

…valConfig instead of AssistantConfig

fix: making processor_kwargs an optional argument

232b4c8

feat: using by default the QuivrQARAGLangGraph class instead of Quivr…

e0a2990

…QARAG

refactor: in LLMEndpointConfig, renaming max_tokens --> max_output_to…

dddfacd

…kens and max_input --> max_input_tokens

style: adding arguments to simplify code understanding

8ee316b

feat: fixing ruff errors, using Rerankers settings from yaml configur…

ca9ec90

…ation file

jacopo-chevallard added 16 commits September 18, 2024 12:07

docs: added important comment to the workflow configuration file

e3a1a87

feat: removing ChatLLMService since we're now using RAGService for ch…

7394e78

…at-with-llm modalities

feat: removing ChatLLM since we're now using QuivrQARAGLangGraph or c…

d5de45f

…hat-with-llm modalities

feat: removing tests of ChatLLM

6f8942b

feat: removing supplier/model setting, since they are passed directly…

d760b41

… by the front for the chat-with-llm modality

feat: enabling explicitly setting the model name after the RetrievalC…

f82892c

…onfig has been loaded from a yaml file

feat: removing ChatLLM

8ed078f

chore: adding sentencepiece dependency, necessary to run the tokenize…

8473f39

…r v3 from Mistral

refactor: moved yaml file

c9e9982

feat: updating list of supported models, enabling simpler merging of …

1291143

…yaml configuration with the configuration pulled from the front

feat: loading the yaml configuration in quivr-api / chat_routes and m…

779ff43

…erging it with the user-made configuration setup in the front

feat: re-establishing the use of the LangChain ChatOpenAI interface f…

2ff0786

…or models (e.g. Mistral, Groq) which don't have a specific interface

feat: updated example .env file

b50a901

feat: moving into a function the logic of loading the yaml configurat…

8ac4987

…ion and merging it with the configuration coming from the front

test: fixing test errors

6fb7bc7

test: fixing test errors

1a0c984

dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Sep 18, 2024

jacopo-chevallard requested review from AmineDiro and StanGirard September 18, 2024 16:57

dosubot bot added the area: backend Related to backend functionality or under the /backend directory label Sep 18, 2024

vercel bot deployed to Preview September 18, 2024 16:59 View deployment

jacopo-chevallard added 2 commits September 19, 2024 10:05

Merge branch 'main' into feat/workflows

d6b3387

fix: fixing wrong match between model supplier (OpenAI/Anthropic) and…

ea46138

… API key env variable

vercel bot deployed to Preview September 19, 2024 08:30 View deployment

AmineDiro reviewed Sep 19, 2024

View reviewed changes

StanGirard reviewed Sep 19, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: introducing configurable retrieval workflows #3227

feat: introducing configurable retrieval workflows #3227

jacopo-chevallard commented Sep 18, 2024 •

edited

Loading

vercel bot commented Sep 18, 2024 •

edited

Loading

jacopo-chevallard commented Sep 18, 2024

jacopo-chevallard commented Sep 19, 2024

linear bot commented Sep 19, 2024

AmineDiro left a comment

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

AmineDiro Sep 19, 2024

StanGirard Sep 19, 2024

StanGirard Sep 19, 2024

StanGirard Sep 19, 2024

StanGirard Sep 19, 2024

		megaparse_config: MegaparseConfig = MegaparseConfig(),
		) -> None:

		@@ -271,6 +282,8 @@ async def create_stream_question_handler(
		for model in models:
		if brain_id == generate_uuid_from_string(model.name):

feat: introducing configurable retrieval workflows #3227

Are you sure you want to change the base?

feat: introducing configurable retrieval workflows #3227

Conversation

jacopo-chevallard commented Sep 18, 2024 • edited Loading

Description

Checklist before requesting a review

Screenshots (if appropriate):

vercel bot commented Sep 18, 2024 • edited Loading

jacopo-chevallard commented Sep 18, 2024

jacopo-chevallard commented Sep 19, 2024

linear bot commented Sep 19, 2024

AmineDiro left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jacopo-chevallard commented Sep 18, 2024 •

edited

Loading

vercel bot commented Sep 18, 2024 •

edited

Loading