This demo application showcases a RAG-System in it's most simple form.
To start, you need to install the JS dependencies:
yarn installAdditionally install ollama and the used models:
ollama pull llama3
ollama pull nomic-embed-textAfter installing these dependencies, start the ui:
yarn devand visit your localhost to get access to the RAG-based chat.
Some of the suggested enhancements are implemented in the enhancements directory. To run them, use the respective node script from package.json. E.g. run yarn embeddings
- embeddings Shows how a different embedding e.g. for german language could drastically improve the quality of the RAG-System
- sources Also print used sources to enable the user to see what documents were used
- split Improve input by splitting text into smaller chunks
To evaluate the quality of the current RAG-Configuration a set of tools was provided. To separate the code from the minimal usecase, all chain definitions are defined in the evaluations directory. To serve the evaluated RAG-Pipeline run yarn serve-for-evaluation
For evaluation a strong LLM is needed to get the best results. Per default this implementation uses Azure OpenAI instances. To connect them, you need to locally add the file azure.env with the following content:
export AZURE_OPENAI_API_INSTANCE_NAME=<AZURE_REGION>
export AZURE_OPENAI_API_DEPLOYMENT_NAME=<AZURE_DEPLOYMENT>
export AZURE_OPENAI_API_KEY=<AZURE_AUTH_KEY> # One of either Key1 / Key2 from your azure openAI instance
export AZURE_OPENAI_API_VERSION="2024-02-01"
export AZURE_OPENAI_BASE_PATH=https://<AZURE_DOMAIN>/openai/deployments- Manually generate some test data and store them in
evaluations/data/questions.json. This can be done with ChatGPT as described in the blog article. Alternatively simply use the questions provided. - Generate test data using the current RAG-Configuration:
yarn generate-test-data <name-of-test> - Evaluate the quality of the RAG:
yarn evaluate <name-of-test> - Check the results with:
yarn summarize <name-of-test>
Evaluation criterias can be found under evaluation/evaluators
It was built using