Question about the RAG process

[hamzachebbigit]

Hello :) I'm new to Haystack and want to understand how RAG works.

So every step I completely understood in your tutorial except the part about the prompt: You see in your documentation you said that the prompt should have 2 inputs: the documents and the question.... but I don't understand how it works exactly because when I tested it with Pinecone, it fetches the content of the chunks inside the Pinecone vector dataset and puts them into the prompt, then passes them to chatgpt (the llm in this case); If my understanding is correct, and please explain further if I'm wrong hhh what's the point of all this hassle to embed the chunks and store them just to give the exact text in the prompt? I could do the same thing, just copy and paste the text and I have a problem with the maximum number of tokens in the prompt. Maybe Iam missing something. I thought the idea of RAG is to avoid all this and tell the llm to fetch the content for each time with ID maybe and take the content of it...would this be possible with defining new pipelines ? Thank you in advance :)

[LoGic142]

Are you running into a problem with the max tokens allowed? Or like you want a general description of a typical RAG pipeline?

[hamzachebbigit]

Hello thank you for the reply :D
Actually both hhhh. I got the error of the max tokens and frankly I want a new solution; I want a breakthrough of this typical {doc in documents} in every prompt maybe something using RAG pipelines like we need to send query for example for the ID of the vector/vectors from the database that are similar to the question asked and the LLM can access the content of the ID of the same vector. I dont know maybe I am saying things that are not logical bare with me please hhh but I dont think this way of introducing content inside the prompt is very optimal specially if you work with a business that have a lot of information 👍 Thanks in advance

[feinmann]

It is mentioned in the tutorial notebook 27_First_RAG_Pipeline.ipynb: "This Retriever will get the relevant documents to the query." The documents get embedded, because the embeddings can be used to compare the content of the documents with the query (also embedded) and thereby filter the relevant documents, that then get included into the prompt.