DuckDuckLLM = DuckDuckGo + Open-source LLM
It simply searches duckduckgo for the query and then creates a stuffed prompt for the LLM. The LLM generates a response based on the user query and the reference articles.
For v0, we simply pass the title and the body of the top-3 articles to the LLM.
- Python 3.11+
- Poetry
- LM Studio
- Download model: You could use any model you want. For this app, I am using openhermes-2.5-mistral-7b.Q5_K_M.gguf.
- Start LM Studio Server (OpenAI-compatible API)
- Run streamlit app
poetry shell poetry install
streamlit run app.py