nemo-reranker.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "26161adb-2c91-4298-af93-3d0b8a67d12d",
   "metadata": {},
   "source": [
    "# Using the NVIDIA Reranker to optimize retrieval pipelines\n",
    "\n",
    "Reranking is crucial for achieving high accuracy and efficiency in retrieval pipelines. It plays a vital role, particularly when the pipeline incorporates citations from diverse datastores, where each datastore may employ its own unique similarity scoring algorithm. Reranking serves two primary purposes:\n",
    "\n",
    "1. Improving accuracy for individual citations within each datastore.\n",
    "2. Integrating results from multiple datastores to provide a cohesive and relevant set of citations.\n",
    "\n",
    "This playbook goes over how to use the NeMo Retriever Text Reranking NIM (Text Reranking NIM) with LangChain for document compression and retrieval via the `NVIDIARerank` class."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c5fca55-e86a-42db-b932-aa4d17c2a6b1",
   "metadata": {},
   "source": [
    "## Step 1 - Install the requirements\n",
    "\n",
    "For this example, we install langchain packages and other dependancies needed to build our pipeline. Additionally, we download the reranker model that we will be using for our examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "72d3d825-57d0-4957-83f3-833e5dcda185",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: faiss_cpu==1.8.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (1.8.0)\n",
      "Requirement already satisfied: numpy in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from faiss_cpu==1.8.0) (1.26.4)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: fastapi==0.104.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.104.1)\n",
      "Requirement already satisfied: typing-extensions>=4.8.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from fastapi==0.104.1) (4.12.2)\n",
      "Requirement already satisfied: starlette<0.28.0,>=0.27.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from fastapi==0.104.1) (0.27.0)\n",
      "Requirement already satisfied: anyio<4.0.0,>=3.7.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from fastapi==0.104.1) (3.7.1)\n",
      "Requirement already satisfied: pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from fastapi==0.104.1) (2.8.2)\n",
      "Requirement already satisfied: idna>=2.8 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi==0.104.1) (3.7)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi==0.104.1) (1.2.2)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<4.0.0,>=3.7.1->fastapi==0.104.1) (1.3.1)\n",
      "Requirement already satisfied: pydantic-core==2.20.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi==0.104.1) (2.20.1)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic!=1.8,!=1.8.1,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0,>=1.7.4->fastapi==0.104.1) (0.7.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: langchain==0.1.11 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.1.11)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (8.5.0)\n",
      "Requirement already satisfied: langchain-text-splitters<0.1,>=0.0.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (0.0.2)\n",
      "Requirement already satisfied: numpy<2,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (1.26.4)\n",
      "Requirement already satisfied: async-timeout<5.0.0,>=4.0.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (4.0.3)\n",
      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (0.6.7)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (3.10.1)\n",
      "Requirement already satisfied: langchain-community<0.1,>=0.0.25 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (0.0.25)\n",
      "Requirement already satisfied: requests<3,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (2.32.3)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (2.0.32)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (6.0.1)\n",
      "Requirement already satisfied: langsmith<0.2.0,>=0.1.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (0.1.96)\n",
      "Requirement already satisfied: langchain-core<0.2,>=0.1.29 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (0.1.29)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (2.8.2)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain==0.1.11) (1.33)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (23.2.0)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (2.3.4)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (1.4.1)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (1.9.4)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (1.3.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain==0.1.11) (6.0.5)\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.1.11) (3.21.3)\n",
      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain==0.1.11) (0.9.0)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain==0.1.11) (3.0.0)\n",
      "Requirement already satisfied: packaging<24.0,>=23.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2,>=0.1.29->langchain==0.1.11) (23.2)\n",
      "Requirement already satisfied: anyio<5,>=3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2,>=0.1.29->langchain==0.1.11) (3.7.1)\n",
      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langsmith<0.2.0,>=0.1.17->langchain==0.1.11) (3.10.6)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain==0.1.11) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.20.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain==0.1.11) (2.20.1)\n",
      "Requirement already satisfied: typing-extensions>=4.6.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain==0.1.11) (4.12.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain==0.1.11) (2.2.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain==0.1.11) (3.7)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain==0.1.11) (3.3.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain==0.1.11) (2024.7.4)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain==0.1.11) (3.0.3)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain==0.1.11) (1.2.2)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.2,>=0.1.29->langchain==0.1.11) (1.3.1)\n",
      "Requirement already satisfied: mypy-extensions>=0.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain==0.1.11) (1.0.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: langchain-community==0.0.25 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.0.25)\n",
      "Requirement already satisfied: SQLAlchemy<3,>=1.4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (2.0.32)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (8.5.0)\n",
      "Requirement already satisfied: requests<3,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (2.32.3)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.8.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (3.10.1)\n",
      "Requirement already satisfied: dataclasses-json<0.7,>=0.5.7 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (0.6.7)\n",
      "Requirement already satisfied: langsmith<0.2.0,>=0.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (0.1.96)\n",
      "Requirement already satisfied: numpy<2,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (1.26.4)\n",
      "Requirement already satisfied: langchain-core<0.2.0,>=0.1.28 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (0.1.29)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-community==0.0.25) (6.0.1)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (2.3.4)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (1.9.4)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (1.4.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (6.0.5)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (4.0.3)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.8.3->langchain-community==0.0.25) (1.3.1)\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community==0.0.25) (3.21.3)\n",
      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json<0.7,>=0.5.7->langchain-community==0.0.25) (0.9.0)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (2.8.2)\n",
      "Requirement already satisfied: anyio<5,>=3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (3.7.1)\n",
      "Requirement already satisfied: packaging<24.0,>=23.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (23.2)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (1.33)\n",
      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langsmith<0.2.0,>=0.1.0->langchain-community==0.0.25) (3.10.6)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-community==0.0.25) (3.3.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-community==0.0.25) (2024.7.4)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-community==0.0.25) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-community==0.0.25) (2.2.2)\n",
      "Requirement already satisfied: greenlet!=0.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain-community==0.0.25) (3.0.3)\n",
      "Requirement already satisfied: typing-extensions>=4.6.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from SQLAlchemy<3,>=1.4->langchain-community==0.0.25) (4.12.2)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (1.3.1)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (1.2.2)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (3.0.0)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.20.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core<0.2.0,>=0.1.28->langchain-community==0.0.25) (2.20.1)\n",
      "Requirement already satisfied: mypy-extensions>=0.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json<0.7,>=0.5.7->langchain-community==0.0.25) (1.0.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: langchain-core==0.1.29 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.1.29)\n",
      "Requirement already satisfied: anyio<5,>=3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (3.7.1)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (6.0.1)\n",
      "Requirement already satisfied: requests<3,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (2.32.3)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (8.5.0)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (2.8.2)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (1.33)\n",
      "Requirement already satisfied: langsmith<0.2.0,>=0.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (0.1.96)\n",
      "Requirement already satisfied: packaging<24.0,>=23.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core==0.1.29) (23.2)\n",
      "Requirement already satisfied: idna>=2.8 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core==0.1.29) (3.7)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core==0.1.29) (1.2.2)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core==0.1.29) (1.3.1)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain-core==0.1.29) (3.0.0)\n",
      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langsmith<0.2.0,>=0.1.0->langchain-core==0.1.29) (3.10.6)\n",
      "Requirement already satisfied: pydantic-core==2.20.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core==0.1.29) (2.20.1)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core==0.1.29) (0.7.0)\n",
      "Requirement already satisfied: typing-extensions>=4.6.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core==0.1.29) (4.12.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core==0.1.29) (2.2.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core==0.1.29) (2024.7.4)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core==0.1.29) (3.3.2)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: langchain-nvidia-ai-endpoints==0.1.4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.1.4)\n",
      "Requirement already satisfied: pillow<11.0.0,>=10.0.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-nvidia-ai-endpoints==0.1.4) (10.4.0)\n",
      "Requirement already satisfied: aiohttp<4.0.0,>=3.9.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-nvidia-ai-endpoints==0.1.4) (3.10.1)\n",
      "Requirement already satisfied: langchain-core<0.3,>=0.1.27 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-nvidia-ai-endpoints==0.1.4) (0.1.29)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (1.9.4)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (6.0.5)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (23.2.0)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (1.4.1)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (4.0.3)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (1.3.1)\n",
      "Requirement already satisfied: aiohappyeyeballs>=2.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from aiohttp<4.0.0,>=3.9.1->langchain-nvidia-ai-endpoints==0.1.4) (2.3.4)\n",
      "Requirement already satisfied: jsonpatch<2.0,>=1.33 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (1.33)\n",
      "Requirement already satisfied: requests<3,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (2.32.3)\n",
      "Requirement already satisfied: tenacity<9.0.0,>=8.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (8.5.0)\n",
      "Requirement already satisfied: PyYAML>=5.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (6.0.1)\n",
      "Requirement already satisfied: langsmith<0.2.0,>=0.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (0.1.96)\n",
      "Requirement already satisfied: packaging<24.0,>=23.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (23.2)\n",
      "Requirement already satisfied: pydantic<3,>=1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (2.8.2)\n",
      "Requirement already satisfied: anyio<5,>=3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (3.7.1)\n",
      "Requirement already satisfied: sniffio>=1.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (1.3.1)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (1.2.2)\n",
      "Requirement already satisfied: idna>=2.8 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio<5,>=3->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (3.7)\n",
      "Requirement already satisfied: jsonpointer>=1.9 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from jsonpatch<2.0,>=1.33->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (3.0.0)\n",
      "Requirement already satisfied: orjson<4.0.0,>=3.9.14 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langsmith<0.2.0,>=0.1.0->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (3.10.6)\n",
      "Requirement already satisfied: annotated-types>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (0.7.0)\n",
      "Requirement already satisfied: pydantic-core==2.20.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (2.20.1)\n",
      "Requirement already satisfied: typing-extensions>=4.6.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from pydantic<3,>=1->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (4.12.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (2.2.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (2024.7.4)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests<3,>=2->langchain-core<0.3,>=0.1.27->langchain-nvidia-ai-endpoints==0.1.4) (3.3.2)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: numpy==1.26.4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (1.26.4)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: sentence-transformers==2.2.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (2.2.2)\n",
      "Requirement already satisfied: nltk in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (3.8.1)\n",
      "Requirement already satisfied: torch>=1.6.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (2.4.0)\n",
      "Requirement already satisfied: sentencepiece in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (0.2.0)\n",
      "Requirement already satisfied: numpy in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (1.26.4)\n",
      "Requirement already satisfied: tqdm in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (4.66.5)\n",
      "Requirement already satisfied: torchvision in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (0.19.0)\n",
      "Requirement already satisfied: transformers<5.0.0,>=4.6.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (4.43.4)\n",
      "Requirement already satisfied: scipy in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (1.14.0)\n",
      "Requirement already satisfied: huggingface-hub>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (0.24.5)\n",
      "Requirement already satisfied: scikit-learn in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sentence-transformers==2.2.2) (1.5.1)\n",
      "Requirement already satisfied: fsspec>=2023.5.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (2024.6.1)\n",
      "Requirement already satisfied: packaging>=20.9 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (23.2)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (6.0.1)\n",
      "Requirement already satisfied: requests in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (2.32.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (4.12.2)\n",
      "Requirement already satisfied: filelock in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (3.15.4)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.0.106)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==9.1.0.70 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (9.1.0.70)\n",
      "Requirement already satisfied: triton==3.0.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (3.0.0)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (11.0.2.54)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (2.20.5)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.105)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (10.3.2.106)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.3.1)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (11.4.5.107)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.105)\n",
      "Requirement already satisfied: sympy in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (1.13.1)\n",
      "Requirement already satisfied: jinja2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (3.1.4)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (12.1.105)\n",
      "Requirement already satisfied: networkx in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torch>=1.6.0->sentence-transformers==2.2.2) (3.3)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.6.0->sentence-transformers==2.2.2) (12.6.20)\n",
      "Requirement already satisfied: tokenizers<0.20,>=0.19 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers==2.2.2) (0.19.1)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers==2.2.2) (0.4.4)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from transformers<5.0.0,>=4.6.0->sentence-transformers==2.2.2) (2024.7.24)\n",
      "Requirement already satisfied: joblib in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->sentence-transformers==2.2.2) (1.4.2)\n",
      "Requirement already satisfied: click in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->sentence-transformers==2.2.2) (8.1.7)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from scikit-learn->sentence-transformers==2.2.2) (3.5.0)\n",
      "Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from torchvision->sentence-transformers==2.2.2) (10.4.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from jinja2->torch>=1.6.0->sentence-transformers==2.2.2) (2.1.5)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (2.2.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (3.3.2)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (2024.7.4)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->huggingface-hub>=0.4.0->sentence-transformers==2.2.2) (3.7)\n",
      "Requirement already satisfied: mpmath<1.4,>=1.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from sympy->torch>=1.6.0->sentence-transformers==2.2.2) (1.3.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n",
      "Requirement already satisfied: unstructured==0.11.8 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (0.11.8)\n",
      "Requirement already satisfied: langdetect in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (1.0.9)\n",
      "Requirement already satisfied: chardet in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (5.2.0)\n",
      "Requirement already satisfied: filetype in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (1.2.0)\n",
      "Requirement already satisfied: beautifulsoup4 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (4.12.3)\n",
      "Requirement already satisfied: python-magic in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (0.4.27)\n",
      "Requirement already satisfied: dataclasses-json in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (0.6.7)\n",
      "Requirement already satisfied: unstructured-client in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (0.25.2)\n",
      "Requirement already satisfied: nltk in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (3.8.1)\n",
      "Requirement already satisfied: typing-extensions in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (4.12.2)\n",
      "Requirement already satisfied: requests in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (2.32.3)\n",
      "Requirement already satisfied: python-iso639 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (2024.4.27)\n",
      "Requirement already satisfied: backoff in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (2.2.1)\n",
      "Requirement already satisfied: wrapt in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (1.16.0)\n",
      "Requirement already satisfied: numpy in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (1.26.4)\n",
      "Requirement already satisfied: tabulate in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (0.9.0)\n",
      "Requirement already satisfied: rapidfuzz in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (3.9.5)\n",
      "Requirement already satisfied: lxml in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (5.2.2)\n",
      "Requirement already satisfied: emoji in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured==0.11.8) (2.12.1)\n",
      "Requirement already satisfied: soupsieve>1.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from beautifulsoup4->unstructured==0.11.8) (2.5)\n",
      "Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json->unstructured==0.11.8) (0.9.0)\n",
      "Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from dataclasses-json->unstructured==0.11.8) (3.21.3)\n",
      "Requirement already satisfied: six in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from langdetect->unstructured==0.11.8) (1.16.0)\n",
      "Requirement already satisfied: click in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->unstructured==0.11.8) (8.1.7)\n",
      "Requirement already satisfied: regex>=2021.8.3 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->unstructured==0.11.8) (2024.7.24)\n",
      "Requirement already satisfied: joblib in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->unstructured==0.11.8) (1.4.2)\n",
      "Requirement already satisfied: tqdm in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from nltk->unstructured==0.11.8) (4.66.5)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->unstructured==0.11.8) (2024.7.4)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->unstructured==0.11.8) (3.3.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->unstructured==0.11.8) (2.2.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from requests->unstructured==0.11.8) (3.7)\n",
      "Requirement already satisfied: requests-toolbelt>=1.0.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (1.0.0)\n",
      "Requirement already satisfied: mypy-extensions>=1.0.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (1.0.0)\n",
      "Requirement already satisfied: pypdf>=4.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (4.3.1)\n",
      "Requirement already satisfied: deepdiff>=6.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (7.0.1)\n",
      "Requirement already satisfied: jsonpath-python>=1.0.6 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (1.0.6)\n",
      "Requirement already satisfied: nest-asyncio>=1.6.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (1.6.0)\n",
      "Requirement already satisfied: packaging>=23.1 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (23.2)\n",
      "Requirement already satisfied: httpx>=0.27.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (0.27.0)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from unstructured-client->unstructured==0.11.8) (2.9.0.post0)\n",
      "Requirement already satisfied: ordered-set<4.2.0,>=4.1.0 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from deepdiff>=6.0->unstructured-client->unstructured==0.11.8) (4.1.0)\n",
      "Requirement already satisfied: anyio in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpx>=0.27.0->unstructured-client->unstructured==0.11.8) (3.7.1)\n",
      "Requirement already satisfied: httpcore==1.* in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpx>=0.27.0->unstructured-client->unstructured==0.11.8) (1.0.5)\n",
      "Requirement already satisfied: sniffio in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpx>=0.27.0->unstructured-client->unstructured==0.11.8) (1.3.1)\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from httpcore==1.*->httpx>=0.27.0->unstructured-client->unstructured==0.11.8) (0.14.0)\n",
      "Requirement already satisfied: exceptiongroup in /home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages (from anyio->httpx>=0.27.0->unstructured-client->unstructured==0.11.8) (1.2.2)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.2\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpip install --upgrade pip\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!pip install faiss_cpu==1.8.0\n",
    "!pip install fastapi==0.104.1\n",
    "!pip install langchain==0.1.11\n",
    "!pip install langchain-community==0.0.25\n",
    "!pip install langchain-core==0.1.29\n",
    "!pip install langchain-nvidia-ai-endpoints==0.1.4\n",
    "!pip install numpy==1.26.4\n",
    "!pip install sentence-transformers==2.2.2\n",
    "!pip install unstructured==0.11.8"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "a6cd23ba-fedc-4b45-b1d0-5302bb734c58",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "--2024-08-05 22:29:22--  https://raw.githubusercontent.com/brevdev/notebooks/main/assets/reranker-launchable/install-docker.sh\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.111.133, 185.199.109.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 852 [text/plain]\n",
      "Saving to: ‘install-docker’\n",
      "\n",
      "     0K                                                       100%  105M=0s\n",
      "\n",
      "2024-08-05 22:29:22 (105 MB/s) - ‘install-docker’ saved [852/852]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "wget https://raw.githubusercontent.com/brevdev/notebooks/main/assets/reranker-launchable/install-docker.sh -O install-docker\n",
    "chmod +x install-docker\n",
    "sudo ./install-docker"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f8321bc-4aef-4849-a890-e01a4d443953",
   "metadata": {},
   "source": [
    "This step requires your NGC API Key. Additionally, it could take up to 5 minutes to pull the model. The script will continue checking and exit when the model is succesfully running"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "112984e2-be96-4bbb-9518-7b7015933770",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "--2024-08-05 22:30:31--  https://raw.githubusercontent.com/brevdev/notebooks/main/assets/reranker-launchable/setup-nim.sh\n",
      "Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ...\n",
      "Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 1060 (1.0K) [text/plain]\n",
      "Saving to: ‘setup-nim’\n",
      "\n",
      "     0K .                                                     100%  125M=0s\n",
      "\n",
      "2024-08-05 22:30:32 (125 MB/s) - ‘setup-nim’ saved [1060/1060]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "%%bash\n",
    "\n",
    "wget https://raw.githubusercontent.com/brevdev/notebooks/main/assets/reranker-launchable/setup-nim.sh -O setup-nim\n",
    "chmod +x setup-nim\n",
    "export NGC_API_KEY=<enter key here>\n",
    "sudo -E ./setup-nim"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7a96fd96-dc96-47c0-847b-fc3675471e5f",
   "metadata": {},
   "source": [
    "# Step 2 - Building a pipeline using Langchain\n",
    "\n",
    "NVIDIA offers a `ChatNVIDIA` package that lets you work with different NVIDIA NIM models. For this example, we run the reranker NIM locally and use the API catalog to access Meta Llama 3.1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "1b724f59-cc93-41a4-9790-38254984b1f5",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/ubuntu/.pyenv/versions/3.10.14/lib/python3.10/site-packages/langchain_nvidia_ai_endpoints/_common.py:486: UserWarning: Found meta/llama-3.1-8b-instruct in available_models, but type is unknown and inference may fail.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from langchain_nvidia_ai_endpoints import ChatNVIDIA\n",
    "\n",
    "os.environ[\"NVIDIA_API_KEY\"] = \"enter api key here\"\n",
    "llm = ChatNVIDIA(model=\"meta/llama-3.1-8b-instruct\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "b9b9963f-901b-4ed4-abec-94b9731a2298",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "prompt = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", (\n",
    "        \"You are a helpful and friendly AI!\"\n",
    "        \"Your responses should be concise and no longer than two sentences.\"\n",
    "        \"Say you don't know if you don't have this information.\"\n",
    "    )),\n",
    "    (\"user\", \"{question}\")\n",
    "])\n",
    "\n",
    "chain = prompt | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "a9379422-459b-4b1f-9300-4ad16a27478a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A CPU (Central Processing Unit) is the primary component that performs calculations and executes instructions for a computer, whereas a GPU (Graphics Processing Unit) is specifically designed to handle large amounts of graphical data and complex calculations, typically found in gaming and graphics-intensive applications.\n"
     ]
    }
   ],
   "source": [
    "print(chain.invoke({\"question\": \"What's the difference between a GPU and a CPU?\"}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "33033926-a7d3-41aa-8e29-b57aa2affde9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unfortunately, I don't have information on what the H in NVIDIA H200 specifically stands for. However, the \"H\" in NVIDIA H100, H200, and other related models is likely a classification or a codename indicating a high-end or high-performance variant.\n"
     ]
    }
   ],
   "source": [
    "print(chain.invoke({\"question\": \"What does the H in the NVIDIA H200 stand for?\"}))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "154a052d-cb2d-40d2-9551-cf436e256bdd",
   "metadata": {},
   "source": [
    "# Reranking with Text Reranking NIM\n",
    "To answer the previous question, lets build a simple retrieval and reranking pipeline to find the most relevant piece of information to the query.\n",
    "\n",
    "Load the NVIDIA H200 Datasheet to use in the retrieval pipeline. LangChain provides a variety of document loaders for various types of documents, such as HTML, PDF, and code, from sources and locations such as private S3 buckets and public websites. The following example uses a LangChain PyPDFLoader to load a datasheet about the NVIDIA H200 Tensor Core GPU"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "fe82baef-e34a-4276-ad34-127782dd4202",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Document(page_content='NVIDIA H200 Tensor Core GPU\\u2002|\\u2002Datasheet\\u2002|\\u2002 1NVIDIA H200 Tensor Core GPU\\nSupercharging AI and HPC workloads.\\nHigher Performance With Larger, Faster Memory\\nThe NVIDIA H200 Tensor Core GPU supercharges generative AI and high-\\nperformance computing (HPC) workloads with game-changing performance  \\nand memory capabilities. \\nBased on the NVIDIA Hopper™ architecture , the NVIDIA H200 is the first GPU to \\noffer 141 gigabytes (GB) of HBM3e memory at 4.8 terabytes per second (TB/s)—\\nthat’s nearly double the capacity of the NVIDIA H100 Tensor Core GPU  with \\n1.4X more memory bandwidth. The H200’s larger and faster memory accelerates \\ngenerative AI and large language models, while advancing scientific computing for \\nHPC workloads with better energy efficiency and lower total cost of ownership. \\nUnlock Insights With High-Performance LLM Inference\\nIn the ever-evolving landscape of AI, businesses rely on large language models to \\naddress a diverse range of inference needs. An AI inference  accelerator must deliver the \\nhighest throughput at the lowest TCO when deployed at scale for a massive user base. \\nThe H200 doubles inference performance compared to H100 GPUs when handling \\nlarge language models such as Llama2 70B.\\n.\\nPreliminary specifications. May be subject to change.\\nLlama2 13B: ISL 128, OSL 2K | Throughput | H100 SXM 1x GPU BS 64 | H200 SXM 1x GPU BS 128\\nGPT-3 175B: ISL 80, OSL 200 | x8 H100 SXM GPUs BS 64 | x8 H200 SXM GPUs BS 128\\nLlama2 70B: ISL 2K, OSL 128 | Throughput | H100 SXM 1x GPU BS 8 | H200 SXM 1x GPU BS 32.Key Features\\n >141GB of HBM3e GPU memory\\n >4.8TB/s of memory bandwidth\\n >4 petaFLOPS of FP8 performance\\n >2X LLM inference performance\\n >110X HPC performance\\nDatasheet', metadata={'source': 'https://nvdam.widen.net/content/udc6mzrk7a/original/hpc-datasheet-sc23-h200-datasheet-3002446.pdf', 'page': 0})"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from langchain_community.document_loaders import PyPDFLoader\n",
    "\n",
    "loader = PyPDFLoader(\"https://nvdam.widen.net/content/udc6mzrk7a/original/hpc-datasheet-sc23-h200-datasheet-3002446.pdf\")\n",
    "\n",
    "document = loader.load()\n",
    "document[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6415c4b0-50c5-497d-8006-02bd3562f60c",
   "metadata": {},
   "source": [
    "Once documents have been loaded, they are often transformed. One method of transformation is known as **chunking, which breaks down large pieces of text, such as a long document, into smaller segments. This technique is valuable because it helps **optimize the relevance of the content returned from the vector database.**\n",
    "\n",
    "LangChain provides a variety of document transformers, such as text splitters. The following example uses a `RecursiveCharacterTextSplitter`. The RecursiveCharacterTextSplitter is divides a large body of text into smaller chunks based on a specified chunk size. It employs recursion as its core mechanism for splitting text, utilizing a predefined set of characters, such as “\\n\\n”, “\\n”, ” “, and “”, to determine where splits should occur. The process begins by attempting to split the text using the first character in the set. If the resulting chunks are still larger than the desired chunk size, it proceeds to the next character in the set and attempts to split again. This process continues until all chunks adhere to the specified maximum chunk size.\n",
    "\n",
    "There are some nuanced complexities to text splitting since, in theory, semantically related text should be kept together."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "bb7b3b3a-6394-4c89-bb58-1b1fe662f505",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of chunks from the document: 16\n"
     ]
    }
   ],
   "source": [
    "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
    "\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    chunk_size=500,\n",
    "    chunk_overlap=100,\n",
    "    separators=[\"\\n\\n\", \"\\n\", \".\", \";\", \",\", \" \", \"\"],\n",
    ")\n",
    "\n",
    "document_chunks = text_splitter.split_documents(document)\n",
    "print(\"Number of chunks from the document:\", len(document_chunks))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "42157653-c20f-4a4e-98d6-fa159ee5819c",
   "metadata": {},
   "source": [
    "The following example shows how to use LangChain to interact with Text Reranking NIM using the `NVIDIAReranking` class from the same `langchain-nvidia-ai-endpoints` package as the first example"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "41eb3afd-c525-4618-9813-efcaca0bd22c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_nvidia_ai_endpoints import NVIDIARerank\n",
    "\n",
    "query = \"What does the H in the NVIDIA H200 stand for?\"\n",
    "\n",
    "# Initialize and connect to a NeMo Retriever Text Reranking NIM running at localhost:8000\n",
    "reranker = NVIDIARerank(model=\"nvidia/nv-rerankqa-mistral-4b-v3\",\n",
    "                        base_url=\"http://localhost:8000/v1\")\n",
    "\n",
    "reranked_chunks = reranker.compress_documents(query=query,\n",
    "                                              documents=document_chunks)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "37480671-4ce8-4722-ba73-88d1be22b1ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Relevance Score:16.625, Page Content:NVIDIA H200 Tensor Core GPU | Datasheet |  1NVIDIA H200 Tensor Core GPU\n",
      "Supercharging AI and HPC workloads.\n",
      "Higher Performance With Larger, Faster Memory\n",
      "The NVIDIA H200 Tensor Core GPU supercharges generative AI and high-\n",
      "performance computing (HPC) workloads with game-changing performance  \n",
      "and memory capabilities. \n",
      "Based on the NVIDIA Hopper™ architecture , the NVIDIA H200 is the first GPU to \n",
      "offer 141 gigabytes (GB) of HBM3e memory at 4.8 terabytes per second (TB/s)—...\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Relevance Score:11.5078125, Page Content:NVIDIA H200 Tensor Core GPU | Datasheet |  3Unleashing AI Acceleration for Mainstream Enterprise Servers \n",
      "With H200 NVL\n",
      "The NVIDIA H200 NVL is the ideal choice for customers with space constraints within  \n",
      "the data center, delivering acceleration for every AI and HPC workload regardless of size. \n",
      "With a 1.5X memory increase and a 1.2X bandwidth increase over the previous generation, \n",
      "customers can fine-tune LLMs within a few hours and experience LLM inference 1.8X faster....\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Relevance Score:8.2578125, Page Content:NVIDIA H200 Tensor Core GPU | Datasheet |  2Supercharge High-Performance Computing\n",
      "Memory bandwidth is crucial for HPC applications, as it enables faster data \n",
      "transfer and reduces complex processing bottlenecks. For memory-intensive \n",
      "HPC applications like simulations, scientific research, and artificial intelligence, \n",
      "the H200’s higher memory bandwidth ensures that data can be accessed and \n",
      "manipulated efficiently, leading to 110X faster time to results....\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Relevance Score:4.6875, Page Content:Certified Systems™ with 4 or 8 GPUsNVIDIA MGX™ H200 NVL partner and \n",
      "NVIDIA-Certified Systems with up to 8 GPUs\n",
      "NVIDIA AI Enterprise Add-on Included\n",
      "1. Preliminary specifications. May be subject to change. \n",
      "2. With sparsity.\n",
      "Ready to Get Started?\n",
      "To learn more about the NVIDIA H200 Tensor Core GPU,  \n",
      "visit nvidia.com/h200\n",
      "© 2024 NVIDIA Corporation and affiliates. All rights reserved. NVIDIA, the NVIDIA logo, HGX, Hopper, MGX,  NVIDIA-...\n",
      "----------------------------------------------------------------------------------------------------\n",
      "Relevance Score:4.53515625, Page Content:Reduce Energy and TCO\n",
      "With the introduction of H200, energy efficiency and TCO reach new levels. This \n",
      "cutting-edge technology offers unparalleled performance, all within the same power \n",
      "profile as the H100 Tensor Core GPU . AI factories and supercomputing systems that \n",
      "are not only faster but also more eco-friendly deliver an economic edge that propels \n",
      "the AI and scientific communities forward.\n",
      "Preliminary specifications. May be subject to change....\n",
      "----------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for chunks in reranked_chunks:\n",
    "\n",
    "    # Access the metadata of the document\n",
    "    metadata = chunks.metadata\n",
    "\n",
    "    # Get the page content\n",
    "    page_content = chunks.page_content\n",
    "    \n",
    "    # Print the relevance score if it exists in the metadata, followed by page content\n",
    "    if 'relevance_score' in metadata:\n",
    "        print(f\"Relevance Score:{metadata['relevance_score']}, Page Content:{page_content}...\")\n",
    "    print(f\"{'-' * 100}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd75663a-65f4-4c46-917a-7bb476c947c1",
   "metadata": {},
   "source": [
    "## Using the reranker for context\n",
    "\n",
    "We can build a similar chain using our top 2 reranked chunks and add them to our pipeline. Lets build one below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "6976c026-4018-4dc4-8846-98418ba382ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_with_context = ChatPromptTemplate.from_messages([\n",
    "    (\"system\", (\n",
    "        \"You are a helpful and friendly AI!\"\n",
    "        \"Your responses should be concise and no longer than two sentences.\"\n",
    "        \"You have the following context for the question: {context}.\"\n",
    "        \"Say you don't know if you don't have this information.\"\n",
    "    )),\n",
    "    (\"user\", \"{question}\")\n",
    "])\n",
    "\n",
    "chain = prompt_with_context | llm | StrOutputParser()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "5a14a971-68aa-45b9-a8a6-134e379531b1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Unfortunately, I don't have information on what the H in NVIDIA H200 specifically stands for. However, the \"H\" in NVIDIA H100, H200, and other related models is likely a classification or a codename indicating a high-end or high-performance variant.\n"
     ]
    }
   ],
   "source": [
    "print(chain.invoke({\"question\": \"What does the H in the NVIDIA H200 stand for?\"}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "f02322d5-0720-4087-8bc1-11d56b9510d6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The \"H\" in NVIDIA H200 stands for \"Hopper\", which is the architecture on which the H200 is based.\n"
     ]
    }
   ],
   "source": [
    "print(chain.invoke({\"context\": reranked_chunks[0].page_content, \"question\": \"What does the H in the NVIDIA H200 stand for?\"}))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}