01-langchain-basis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Langchain Basis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By end of this notebook, you should understand some basic concepts, such as Large Language Model (LLM), prompt, memory/chat history. These are the basis when designing Gen AI applications."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LLM"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One of the main reasons of using LangChain is that it provides a standard interface for interacting with many different LLMs (although it is not the only one). \n",
    "\n",
    "The quick way to try LangChain is to use OpenAI, you don't need to have a powerful GPU in your machine. Below is an example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "\n",
    "from langchain.llms import OpenAI\n",
    "\n",
    "# Default model is \"text-davinci-003\"\n",
    "llm = OpenAI()\n",
    "\n",
    "# or\n",
    "llm = OpenAI(openai_api_key=\"sk-xxx\", model_name=\"text-davinci-003\", temperature=1.0, max_tokens=512)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apart from OpenAI, langchain also has integrations to many other language models. (Check for the full list from: https://python.langchain.com/docs/integrations/llms/). From AWS side, it supports Bedrock, Sagemaker Endpoint, API Gateway etc.\n",
    "\n",
    "In this notebook, we will use open source language model (`meta-llama/Llama-2-7b-chat-hf`) instead. You will need to accept the Llama license, and then download the model by providing huggingface hub token. Check more details from https://huggingface.co/meta-llama/Llama-2-7b-chat-hf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/aiden/miniconda3/envs/langchain/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
      "  from .autonotebook import tqdm as notebook_tqdm\n",
      "Loading checkpoint shards: 100%|██████████| 2/2 [00:04<00:00,  2.33s/it]\n"
     ]
    }
   ],
   "source": [
    "import torch\n",
    "from transformers import pipeline\n",
    "from langchain.llms import HuggingFacePipeline\n",
    "\n",
    "pipe = pipeline(\n",
    "    task=\"text-generation\",\n",
    "    model=\"meta-llama/Llama-2-7b-chat-hf\",\n",
    "    torch_dtype=torch.bfloat16,\n",
    "    device_map=\"auto\",\n",
    "    temperature=0.0, \n",
    "    max_new_tokens=1024,\n",
    "    repetition_penalty=1.1,\n",
    ")\n",
    "\n",
    "llm = HuggingFacePipeline(pipeline=pipe)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The basic usage of LLM is text generation, once you have define the `llm`, you can use it to start generate new texts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "', there was a young woman named Sophia who lived in a small village nestled in the rolling hills of Tuscany. She had always been fascinated by the ancient art of alchemy, and spent many hours studying the dusty tomes in the local library.\\nOne day, while out on a walk, Sophia stumbled upon an old, mysterious-looking book hidden away in a forgotten corner of the library. The cover was worn and faded, but as she opened it, she found that the pages were filled with strange symbols and illustrations that seemed to glow with an otherworldly light.\\nIntrigued, Sophia took the book home and began to study it more closely. As she delved deeper into its secrets, she discovered that it contained the knowledge of a powerful alchemist who had lived centuries before.\\nSophia spent many long days and nights pouring over the book, learning the intricacies of alchemy and practicing the techniques described within its pages. And as she did so, she began to notice a change within herself - her body was transforming, her senses growing sharper, and her mind becoming more clear and focused.\\nAs the weeks passed, Sophia became increasingly skilled in the art of alchemy, and people from all over the region came to seek her counsel and guidance. She became known throughout the land as a wise and powerful alchemist, and her reputation only grew as the years went by.\\nBut Sophia never forgot the old, mysterious book that had first sparked her interest in alchemy, and she continued to study it throughout her life, uncovering new secrets and insights with each passing year. For in the world of alchemy, there is always more to learn, and more secrets to uncover.'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.predict(\"Once upon a time\")\n",
    "\n",
    "# This is same as\n",
    "# llm(\"Once upon a time\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OpenAI provides different models for general text generation (e.g. text-davinci-003) and chat generation (e.g. \"gpt-3.5-turbo\"), langchain also has a concept of `Chat Models`. \n",
    "\n",
    "You can try that out by yourself, below is an quick example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PromptTemplate and Chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's try some instructions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'\\n\\nHere is the translation of \"I love programming\" in Chinese:\\n\\n我爱编程。\\n\\nNote: The word \"编程\" (pinyin: biān chéng) means \"programming\" in Chinese.'"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.predict(\"Translate this sentence into Chinese: I love programming.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Llama 2 Chat is already fine-tuned to perform well on instructions, we can use the predefined prompt template as below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"\"\"[INST] <<SYS>>\n",
    " You are a helpful, respectful and honest assistant.\n",
    " <</SYS>>\n",
    "\n",
    "Translate this sentence into Chinese: I love programming.[/INST] \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' Sure! Here\\'s the translation of \"I love programming\" in Chinese:\\n\\n我爱编程 (wǒ ài biǎo chéng)\\n\\nNote: In Chinese, the word for \"love\" is 爱 (ài), and the word for \"programming\" is 编程 (biǎo chéng).'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.predict(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, what if we want to predict for other instructions and we don't want to pass the system prompt everytime.  Of course, we can use the `str.format()` in Python."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[INST] <<SYS>>\\n You are a helpful, respectful and honest assistant.\\n <</SYS>>\\n\\nTranslate this sentence into Chinese: I love programming.[/INST] '"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "text_prompt_template = \"\"\"[INST] <<SYS>>\n",
    " You are a helpful, respectful and honest assistant.\n",
    " <</SYS>>\n",
    "\n",
    "{instruction}[/INST] \"\"\"\n",
    "prompt = text_prompt_template.format(instruction=\"Translate this sentence into Chinese: I love programming.\")\n",
    "prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' Sure! Here\\'s the translation of \"I love programming\" in Chinese:\\n\\n我爱编程 (wǒ ài biǎo chéng)\\n\\nNote: In Chinese, the word for \"love\" is 爱 (ài), and the word for \"programming\" is 编程 (biǎo chéng).'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm.predict(prompt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Langchain, you can achieve the same by usign PromptTemplate and Chains. A `Chain` is` very generically as a sequence of calls to components, which can include other chains. \n",
    "\n",
    "The basic chain is `LLMChain` which takes in a prompt template, formats it with the user input and returns the response from an LLM. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "from langchain import PromptTemplate, LLMChain\n",
    "\n",
    "prompt_template = PromptTemplate(\n",
    "    input_variables=[\"instruction\"],\n",
    "    template=text_prompt_template,\n",
    ")\n",
    "llm_chain = LLMChain(llm=llm, prompt=prompt_template)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' Sure! Here\\'s the translation of \"I love programming\" in Chinese:\\n\\n我爱编程 (wǒ ài biǎo chéng)\\n\\nNote: In Chinese, the word for \"love\" is 爱 (ài), and the word for \"programming\" is 编程 (biǎo chéng).'"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain.run(instruction=\"Translate this sentence into Chinese: I love programming.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are some other ways of using the chain"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```python\n",
    "# Use llm_chain.predict(...)\n",
    "llm_chain.predict(instruction=\"Translate this sentence into Chinese: I love programming.\")\n",
    "\n",
    "# Or\n",
    "llm_chain.run({\"instruction\": \"Translate this sentence into Chinese: I love programming.\"})\n",
    "\n",
    "# Or \n",
    "llm_chain(inputs={\"instruction\": \"Translate this sentence into Chinese: I love programming.\"})\n",
    "\n",
    "# If it only has one parameter, you can simply use:\n",
    "llm_chain.run(\"Translate this sentence into Chinese: I love programming.\")\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PromptTemplate(input_variables=['instruction'], output_parser=None, partial_variables={}, template='[INST] <<SYS>>\\n You are a helpful, respectful and honest assistant.\\n <</SYS>>\\n\\n{instruction}[/INST] ', template_format='f-string', validate_template=True)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#  We can check the prompt template of a chain. This is useful as sometimes the prompt template is not defined by us.\n",
    "llm_chain.prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can add `verbose=True` to show more details (can be used for debug)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm_chain = LLMChain(llm=llm, prompt=prompt_template, verbose=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3m[INST] <<SYS>>\n",
      " You are a helpful, respectful and honest assistant.\n",
      " <</SYS>>\n",
      "\n",
      "Translate this sentence into Chinese: I love programming.[/INST] \u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Sure! Here\\'s the translation of \"I love programming\" in Chinese:\\n\\n我爱编程 (wǒ ài biǎo chéng)\\n\\nNote: In Chinese, the word for \"love\" is 爱 (ài), and the word for \"programming\" is 编程 (biǎo chéng).'"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain.run(\"Translate this sentence into Chinese: I love programming.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Memory"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's try to translate that into a different lauguage, only we don't provide the sentence again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3m[INST] <<SYS>>\n",
      " You are a helpful, respectful and honest assistant.\n",
      " <</SYS>>\n",
      "\n",
      "Translate the sentence into French.[/INST] \u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Of course! Here is the translation of the sentence \"You are a helpful, respectful and honest assistant\" into French:\\n\\nVous êtes un assistante utile, respectueuse et honnête.'"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain.run(\"Translate the sentence into French.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clearly it doesn't know what which `sentence` wa are referring to, as it doesn't know the previous conversations (chat history). To resolve this issue, we will add memory to the chain (or llm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_prompt_template_with_history = \"\"\"[INST] <<SYS>>\n",
    " You are a helpful, respectful and honest assistant.\n",
    " <</SYS>>\n",
    "\n",
    "Previous conversation:\n",
    "{chat_history}\n",
    "\n",
    "{instruction}[/INST] \"\"\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.memory import ConversationBufferMemory\n",
    "\n",
    "prompt = PromptTemplate.from_template(text_prompt_template_with_history)\n",
    "# Notice that we need to align the `memory_key`\n",
    "memory = ConversationBufferMemory(memory_key=\"chat_history\")\n",
    "llm_chain_with_memory = LLMChain(\n",
    "    llm=llm,\n",
    "    prompt=prompt,\n",
    "    memory=memory,\n",
    "    verbose=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3m[INST] <<SYS>>\n",
      " You are a helpful, respectful and honest assistant.\n",
      " <</SYS>>\n",
      "\n",
      "Previous conversation:\n",
      "\n",
      "\n",
      "Translate this sentence into Chinese: I love programming.[/INST] \u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Of course! Here\\'s the translation of \"I love programming\" in Chinese:\\n\\n我爱编程 (wǒ ài biǎo chéng)\\n\\nI hope that helps! Let me know if you have any other questions or if there\\'s anything else I can assist you with.'"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain_with_memory.run(\"Translate this sentence into Chinese: I love programming.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\u001b[1m> Entering new LLMChain chain...\u001b[0m\n",
      "Prompt after formatting:\n",
      "\u001b[32;1m\u001b[1;3m[INST] <<SYS>>\n",
      " You are a helpful, respectful and honest assistant.\n",
      " <</SYS>>\n",
      "\n",
      "Previous conversation:\n",
      "Human: Translate this sentence into Chinese: I love programming.\n",
      "AI:  Of course! Here's the translation of \"I love programming\" in Chinese:\n",
      "\n",
      "我爱编程 (wǒ ài biǎo chéng)\n",
      "\n",
      "I hope that helps! Let me know if you have any other questions or if there's anything else I can assist you with.\n",
      "\n",
      "Translate the sentence into French.[/INST] \u001b[0m\n",
      "\n",
      "\u001b[1m> Finished chain.\u001b[0m\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "' Of course! Here\\'s the translation of \"I love programming\" in French:\\nJe suis amoureux de la programmation.\\nI hope that helps! Let me know if you have any other questions or if there\\'s anything else I can assist you with.'"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain_with_memory.run(\"Translate the sentence into French.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see the chat history is added into the prompt for new instructions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`ConversationBufferMemory` basically put all histories into the prompt, and we know there are always limitions on context length for large language model, e.g. for Llama2, the context length is only 4096. Langchain provides many other types of memories.\n",
    "\n",
    "For example, So we may only want to store the most recent chats into the prompt only, we can use `ConversationBufferWindowMemory`, such as memory=ConversationBufferWindowMemory(k=2) for the last two chats only.  You can try more yourself.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you don't want to define the prompt template yourself, here is another chain you can use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "\n",
    "conversation = ConversationChain(\n",
    "    llm=llm,\n",
    "    memory=ConversationBufferMemory()\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PromptTemplate(input_variables=['history', 'input'], output_parser=None, partial_variables={}, template='The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.\\n\\nCurrent conversation:\\n{history}\\nHuman: {input}\\nAI:', template_format='f-string', validate_template=True)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check the prompt template it used\n",
    "conversation.prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Context"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Llama 2 is pretrained with a lot of data, that doesnt mean it know everything. Let's try to ask some question about the new AWS product: Amazon Bedrock."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_prompt_template = \"\"\"[INST] <<SYS>>\n",
    " You are a helpful, respectful and honest assistant. If you don't know the answer to a question, please don't share false information.\n",
    " <</SYS>>\n",
    "\n",
    "{question} [/INST] \"\"\"\n",
    "\n",
    "prompt_template = PromptTemplate(\n",
    "    template=text_prompt_template, input_variables=[\"question\"]\n",
    ")\n",
    "\n",
    "llm_chain = LLMChain(llm=llm, prompt=prompt_template)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' Thank you for asking! I\\'m happy to help. However, I must inform you that \"Bedrock\" is not a real place or location. It is actually a fictional setting from the classic cartoon show \"The Flintstones,\" which was set in the prehistoric town of Bedrock. The show was created in the 1960s and featured a Stone Age family called the Flintstones, who lived in the town of Bedrock with their friends and neighbors. So, while Bedrock may not be a real place, it has become a cultural reference point and a beloved part of popular culture. Is there anything else I can help you with?'"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain.run(question=\"What is Bedrock?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One way to resolve this is that we can add more contexts into the prompt."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_prompt_template_with_context = \"\"\"[INST] <<SYS>>\n",
    " You are a helpful, respectful and honest assistant. If you don't know the answer to a question, please don't share false information. \n",
    " <</SYS>>\n",
    "\n",
    "Here is the context\n",
    "{context}\n",
    "\n",
    "{question} [/INST] \"\"\"\n",
    "\n",
    "prompt_template = PromptTemplate(\n",
    "    template=text_prompt_template_with_context, input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "llm_chain_with_context = LLMChain(llm=llm, prompt=prompt_template)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/aiden/miniconda3/envs/langchain/lib/python3.10/site-packages/transformers/pipelines/base.py:1083: UserWarning: You seem to be using the pipelines sequentially on GPU. In order to maximize efficiency please use a dataset\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "\" Hello! I'm here to help you understand what Bedrock is. Based on the context provided, Bedrock appears to be a fully managed service offered by Amazon that provides access to foundation models (FMs) from Amazon and leading AI startups through an API. This means that developers can choose from various FMs to find the one that best suits their needs and use case.\\nWith Bedrock, developers can quickly get started, easily experiment with FMs, privately customize FMs with their own data, and seamlessly integrate and deploy them into their applications using AWS tools and capabilities. Additionally, agents for Bedrock are fully managed, making it easier for developers to create generative-AI applications that can deliver up-to-date answers based on proprietary knowledge sources and complete tasks for a wide range of use cases.\\nIn summary, Bedrock is a service offered by Amazon that provides easy access to foundation models for generative-AI applications, allowing developers to quickly and easily build and deploy AI-powered applications.\""
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Copy the information from https://aws.amazon.com/bedrock/?nc1=h_ls as context\n",
    "\n",
    "context = \"Amazon Bedrock is a fully managed service that makes foundation models (FMs) from Amazon and leading AI startups available through an API, so you can choose from various FMs to find the model that's best suited for your use case. With the Amazon Bedrock serverless experience, you can quickly get started, easily experiment with FMs, privately customize FMs with your own data, and seamlessly integrate and deploy them into your applications using AWS tools and capabilities. Agents for Amazon Bedrock are fully managed and make it easier for developers to create generative-AI applications that can deliver up-to-date answers based on proprietary knowledge sources and complete tasks for a wide range of use cases.\"\n",
    "\n",
    "llm_chain_with_context.run(context=context, question=\"What is Bedrock?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sounds about right, because all the information is added into prompt. But let's ask a different question."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "' As a responsible and honest assistant, I must inform you that I cannot provide a definitive list of FMs supported by Amazon Bedrock as the information is not publicly available. The Amazon Bedrock website does not mention any specific FMs that are supported, and there is no official documentation or announcement regarding the matter.\\nHowever, based on the description provided in the context, it seems that Amazon Bedrock offers a variety of foundation models (FMs) from Amazon and leading AI startups through its API. These FMs may be tailored to different use cases and can be easily integrated and deployed into applications using AWS tools and capabilities.\\nIf you have any further questions or need more information, please feel free to ask!'"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "llm_chain_with_context.run(context=context, question=\"What FMs are supported by bedrock?\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, we don't provide enough context to answer the questions, however, we can not predict what the questions are and we also can't put all information into the prompt as context.\n",
    "\n",
    "To resolve this issues, there are two other ways: \n",
    " \n",
    "1. Retrieval Augmented Generation\n",
    "2. Using Agent/Tools\n",
    "\n",
    "Check out the next notebooks for more details.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langchain",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}