diff --git a/README.md b/README.md index 609f8d3bf..48faac89a 100644 --- a/README.md +++ b/README.md @@ -6,28 +6,9 @@ [![Deploy with GitHub Codespaces](https://github.com/codespaces/badge.svg)](#deploy-with-github-codespaces) -![sample](assets/chabot-sample.gif "AWS GenAI Chatbot") +#### [Full documentation](https://aws-samples.github.io/aws-genai-llm-chatbot/) -## Table of content - -- [Features](#features) -- [Precautions](#precautions) -- [Prequirements](#requirements) - - [Self hosted models on Amazon SageMaker](#amazon-sagemaker-requirements-for-self-hosted-models-only) - - [Amazon Bedrock](#amazon-bedrock-requirements) - - [Third party models](#third-party-models-requirements) -- [Deploy](#deploy) - - [With AWS Cloud9](#deploy-with-aws-cloud9) - - [With GitHub Codespaces](#deploy-with-github-codespaces) - - [Local deployment](#local-deployment) -- [Clean up](#clean-up) -- [Authors](#authors) -- [Credits](#credits) -- [License](#license) - -# Features - -## Modular, comprehensive and ready to use +![sample](docs/about/assets/chabot-sample.gif "AWS GenAI Chatbot") This solution provides ready-to-use code so you can start **experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts** in your own AWS account. @@ -37,330 +18,6 @@ Supported model providers: - [Amazon SageMaker](https://aws.amazon.com/sagemaker/) self-hosted models from Foundation, Jumpstart and HuggingFace. - Third-party providers via API such as Anthropic, Cohere, AI21 Labs, OpenAI, etc. [See available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list. -## Experiment with multimodal models - -Deploy [IDEFICS](https://huggingface.co/blog/idefics) models on [Amazon SageMaker](https://aws.amazon.com/sagemaker/) and see how the chatbot can answer questions about images, describe visual content, generate text grounded in multiple images. - -![sample](assets/multimodal-sample.gif "AWS GenAI Chatbot") - -Currently, the following multimodal models are supported: - -- [IDEFICS 9b Instruct](https://huggingface.co/HuggingFaceM4/idefics-9b) - - Requires `ml.g5.12xlarge` instance. -- [IDEFICS 80b Instruct](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) - - Requires `ml.g5.48xlarge` instance. - -To have the right instance types and how to request them, read [Amazon SageMaker requirements](#amazon-sagemaker-requirements-for-self-hosted-models-only) - -> NOTE: Make sure to review [IDEFICS models license sections](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct#license). - -To deploy a multimodal model, follow the [deploy instructions](#deploy) -and select one of the supported models (press Space to select/deselect) from the magic-create CLI step and deploy as [instructed in the above section](<(#deployment-dependencies-installation)>). - -> ⚠️ NOTE ⚠️ Amazon SageMaker are billed by the hour. Be aware of not letting this model run unused to avoid unnecessary costs. - -## Multi-Session Chat: evaluate multiple models at once - -Send the same query to 2 to 4 separate models at once and see how each one responds based on its own learned history, context and access to the same powerful document retriever, so all requests can pull from the same up-to-date knowledge. - -![sample](assets/multichat-sample.gif "AWS GenAI Chatbot") - -## Experiment with multiple RAG options with Workspaces - -A workspace is a logical namespace where you can upload files for indexing and storage in one of the vector databases. You can select the embeddings model and text-splitting configuration of your choice. - -![sample](assets/create-workspace-sample.gif "AWS GenAI Chatbot") - -## Unlock RAG potentials with Workspaces Debugging Tools - -The solution comes with several debugging tools to help you debug RAG scenarios: - -- Run RAG queries without chatbot and analyse results, scores, etc. -- Test different embeddings models directly in the UI -- Test cross encoders and analyse distances from different functions between sentences. - -![sample](assets/workspace-debug-sample.gif "AWS GenAI Chatbot") - -## Full-fledged User Interface - -The repository includes a CDK construct to deploy a **full-fledged UI** built with [React](https://react.dev/) to interact with the deployed LLMs/MLMs as chatbots. Hosted on [Amazon S3](https://aws.amazon.com/s3/) and distributed with [Amazon CloudFront](https://aws.amazon.com/cloudfront/). - -Protected with [Amazon Cognito Authentication](https://aws.amazon.com/cognito/) to help you interact and experiment with multiple LLMs/MLMs, multiple RAG engines, conversational history support and document upload/progress. - -The interface layer between the UI and backend is built with [AppSync](https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html) for management requests and for realtime interaction with chatbot (messages and responses) using GraphQL subscriptions. - -Design system provided by [AWS Cloudscape Design System](https://cloudscape.design/). - -# ⚠️ Precautions ⚠️ - -Before you begin using the solution, there are certain precautions you must take into account: - -- **Cost Management with self-hosted models on SageMaker**: Be mindful of the costs associated with AWS resources, especially with SageMaker models billed by the hour. While the sample is designed to be cost-effective, leaving serverful resources running for extended periods or deploying numerous LLMs/MLMs can quickly lead to increased costs. - -- **Licensing obligations**: If you choose to use any datasets or models alongside the provided samples, ensure you check the LLM code and comply with all licensing obligations attached to them. - -- **This is a sample**: the code provided in this repository shouldn't be used for production workloads without further reviews and adaptation. - -# Amazon SageMaker requirements (for self-hosted models only) - -**Instance type quota increase** - -If you are looking to self-host models on Amazon SageMaker, you'll likely need to request an increase in service quota for specific SageMaker instance types, such as the `ml.g5` instance type. This will give access to the latest generation of GPU/Multi-GPU instance types. [You can do this from the AWS console](https://console.aws.amazon.com/servicequotas/home/services/sagemaker/quotas) - -# Amazon Bedrock requirements - -**Base Models Access** - -If you are looking to interact with models from Amazon Bedrock, you need to [request access to the base models in one of the regions where Amazon Bedrock is available](https://console.aws.amazon.com/bedrock/home?#/modelaccess). Make sure to read and accept models' end-user license agreements or EULA. - -Note: - -- You can deploy the solution to a different region from where you requested Base Model access. -- **While the Base Model access approval is instant, it might take several minutes to get access and see the list of models in the UI.** - -![sample](assets/enable-models.gif "AWS GenAI Chatbot") - -# Third-party models requirements - -You can also interact with external providers via their API, such as AI21 Labs, Cohere, OpenAI, etc. - -The provider must be supported in the [Model Interface](./lib/model-interfaces/langchain/functions/request-handler/index.py), [see available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list of providers. - -Usually, an `API_KEY` is required to integrate with 3P models. To do so, the [Model Interface](./lib/model-interfaces/langchain/index.ts) deployes a Secrets in [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/), intially with an empty JSON `{}`, where you can add your API KEYS for one or more providers. - -These keys will be injected at runtime into the Lambda function Environment Variables; they won't be visible in the AWS Lambda Console. - -For example, if you wish to be able to interact with AI21 Labs., OpenAI's and Cohere endpoints: - -- Open the [Model Interface Keys Secret](./lib/model-interfaces/langchain/index.ts#L38) in Secrets Manager. You can find the secret name in the stack output, too. -- Update the Secrets by adding a key to the JSON - -```json -{ - "AI21_API_KEY": "xxxxx", - "OPENAI_API_KEY": "sk-xxxxxxxxxxxxxxx", - "COHERE_API_KEY": "xxxxx" -} -``` - -N.B: In case of no keys needs, the secret value must be an empty JSON `{}`, NOT an empty string `''`. - -make sure that the environment variable matches what is expected by the framework in use, like Langchain ([see available langchain integrations](https://python.langchain.com/docs/integrations/llms/). - -# Deploy - -### Environment setup - -#### Deploy with AWS Cloud9 - -We recommend deploying with [AWS Cloud9](https://aws.amazon.com/cloud9/). -If you'd like to use Cloud9 to deploy the solution, you will need the following before proceeding: - -- select at least `m5.large` as Instance type. -- use `Ubuntu Server 22.04 LTS` as the platform. - -#### Deploy with Github Codespaces - -If you'd like to use [GitHub Codespaces](https://github.com/features/codespaces) to deploy the solution, you will need the following before proceeding: - -1. An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) -2. An [IAM User](https://console.aws.amazon.com/iamv2/home?#/users/create) with: - -- `AdministratorAccess` policy granted to your user (for production, we recommend restricting access as needed) -- Take note of `Access key` and `Secret access key`. - -To get started, click on the button below. - -[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/aws-samples/aws-genai-llm-chatbot) - -Once in the Codespaces terminal, set up the AWS Credentials by running - -```shell -aws configure -``` - -```shell -AWS Access Key ID [None]: -AWS Secret Access Key [None]: -Default region name: -Default output format: json -``` - -You are all set for deployment; you can now jump to [.3 of the deployment section below](#deployment-dependencies-installation). - -#### Local deployment - -If you have decided not to use AWS Cloud9 or GitHub Codespaces, verify that your environment satisfies the following prerequisites: - -You have: - -1. An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) -2. `AdministratorAccess` policy granted to your AWS account (for production, we recommend restricting access as needed) -3. Both console and programmatic access -4. [NodeJS 16 or 18](https://nodejs.org/en/download/) installed - - - If you are using [`nvm`](https://github.com/nvm-sh/nvm) you can run the following before proceeding - - ``` - nvm install 16 && nvm use 16 - - or - - nvm install 18 && nvm use 18 - ``` - -5. [AWS CLI](https://aws.amazon.com/cli/) installed and configured to use with your AWS account -6. [Typescript 3.8+](https://www.typescriptlang.org/download) installed -7. [AWS CDK CLI](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html) installed -8. [Docker](https://docs.docker.com/get-docker/) installed - - N.B. [`buildx`](https://github.com/docker/buildx) is also required. For Windows and macOS `buildx` [is included](https://github.com/docker/buildx#windows-and-macos) in [Docker Desktop](https://docs.docker.com/desktop/) -9. [Python 3+](https://www.python.org/downloads/) installed - -### Deployment - -1. Clone the repository - -```bash -git clone https://github.com/aws-samples/aws-genai-llm-chatbot -``` - -2. Move into the cloned repository - -```bash -cd aws-genai-llm-chatbot -``` - -#### (Optional) Only for Cloud9 - -If you use Cloud9, increase the instance's EBS volume to at least 100GB. -To do this, run the following command from the Cloud9 terminal: - -``` -./scripts/cloud9-resize.sh -``` - -See the documentation for more details [on environment resize](https://docs.aws.amazon.com/cloud9/latest/user-guide/move-environment.html#move-environment-resize). - - 3. Install the project dependencies and build the project by running this command - -```bash -npm install && npm run build -``` - -4. Once done, run the magic-create CLI to help you set up the solution with the features you care most: - -```bash -npm run create -``` - -You'll be prompted to configure the different aspects of the solution, such as: - -- The LLMs or MLMs to enable (we support all models provided by Bedrock along with SageMaker hosted Idefics, FalconLite, Mistral and more to come) -- Setup of the RAG system: engine selection (i.e. Aurora w/ pgvector, OpenSearch, Kendra..) embeddings selection and more to come. - -When done, answer `Y` to create a new configuration. - -![sample](assets/magic-create-sample.gif "CLI sample") - -Your configuration is now stored under `bin/config.json`. You can re-run the magic-create command as needed to update your `config.json` - -5. (Optional) Bootstrap AWS CDK on the target account and region - -> **Note**: This is required if you have never used AWS CDK on this account and region combination. ([More information on CDK bootstrapping](https://docs.aws.amazon.com/cdk/latest/guide/cli.html#cli-bootstrap)). - -```bash -npx cdk bootstrap aws://{targetAccountId}/{targetRegion} -``` - -You can now deploy by running: - -```bash -npx cdk deploy -``` - -> **Note**: This step duration can vary greatly, depending on the Constructs you are deploying. - -You can view the progress of your CDK deployment in the [CloudFormation console](https://console.aws.amazon.com/cloudformation/home) in the selected region. - -6. Once deployed, take note of the `User Interface`, `User Pool` and, if you want to interact with [3P models providers](#3p-models-providers), the `Secret` that will, eventually, hold the various `API_KEYS` should you want to experiment with 3P providers. - -```bash -... -Outputs: -GenAIChatBotStack.UserInterfaceUserInterfaceDomanNameXXXXXXXX = dxxxxxxxxxxxxx.cloudfront.net -GenAIChatBotStack.AuthenticationUserPoolLinkXXXXX = https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx -GenAIChatBotStack.ApiKeysSecretNameXXXX = ApiKeysSecretName-xxxxxx -... -``` - -7. Open the generated **Cognito User Pool** Link from outputs above i.e. `https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx` - -8. Add a user that will be used to log into the web interface. - -9. Open the `User Interface` Url for the outputs above, i.e. `dxxxxxxxxxxxxx.cloudfront.net` - -10. Login with the user created in .8; you will be asked to change the password. - -# Run user interface locally - -See instructions in the README file of the [`lib/user-interface/react-app`](./lib/user-interface/react-app/README.md) folder. - -# Using Kendra with a non-english index - -If you're using Kendra with an index in a language other than English, you will need to make some code modifications. - -You'll need to modify the filters in the file `lib/shared/layers/python-sdk/python/genai_core/kendra/query.py`. - -Example for french : - -```python - if kendra_index_external or kendra_use_all_data: - result = kendra.retrieve( - IndexId=kendra_index_id, - QueryText=query, - PageSize=limit, - PageNumber=1, - AttributeFilter={'AndAllFilters': [{"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}]} - ) - else: - result = kendra.retrieve( - IndexId=kendra_index_id, - QueryText=query, - PageSize=limit, - PageNumber=1, - AttributeFilter={'AndAllFilters': - [ - {"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}, - {"EqualsTo": {"Key": "workspace_id","Value": {"StringValue": workspace_id}}} - ] - } - ) -``` - -Please note: If these adjustments are made post-deployment, it's essential to rebuild and redeploy. If done prior to deployment, you can proceed with the walkthrough as usual. - -```bash -npm install && npm run build -npx cdk deploy -``` - -# Clean up - -You can remove the stacks and all the associated resources created in your AWS account by running the following command: - -```bash -npx cdk destroy -``` - -> **Note**: Depending on which resources have been deployed. Destroying the stack might take a while, up to 45m. If the deletion fails multiple times, please manually delete the remaining stack's ENIs; you can filter ENIs by VPC/Subnet/etc using the search bar [here](https://console.aws.amazon.com/ec2/home#NIC) in the AWS console) and re-attempt a stack deletion. - -# Architecture - -This repository comes with several reusable CDK constructs. Giving you the freedom to decide what to deploy and what not. - -Here's an overview: - -![sample](assets/architecture.png "Architecture Diagram") - # Authors - [Bigad Soleiman](https://www.linkedin.com/in/bigadsoleiman/) diff --git a/assets/architecture.png b/assets/architecture.png deleted file mode 100644 index 007dfce29..000000000 Binary files a/assets/architecture.png and /dev/null differ diff --git a/docs/.vitepress/config.mts b/docs/.vitepress/config.mts index d5d7f5574..2f4dc4b18 100644 --- a/docs/.vitepress/config.mts +++ b/docs/.vitepress/config.mts @@ -7,25 +7,48 @@ export default defineConfig({ base: "/aws-genai-llm-chatbot/", themeConfig: { // https://vitepress.dev/reference/default-theme-config + search: { + provider: 'local' + }, socialLinks: [ { icon: 'github', link: 'https://github.com/aws-samples/aws-genai-llm-chatbot' } ], nav: [ { text: 'Home', link: '/' }, - { text: 'Guide', link: '/guide/getting-started' } + { text: 'About', link: '/about/welcome' }, + { + text: 'Guide', + items: [ + { text: 'Deploy', link: '/guide/deploy' }, + { text: 'Developer Guide', link: '/guide/developers' }, + ] + }, + { text: 'Documentation', link: '/documentation/model-requirements' } ], sidebar: [ + { text: 'About', items: [ + { text: 'The Project', link: '/about/welcome' }, + { text: 'Features', link: '/about/features' }, + { text: 'Architecture', link: '/about/architecture' }, + { text: 'Authors & Credits', link: '/about/authors' }, + { text: 'License Information', link: '/about/license' }, + ] + }, { - text: 'Getting Started', + text: 'Guide', items: [ - { text: 'Welcome', link: '/guide/getting-started' } + { text: 'Deploy', link: '/guide/deploy' }, + { text: 'Developer Guide', link: '/guide/developers' } ] }, { - text: 'Example', + text: 'Documentation', items: [ - { text: 'Lipsum', link: '/guide/lipsum' }, - { text: 'Lipsum 2', link: '/guide/lipsum2' } + { text: 'Model Requirements', link: '/documentation/model-requirements' }, + { text: 'Inference Script', link: '/documentation/inference-script' }, + { text: 'Document Retrieval', link: '/documentation/retriever' }, + { text: 'AppSync', link: '/documentation/appsync' }, + { text: 'Precautions', link: '/documentation/precautions' } ] } ], diff --git a/docs/about/architecture.md b/docs/about/architecture.md new file mode 100644 index 000000000..19eb87e27 --- /dev/null +++ b/docs/about/architecture.md @@ -0,0 +1,7 @@ +# Architecture + +This repository comes with several reusable CDK constructs. Giving you the freedom to decide what to deploy and what not. + +Here's an overview: + +![sample](./assets/architecture.png "Architecture Diagram") \ No newline at end of file diff --git a/docs/assets/architecture.png b/docs/about/assets/architecture.png similarity index 100% rename from docs/assets/architecture.png rename to docs/about/assets/architecture.png diff --git a/assets/chabot-sample.gif b/docs/about/assets/chabot-sample.gif similarity index 100% rename from assets/chabot-sample.gif rename to docs/about/assets/chabot-sample.gif diff --git a/assets/create-workspace-sample.gif b/docs/about/assets/create-workspace-sample.gif similarity index 100% rename from assets/create-workspace-sample.gif rename to docs/about/assets/create-workspace-sample.gif diff --git a/assets/multichat-sample.gif b/docs/about/assets/multichat-sample.gif similarity index 100% rename from assets/multichat-sample.gif rename to docs/about/assets/multichat-sample.gif diff --git a/assets/multimodal-sample.gif b/docs/about/assets/multimodal-sample.gif similarity index 100% rename from assets/multimodal-sample.gif rename to docs/about/assets/multimodal-sample.gif diff --git a/assets/workspace-debug-sample.gif b/docs/about/assets/workspace-debug-sample.gif similarity index 100% rename from assets/workspace-debug-sample.gif rename to docs/about/assets/workspace-debug-sample.gif diff --git a/docs/about/authors.md b/docs/about/authors.md new file mode 100644 index 000000000..cee42a315 --- /dev/null +++ b/docs/about/authors.md @@ -0,0 +1,12 @@ +# Authors + +- [Bigad Soleiman](https://www.linkedin.com/in/bigadsoleiman/) +- [Sergey Pugachev](https://www.linkedin.com/in/spugachev/) + +# Credits + +This sample was made possible thanks to the following libraries: + +- [langchain](https://python.langchain.com/docs/get_started/introduction.html) from [LangChain AI](https://github.com/langchain-ai) +- [unstructured](https://github.com/Unstructured-IO/unstructured) from [Unstructured-IO](https://github.com/Unstructured-IO/unstructured) +- [pgvector](https://github.com/pgvector/pgvector) from [Andrew Kane](https://github.com/ankane) \ No newline at end of file diff --git a/docs/about/features.md b/docs/about/features.md new file mode 100644 index 000000000..6c00b303e --- /dev/null +++ b/docs/about/features.md @@ -0,0 +1,65 @@ +# Features + +## Modular, comprehensive and ready to use + +This solution provides ready-to-use code so you can start **experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts** in your own AWS account. + +Supported model providers: + +- [Amazon Bedrock](https://aws.amazon.com/bedrock/) +- [Amazon SageMaker](https://aws.amazon.com/sagemaker/) self-hosted models from Foundation, Jumpstart and HuggingFace. +- Third-party providers via API such as Anthropic, Cohere, AI21 Labs, OpenAI, etc. [See available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list. + +## Experiment with multimodal models + +Deploy [IDEFICS](https://huggingface.co/blog/idefics) models on [Amazon SageMaker](https://aws.amazon.com/sagemaker/) and see how the chatbot can answer questions about images, describe visual content, generate text grounded in multiple images. + +![sample](./assets/multimodal-sample.gif "AWS GenAI Chatbot") + +Currently, the following multimodal models are supported: + +- [IDEFICS 9b Instruct](https://huggingface.co/HuggingFaceM4/idefics-9b) + - Requires `ml.g5.12xlarge` instance. +- [IDEFICS 80b Instruct](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct) + - Requires `ml.g5.48xlarge` instance. + +To have the right instance types and how to request them, read [Amazon SageMaker requirements](../documentation/model-requirements#amazon-sagemaker-requirements-for-self-hosted-models-only) + +> NOTE: Make sure to review [IDEFICS models license sections](https://huggingface.co/HuggingFaceM4/idefics-80b-instruct#license). + +To deploy a multimodal model, follow the [deploy instructions](../guide/deploy) +and select one of the supported models (press Space to select/deselect) from the magic-create CLI step and deploy as instructed in the above section. + +> ⚠️ NOTE ⚠️ Amazon SageMaker are billed by the hour. Be aware of not letting this model run unused to avoid unnecessary costs. + +## Multi-Session Chat: evaluate multiple models at once + +Send the same query to 2 to 4 separate models at once and see how each one responds based on its own learned history, context and access to the same powerful document retriever, so all requests can pull from the same up-to-date knowledge. + +![sample](./assets/multichat-sample.gif "AWS GenAI Chatbot") + +## Experiment with multiple RAG options with Workspaces + +A workspace is a logical namespace where you can upload files for indexing and storage in one of the vector databases. You can select the embeddings model and text-splitting configuration of your choice. + +![sample](./assets/create-workspace-sample.gif "AWS GenAI Chatbot") + +## Unlock RAG potentials with Workspaces Debugging Tools + +The solution comes with several debugging tools to help you debug RAG scenarios: + +- Run RAG queries without chatbot and analyse results, scores, etc. +- Test different embeddings models directly in the UI +- Test cross encoders and analyse distances from different functions between sentences. + +![sample](./assets/workspace-debug-sample.gif "AWS GenAI Chatbot") + +## Full-fledged User Interface + +The repository includes a CDK construct to deploy a **full-fledged UI** built with [React](https://react.dev/) to interact with the deployed LLMs/MLMs as chatbots. Hosted on [Amazon S3](https://aws.amazon.com/s3/) and distributed with [Amazon CloudFront](https://aws.amazon.com/cloudfront/). + +Protected with [Amazon Cognito Authentication](https://aws.amazon.com/cognito/) to help you interact and experiment with multiple LLMs/MLMs, multiple RAG engines, conversational history support and document upload/progress. + +The interface layer between the UI and backend is built with [AppSync](https://docs.aws.amazon.com/appsync/latest/devguide/what-is-appsync.html) for management requests and for realtime interaction with chatbot (messages and responses) using GraphQL subscriptions. + +Design system provided by [AWS Cloudscape Design System](https://cloudscape.design/). diff --git a/docs/about/license.md b/docs/about/license.md new file mode 100644 index 000000000..de3348af1 --- /dev/null +++ b/docs/about/license.md @@ -0,0 +1,8 @@ +# License + +This library is licensed under the MIT-0 License. See the LICENSE file. + +- [Changelog](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/CHANGELOG.md) of the project. +- [License](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/LICENSE) of the project. +- [Code of Conduct](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/CODE_OF_CONDUCT.md) of the project. +- [CONTRIBUTING](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/CONTRIBUTING.md#security-issue-notifications) for more information. \ No newline at end of file diff --git a/docs/about/welcome.md b/docs/about/welcome.md new file mode 100644 index 000000000..ca087362b --- /dev/null +++ b/docs/about/welcome.md @@ -0,0 +1,21 @@ +--- +layout: doc +--- + +# Deploying a Multi-Model and Multi-RAG Powered Chatbot Using AWS CDK on AWS + +[![Release Notes](https://img.shields.io/github/v/release/aws-samples/aws-genai-llm-chatbot)](https://github.com/aws-samples/aws-genai-llm-chatbot/releases) + +[![GitHub star chart](https://img.shields.io/github/stars/aws-samples/aws-genai-llm-chatbot?style=social)](https://star-history.com/#aws-samples/aws-genai-llm-chatbot) + +[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) + +[![Deploy with GitHub Codespaces](https://github.com/codespaces/badge.svg)](#deploy-with-github-codespaces) + +The AWS GenAI LLM Chatbot provides ready-to-use code so you can start experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts in your own AWS account. + +![sample](./assets/chabot-sample.gif "AWS GenAI Chatbot") + +Want to find out more? Continue to [Features](./features). + +Want to get started? Head to the [Deployment Guide](../guide/deploy). \ No newline at end of file diff --git a/docs/assets/icon-dark.png b/docs/assets/icon-dark.png deleted file mode 100644 index 546a796cd..000000000 Binary files a/docs/assets/icon-dark.png and /dev/null differ diff --git a/docs/assets/icon-light.png b/docs/assets/icon-light.png deleted file mode 100644 index 14c5f6cc0..000000000 Binary files a/docs/assets/icon-light.png and /dev/null differ diff --git a/docs/appsync/appsync.md b/docs/documentation/appsync.md similarity index 100% rename from docs/appsync/appsync.md rename to docs/documentation/appsync.md diff --git a/assets/enable-models.gif b/docs/documentation/assets/enable-models.gif similarity index 100% rename from assets/enable-models.gif rename to docs/documentation/assets/enable-models.gif diff --git a/docs/sagemker-hosting/inference-script.md b/docs/documentation/inference-script.md similarity index 100% rename from docs/sagemker-hosting/inference-script.md rename to docs/documentation/inference-script.md diff --git a/docs/documentation/model-requirements.md b/docs/documentation/model-requirements.md new file mode 100644 index 000000000..8ba498f7c --- /dev/null +++ b/docs/documentation/model-requirements.md @@ -0,0 +1,48 @@ +# Model Requirements + +## Amazon SageMaker requirements (for self-hosted models only) + +**Instance type quota increase** + +If you are looking to self-host models on Amazon SageMaker, you'll likely need to request an increase in service quota for specific SageMaker instance types, such as the `ml.g5` instance type. This will give access to the latest generation of GPU/Multi-GPU instance types. [You can do this from the AWS console](https://console.aws.amazon.com/servicequotas/home/services/sagemaker/quotas) + +## Amazon Bedrock requirements + +**Base Models Access** + +If you are looking to interact with models from Amazon Bedrock, you need to [request access to the base models in one of the regions where Amazon Bedrock is available](https://console.aws.amazon.com/bedrock/home?#/modelaccess). Make sure to read and accept models' end-user license agreements or EULA. + +Note: + +- You can deploy the solution to a different region from where you requested Base Model access. +- **While the Base Model access approval is instant, it might take several minutes to get access and see the list of models in the UI.** + +![sample](./assets/enable-models.gif "AWS GenAI Chatbot") + +## Third-party models requirements + +You can also interact with external providers via their API, such as AI21 Labs, Cohere, OpenAI, etc. + +The provider must be supported in the [Model Interface](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/lib/model-interfaces/langchain/functions/request-handler/index.py), [see available langchain integrations](https://python.langchain.com/docs/integrations/llms/) for a comprehensive list of providers. + +Usually, an `API_KEY` is required to integrate with 3P models. To do so, the [Model Interface](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/lib/model-interfaces/langchain/index.ts) deployes a Secrets in [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/), intially with an empty JSON `{}`, where you can add your API KEYS for one or more providers. + +These keys will be injected at runtime into the Lambda function Environment Variables; they won't be visible in the AWS Lambda Console. + +For example, if you wish to be able to interact with AI21 Labs., OpenAI's and Cohere endpoints: + +- Open the [Model Interface Keys Secret](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/lib/model-interfaces/langchain/index.ts#L38) in Secrets Manager. You can find the secret name in the stack output, too. +- Update the Secrets by adding a key to the JSON + +```json +{ + "AI21_API_KEY": "xxxxx", + "OPENAI_API_KEY": "sk-xxxxxxxxxxxxxxx", + "COHERE_API_KEY": "xxxxx" +} +``` + + +N.B: In case of no keys needs, the secret value must be an empty JSON `{}`, NOT an empty string `''`. + +make sure that the environment variable matches what is expected by the framework in use, like Langchain ([see available langchain integrations](https://python.langchain.com/docs/integrations/llms/)). diff --git a/docs/documentation/precautions.md b/docs/documentation/precautions.md new file mode 100644 index 000000000..b6b0bc6b8 --- /dev/null +++ b/docs/documentation/precautions.md @@ -0,0 +1,9 @@ +# ⚠️ Precautions ⚠️ + +Before you begin using the solution, there are certain precautions you must take into account: + +- **Cost Management with self-hosted models on SageMaker**: Be mindful of the costs associated with AWS resources, especially with SageMaker models billed by the hour. While the sample is designed to be cost-effective, leaving serverful resources running for extended periods or deploying numerous LLMs/MLMs can quickly lead to increased costs. + +- **Licensing obligations**: If you choose to use any datasets or models alongside the provided samples, ensure you check the LLM code and comply with all licensing obligations attached to them. + +- **This is a sample**: the code provided in this repository shouldn't be used for production workloads without further reviews and adaptation. \ No newline at end of file diff --git a/docs/document-retrieval/retriever.md b/docs/documentation/retriever.md similarity index 100% rename from docs/document-retrieval/retriever.md rename to docs/documentation/retriever.md diff --git a/assets/magic-create-sample.gif b/docs/guide/assets/magic-create-sample.gif similarity index 100% rename from assets/magic-create-sample.gif rename to docs/guide/assets/magic-create-sample.gif diff --git a/docs/guide/deploy.md b/docs/guide/deploy.md new file mode 100644 index 000000000..cbfd49663 --- /dev/null +++ b/docs/guide/deploy.md @@ -0,0 +1,205 @@ +# Deploy + +## Environment setup + +### Deploy with AWS Cloud9 + +We recommend deploying with [AWS Cloud9](https://aws.amazon.com/cloud9/). +If you'd like to use Cloud9 to deploy the solution, you will need the following before proceeding: + +- select at least `m5.large` as Instance type. +- use `Ubuntu Server 22.04 LTS` as the platform. + +### Deploy with Github Codespaces + +If you'd like to use [GitHub Codespaces](https://github.com/features/codespaces) to deploy the solution, you will need the following before proceeding: + +1. An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) +2. An [IAM User](https://console.aws.amazon.com/iamv2/home?#/users/create) with: + +- `AdministratorAccess` policy granted to your user (for production, we recommend restricting access as needed) +- Take note of `Access key` and `Secret access key`. + +To get started, click on the button below. + +[![Open in GitHub Codespaces](https://github.com/codespaces/badge.svg)](https://codespaces.new/aws-samples/aws-genai-llm-chatbot) + +Once in the Codespaces terminal, set up the AWS Credentials by running + +```shell +aws configure +``` + +```shell +AWS Access Key ID [None]: +AWS Secret Access Key [None]: +Default region name: +Default output format: json +``` + +You are all set for deployment; you can now jump to [.3 of the deployment section below](#deployment-dependencies-installation). + +### Local deployment + +If you have decided not to use AWS Cloud9 or GitHub Codespaces, verify that your environment satisfies the following prerequisites: + +You have: + +1. An [AWS account](https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/) +2. `AdministratorAccess` policy granted to your AWS account (for production, we recommend restricting access as needed) +3. Both console and programmatic access +4. [NodeJS 16 or 18](https://nodejs.org/en/download/) installed + + - If you are using [`nvm`](https://github.com/nvm-sh/nvm) you can run the following before proceeding + - ``` + nvm install 16 && nvm use 16 + + or + + nvm install 18 && nvm use 18 + ``` + +5. [AWS CLI](https://aws.amazon.com/cli/) installed and configured to use with your AWS account +6. [Typescript 3.8+](https://www.typescriptlang.org/download) installed +7. [AWS CDK CLI](https://docs.aws.amazon.com/cdk/latest/guide/getting_started.html) installed +8. [Docker](https://docs.docker.com/get-docker/) installed + - N.B. [`buildx`](https://github.com/docker/buildx) is also required. For Windows and macOS `buildx` [is included](https://github.com/docker/buildx#windows-and-macos) in [Docker Desktop](https://docs.docker.com/desktop/) +9. [Python 3+](https://www.python.org/downloads/) installed + +## Deployment + +1. Clone the repository + +```bash +git clone https://github.com/aws-samples/aws-genai-llm-chatbot +``` + +2. Move into the cloned repository + +```bash +cd aws-genai-llm-chatbot +``` + +#### (Optional) Only for Cloud9 + +If you use Cloud9, increase the instance's EBS volume to at least 100GB. +To do this, run the following command from the Cloud9 terminal: + +``` +./scripts/cloud9-resize.sh +``` + +See the documentation for more details [on environment resize](https://docs.aws.amazon.com/cloud9/latest/user-guide/move-environment.html#move-environment-resize). + + 3. Install the project dependencies and build the project by running this command + +```bash +npm install && npm run build +``` + +4. Once done, run the magic-create CLI to help you set up the solution with the features you care most: + +```bash +npm run create +``` + +You'll be prompted to configure the different aspects of the solution, such as: + +- The LLMs or MLMs to enable (we support all models provided by Bedrock along with SageMaker hosted Idefics, FalconLite, Mistral and more to come) +- Setup of the RAG system: engine selection (i.e. Aurora w/ pgvector, OpenSearch, Kendra..) embeddings selection and more to come. + +When done, answer `Y` to create a new configuration. + +![sample](./assets/magic-create-sample.gif "CLI sample") + +Your configuration is now stored under `bin/config.json`. You can re-run the magic-create command as needed to update your `config.json` + +5. (Optional) Bootstrap AWS CDK on the target account and region + +> **Note**: This is required if you have never used AWS CDK on this account and region combination. ([More information on CDK bootstrapping](https://docs.aws.amazon.com/cdk/latest/guide/cli.html#cli-bootstrap)). + +```bash +npx cdk bootstrap aws://{targetAccountId}/{targetRegion} +``` + +You can now deploy by running: + +```bash +npx cdk deploy +``` + +> **Note**: This step duration can vary greatly, depending on the Constructs you are deploying. + +You can view the progress of your CDK deployment in the [CloudFormation console](https://console.aws.amazon.com/cloudformation/home) in the selected region. + +6. Once deployed, take note of the `User Interface`, `User Pool` and, if you want to interact with [3P models providers](#3p-models-providers), the `Secret` that will, eventually, hold the various `API_KEYS` should you want to experiment with 3P providers. + +```bash +... +Outputs: +GenAIChatBotStack.UserInterfaceUserInterfaceDomanNameXXXXXXXX = dxxxxxxxxxxxxx.cloudfront.net +GenAIChatBotStack.AuthenticationUserPoolLinkXXXXX = https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx +GenAIChatBotStack.ApiKeysSecretNameXXXX = ApiKeysSecretName-xxxxxx +... +``` + +7. Open the generated **Cognito User Pool** Link from outputs above i.e. `https://xxxxx.console.aws.amazon.com/cognito/v2/idp/user-pools/xxxxx_XXXXX/users?region=xxxxx` + +8. Add a user that will be used to log into the web interface. + +9. Open the `User Interface` Url for the outputs above, i.e. `dxxxxxxxxxxxxx.cloudfront.net` + +10. Login with the user created in .8; you will be asked to change the password. + +## Run user interface locally + +See instructions in the README file of the [`lib/user-interface/react-app`](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/lib/user-interface/react-app/README.md) folder. + +## Using Kendra with a non-english index + +If you're using Kendra with an index in a language other than English, you will need to make some code modifications. + +You'll need to modify the filters in the file `lib/shared/layers/python-sdk/python/genai_core/kendra/query.py`. + +Example for french : + +```python + if kendra_index_external or kendra_use_all_data: + result = kendra.retrieve( + IndexId=kendra_index_id, + QueryText=query, + PageSize=limit, + PageNumber=1, + AttributeFilter={'AndAllFilters': [{"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}]} + ) + else: + result = kendra.retrieve( + IndexId=kendra_index_id, + QueryText=query, + PageSize=limit, + PageNumber=1, + AttributeFilter={'AndAllFilters': + [ + {"EqualsTo": {"Key": "_language_code","Value": {"StringValue": "fr"}}}, + {"EqualsTo": {"Key": "workspace_id","Value": {"StringValue": workspace_id}}} + ] + } + ) +``` + +Please note: If these adjustments are made post-deployment, it's essential to rebuild and redeploy. If done prior to deployment, you can proceed with the walkthrough as usual. + +```bash +npm install && npm run build +npx cdk deploy +``` + +## Clean up + +You can remove the stacks and all the associated resources created in your AWS account by running the following command: + +```bash +npx cdk destroy +``` + +> **Note**: Depending on which resources have been deployed. Destroying the stack might take a while, up to 45m. If the deletion fails multiple times, please manually delete the remaining stack's ENIs; you can filter ENIs by VPC/Subnet/etc using the search bar [here](https://console.aws.amazon.com/ec2/home#NIC) in the AWS console) and re-attempt a stack deletion. \ No newline at end of file diff --git a/docs/guide/developers.md b/docs/guide/developers.md new file mode 100644 index 000000000..650caf967 --- /dev/null +++ b/docs/guide/developers.md @@ -0,0 +1,59 @@ +# Contributing Guidelines + +Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional +documentation, we greatly value feedback and contributions from our community. + +Please read through this document before submitting any issues or pull requests to ensure we have all the necessary +information to effectively respond to your bug report or contribution. + + +## Reporting Bugs/Feature Requests + +We welcome you to use the GitHub issue tracker to report bugs or suggest features. + +When filing an issue, please check existing open, or recently closed, issues to make sure somebody else hasn't already +reported the issue. Please try to include as much information as you can. Details like these are incredibly useful: + +* A reproducible test case or series of steps +* The version of our code being used +* Any modifications you've made relevant to the bug +* Anything unusual about your environment or deployment + + +## Contributing via Pull Requests +Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: + +1. You are working against the latest source on the *main* branch. +2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already. +3. You open an issue to discuss any significant work - we would hate for your time to be wasted. + +To send us a pull request, please: + +1. Fork the repository. +2. Modify the source; please focus on the specific change you are contributing. If you also reformat all the code, it will be hard for us to focus on your change. +3. Ensure local tests pass. +4. Commit to your fork using clear commit messages. +5. Send us a pull request, answering any default questions in the pull request interface. +6. Pay attention to any automated CI failures reported in the pull request, and stay involved in the conversation. + +GitHub provides additional document on [forking a repository](https://help.github.com/articles/fork-a-repo/) and +[creating a pull request](https://help.github.com/articles/creating-a-pull-request/). + + +## Finding contributions to work on +Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any 'help wanted' issues is a great place to start. + + +## Code of Conduct +This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). +For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact +opensource-codeofconduct@amazon.com with any additional questions or comments. + + +## Security issue notifications +If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. + + +## Licensing + +See the [LICENSE](https://github.com/aws-samples/aws-genai-llm-chatbot/blob/main/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. \ No newline at end of file diff --git a/docs/guide/getting-started.md b/docs/guide/getting-started.md deleted file mode 100644 index 77ceca8e6..000000000 --- a/docs/guide/getting-started.md +++ /dev/null @@ -1,9 +0,0 @@ ---- -layout: doc ---- - -# Getting Started - -This solution provides ready-to-use code so you can start **experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts** in your own AWS account. - -![Architecture](../assets/architecture.png) \ No newline at end of file diff --git a/docs/guide/lipsum.md b/docs/guide/lipsum.md deleted file mode 100644 index 48525b7a3..000000000 --- a/docs/guide/lipsum.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -layout: doc -title: Title ---- - -# Lorem Ipsum - - -## 1 -Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis sit amet justo nunc. Cras porta consectetur lacus, et tempus dui consectetur in. Nullam a sem sed neque sollicitudin tempus. Curabitur commodo sapien at sodales scelerisque. Vestibulum rutrum non lectus nec egestas. Duis sodales, nisl eu molestie blandit, ligula nulla fringilla nunc, eget imperdiet erat orci posuere lacus. Nullam elit mauris, egestas eu fermentum nec, feugiat at tortor. Aliquam vitae orci id mauris accumsan vehicula eu a ligula. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. - -Curabitur sollicitudin mollis interdum. Curabitur volutpat ipsum sed nibh pharetra vestibulum. Morbi imperdiet lorem ac varius consequat. Vivamus arcu lacus, dictum a libero ac, bibendum finibus mauris. Phasellus ut venenatis leo, id ultricies eros. Curabitur fermentum orci quis sagittis hendrerit. Vestibulum ut lorem vitae lectus sollicitudin pellentesque. - -Nunc urna augue, varius at auctor ut, dapibus et ligula. Curabitur cursus odio nec tellus tempus, et gravida tellus rhoncus. Nunc vel dapibus nunc. Proin scelerisque vitae tellus eu dictum. Nunc posuere eleifend elit non pulvinar. Sed malesuada est sed turpis vulputate laoreet. Proin venenatis tortor vitae facilisis malesuada. Curabitur sed nunc justo. Donec et feugiat sem, eu gravida turpis. - -## 2 -Sed a sodales lorem, at tempor quam. Nam neque eros, vestibulum vitae velit aliquet, vestibulum convallis nisl. Fusce posuere dictum justo nec vehicula. Sed turpis nisi, pulvinar vel maximus at, ullamcorper at risus. Praesent congue ullamcorper erat. Etiam sed turpis et eros imperdiet congue id quis est. Cras commodo enim eget massa hendrerit, a ultrices leo vulputate. Morbi mollis tincidunt massa, sit amet consequat erat fermentum id. In vel libero id ligula pulvinar semper a vel dolor. Etiam consectetur, enim at imperdiet vehicula, tortor metus congue sem, a pharetra metus lectus quis ante. Donec elit dolor, euismod in finibus vitae, hendrerit quis risus. Suspendisse a felis ultrices dolor placerat convallis. - -Phasellus in elit lectus. Vestibulum commodo mi at nisi cursus aliquam. Nam facilisis justo ut lacus lobortis bibendum. Maecenas non consequat est, quis tempus tellus. Praesent lacinia dolor neque, in mollis felis egestas nec. Quisque volutpat malesuada augue, quis ornare lectus. Mauris a pretium est. Curabitur semper suscipit erat commodo posuere. Duis vel nunc nibh. Vivamus et lectus tincidunt, vestibulum nulla vitae, scelerisque nisl. Nam dolor libero, blandit sed nunc quis, luctus blandit enim. Quisque vitae dui commodo, tristique nibh dapibus, ullamcorper dui. Praesent odio neque, lacinia sit amet turpis sit amet, porttitor commodo orci. Donec a volutpat mi. Aliquam erat volutpat. diff --git a/docs/guide/lipsum2.md b/docs/guide/lipsum2.md deleted file mode 100644 index 194dc7b9a..000000000 --- a/docs/guide/lipsum2.md +++ /dev/null @@ -1,19 +0,0 @@ ---- -layout: doc -title: Title ---- - -# Lorem Ipsum 2 - - -## 3 -Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis sit amet justo nunc. Cras porta consectetur lacus, et tempus dui consectetur in. Nullam a sem sed neque sollicitudin tempus. Curabitur commodo sapien at sodales scelerisque. Vestibulum rutrum non lectus nec egestas. Duis sodales, nisl eu molestie blandit, ligula nulla fringilla nunc, eget imperdiet erat orci posuere lacus. Nullam elit mauris, egestas eu fermentum nec, feugiat at tortor. Aliquam vitae orci id mauris accumsan vehicula eu a ligula. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos himenaeos. - -Curabitur sollicitudin mollis interdum. Curabitur volutpat ipsum sed nibh pharetra vestibulum. Morbi imperdiet lorem ac varius consequat. Vivamus arcu lacus, dictum a libero ac, bibendum finibus mauris. Phasellus ut venenatis leo, id ultricies eros. Curabitur fermentum orci quis sagittis hendrerit. Vestibulum ut lorem vitae lectus sollicitudin pellentesque. - -Nunc urna augue, varius at auctor ut, dapibus et ligula. Curabitur cursus odio nec tellus tempus, et gravida tellus rhoncus. Nunc vel dapibus nunc. Proin scelerisque vitae tellus eu dictum. Nunc posuere eleifend elit non pulvinar. Sed malesuada est sed turpis vulputate laoreet. Proin venenatis tortor vitae facilisis malesuada. Curabitur sed nunc justo. Donec et feugiat sem, eu gravida turpis. - -## 4 -Sed a sodales lorem, at tempor quam. Nam neque eros, vestibulum vitae velit aliquet, vestibulum convallis nisl. Fusce posuere dictum justo nec vehicula. Sed turpis nisi, pulvinar vel maximus at, ullamcorper at risus. Praesent congue ullamcorper erat. Etiam sed turpis et eros imperdiet congue id quis est. Cras commodo enim eget massa hendrerit, a ultrices leo vulputate. Morbi mollis tincidunt massa, sit amet consequat erat fermentum id. In vel libero id ligula pulvinar semper a vel dolor. Etiam consectetur, enim at imperdiet vehicula, tortor metus congue sem, a pharetra metus lectus quis ante. Donec elit dolor, euismod in finibus vitae, hendrerit quis risus. Suspendisse a felis ultrices dolor placerat convallis. - -Phasellus in elit lectus. Vestibulum commodo mi at nisi cursus aliquam. Nam facilisis justo ut lacus lobortis bibendum. Maecenas non consequat est, quis tempus tellus. Praesent lacinia dolor neque, in mollis felis egestas nec. Quisque volutpat malesuada augue, quis ornare lectus. Mauris a pretium est. Curabitur semper suscipit erat commodo posuere. Duis vel nunc nibh. Vivamus et lectus tincidunt, vestibulum nulla vitae, scelerisque nisl. Nam dolor libero, blandit sed nunc quis, luctus blandit enim. Quisque vitae dui commodo, tristique nibh dapibus, ullamcorper dui. Praesent odio neque, lacinia sit amet turpis sit amet, porttitor commodo orci. Donec a volutpat mi. Aliquam erat volutpat. diff --git a/docs/index.md b/docs/index.md index a032e0104..fb4438ecc 100644 --- a/docs/index.md +++ b/docs/index.md @@ -12,12 +12,16 @@ hero: text: View on GitHub link: https://github.com/aws-samples/aws-genai-llm-chatbot features: - - icon: 🛠️ - title: Feature - details: Lorem ipsum... - - icon: - dark: /assets/icon-light.png - light: /assets/icon-dark.png - title: Another cool feature - details: Lorem ipsum... + - title: Modular, comprehensive and ready to use + details: This solution provides ready-to-use code so you can start experimenting with a variety of Large Language Models and Multimodal Language Models, settings and prompts in your own AWS account. + - title: Experiment with multimodal models + details: Deploy IDEFICS models on Amazon SageMazker and see how the chatbot can answer questions about images, describe visual content, and generate text grounded in multiple images. + - title: Multi-Session Chat - evaluate multiple models at once + details: Send the same query to 2 to 4 separate models at once and see how each one responds based on its own learned history, context and access to the same powerful document retriever. + - title: Experiment with multiple RAG options with Workspaces + details: A workspace is a logical namespace where you can upload files for indexing and storage in one of the vector databases. You can select the embeddings model and text-splitting configuration of your choice. + - title: Unlock RAG potentials with Workspaces Debugging Tools + details: The solution comes with several debugging tools to help you debug RAG scenarios. + - title: Full-fledged User Interface + details: The repository includes a CDK construct to deploy a full-fledged UI built with React to interact with the deployed LLMs/MLMs as chatbots. Hosted on Amazon S3 and distributed with Amazon CloudFront. ---