Skip to content

Use OpenAI with HuggingChat by emulating the text_generation_inference_server

License

Notifications You must be signed in to change notification settings

gururise/openai_text_generation_inference_server

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

21 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HuggingChat with OpenAI Integration

This repository contains code that enables the integration of the Hugging Chat UI with the OpenAI API. The provided Python FastAPI code allows you to generate chat completions using the OpenAI GPT-3.5-turbo model and stream the results to the Hugging Chat UI in real-time.

Features

  • Integration of the Hugging Chat UI with the OpenAI API for chat completions.
  • Streaming of chat completion results to the UI in real-time.
  • Customizable parameters for generating chat completions.
  • Easy setup and deployment with FastAPI.

Requirements

To run the code in this repository, you need the following:

  • Python 3.7 or higher
  • FastAPI
  • Pydantic
  • OpenAI Python SDK
  • asyncio

You will also need a valid OpenAI API key to authenticate your requests.

Installation

  1. Clone the repository

  2. install the required dependencies: pip install -r requirements.txt

  3. Set up your OpenAI API key:

    Sign up for an OpenAI account and obtain an API key.

    Set your API key as the OPENAI_API_KEY env variable.

Usage

  1. Start the FastAPI server by running the following command:

    uvicorn server:app --reload

    The server will start running on http://localhost:8000 by default.

  2. Update the .env.local file on your HuggingChat UI installation to include:

    MODELS=`[
    {
        "name": "ChatGPT 3.5 Model",
        "endpoints": [{"url": "http://127.0.0.1:8000/generate_stream"}],
        "userMessageToken": "User: ",
        "assistantMessageToken": "Assistant: ",
        "messageEndToken": "\n",
        "preprompt": "You are a helpful assistant.",
        "parameters": {
        "temperature": 0.9,
        "max_new_tokens": 50,
        "truncate": 1000
        }
    }
    ]`

Testing

Make a POST request to http://localhost:8000/generate_stream with the following JSON payload:

{
"inputs": "User input message",
    "parameters": {
        "temperature": 0.7,
        "max_tokens": 100
    }
}

The chat completions generated by the OpenAI GPT-3.5-turbo model will be streamed to the UI in real-time.

  1. Using Curl, you can test via:
    curl 127.0.0.1:8000/generate_stream \
        -X POST \
        -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17}}' \
        -H 'Content-Type: application/json'

License

This project is licensed under the GPL3 License.

About

Use OpenAI with HuggingChat by emulating the text_generation_inference_server

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published