Skip to content

Commit

Permalink
Merge branch 'main' into app-template
Browse files Browse the repository at this point in the history
  • Loading branch information
@shhdgit
shhdgit authored Mar 28, 2024
2 parents 0c28631 + 27815ce commit 7663d6c
Show file tree
Hide file tree
Showing 3 changed files with 93 additions and 92 deletions.
125 changes: 35 additions & 90 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,120 +2,65 @@

# LinguFlow

## Local Deployment

You can run LinguFlow on your local machine using docker compose. This method is ideal for developing & testing LinguFlow and troubleshooting integration issues.

Requirements: docker and docker compose, both are included in [Docker Desktop](https://docs.docker.com/get-docker/) for Mac or Windows users.

### Getting Started

```sh
git clone [email protected]:pingcap/LinguFlow.git
cd LinguFlow

# start the ui and api server
docker-compose -f docker-compose.dev.yaml up
```

-> Now, you can access Langfuse at http://localhost:5173

### Updating Langfuse
## What is LinguFlow

Most of the time, you only need to execute `git pull` to use the latest code of LinguFlow locally, except for two exceptions:
LinguFlow, a low-code tool designed for LLM application development, simplifies the building, debugging, and deployment process for developers. It utilizes a DAG-based message flow for business logic, requiring only minimal familiarity with LinguFlow blocks to effectively use.

- The dependencies of `LinguFlow` have been updated (in requirements.txt)
- The model of `LinguFlow` has been updated (in model.py)
### Why we need LinguFlow?

In these two cases, you need to execute `docker-compose -f docker-compose.dev.yaml build` to rebuild the image of LinguFlow.
When attempting to apply LLM to real-world business scenarios, the limitations of using a simple LLM Wrapper become evident. These limitations include:

## Self-Hosting Guide
- Difficulty in further improving accuracy.
- The inability to restrict the conversation to business-relevant topics only.
- Challenges in handling complex business processes.

LinguFlow Server, which includes the API and Web UI, is open-source and can be self-hosted using Docker.
LinguFlow is needed precisely to overcome these challenges, offering a platform that enables the structured building of LLM applications tailored to specific business needs and enhancing their accuracy over time. The most classic approach to deploying applications with LLM (Large Language Models) is through the construction of a [DAG (Directed Acyclic Graph)](https://en.wikipedia.org/wiki/Directed_acyclic_graph).

### Prerequisites: Database
### the features with LinguFlow

A database is required to store the business data of LinguFlow, including applications, versions, and interactions.
Thus, the features of applications developed with LinguFlow include:

LinguFlow can support [these](https://docs.sqlalchemy.org/en/20/dialects/index.html#support-levels-for-included-dialects) databases, and it is recommended to use [TiDB Serverless](https://www.pingcap.com/tidb-serverless/) produced by PingCAP. It is a fully-managed MySQL compatible database that can automatically scale and provides free quotas, making it the first choice for small development teams.
- **Technical Characteristics**:
- Construction based on a DAG of information flows.
- Multiple interactions with an LLM (for example, a Chatbot might interact with an LLM four times to answer a single customer query) where each interaction addresses a specific issue, such as intent determination, rephrasing, answering, or critique. This approach effectively overcomes the limitations of single interactions and supports the development of relatively complex applications.

No matter which database you choose, once the database is ready, keep the connection string handy.
- **Business Characteristics**:
- LinguFlow is suitable for those with a clear understanding of how to solve their business problems using LLM, particularly when supporting more complex logic and requiring higher accuracy. As LinguFlow is based on the construction of DAG, similar to traditional application development, it is also well-suited for diving into complex business scenarios.

### Configure the Database
In essence, LinguFlow's design and implementation method offer a structured and logical framework for integrating LLMs into complex business processes, emphasizing the accuracy and logic-specific solutions of LLM interactions.

LinguFlow's model supports the automatic creation of table structures using [alembic](https://alembic.sqlalchemy.org/en/latest/). However, before this, you need to manually create the database. For TiDB Serverless, you can directly execute the SQL `create database linguflow` on the Chat2Query page of its web console. For other databases, you can use the corresponding client of the database to connect and execute `create database linguflow` to create it.
## Get Started

Once the database is created, you can start the automatic creation of table structures.
### Localhost (docker)

Execute the following commands to initialize the database:
You can run LinguFlow on your local machine using [docker](https://docs.docker.com/get-docker/) compose. This setup is perfect for developing, testing LinguFlow applications, and diagnosing integration issues.

```sh
# First, install alembic
pip install alembic

# Initialize alembic
# Clone the LinguFlow repository
git clone [email protected]:pingcap/LinguFlow.git
# Navigate into the LinguFlow directory
cd LinguFlow
alembic init alembic
sed -i '1s|^|import model\n|' alembic/env.py
sed -i "s|target_metadata =.*|target_metadata = model.Base.metadata|" alembic/env.py
sed -i "s|sqlalchemy.url =.*|sqlalchemy.url = <database_url>|" alembic.ini
alembic revision --autogenerate -m "init"
alembic upgrade head
```

Note: Please replace `<database_url>` in the above command with the actual SQLAIchemy format database connection address. For TiDB Serverless, it looks like:

```
mysql+pymysql://2ogML5E9iFqsVWA.root:<PASSWORD>@gateway01.us-west-2.prod.aws.tidbcloud.com:4000/linguflow?ssl_ca=/etc/ssl/certs/ca-certificates.crt&ssl_verify_cert=true&ssl_verify_identity=true
```

### Deploying the Application

To boostrap the application, you should firstly edit `docker-compose.yaml` and change the `DATABASE_URL` environment to the above real database url. Then on the production host:

```sh
docker-compose up -d
```

Then you can access LinguFlow page on http://{your-public-ip}.

### How to update

To update the application:

```sh
cd LinguFlow
docker-compose down
git pull
docker-compose build --no-cache
alembic revision --autogenerate -m "update schema if any necessary database migrations"
alembic upgrade head
docker-compose up
# Start the UI and API server
docker-compose -f docker-compose.dev.yaml up
```

## What is LinguFlow
-> You can now access LinguFlow at http://localhost:5173. More about [deploying locally](https://www.linguflow.com/docs/deployment/local).

LinguFlow, a low-code tool designed for LLM application development, simplifies the building, debugging, and deployment process for developers. It utilizes a DAG-based message flow for business logic, requiring only minimal familiarity with LinguFlow blocks to effectively use.
### Self-Hosting (docker)

### Why we need LinguFlow?

When attempting to apply LLM to real-world business scenarios, the limitations of using a simple LLM Wrapper become evident. These limitations include:

- Difficulty in further improving accuracy.
- The inability to restrict the conversation to business-relevant topics only.
- Challenges in handling complex business processes.
LinguFlow Server, which includes the API and Web UI, is open-source and can be self-hosted using Docker.

LinguFlow is needed precisely to overcome these challenges, offering a platform that enables the structured building of LLM applications tailored to specific business needs and enhancing their accuracy over time. The most classic approach to deploying applications with LLM (Large Language Models) is through the construction of a [DAG (Directed Acyclic Graph)](https://en.wikipedia.org/wiki/Directed_acyclic_graph).
-> More about [deploying self-host](https://www.linguflow.com/docs/deployment/self_host).

### the features with LinguFlow
### API Call

Thus, the features of applications developed with LinguFlow include:
1. Click the Connect App button within the App.
2. Follow the instructions to use the POST API to call the asynchronous interface, obtaining the interaction id for this interaction.
3. Use the GET API to query the previously obtained interaction id, retrieving the final response from the LinguFlow application.

- **Technical Characteristics**:
- Construction based on a DAG of information flows.
- Multiple interactions with an LLM (for example, a Chatbot might interact with an LLM four times to answer a single customer query) where each interaction addresses a specific issue, such as intent determination, rephrasing, answering, or critique. This approach effectively overcomes the limitations of single interactions and supports the development of relatively complex applications.
-> More about [APIs](https://www.linguflow.com/docs/run/call_an_application).

- **Business Characteristics**:
- LinguFlow is suitable for those with a clear understanding of how to solve their business problems using LLM, particularly when supporting more complex logic and requiring higher accuracy. As LinguFlow is based on the construction of DAG, similar to traditional application development, it is also well-suited for diving into complex business scenarios.
## License

In essence, LinguFlow's design and implementation method offer a structured and logical framework for integrating LLMs into complex business processes, emphasizing the accuracy and logic-specific solutions of LLM interactions.
This repository is MIT licensed, except for the ee/ folder. See [LICENSE](LICENSE) for more details.
4 changes: 2 additions & 2 deletions api/application.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -335,8 +335,8 @@ def score_interaction(
)

# XXX:
# The Langfuse SDK executes network requests in the background on a separate thread.
# Any exception on that thread cannot be caught here. Therefore, a response with success=True
# The Langfuse SDK executes network requests in the background on a separate thread.
# Any exception on that thread cannot be caught here. Therefore, a response with success=True
# does not necessarily mean that the Langfuse server has accepted the score.
return ItemCreateResponse(success=True, message=f"Score has been created.")

Expand Down
56 changes: 56 additions & 0 deletions observability.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ def wrapper(*args, **kwargs):
token = ctx.push("langfuse", Langfuse(**decorator_kwargs))
try:
return func(*args, **kwargs)
except Exception as e:
ctx.langfuse.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.langfuse.flush()
ctx.pop(token)
Expand All @@ -58,6 +65,13 @@ async def async_wrapper(*args, **kwargs):
token = ctx.push("langfuse", Langfuse(**decorator_kwargs))
try:
return await func(*args, **kwargs)
except Exception as e:
ctx.langfuse.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.langfuse.flush()
ctx.pop(token)
Expand Down Expand Up @@ -90,6 +104,13 @@ def wrapper(*args, **kwargs):
output = func(*args, **kwargs)
ctx.this_observation.update(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand All @@ -112,6 +133,13 @@ async def async_wrapper(*args, **kwargs):
output = await func(*args, **kwargs)
ctx.this_observation.update(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand Down Expand Up @@ -144,6 +172,13 @@ def wrapper(*args, **kwargs):
output = func(*args, **kwargs)
ctx.this_observation.end(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand All @@ -166,6 +201,13 @@ async def async_wrapper(*args, **kwargs):
output = await func(*args, **kwargs)
ctx.this_observation.end(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand Down Expand Up @@ -204,6 +246,13 @@ def wrapper(*args, **kwargs):
else:
ctx.this_observation.end(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand Down Expand Up @@ -231,6 +280,13 @@ async def async_wrapper(*args, **kwargs):
else:
ctx.this_observation.end(output=output_fn(output))
return output
except Exception as e:
ctx.this_observation.event(
name=type(e).__name__,
level="ERROR",
status_message=str(e),
)
raise e
finally:
ctx.last_observation = ctx.this_observation
ctx.pop(token)
Expand Down

0 comments on commit 7663d6c

Please sign in to comment.