You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I am looking for guidance on two key aspects of working with the vLLM engine:
Debugging the vLLM Engine:
What are the recommended steps and tools for debugging the vLLM engine?
Are there any specific logs or configurations that can help in identifying performance bottlenecks or errors?
Measuring the Backend vLLM Engine Performance:
How can I measure the performance of the vLLM backend engine in isolation?
Are there any built-in metrics or tools provided by vLLM for this purpose, or would external profiling tools be more appropriate?
Right now what I am doing is through the web client application, is there anyway to do it with the vLLM Engine alone?
Thank you for your assistance!
Before submitting a new issue...
Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
The text was updated successfully, but these errors were encountered:
htang2012
changed the title
[Usage]: Debugging and Measuring the vLLM Engine Backend
[Usage]: Standalone Debugging and Measuring the vLLM Engine Backend
Sep 19, 2024
I am looking for guidance on two key aspects of working with the vLLM engine:
Debugging the vLLM Engine:
What are the recommended steps and tools for debugging the vLLM engine?
Are there any specific logs or configurations that can help in identifying performance bottlenecks or errors?
Measuring the Backend vLLM Engine Performance:
How can I measure the performance of the vLLM backend engine in isolation?
Are there any built-in metrics or tools provided by vLLM for this purpose, or would external profiling tools be more appropriate?
Right now what I am doing is through the web client application, is there anyway to do it with the vLLM Engine alone?
Thank you for your assistance!
Before submitting a new issue...
The text was updated successfully, but these errors were encountered: