inference gradio web reponse random words for deepseek instrcuct model

[KinglyWayne]

inference gradio web reponse random words for deepseek instrcuct model

while using rest api, everything seems to be OK:

curl -H "Content-Type: application/json" -X POST -d '{"prompt": "写一个快排吧"}' "http://127.0.0.1:8000/api/v1/default/opencsg--opencsg-deepseek-coder-1.3b-v0.1/run/predict" {"generated_text":"}\n\n\n# 快排\ndef quicksort(arr):\n if len(arr) <= 1:\n return arr\n pivot = arr[len(arr) // 2]\n left = [x for x in arr if x < pivot]\n middle = [x for x in arr if x == pivot]\n right = [x for x in arr if x > pivot]\n return quicksort(left) + middle + quicksort(right)\n\n# 测试\nprint(quicksort(arr))\n\n# 输出: [1, 2, 3, 4, 5, 6, 7, 8, 9]\n```\n\n这个程序使用了快速排序算法，它是一种高效的排序算法，基于分治法的原理。它选择一个元素作为枢轴，并根据它们与枢轴的大小将其他元素分成两个子数组，然后递归地对子数组进行排序。\n\n快速排序的平均时间复杂度为O(n log n)，最坏情况下的时间复杂度为O(n^2)，但这种情况很少发生。","num_input_tokens":16,"num_input_tokens_batch":16,"num_generated_tokens":267,"num_generated_tokens_batch":267,"preprocessing_time":0.008793507993686944,"generation_time":2.4766286090016365,"postprocessing_time":0.0009328589949291199,"generation_time_per_token":0.008751337840995181,"generation_time_per_token_batch":0.008751337840995181,"num_total_tokens":283,"num_total_tokens_batch":283,"total_time":2.4863549759902526,"total_time_per_token":0.008785706628940822,"total_time_per_token_batch":0.008785706628940822}(.llm-inference) root@opencsg-gpu1-4090:~/pl/workspace/depenglee/llm-inference#

[depenglee1707]

That's weird, base on the implements, actually the UI and rest ones share same implements, will dig in soon