Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

there is no "usage" in the "metadata" #8544

Closed
5 tasks done
joneqian opened this issue Sep 19, 2024 · 5 comments · Fixed by #9167
Closed
5 tasks done

there is no "usage" in the "metadata" #8544

joneqian opened this issue Sep 19, 2024 · 5 comments · Fixed by #9167
Assignees
@JohnJyong
Labels
🐞 bug Something isn't working

Comments

@joneqian
Copy link

Self Checks

  • This is only for bug report, if you would like to ask a question, please head to Discussions.
  • I have searched for existing issues search for existing issues, including closed ones.
  • I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
  • [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
  • Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

In "advance-chat", the request parameter "response_mode" is "blocking" or "streaming", and there is no "usage" in the "metadata" of the returned result

✔️ Expected Behavior

return "usage" in the "metadata"

❌ Actual Behavior

No response
@dosubot DosuBot
Copy link

dosubot bot commented Sep 19, 2024

Hello, @joneqian! I'm here to assist you with any bugs, questions, or contributions. Let's work together to resolve the issue you're experiencing.

The "usage" information should be included in the "metadata" of the returned result for both "blocking" and "streaming" response modes in "advance-chat".

For "Blocking Mode" responses, the "usage" information is typically structured as follows:

{
    "event": "message",
    "message_id": "9da23599-e713-473b-982c-4328d4f5c78a",
    "mode": "completion",
    "answer": "Hello World!...",
    "metadata": {
        "usage": {
            "prompt_tokens": 1033,
            "prompt_unit_price": "0.001",
            "prompt_price_unit": "0.001",
            "prompt_price": "0.0010330",
            "completion_tokens": 128,
            "completion_unit_price": "0.002",
            "completion_price_unit": "0.001",
            "completion_price": "0.0002560",
            "total_tokens": 1161,
            "total_price": "0.0012890",
            "currency": "USD",
            "latency": 0.7682376249867957
        }
    },
    "created_at": 1705407629
}

For "Streaming Mode" responses, the "usage" information is included in the "metadata" of the final chunk:

data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " I", "created_at": 1679586595}
data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": "'m", "created_at": 1679586595}
data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " glad", "created_at": 1679586595}
data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " to", "created_at": 1679586595}
data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " meet", "created_at": 1679586595}
data: {"event": "message", "message_id": "5ad4cb98-f0c7-4085-b384-88c403be6290", "answer": " you", "created_at": 1679586595}
data: {"event": "metadata", "metadata": {"usage": {"total_tokens": 63127864, "total_price": 2.378, "currency": "USD"}}}

If the "usage" information is missing, it could be due to a bug or misconfiguration. You might want to check the specific logging or debugging information available in the Dify codebase, such as the ensure_dict method in the LangfuseTrace, LangfuseSpan, and LangfuseGeneration classes, and the GenerationUsage class, which captures detailed usage information [1][2].

To continue talking to Dosu, mention @dosu.

@dosubot dosubot bot added the 🐞 bug Something isn't working label Sep 19, 2024
@joneqian
Copy link
Author

I get this response data, there is no usage info in metadata：

{
        "event": "message",
        "message_id": "d62483e5-62ff-4885-a212-ee35a7eac9fb",
        "conversation_id": "a58e0296-528c-4b7c-950b-14b96dabbd20",
        "answer": "空腹血糖值低于正常范围（低于3.9 mmol/L）时，需要采取适当措施以避免低血糖的风险。对于3.5 mmol/L的空腹血糖值，建议采取以下步骤：\n\n1. **立即摄入糖分**：可以选择喝15克的糖水或吃2颗糖果，迅速提高血糖水平。\n2. **休息**：在摄入糖分后，坐下休息15分钟，然后重新测量血糖。\n3. **监测血糖变化**：确保血糖恢复到正常范围内，如果情况持续或出现症状，可能需要进一步关注或调整管理方案。\n\n注意：长期低血糖的情况需要与专业人士讨论，以便找到合适的管理方案。",
        "created_at": 1726725811,
        "metadata": {
            "retriever_resources": [
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "a921172d-52bc-4a04-a4de-d26a54432e0c",
                    "document_name": "糖尿病防治实用指导 第3版.pdf-2024-05-12-23-40-37-122.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "34dee678-ef0c-44a2-be1a-1463e5fec6d3",
                    "retriever_from": "workflow",
                    "score": 0.7689991004193767,
                    "hit_count": 10,
                    "word_count": 14,
                    "segment_position": 1046,
                    "index_node_hash": "0ab55e371ff45c1a63f7927df413a30e90d0036a6b49cb60edf121918bc7605c",
                    "content": "| 空腹 | >5. 1 |"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "a921172d-52bc-4a04-a4de-d26a54432e0c",
                    "document_name": "糖尿病防治实用指导 第3版.pdf-2024-05-12-23-40-37-122.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "7fc31df6-e172-4505-8f28-a03b7441a7d5",
                    "retriever_from": "workflow",
                    "score": 0.47724096715985764,
                    "hit_count": 10,
                    "word_count": 12,
                    "segment_position": 263,
                    "index_node_hash": "e1bddc1e5d8140c19c2dc32bf82c81e7a99ededa09d455baa06957d017bd97af",
                    "content": "检查空腹血糖有什么意义？"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "a921172d-52bc-4a04-a4de-d26a54432e0c",
                    "document_name": "糖尿病防治实用指导 第3版.pdf-2024-05-12-23-40-37-122.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "7a5dc1fa-c57f-46b7-ae9d-9db33c1e7a38",
                    "retriever_from": "workflow",
                    "score": 0.45920210644962994,
                    "hit_count": 10,
                    "word_count": 14,
                    "segment_position": 265,
                    "index_node_hash": "39dbb4f926b7a767583f3603442f0f2a5cf635f747c07369bdf8d7862c2c59f2",
                    "content": "测空腹血糖前应注意哪些问题？"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "9cc31ce4-5e5c-4cfa-9483-63c6e0634729",
                    "document_name": "国家基层糖尿病防治管理指南2022.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "b8138bb9-2861-418a-bfce-3b3ad94ea47e",
                    "retriever_from": "workflow",
                    "score": 0.4542735352545543,
                    "hit_count": 10,
                    "word_count": 16,
                    "segment_position": 111,
                    "index_node_hash": "8999b138b8236221c41ac5ebbfd7e0aedc82f5ea5d281d76d4441dfadf3b6a60",
                    "content": "空腹血糖、餐后2h血糖、甘油三酯"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "c8c9a856-6441-4bff-8862-c72e92b3a5f5",
                    "document_name": "糖尿病个体化诊治策略.pdf-2024-05-13-11-14-12-915.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "155c7389-45b3-44b2-8976-a96f55f761af",
                    "retriever_from": "workflow",
                    "score": 0.36999078088615517,
                    "hit_count": 4,
                    "word_count": 57,
                    "segment_position": 651,
                    "index_node_hash": "449387cf748dc9147e124c80a89ebfaa6912a78f51d891d684f6baee7249403f",
                    "content": "| 空腹或餐前 | 3.9 〜5.6 | 5.0 〜8.0 | > 8.0 | >9.0 | 3.9 〜7.2 |"
                },
                {
                    "position": 1,
                    "dataset_id": "e0584c4e-6a38-4ae4-9ed0-d9be07758ee3",
                    "dataset_name": "山丘-血糖标准话术",
                    "document_id": "91206763-0b1e-4f35-b13a-432281da94d1",
                    "document_name": "血糖标准话术.md",
                    "data_source_type": "upload_file",
                    "segment_id": "5d4a462c-c7b1-4b76-a644-1a4c4eca1b82",
                    "retriever_from": "workflow",
                    "score": 0.2869762583699582,
                    "hit_count": 26,
                    "word_count": 466,
                    "segment_position": 1,
                    "index_node_hash": "aecaf2050c8a333857978cc544c586c6c5ef2d4e602d507aa5d95a8108599114",
                    "content": "- 血糖类型：空腹血糖：每日 06:00 至 8:59 测量的血糖值，一般指起床空腹状态下，吃早餐前测量的血糖值 - 区间：低值：＜ 3.9mmol/L - 话术：目前空腹血糖低于正常值，建议可以服用 15g 的糖水或者吃 2 颗糖果，坐下休息 15 分钟后，在测一下血糖，避免出现风险 - 区间：警示值：3.9-4.3mmol/L - 话术：目前血糖略微偏低，建议尽快吃早餐，避免出现低血糖 - 区间：正常值：4.4-6.1mmol/L - 话术：血糖很好，继续努力 - 区间：正常值：4.4-6.1mmol/L - 话术：血糖很好，继续保持 - 区间：正常值：4.4-6.1mmol/L - 话术：血糖很好，继续加油 - 区间：偏高：6.2-7mmol/L - 话术：空腹血糖略微偏高，建议继续保持当前的饮食结构，给胰岛功能创造修复空间，继续加油 - 区间：高值：＞ 7mmol/L - 话术：空腹血糖是反应基础胰岛功能的，目前的空腹血糖偏高，建议晚餐后可以增加规律的运动，保证充足的睡眠，规律用药，对于改善空腹血糖都是有帮助的"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "a921172d-52bc-4a04-a4de-d26a54432e0c",
                    "document_name": "糖尿病防治实用指导 第3版.pdf-2024-05-12-23-40-37-122.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "aef5414f-dd54-48bb-af9a-34aff5337c85",
                    "retriever_from": "workflow",
                    "score": 0.2858193492,
                    "hit_count": 4,
                    "word_count": 20,
                    "segment_position": 332,
                    "index_node_hash": "d33e9f593e1366205637e7efed0b9ca0dd80ef5970934b9c1add9a1f1889ef12",
                    "content": "平时饭量比糖尿病饮食计算的量大时怎么办？"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "4b8a8a38-c79a-48e1-8fca-514e0a3cfff5",
                    "document_name": "逆转糖尿病.pdf-2024-05-13-14-26-07-156.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "2521ae90-a4be-442c-b6b4-6ea347f47a1c",
                    "retriever_from": "workflow",
                    "score": 0.27782250719999996,
                    "hit_count": 4,
                    "word_count": 5,
                    "segment_position": 3073,
                    "index_node_hash": "cee4c514b1d424840ecac7f858fcfa94df2d2f84e681ef79e74475b76bb81ae0",
                    "content": "血糖检测3"
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "4b8a8a38-c79a-48e1-8fca-514e0a3cfff5",
                    "document_name": "逆转糖尿病.pdf-2024-05-13-14-26-07-156.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "6c817e5c-6a9e-48ac-bba0-e7e23318737b",
                    "retriever_from": "workflow",
                    "score": 0.27427096199999995,
                    "hit_count": 4,
                    "word_count": 15,
                    "segment_position": 269,
                    "index_node_hash": "e87b80e18d6f11c9272ae665cfafea55cc97ac9066c56f2adfe9b8b868aebe6c",
                    "content": "口服葡萄糖2小吋血糖值的意义："
                },
                {
                    "position": 1,
                    "dataset_id": "31788454-626b-48cc-bc1b-c5619cfbc788",
                    "dataset_name": "山丘-糖尿病知识库",
                    "document_id": "a921172d-52bc-4a04-a4de-d26a54432e0c",
                    "document_name": "糖尿病防治实用指导 第3版.pdf-2024-05-12-23-40-37-122.docx",
                    "data_source_type": "upload_file",
                    "segment_id": "d2d82d5f-9b0c-4ba0-baef-e64b38f009f1",
                    "retriever_from": "workflow",
                    "score": 0.27407474639999996,
                    "hit_count": 7,
                    "word_count": 18,
                    "segment_position": 271,
                    "index_node_hash": "ab3f53ea2af1e2c94572875695d36968250885da099383750d0ec5ca619fe8af",
                    "content": "为什么要查餐后血糖，有哪些注意事项？"
                }
            ]
        }

@DayoWang
Copy link

DayoWang commented Sep 20, 2024 •
edited
Loading

The same issue, and sometimes under the same workflow (multiple LLMs and knowledge bases are connected in series), after upgrading to 0.8.2, no retriever_resources is returned.

@crazywoola crazywoola mentioned this issue Sep 21, 2024
Closed
5 tasks
@crazywoola
Copy link
Member

Link #8530

@crazywoola
Copy link
Member

Seems to be fixed in #8663

@joneqian joneqian mentioned this issue Oct 9, 2024
Closed
5 tasks
@takatost takatost mentioned this issue Oct 10, 2024
Merged
11 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JohnJyong JohnJyong

Labels
🐞 bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: missing usage of metadata in the chatflow app langgenius/dify
4 participants
@DayoWang @joneqian @JohnJyong @crazywoola