New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] mmlupro 正则提取错误 #1661

Open

2 tasks done

bittersweet1999 opened this issue Nov 4, 2024 · 0 comments

Assignees

Collaborator

bittersweet1999 commented Nov 4, 2024

先决条件

我已经搜索过问题和讨论但未得到预期的帮助。
错误在最新版本中尚未被修复。

问题类型

我正在使用官方支持的任务/模型/数据集进行评估。

环境

None

重现问题 - 代码/配置示例

https://github.com/open-compass/opencompass/blob/main/configs/datasets/mmlu_pro/mmlu_pro_gen_cdbebf.py

重现问题 - 命令或脚本

None

重现问题 - 错误信息

正则提取错误导致得分不可信

其他信息

No response

mm-assistant bot assigned bittersweet1999

bittersweet1999 assigned tonysy and unassigned bittersweet1999

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment