Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Task] 做一个面向开源领域的 Benchmark 框架 #40

Open
will-ww opened this issue Jun 25, 2023 · 0 comments
Open

[Task] 做一个面向开源领域的 Benchmark 框架 #40

will-ww opened this issue Jun 25, 2023 · 0 comments

Comments

@will-ww
Copy link
Contributor

will-ww commented Jun 25, 2023

基于部分以前的讨论:

准备做一个标准的、可扩展的 Benchmark 框架,方便实验室的各位同学进行套用,推动不同方面的工作。

例如一个 OpenPerf 中的 Benchmark 可以包括:

  • Benchmark 名称:例如,OPB-Bot-Classification
  • Benchmark 类别:【discussion】benchmark 类型讨论 #31 中的五个类型之一,先从第三类做起(数据类)
  • Benchmark 应用场景:开源纲目中的某个应用场景(需要重构与优化)
  • Benchmark 任务:某个具体的任务,例如分类
  • 数据集:所需要进行评测的数据集
  • 数据类型:时间序列、图数据、NLP、多媒体等
  • 评价指标:对应任务上的评价指标,例如 Precision、Recall
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant