[Feature] Upgrade the needle-in-a-haystack experiment to Needlebench (#…

…913) * add needlebench * simplify needlebench 32k, 128k, 200k for eval * update act prompt * fix bug in needlebench summarizer * add needlebench intro, fix summarizer * lint summarizer * fix linting error * move readme.md * update readme for needlebench * update docs of needlebench * simplify needlebench summarizers
open-compass · Mar 4, 2024 · 8142f39 · 8142f39
1 parent 120bf8b
commit 8142f39
Show file tree

Hide file tree

Showing 40 changed files with 5,056 additions and 730 deletions.
diff --git a/configs/datasets/cdme/cdme200k.py b/configs/datasets/cdme/cdme200k.py
diff --git a/configs/datasets/cdme/cdme32k.py b/configs/datasets/cdme/cdme32k.py
diff --git a/configs/datasets/cdme/cdme8k.py b/configs/datasets/cdme/cdme8k.py
diff --git a/configs/datasets/cdme/multi_needle/cdme8k_cot2_italy.py b/configs/datasets/cdme/multi_needle/cdme8k_cot2_italy.py