Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Open Arabic LLM Leaderboard Benchmarks (Full and Light Version) #2232

Merged
merged 68 commits into from
Sep 10, 2024
Merged

Add Open Arabic LLM Leaderboard Benchmarks (Full and Light Version) #2232

merged 68 commits into from
Sep 10, 2024

Commits on Aug 12, 2024

  1. arabic leaferboard yaml file is added

    @Malikeh97
    Malikeh97 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    ef82779 View commit details
    Browse the repository at this point in the history

  2. arabic toxigen is implemented

    @Malikeh97
    Malikeh97 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    09d428f View commit details
    Browse the repository at this point in the history

  3. Dataset library is imported

    @Malikeh97
    Malikeh97 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    8d78ff4 View commit details
    Browse the repository at this point in the history

  4. arabic sciq is added

    @Malikeh97
    Malikeh97 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    53f107b View commit details
    Browse the repository at this point in the history

  5. util file of arabic toxigen is updated

    @Malikeh97
    Malikeh97 committed Aug 12, 2024
    Configuration menu
    Copy the full SHA
    07396a2 View commit details
    Browse the repository at this point in the history

Commits on Aug 13, 2024

  1. arabic race is added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    71dfe8b View commit details
    Browse the repository at this point in the history

  2. arabic piqa is implemented

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    71534a3 View commit details
    Browse the repository at this point in the history

  3. arabic open qa is added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    6118a55 View commit details
    Browse the repository at this point in the history

  4. arabic copa is implemented

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    49e4011 View commit details
    Browse the repository at this point in the history

  5. arabic boolq ia added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    38ff060 View commit details
    Browse the repository at this point in the history

  6. arabic arc easy is added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    9720eb5 View commit details
    Browse the repository at this point in the history

  7. arabic arc challenge is added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    045b383 View commit details
    Browse the repository at this point in the history

  8. arabic exams benchmark is implemented

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    850dca3 View commit details
    Browse the repository at this point in the history

  9. arabic hellaswag is added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    d5c2e55 View commit details
    Browse the repository at this point in the history

  10. arabic leaderboard yaml file metrics are updated

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    d041ae9 View commit details
    Browse the repository at this point in the history

  11. arabic mmlu benchmarks are added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    1c63080 View commit details
    Browse the repository at this point in the history

  12. arabic mmlu group yaml file is updated

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    5feec66 View commit details
    Browse the repository at this point in the history

  13. alghafa benchmarks are added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    27f07d4 View commit details
    Browse the repository at this point in the history

  14. acva benchmarks are added

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    10b758f View commit details
    Browse the repository at this point in the history

  15. acva utils.py is updated

    @Malikeh97
    Malikeh97 committed Aug 13, 2024
    Configuration menu
    Copy the full SHA
    40fe782 View commit details
    Browse the repository at this point in the history

Commits on Aug 14, 2024

  1. light version of arabic leaderboard benchmarks are added

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    c3396db View commit details
    Browse the repository at this point in the history

  2. bugs fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    b48bc10 View commit details
    Browse the repository at this point in the history

  3. bug fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    38bd735 View commit details
    Browse the repository at this point in the history

  4. bug fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    860b0dd View commit details
    Browse the repository at this point in the history

  5. bug fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    d92c684 View commit details
    Browse the repository at this point in the history

  6. bug fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    6f3f692 View commit details
    Browse the repository at this point in the history

  7. bug fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    89c5e31 View commit details
    Browse the repository at this point in the history

  8. library import bug is fixed

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    634ac08 View commit details
    Browse the repository at this point in the history

  9. doc to target updated

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    821e101 View commit details
    Browse the repository at this point in the history

  10. bash file is deleted

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    1992457 View commit details
    Browse the repository at this point in the history

  11. results folder is deleted

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    01fe836 View commit details
    Browse the repository at this point in the history

  12. leaderboard groups are added

    @Malikeh97
    Malikeh97 committed Aug 14, 2024
    Configuration menu
    Copy the full SHA
    223ee4b View commit details
    Browse the repository at this point in the history

Commits on Aug 15, 2024

  1. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @Malikeh97
    Malikeh97 committed Aug 15, 2024
    Configuration menu
    Copy the full SHA
    c505e63 View commit details
    Browse the repository at this point in the history

Commits on Aug 16, 2024

  1. full arabic leaderboard groups are added, plus some bug fixes to the … 

    …light version
    @shahrzads
    shahrzads committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    15dfe96 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @Malikeh97
    Malikeh97 committed Aug 16, 2024
    Configuration menu
    Copy the full SHA
    729a34f View commit details
    Browse the repository at this point in the history

Commits on Aug 20, 2024

  1. Create README.md 

    README.md for arabic_leaderboard_complete
    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    7b1c4db View commit details
    Browse the repository at this point in the history

  2. Create README.md 

    README.md for arabic_leaderboard_light
    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    28c4c85 View commit details
    Browse the repository at this point in the history

  3. Delete lm_eval/tasks/arabic_leaderboard directory

    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    9cb8a1e View commit details
    Browse the repository at this point in the history

  4. Update README.md

    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    cb9d718 View commit details
    Browse the repository at this point in the history

  5. Update README.md 

    adding the Arabic leaderboards to the library
    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    d8fc640 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    80e7ba3 View commit details
    Browse the repository at this point in the history

  7. Update README.md 

    10% of the training set
    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    0072ded View commit details
    Browse the repository at this point in the history

  8. Update README.md 

    10% of the training set
    @shahrzads
    shahrzads committed Aug 20, 2024
    Configuration menu
    Copy the full SHA
    c3e089b View commit details
    Browse the repository at this point in the history

Commits on Aug 23, 2024

  1. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    72665a0 View commit details
    Browse the repository at this point in the history

  2. revert .gitignore to prev version

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    72b2865 View commit details
    Browse the repository at this point in the history

  3. Update lm_eval/tasks/README.md 

    Co-authored-by: Hailey Schoelkopf <[email protected]>
    @shahrzads @haileyschoelkopf
    shahrzads and haileyschoelkopf committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    7689a8d View commit details
    Browse the repository at this point in the history

  4. updated main README.md

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    4adec15 View commit details
    Browse the repository at this point in the history

  5. Update lm_eval/tasks/README.md

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    1d55cde View commit details
    Browse the repository at this point in the history

  6. specify machine translated benchmarks (complete)

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    cc48912 View commit details
    Browse the repository at this point in the history

  7. specify machine translated benchmarks (light version)

    @shahrzads
    shahrzads committed Aug 23, 2024
    Configuration menu
    Copy the full SHA
    6b27890 View commit details
    Browse the repository at this point in the history

Commits on Aug 26, 2024

  1. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @shahrzads
    shahrzads committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    0a4adb9 View commit details
    Browse the repository at this point in the history

  2. add alghafa to the related task names (complete and light)

    @shahrzads
    shahrzads committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    7151859 View commit details
    Browse the repository at this point in the history

  3. add 'acva' to the related task names (complete and light)

    @shahrzads
    shahrzads committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    e65f546 View commit details
    Browse the repository at this point in the history

  4. add 'arabic_leaderboard' to all the groups (complete and light)

    @shahrzads
    shahrzads committed Aug 26, 2024
    Configuration menu
    Copy the full SHA
    3c0d737 View commit details
    Browse the repository at this point in the history

Commits on Aug 27, 2024

  1. all dataset - not a random sample

    @shahrzads
    shahrzads committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    7d4d0e9 View commit details
    Browse the repository at this point in the history

  2. added more accurate details to the readme file

    @shahrzads
    shahrzads committed Aug 27, 2024
    Configuration menu
    Copy the full SHA
    8c24721 View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. added mt_mmlu from okapi

    @shahrzads
    shahrzads committed Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7004776 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @Malikeh97
    Malikeh97 committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    fdaa3c1 View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2024

  1. Update lm_eval/tasks/README.md 

    Co-authored-by: Hailey Schoelkopf <[email protected]>
    @shahrzads @haileyschoelkopf
    shahrzads and haileyschoelkopf committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    d0bcbed View commit details
    Browse the repository at this point in the history

  2. Update lm_eval/tasks/README.md 

    Co-authored-by: Hailey Schoelkopf <[email protected]>
    @shahrzads @haileyschoelkopf
    shahrzads and haileyschoelkopf committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    547ce12 View commit details
    Browse the repository at this point in the history

  3. updated mt_mmlu readme

    @shahrzads
    shahrzads committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    014473c View commit details
    Browse the repository at this point in the history

  4. renaming 'alghafa' full and light

    @shahrzads
    shahrzads committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    c7aa2ff View commit details
    Browse the repository at this point in the history

  5. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @shahrzads
    shahrzads committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    b051008 View commit details
    Browse the repository at this point in the history

  6. renaming 'arabic_mmlu' light and full

    @shahrzads
    shahrzads committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    7c641b7 View commit details
    Browse the repository at this point in the history

  7. renaming 'acva' full and light

    @shahrzads
    shahrzads committed Aug 30, 2024
    Configuration menu
    Copy the full SHA
    e0f9fd2 View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. update readme and standardize dir/file names

    @shahrzads
    shahrzads committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    b0366af View commit details
    Browse the repository at this point in the history

  2. running pre-commit

    @shahrzads
    shahrzads committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    33237fe View commit details
    Browse the repository at this point in the history

Commits on Sep 10, 2024

  1. Merge branch 'EleutherAI:main' into arabic-leaderboard

    @Malikeh97
    Malikeh97 committed Sep 10, 2024
    Configuration menu
    Copy the full SHA
    f590408 View commit details
    Browse the repository at this point in the history