Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Initial assessment of accuracy of LLM label information extraction and checklist evaluation #1

Closed
13 of 14 tasks
SamuelPelletierEvraire opened this issue Mar 26, 2024 · 0 comments · Fixed by #4
Closed
13 of 14 tasks

Initial assessment of accuracy of LLM label information extraction and checklist evaluation #1

SamuelPelletierEvraire opened this issue Mar 26, 2024 · 0 comments · Fixed by #4

Comments

@SamuelPelletierEvraire
Copy link
Contributor

SamuelPelletierEvraire commented Mar 26, 2024 •
edited
Loading

  • checkin test images (one product per folder)
  • create input spreadsheet for checklist (confidential so not checked in)
    • extract checklist into spreadsheet
    • provide reformulate questions and key names for every question
    • export to .csv as input to the script
  • iterate through product folders to generate LLM queries
    • read input .csv and generate n queries per product
  • testing
    • Testing with original checklist question
    • Testing with the modified question
    • Testing with different degree of certitude
    • Testing with different way to ask question
  • for each test run
    • Output YYYY-mm-dd-result.json
    • Compile result to get % result of correct answers.
@rngadam rngadam changed the title Testing every step to do Initial assessment of accuracy of LLM label information extraction and checklist evaluation Mar 26, 2024
@SamuelPelletierEvraire SamuelPelletierEvraire linked a pull request Apr 12, 2024 that will close this issue
Adding unit test for comparing json file #4
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
FertiScan
Status: Done
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Adding unit test for comparing json file ai-cfia/fertiscan-frontend
1 participant
@SamuelPelletierEvraire