Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Evaluation Schema + a necessary revamp #118

Merged
merged 24 commits into from
Feb 12, 2023
Merged
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8471190
fix: add upsc mock; refactor; create evaluation-schema
Udayraj123 Dec 30, 2022
a796fe6
fix: refactor; change defaults to .py; pickup config.json recursively…
Udayraj123 Jan 1, 2023
f41d128
fix: update evaluation-schema and sample json wip
Udayraj123 Jan 2, 2023
e74394e
refactor: separate static vs instance ImageUtils; breakfown files fur…
Udayraj123 Jan 3, 2023
c69b978
fix: load json error handling; dynamic config json working for sample5
Udayraj123 Jan 5, 2023
91fc506
fix: use formatted strings at all places necessary
Udayraj123 Jan 5, 2023
3e7450f
fix: correct config schema
Udayraj123 Jan 5, 2023
453680b
fix: working evaluation code
Udayraj123 Jan 5, 2023
f8b97c3
fix: add range operator support; add validations for marking scheme a…
Udayraj123 Jan 7, 2023
aa69b50
fix: add a sample with answer_key.csv
Udayraj123 Jan 7, 2023
d16b647
fix: minor fixes
Udayraj123 Jan 8, 2023
9bd65dd
fix: foundation for answer_key_image_path; refactoring; minor changes
Udayraj123 Jan 8, 2023
cb6817d
fix: remove unnecessary comments; fix minor comments
Udayraj123 Jan 8, 2023
f8a7d91
fix: further on foundation for csv_from_image feature; refactoring
Udayraj123 Jan 8, 2023
8169a81
fix: stronger validations; conditionality in streak scheme; joinpath …
Udayraj123 Jan 12, 2023
54383b3
fix: support multi-correct and basic multi-weighted; update explanations
Udayraj123 Jan 13, 2023
b715c8f
fix: csv load custom key support
Udayraj123 Jan 13, 2023
ea75858
fix: add evaluation configs in some samples
Udayraj123 Jan 13, 2023
3692500
fix: load answer key from image with working samples
Udayraj123 Jan 23, 2023
fb15654
fix: compress images
Udayraj123 Jan 26, 2023
d308609
fix: update samples; add readme for template shift sample
Udayraj123 Jan 26, 2023
1f54a6e
Merge branch 'master' into feature/evalution-refactor
Udayraj123 Feb 1, 2023
b2f394b
fix: reduce images; minor changes
Udayraj123 Feb 1, 2023
426b7a6
[Feature] setup pytest on existing samples (#124)
Udayraj123 Feb 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 11 additions & 12 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,15 +1,14 @@
**/__pycache__
**/CheckedOMRs
**/ignore
**/.DS_Store
**/.vscode
venv/
.idea/
OMRChecker.wiki/
# Any directory starting with a dot
**/\.*/
# Except .github
!.github/

# Everything in inputs/ and outputs/
inputs/*
outputs/*
# Except *.json and OMR_Files/
# !inputs/OMR_Files/
# !inputs/*.json
# !inputs/omr_marker.jpg

# Misc
**/.DS_Store
**/__pycache__
venv/
OMRChecker.wiki/
3 changes: 2 additions & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ repos:
hooks:
- id: check-yaml
- id: end-of-file-fixer
exclude_types: ["csv", "json"]
- id: trailing-whitespace
- repo: https://github.com/pycqa/isort
rev: 5.5.4
Expand All @@ -20,4 +21,4 @@ repos:
- id: flake8
args:
- "F401,F841" # unused imports, unused variables
- "--ignore=E501,W503,E203" # Line too long, Line break occurred before a binary operator, Whitespace before ':'
- "--ignore=E501,W503,E203,E741" # Line too long, Line break occurred before a binary operator, Whitespace before ':'
9 changes: 5 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,20 +220,21 @@ Command: <code>python3 -m pip install --user --upgrade pip</code>
## Full Usage

```
python3 main.py [--setLayout] [--autoAlign] [--inputDir dir1] [--outputDir dir1]
python3 main.py [--setLayout] [--inputDir dir1] [--outputDir dir1]
```

Explanation for the arguments:

`--setLayout`: Set up OMR template layout - modify your json file and run again until the template is set.

`--autoAlign`: (experimental) Enables automatic template alignment - use if the scans show slight misalignments.

`--inputDir`: Specify an input directory.

`--outputDir`: Specify an output directory.

**Note:** The `--noCropping` flag has been replaced with including/excluding a 'CropPage' plugin in "preProcessors" of the template.json(see [samples](https://github.com/Udayraj123/OMRChecker/tree/master/samples)).
**Notes:**

- The old `--noCropping` flag has been replaced with the 'CropPage' plugin in "preProcessors" of the template.json(see [samples](https://github.com/Udayraj123/OMRChecker/tree/master/samples)).
- The old `--autoAlign` flag can now be toggled from config.json

<!-- #### Testing the code
Datasets to test on :
Expand Down
19 changes: 2 additions & 17 deletions main.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,11 @@

"""

from src.logger import logger

logger.info(f"Loading OMRChecker modules...")
# It takes a few seconds for the imports

import argparse
from pathlib import Path

from src.core import entry_point
from src.entry import entry_point
from src.logger import logger

# construct the argument parse and parse the arguments
argparser = argparse.ArgumentParser()
Expand All @@ -41,17 +37,6 @@
help="Specify an output directory.",
)

# TODO: separate the interactive modes from main code
argparser.add_argument(
"-a",
"--autoAlign",
required=False,
dest="autoAlign",
action="store_true",
help="(experimental) Enables automatic template alignment - \
use if the scans show slight misalignments.",
)

argparser.add_argument(
"-l",
"--setLayout",
Expand Down
13 changes: 7 additions & 6 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
matplotlib>=3.3.1
numpy>=1.16.1
pandas>=1.1.1
deepmerge>=0.1.0
dotmap>=1.3.17
jsonschema>=4.3.3
deepmerge>=1.0.1
dotmap>=1.3.30
jsonschema>=4.16.0
matplotlib>=3.5.3
numpy>=1.23.2
pandas>=1.4.3
rich>=12.6.0
screeninfo>=0.8.1
Binary file modified samples/community/Sandeep-1507/omr-1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 0 additions & 2 deletions samples/community/Sandeep-1507/template.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
{
"Globals": {
"window_width": 1189,
Udayraj123 marked this conversation as resolved.
Show resolved Hide resolved
"window_height": 1682,
"display_width": 1189,
"display_height": 1682
},
Expand Down
Binary file added samples/community/UPSC-mock/answer_key.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
11 changes: 11 additions & 0 deletions samples/community/UPSC-mock/config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"dimensions": {
"display_height": 1800,
"display_width": 2400,
"processing_height": 2400,
"processing_width": 1800
},
"outputs": {
"show_image_level": 0
}
}
12 changes: 12 additions & 0 deletions samples/community/UPSC-mock/evaluation.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"source_type": "csv",
"options": {
"answer_key_csv_path": "answer_key.csv",
"answer_key_image_path": "answer_key.jpg",
"questions_in_order": ["q1..100"],
"should_explain_scoring": true
},
"marking_scheme": {
"DEFAULT": {"correct": "2", "incorrect": "-2/3", "unmarked": "0" }
}
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading