Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Virgo HDF5 file format #240

Draft
wants to merge 24 commits into
base: main
Choose a base branch
from
Draft

Virgo HDF5 file format #240

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
3f7282f
update virgo generated dataset to use hdf5 format
jarlsondre Nov 6, 2024
9a39414
add functionality for selecting output location
jarlsondre Nov 6, 2024
865eda9
set new data format as standard
jarlsondre Nov 6, 2024
01ae8c0
make virgo work with new data loader and add progress bar
jarlsondre Nov 7, 2024
31fdbd2
fix merge conflict
jarlsondre Nov 7, 2024
3e674e6
remove old generation files and add script for concatenating hdf5 files
jarlsondre Nov 7, 2024
4bc8cd6
remove old generation files and add script for concatenating hdf5 files
jarlsondre Nov 7, 2024
ec2e589
Merge branch 'virgo-hdf5' of github.com:interTwin-eu/itwinai into vir…
jarlsondre Nov 7, 2024
35d86ab
rename folder using hyphens
jarlsondre Nov 7, 2024
4dc1dea
remove multiprocessing
jarlsondre Nov 7, 2024
812696f
add multiprocessing at correct place
jarlsondre Nov 7, 2024
c039052
update handling of seed and num processes
jarlsondre Nov 8, 2024
7687be9
Gpu monitoring (#237)
jarlsondre Nov 7, 2024
322acbc
Scalability test wall clock (#239)
jarlsondre Nov 7, 2024
1d1343f
make virgo work with new data loader and add progress bar
jarlsondre Nov 7, 2024
060e572
add contributors
jarlsondre Nov 8, 2024
8c18316
update ruff settings in pyproject
jarlsondre Nov 8, 2024
bed5249
update virgo dataset concatenation
jarlsondre Nov 8, 2024
b7e72a7
fix merge
jarlsondre Nov 8, 2024
46b8c96
add isort option to ruff
jarlsondre Nov 8, 2024
933fce0
break imports on purpose
jarlsondre Nov 8, 2024
c094bb6
break more imports to test
jarlsondre Nov 8, 2024
0017e0d
remove ruff config file
jarlsondre Nov 8, 2024
85d855d
😀
jarlsondre Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/lint.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
PYTHON_FLAKE8_CONFIG_FILE: .flake8
JSCPD_CONFIG_FILE: .jscpd.json
PYTHON_ISORT_CONFIG_FILE: .isort.cfg
# PYTHON_RUFF_CONFIG_FILE: .ruff.toml

VALIDATE_PYTHON_BLACK: false
VALIDATE_PYTHON_MYPY: false
Expand Down
5 changes: 4 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,4 +202,7 @@ cython_debug/
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
#.idea/

# MacOS
.DS_Store
4 changes: 4 additions & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ dependencies = [
# "prov4ml@git+https://github.com/HPCI-Lab/ProvML@main", # Prov4ML
# "prov4ml@git+https://github.com/matbun/ProvML@main",
"pandas",
"seaborn"
]

# dynamic = ["version", "description"]
Expand Down Expand Up @@ -95,3 +96,6 @@ markers = [
[tool.isort]
profile = "black"
known_first_party = ["itwinai"]

[tool.ruff.lint]
select = ["I"]
Loading