Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove comm hash and add per-comm caches #724

Merged
merged 41 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
877def6
Remove hash_comm and add per-comm caches
JDBetteridge Aug 8, 2024
767a21f
Update the TestDiskCachedDecorator test for collective
JDBetteridge Aug 8, 2024
e35a589
Refactor disk_cached wrapper and add warning
JDBetteridge Aug 8, 2024
64f5616
Refactor memory/disk caches in order to remove use of id() in GlobalK…
JDBetteridge Aug 8, 2024
9a31c3d
Rethink memory only cache for non-broadcastable values
JDBetteridge Aug 8, 2024
3f030af
New caching implementation, WIP needs tidying and better tests
JDBetteridge Aug 12, 2024
6aa6554
WIP
JDBetteridge Aug 12, 2024
6dec230
Just notes
JDBetteridge Aug 13, 2024
a7451fc
Remove cached(object) and lint
JDBetteridge Aug 14, 2024
c1a48c9
WIP, need to fix remaining cache tests
JDBetteridge Aug 14, 2024
d1bacf0
Round things out and fix remaining tests
JDBetteridge Aug 14, 2024
a7f43d4
CI debugging
JDBetteridge Aug 15, 2024
783ab12
CI debugging, ssh in?
JDBetteridge Aug 15, 2024
3d13c4f
CI debugging, a fix?
JDBetteridge Aug 15, 2024
756e29e
CI debugging, a fix for 3.12
JDBetteridge Aug 15, 2024
f3decc4
.
JDBetteridge Aug 15, 2024
723468b
.
JDBetteridge Aug 15, 2024
474058d
.
JDBetteridge Aug 15, 2024
aab930a
Remove custom implementation of @cached_property, use the Python(>=3.…
JDBetteridge Aug 15, 2024
f512249
Add a sreial cache with the same interface
JDBetteridge Aug 15, 2024
c12194c
Caching fixes and tweaks
JDBetteridge Aug 16, 2024
c74c7bb
Lint
JDBetteridge Aug 16, 2024
f52449c
Add instrumentation to caches
JDBetteridge Aug 19, 2024
0c2b088
Remove get_so from Compiler base class
JDBetteridge Aug 19, 2024
0cf63f9
Remove comm from Compiler class
JDBetteridge Aug 19, 2024
7ca74eb
Use caching.disk_only_cache for make_so
JDBetteridge Aug 19, 2024
91c25b8
A better solution for wrapping make_so in a disk cache
JDBetteridge Aug 20, 2024
919523a
Update pyop2/caching.py
dham Aug 21, 2024
b1ecd92
WIP: may have reintroduced deadlocks
JDBetteridge Aug 22, 2024
c6225f0
WIP: Fixed the deadlock
JDBetteridge Aug 22, 2024
13bc76c
Add event decorators
JDBetteridge Aug 23, 2024
9bdc418
WIP: Fixing and tidying
JDBetteridge Aug 23, 2024
aeddb09
Add additional cache tests
JDBetteridge Aug 23, 2024
f4f4194
Fix stats printing
JDBetteridge Aug 24, 2024
1c69860
Handle multiple VENVs + instrumenting
JDBetteridge Aug 27, 2024
8d0f319
Add PYOP2_SPMD_STRICT environment variable for checking MPI correctness
JDBetteridge Aug 27, 2024
e8f0859
Actually put mismatched kernels in the directory created
JDBetteridge Sep 1, 2024
212f9d0
Update .github/workflows/ci.yml
dham Oct 3, 2024
5706ddd
Apply code suggestions from review
JDBetteridge Oct 7, 2024
81d0f0a
Consolidate caching tests
JDBetteridge Oct 7, 2024
df14e55
Need to close file descriptor
JDBetteridge Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 22 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
# Don't immediately kill all if one Python version fails
fail-fast: false
matrix:
python-version: ['3.8', '3.9', '3.10', '3.11']
python-version: ['3.9', '3.10', '3.11', '3.12']
env:
CC: mpicc
PETSC_DIR: ${{ github.workspace }}/petsc
Expand Down Expand Up @@ -58,15 +58,15 @@ jobs:
working-directory: ${{ env.PETSC_DIR }}/src/binding/petsc4py
run: |
python -m pip install --upgrade pip
python -m pip install --upgrade wheel 'cython<3' numpy
python -m pip install --upgrade wheel cython numpy
python -m pip install --no-deps .

- name: Checkout PyOP2
uses: actions/checkout@v2
with:
path: PyOP2

- name: Install PyOP2
- name: Install PyOP2 dependencies
shell: bash
working-directory: PyOP2
run: |
Expand All @@ -76,7 +76,21 @@ jobs:
python -m pip install pulp
python -m pip install -U flake8
python -m pip install -U pytest-timeout
python -m pip install .

- name: Install PyOP2 (Python <3.12)
if: ${{ matrix.python-version != '3.12' }}
shell: bash
working-directory: PyOP2
run: python -m pip install .

# Not sure if this is a bug in setuptools or something PyOP2 is doing wrong
- name: Install PyOP2 (Python == 3.12)
if: ${{ matrix.python-version == '3.12' }}
shell: bash
working-directory: PyOP2
run: |
python -m pip install -U setuptools
python setup.py install

- name: Run linting
shell: bash
Expand All @@ -86,7 +100,10 @@ jobs:
- name: Run tests
shell: bash
working-directory: PyOP2
run: pytest --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
run: |
# Running parallel test cases separately works around a bug in pytest-mpi
pytest -k "not parallel" --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
mpiexec -n 3 pytest -k "parallel[3]" --tb=native --timeout=480 --timeout-method=thread -o faulthandler_timeout=540 -v test
timeout-minutes: 10

- name: Build documentation
Expand Down
Loading
Loading