Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytorch 2.4.1, python 3.13, cudnn9 #261

Merged

Conversation

regro-cf-autotick-bot
Copy link
Contributor

@regro-cf-autotick-bot regro-cf-autotick-bot commented Sep 19, 2024

This PR has been triggered in an effort to update python313.

Notes and instructions for merging this PR:

  1. Please merge the PR only after the tests have passed.
  2. Feel free to push to the bot's branch to update this PR if needed.

Please note that if you close this PR we presume that the feedstock has been rebuilt, so if you are going to perform the rebuild yourself don't close this PR until the your rebuild has been merged.


If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase @conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. Feel free to drop us a line if there are any issues! This PR was generated by - please use this URL for debugging.

Also includes...

Fixes #264
Fixes #259

@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@hmaarrfk hmaarrfk mentioned this pull request Sep 19, 2024
1 task
@hmaarrfk
Copy link
Contributor

hmaarrfk commented Sep 20, 2024

if anybody could rebuild OSX that would be great!

Edit: nevermind, i forgot about
#256 and #259

@hmaarrfk hmaarrfk changed the title Rebuild for python 3.13 Rebuild for python 3.13 and cudnn 9 Sep 21, 2024
@hmaarrfk
Copy link
Contributor

cudnn 9 would make this package incompatible with cupy.
conda-forge/cupy-feedstock#286

Would it be appropriate to build out 2 versions, one for cudnn 9, the other for cudnn 8???

@hmaarrfk hmaarrfk force-pushed the rebuild-python313-0-1_h3e3c58 branch 2 times, most recently from 47a96d9 to 2665bfd Compare September 22, 2024 18:40
@hmaarrfk hmaarrfk changed the title Rebuild for python 3.13 and cudnn 9 Rebuild for python 3.13 Sep 22, 2024
@conda-forge-webservices
Copy link
Contributor

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found some lint.

Here's what I've got...

For recipe/meta.yaml:

  • If python is a host requirement, it should be a run requirement.

@conda-forge-webservices
Copy link
Contributor

conda-forge-webservices bot commented Sep 22, 2024

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

@hmaarrfk
Copy link
Contributor

I thought that it i could get things to work by downgrading to 2.4.0 instead of 2.4.1.... but it still doesn't work

@Tobias-Fischer
Copy link
Contributor

It looks like it's going ok so far? :)

@hmaarrfk
Copy link
Contributor

No I ran it locally in docker on a linux-64 host and ran into the same problem as
#256

@hmaarrfk hmaarrfk force-pushed the rebuild-python313-0-1_h3e3c58 branch from 0ffc036 to 4f14b13 Compare October 2, 2024 12:14
@hmaarrfk hmaarrfk changed the title Rebuild for python 3.13 Pytorch 2.4.1, python 3.13, cudnn9 Oct 2, 2024
@hmaarrfk hmaarrfk force-pushed the rebuild-python313-0-1_h3e3c58 branch from 7b062b5 to da0e9fd Compare October 2, 2024 12:18
@h-vetinari h-vetinari force-pushed the rebuild-python313-0-1_h3e3c58 branch from da0e9fd to be0a19c Compare October 2, 2024 13:10
@jakirkham
Copy link
Member

jakirkham commented Oct 4, 2024

Am seeing the following error in this CI job (with attached log):

-- Using third party subdirectory Eigen.
CMake Error at /home/conda/feedstock_root/build_artifacts/libtorch_1728075572459/_build_env/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:233 (message):
  Could NOT find Python (missing: Python_INCLUDE_DIRS Interpreter
  Development.Module NumPy) (found version "3.11.10")
Call Stack (most recent call first):
  /home/conda/feedstock_root/build_artifacts/libtorch_1728075572459/_build_env/share/cmake-3.30/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_FAILURE_MESSAGE)
  /home/conda/feedstock_root/build_artifacts/libtorch_1728075572459/_build_env/share/cmake-3.30/Modules/FindPython.cmake:673 (find_package_handle_standard_args)
  cmake/Dependencies.cmake:857 (find_package)
  CMakeLists.txt:857 (include)

Edit: Another jobs fails with the same error

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

its probably because i hardcoded the numpy include dir and its frozen in.

@jakirkham
Copy link
Member

What if we added a symlink to the NumPy headers using a Python version agnostic location and provided the latter as input?

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

you are welcome to try, i can back off.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

actuallly my guess is that cmake is invoked through a new codepath i didn't capture in my patch.

@jakirkham
Copy link
Member

jakirkham commented Oct 5, 2024

I trust your judgment Mark. Please let me know if I'm just missing things

Edit: Fixed a typo

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

I’m too tired for today. Just giving ideas.

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

I think it is a difference between numpy 2.0 and 2.1.

Is there a reason to to build with 2.0 vs 2.1?

The reason i ask is that the migration pins to 2.0:
https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/migrations/numpy2.yaml#L48

@h-vetinari
Copy link
Member

Is there a reason to to build with 2.0 vs 2.1?

Just because we started out conservatively. You can use 2.1 without issue, or pin to 2 directly

@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 5, 2024

i'm not so convinced about my 2.1 theory anymore. I had successful builds with Linux64 and aarch64 in the past...

@hmaarrfk hmaarrfk removed the automerge Merge the PR when CI passes label Oct 5, 2024
@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 6, 2024

@isuruf it seems the sed “fix” was at the source of the issue. Any other ideas on how to make sed better? Or should we just restart builds periodically.

@hmaarrfk hmaarrfk added the automerge Merge the PR when CI passes label Oct 6, 2024
@hmaarrfk
Copy link
Contributor

hmaarrfk commented Oct 6, 2024

Well thank you all!!!!!

@hmaarrfk hmaarrfk merged commit 31fdb5f into conda-forge:main Oct 6, 2024
26 checks passed
@regro-cf-autotick-bot regro-cf-autotick-bot deleted the rebuild-python313-0-1_h3e3c58 branch October 6, 2024 16:36
@jakirkham
Copy link
Member

Hooray! 🥳

Thanks everyone! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Merge the PR when CI passes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Mitigate performance degradation on linux-aarch64 CUDA builds by incorporating the linker script from upstream
7 participants