Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make sure to raise an error if pick_python_cmd returns False for Python bundles/packages #3430

Merged
merged 14 commits into from
Sep 6, 2024

Conversation

casparvl
Copy link
Contributor

@casparvl casparvl commented Sep 3, 2024

This was already done for PythonPackage, but not for PythonBundle. I ran into this here easybuilders/easybuild-easyconfigs#21307 (comment)

@casparvl
Copy link
Contributor Author

casparvl commented Sep 3, 2024

When used with ReFrame-4.9.2.eb from easybuilders/easybuild-easyconfigs#21307 , with this PR, a proper error is raised:

== 2024-09-03 21:20:47,564 run.py:247 INFO running cmd: /usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'
== 2024-09-03 21:20:47,591 run.py:689 DEBUG cmd "/usr/bin/python3 -c 'import sys; print("%s.%s.%s" % sys.version_info[:3])'" exited with exit code 0 and output:
3.6.8

== 2024-09-03 21:20:47,591 run.py:725 DEBUG Using default regular expression: (?<![(,-]|\w)(?:error|segmentation fault|failed)(?![(,-]|\.?\w)
== 2024-09-03 21:20:47,591 pythonpackage.py:130 DEBUG Minimal requirement for minor Python version not satisfied: 3.6.8 vs 3.7
== 2024-09-03 21:20:47,591 pythonpackage.py:159 DEBUG Python command '/usr/bin/python3' does not satisfy version requirements (maj: 3, min: 7), moving on
== 2024-09-03 21:20:47,611 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/base/exceptions.py:126 in __init__): Failed to pick Python command to use (at easybuild/easyblocks/generic/pythonbundle.py:115 in prepare_step)
== 2024-09-03 21:20:47,612 build_log.py:267 INFO ... (took < 1 sec)
== 2024-09-03 21:20:47,615 config.py:700 DEBUG software install path as specified by 'installpath' and 'subdir_software': /home/it4i-casparl/.local/easybuild/software
== 2024-09-03 21:20:47,615 filetools.py:2013 INFO Removing lock /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.local_easybuild_software_ReFrame_4.6.2.lock...
== 2024-09-03 21:20:47,616 filetools.py:383 INFO Path /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.
local_easybuild_software_ReFrame_4.6.2.lock successfully removed.
== 2024-09-03 21:20:47,616 filetools.py:2017 INFO Lock removed: /home/it4i-casparl/.local/easybuild/software/.locks/_home_it4i-casparl_.local_easybuild_software_ReFrame_4.6.2.lock
== 2024-09-03 21:20:47,616 easyblock.py:4285 WARNING build failed (first 300 chars): Failed to pick Python command to use

Although it is still cryptic (it doesn't specifically complain about the version unless you run with --debug), at least behavior is now the same as for PythonPackage. And: at least EasyBuild doesn't just try to run a python command that doesn't exist on the system.

@casparvl
Copy link
Contributor Author

casparvl commented Sep 3, 2024

Test build in easybuilders/easybuild-easyconfigs#21307 (comment) (ended up in the other PR... but still shows it in the gist all right: https://gist.github.com/casparvl/07e98bf3cb8c43ee1a6f34a38b1eed6d . It now

  1. errors out also for pythonbundle and
  2. the error is more clear on the fact that there is a requirement for the python versions in the EasyConfigs (and that that might be why no valid python command is found)

@bedroge
Copy link
Contributor

bedroge commented Sep 6, 2024

Test report by @bedroge

Overview of tested easyconfigs (in order)

Build succeeded for 0 out of 1 (1 easyconfigs in total)
interactive1 - Linux Rocky Linux 8.9, x86_64, AMD EPYC-Milan Processor (zen3), Python 3.6.8
See https://gist.github.com/bedroge/658cf22fd1c22abb32ca8337a5027899 for a full test report.

This is an expected failure to demonstrate that the added functionality works: I tried building the ReFrame version from easybuilders/easybuild-easyconfigs#21307 using Python 3.6, while it needs at least 3.7.

== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/p251204/ReFrame/4.6.2/system-system): build failed (first 300 chars): Failed to pick 
python command that satisfies requirements in the EasyConfigs (req_py_majver = 3, req_py_minver = 7) (took 0 secs)

@bedroge
Copy link
Contributor

bedroge commented Sep 6, 2024

Test report by @bedroge

Overview of tested easyconfigs (in order)

  • SUCCESS pyBigWig-0.3.22-gfbf-2023a.eb
  • SUCCESS PyYAML-6.0-GCCcore-12.2.0.eb
  • SUCCESS sktime-0.25.0-gfbf-2023a.eb
  • SUCCESS NiBabel-4.0.2-foss-2022a.eb

Build succeeded for 4 out of 4 (4 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/390dbacfe8775eb02397ca838813083c for a full test report.

@bedroge bedroge changed the title Make sure to raise an error if pick_python_cmd returns False make sure to raise an error if pick_python_cmd returns False for Python bundles/packages Sep 6, 2024
@bedroge
Copy link
Contributor

bedroge commented Sep 6, 2024

Test report by @bedroge

Overview of tested easyconfigs (in order)

  • SUCCESS ReFrame-4.6.2.eb

Build succeeded for 1 out of 1 (1 easyconfigs in total)
bob-Latitude-5300 - Linux Ubuntu 24.04.1 LTS (Noble Numbat), x86_64, Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz, Python 3.12.3
See https://gist.github.com/bedroge/ddf841dd1329ad5b5c6f7ac561446ff9 for a full test report.

This is the other easyconfig from easybuilders/easybuild-easyconfigs#21307, which requires at least Python 3.7, and 3.12 is being used. So, it also works as expected.

Copy link
Contributor

@bedroge bedroge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm

@bedroge bedroge added this to the release after 4.9.2 milestone Sep 6, 2024
@bedroge bedroge added the bug fix label Sep 6, 2024
@bedroge bedroge merged commit 020ca79 into easybuilders:develop Sep 6, 2024
41 checks passed
@boegel boegel changed the title make sure to raise an error if pick_python_cmd returns False for Python bundles/packages make sure to raise an error if pick_python_cmd returns False for Python bundles/packages Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants