Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python: fix Py_SetPythonHome deprecated API for newer versions #3322

Open
wants to merge 1 commit into
base: dev
Choose a base branch
from

Conversation

r3yc0n1c
Copy link

Your checklist for this pull request

Detailed description
Prevent Deprecated API call since Python 3.11:

https://docs.python.org/3/c-api/init.html#c.Py_SetPythonHome
https://docs.python.org/3/c-api/init_config.html#init-config

Test plan (required)
Build test for newer versions ( > 3.11 ) of Python

Closing issues
closes #3214

Copy link
Member

@XVilka XVilka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/Users/runner/work/cutter/cutter/src/common/PythonManager.cpp:67:5: error: unknown type name 'PyConfig'; did you mean 'RzConfig'?
    PyConfig config;
    ^~~~~~~~
    RzConfig
/Users/runner/work/cutter/cutter/build/Rizin-prefix/include/librz/rz_config.h:58:3: note: 'RzConfig' declared here
} RzConfig;
  ^
/Users/runner/work/cutter/cutter/src/common/PythonManager.cpp:68:5: error: use of undeclared identifier 'PyConfig_InitPythonConfig'
    PyConfig_InitPythonConfig(&config);
    ^
/Users/runner/work/cutter/cutter/src/common/PythonManager.cpp:72:50: error: no member named 'home' in 'rz_config_t'
        PyConfig_SetBytesString(&config, &config.home, customPythonHome);
                                          ~~~~~~ ^
/Users/runner/work/cutter/cutter/src/common/PythonManager.cpp:93:30: error: use of undeclared identifier 'config'
    Py_InitializeFromConfig(&config);
                             ^
/Users/runner/work/cutter/cutter/src/common/PythonManager.cpp:101:21: error: use of undeclared identifier 'config'
    PyConfig_Clear(&config);
                    ^

https://github.com/rizinorg/cutter/actions/runs/8408745952/job/23025329200?pr=3322#step:7:4826

@karliss
Copy link
Member

karliss commented Mar 25, 2024

Test plan (required)
Build test for newer versions ( > 3.11 ) of Python

This issue needs a better test plan than that. For issue like this the code changes is the easy part, properly testing it ensure it works correctly is the meat of it.

On what systems with what python versions and what Cutter build configurations did you try?

How did you test that the code for changing pythonhome is actually functioning correctly? Since this is optional config (from Pyhthon perspective, not necessarily optional for all usecases) , just having it compile and even Cutter open doesn't automatically prove that the change is correct.

I might have been less strict on this if the new API was a drop in replacement of one function requiring only very localized changes, but in this case it also changes the overall sequence of how Python is initialized.

@r3yc0n1c
Copy link
Author

I used Ubuntu 18.04 with Python 3.11. I had multiple python versions (3.8 (system), 3.9, 3.10) with pyenv.

While testing with multiple versions I faced the Shiboken2 configuration error and when I included the <Python.h> header I got these:

image-tc3mu8cugtb8ibnc5eb7tc9zhe.png

Now, I switched to Ubuntu 22.04 but the configuration error still persists.

@0pendev
Copy link

0pendev commented Apr 14, 2024

Hi everyone,

Interested in this merge for packaging reasons 😄

I used Ubuntu 18.04 with Python 3.11. I had multiple python versions (3.8 (system), 3.9, 3.10) with pyenv.

While testing with multiple versions I faced the Shiboken2 configuration error and when I included the <Python.h> header I got these:

image-tc3mu8cugtb8ibnc5eb7tc9zhe.png

Now, I switched to Ubuntu 22.04 but the configuration error still persists.

The error is weird here. It looks like shibokken is defining structs for compatibility reasons (files prepended with 37) but you are using python38 and it redefines the struct again. How are you including those headers ?

@@ -55,11 +55,23 @@ void PythonManager::initPythonHome()
}
#endif

#if (PY_MAJOR_VERSION <= 3) && (PY_MICRO_VERSION < 11)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at the API and ABI versioning documentation, I would suggest the following:

Suggested change
#if (PY_MAJOR_VERSION <= 3) && (PY_MICRO_VERSION < 11)
#if (PY_MAJOR_VERSION == 3) && (PY_MINOR_VERSION < 11)

@0pendev
Copy link

0pendev commented Apr 20, 2024

Hey @r3yc0n1c. Might have an answer to your problem.
So the cutter project defines Py_LIMITED_API in PythonAPI.h.

The problem is that shibokken redefines structs and functions already defined if this macro is set:
Sat_Apr_20_07:01:03_PM_CEST_2024

So I see two possible fixes:

  • remove the definition in PythonAPI.h because even if it is there the project still manually checks for versioning later on
  • open an issue on shiboken2

@r3yc0n1c
Copy link
Author

Hey @r3yc0n1c. Might have an answer to your problem. So the cutter project defines Py_LIMITED_API in PythonAPI.h.

The problem is that shibokken redefines structs and functions already defined if this macro is set: Sat_Apr_20_07:01:03_PM_CEST_2024

So I see two possible fixes:

  • remove the definition in PythonAPI.h because even if it is there the project still manually checks for versioning later on
  • open an issue on shiboken2

yeah... I was also looking into possible fixes for this... let's discuss it further on Mattermost

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Python: 'Py_SetPythonHome' is deprecated since 3.11
4 participants