[WIP] CP-47926: Preferred locale encoding should ignore case

[liulinC]

The locale module in python has different output for getpreferredencoding between python3.11 and python3.6

• 3.11: 'utf-8'
• 3.6: 'UTF-8'

The case of the return value should be ignored to match both

[rosslagerwall]

Hmm, I'm not seeing that:

On XenServer 8:

[root@lcy2-dt77 ~]# python3 --version
Python 3.6.8
[root@lcy2-dt77 ~]# python3 -c 'import locale; print(locale.getpreferredencoding())'
UTF-8
[root@lcy2-dt77 ~]# echo "$LANG"
en_US.UTF-8

On Fedora 38:

$ python3 --version
Python 3.11.6
$ python3 -c 'import locale; print(locale.getpreferredencoding())'
UTF-8
$ echo "$LANG"
en_US.UTF-8

Is there something different in your environment? Or does the Fedora build of Python 3.11 have some patch that we don't?

[liulinC]

Probably some environment diff.

    def getpreferredencoding(do_setlocale=True):
        """Return the charset that the user is likely using,
        according to the system configuration."""
        import pdb
        pdb.set_trace()
    
        if sys.flags.warn_default_encoding:
            import warnings
            warnings.warn(
                "UTF-8 Mode affects locale.getpreferredencoding(). Consider locale.getencoding() instead.",
                EncodingWarning, 2)
        if sys.flags.utf8_mode:  <<< match in xs9 but not in fedora38
            return 'utf-8'

        if not do_setlocale:
            return getencoding()

xs9
(Pdb) import sys
(Pdb) sys.flags
sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0, dev_mode=False, utf8_mode=1, warn_default_encoding=0, safe_path=False, int_max_str_digits=-1)

fedora38
>>> import sys
>>> sys.flags
sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0, dev_mode=False, utf8_mode=0, warn_default_encoding=0, safe_path=False, int_max_str_digits=-1)

[rosslagerwall]

I would expect that flag to be set either if Python is executed like python3 -X utf8 or if PYTHONUTF8=1 is set in the environment.

[rosslagerwall]

This is the other way round,
XS9 do have that flag set while fedora not.

Indeed. So what causes it to be set?

[liulinC]

Close this RP, fixing by install related locale package.