[python] Retain GIL for exist

[nguyenv]

This continues to partially revert commit 72a4e20 in conjuction with #2800. We are now only retaining releasing the GIL for SOMADataFrame::count.

Due to multiple reports of intermittent segfaults internally and externally, SOMAArray::exists should also be reverted until we have a better way to identify and test releasing the GIL in the pybind11 code. I tested this revert fixed my itermittent segfaults locally by running make test 6 times. Previously, I could get a segfault to occur with 3 make test runs.

[bkmartinjr]

What is the rationale for reverting the GIL release on DataFrame.count? We support the same operation with a GIL release on NDArray, without issue. And this is a win we can land, while we sort out the issues with open/exists.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.02%. Comparing base (055917d) to head (9d53f7e).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2801      +/-   ##
==========================================
+ Coverage   89.84%   90.02%   +0.17%     
==========================================
  Files          37       37              
  Lines        3930     3930              
==========================================
+ Hits         3531     3538       +7     
+ Misses        399      392       -7     

Flag	Coverage Δ
python	90.02% <ø> (+0.17%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
python_api	90.02% <ø> (+0.17%)	⬆️
libtiledbsoma	∅ <ø> (∅)

[ryan-williams]

Edit: failures mentioned previously are reproducible on 3.12, but we don't actually support 3.12 yet, so disregard!

---

I'm seeing test failures in (ubuntu,3.12) in my own GHA tests of this branch.

I'm actually not sure which of the commits from this branch it checked out, just started another run that runs a git log -1 for verification.

Several (edit: All 153) of the failures look like this, in case it rings any bells:

________________ ERROR at setup of test_empty_categorical_query ________________

conftest_pbmc_small_h5ad_path = PosixPath('/home/runner/work/tiledb-scratch/tiledb-scratch/apis/python/testdata/pbmc-small.h5ad')

    @pytest.fixture
    def conftest_pbmc_small_exp(conftest_pbmc_small_h5ad_path) -> Experiment:
        """Ingest an ``AnnData``, yield a ``TestCase`` with the original and new AnnData objects."""
        with TemporaryDirectory() as exp_path:
>           tiledbsoma.io.from_h5ad(
                exp_path, conftest_pbmc_small_h5ad_path, measurement_name="RNA"
            )

tests/conftest.py:31: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/tiledbsoma/io/ingest.py:361: in from_h5ad
    uri = from_anndata(
src/tiledbsoma/io/ingest.py:495: in from_anndata
    conversions.decategoricalize_obs_or_var(anndata.obs),
src/tiledbsoma/io/conversions.py:29: in decategoricalize_obs_or_var
    str(k): to_tiledb_supported_array_type(str(k), v)
src/tiledbsoma/io/conversions.py:53: in to_tiledb_supported_array_type
    target_dtype = _to_tiledb_supported_dtype(x.dtype)  # type: ignore[arg-type]
src/tiledbsoma/io/conversions.py:38: in _to_tiledb_supported_dtype
    def _to_tiledb_supported_dtype(dtype: _DT) -> _DT:
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_functions.py:136: in check_argument_types
    check_type_internal(value, annotation, memo)
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_checkers.py:779: in check_type_internal
    checker(value, origin_type, args, memo)
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_checkers.py:527: in check_typevar
    check_type_internal(value, annotation, memo)
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_checkers.py:779: in check_type_internal
    checker(value, origin_type, args, memo)
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_checkers.py:418: in check_union
    check_type_internal(value, type_, memo)
/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_checkers.py:736: in check_type_internal
    annotation = evaluate_forwardref(annotation, memo)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

forwardref = ForwardRef('ExtensionDtype')
memo = <typeguard.TypeCheckMemo object at 0x7fd974fda940>

    def evaluate_forwardref(forwardref: ForwardRef, memo: TypeCheckMemo) -> Any:
>       return forwardref._evaluate(memo.globals, memo.locals, frozenset())
E       TypeError: ForwardRef._evaluate() missing 1 required keyword-only argument: 'recursive_guard'

/opt/hostedtoolcache/Python/3.12.4/x64/lib/python3.12/site-packages/typeguard/_utils.py:18: TypeError

---

So far I'm not seeing segfaults with this PR, from some extra stress-testing (example)

[johnkerl]

@bkmartinjr are you concerns allayed, or do you think there's more to do here?

[bkmartinjr]

Reverting open/exists seems smart until we further diagnose the crashes. Long term, we need to release the GIL on as many operations as is possible, so I advocate for leaving open a tracking issue.

[johnkerl]

Reverting open/exists seems smart until we further diagnose the crashes. Long term, we need to release the GIL on as many operations as is possible, so I advocate for leaving open a tracking issue.

@bkmartinjr we are on the same page 😎

#2749 (comment)