Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Dummy PR to check maint-18.0.0 status #44367

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from
Draft

WIP: Dummy PR to check maint-18.0.0 status #44367

wants to merge 17 commits into from

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Oct 10, 2024

DO NOT MERGE.

This PR is to track some crossbow jobs to validate status of maintenance branch before creating the first RC.

…44359)

### Rationale for this change

This is not happen on Debian GNU/Linux stable and unstable. This is happen only on Debian GNU/Linux testing. So this may be a temporary problem.

### What changes are included in this PR?

Create a non-existent path manually as a workaround.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* GitHub Issue: #44358

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change

Other file systems such as GCS file system and S3 file system are enabled.

### What changes are included in this PR?

Enabled Azure file system.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.
* GitHub Issue: #44347

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
…n wheel-manylinux-*-cp313t-* (#44362)

### Rationale for this change

If interactive deb configuration is used, CI is blocked by waiting an user input.

### What changes are included in this PR?

Disable interactive deb configuration.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: #44355

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>

This comment was marked as off-topic.

@github-actions github-actions bot added the awaiting committer review Awaiting committer review label Oct 10, 2024
@raulcd

This comment was marked as outdated.

@raulcd

This comment was marked as outdated.

raulcd and others added 2 commits October 10, 2024 18:52
…ckaging (#44148)

## WIP

Creating the PR to validate if / what are the CI failures 

### Rationale for this change
Once the issue has been merged:
- #25118

We do not require numpy as a pyarrow dependency.

### What changes are included in this PR?

Remove numpy as a required dependency

### Are these changes tested?

Via CI

### Are there any user-facing changes?

Yes, pyarrow won't install numpy as a required dependency.
* GitHub Issue: #43846

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
…test wheels on Python 3.13 (#44193)

### Rationale for this change

Now that numpy, cython and pandas all have releases that support Python 3.13, we can remove the extra index and ensure that we are building our wheels with the released version instead of nightly dev version.

* GitHub Issue: #43519

Authored-by: Joris Van den Bossche <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
raulcd and others added 4 commits October 11, 2024 11:10
…test_dataset.py (#44370)

### Rationale for this change

If pyarrow has been built without dataset test collection errors due to pytest.mark.parametrize requiring dataset.

### What changes are included in this PR?

Move dataset requirement outside of test collection by removing it from the parametrization.

### Are these changes tested?

Via archery

### Are there any user-facing changes?

No
* GitHub Issue: #44369

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
…ve macOS 12 pyarrow wheels (#44315)

### Rationale for this change

We did just bump MACOSX_DEPLOYMENT_TARGET from 10 to 12 for pyarrow wheels on 18.0.0, we probably should wait a little before dropping.

### What changes are included in this PR?

Revert moving macOS deployment target to 12

### Are these changes tested?

Will trigger wheel jobs on archery

### Are there any user-facing changes?

Yes, wheels will be available for macOS 12.
* GitHub Issue: #44314

Authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
…4374)

### Rationale for this change

Java-jars are currently failing on macOS to install Python 3.13 because `python@ 3.13` (`python` is an alias of `python@ 3.13` now) isn't installed yet. Our link failure workaround is updating already installed `python@*` with `--overwrite` explicitly. `python@ 3.13` (`python`) isn't installed on GitHub Actions runner yet. So `brew bundle` installs `python` without `--overwrite`.

### What changes are included in this PR?

Install `python` with `--overwrite` explicitly before `brew bundle`.

### Are these changes tested?

Via CI

### Are there any user-facing changes?

No
* GitHub Issue: #44373

Lead-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
### Rationale for this change

When adding Python 3.13 support (#43539), conda was not yet ready to support Python 3.13. But over the last weeks most conda packages having being updated, so trying again.

* GitHub Issue: #43519

Authored-by: Joris Van den Bossche <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
@raulcd

This comment was marked as outdated.

raulcd and others added 2 commits October 12, 2024 12:18
…cation jobs (#44382)

### Rationale for this change

Verification job is currently failing for Ruby.

### What changes are included in this PR?

We can't use Python 3.13 and `glib==2.80.5`. So we use old Python as a workaround.

### Are these changes tested?

Via archery

### Are there any user-facing changes?

No
* GitHub Issue: #44381

Lead-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Raúl Cumplido <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
…fication when using Conda (#44388)

### Rationale for this change

Conda Python 3.13 fails when running verification tasks for Java. It's related to JPype.

See also:
* GH-44389
* https://discuss.python.org/t/api-for-python-3-13-prevents-use-of-3rd-party-gc-allocators/62709/5

### What changes are included in this PR?

Pin Python 3.12 on Conda integration jobs because JPype doesn't work with Python 3.13. We can use Python 3.13 again once JPype supports Python 3.13.

### Are these changes tested?

Yes via CI

### Are there any user-facing changes?

No
* GitHub Issue: #44386

Lead-authored-by: Raúl Cumplido <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@raulcd
Copy link
Member Author

raulcd commented Oct 13, 2024

Revision: 0e184c3

Submitted crossbow builds: ursacomputing/crossbow @ tests-maint-18.0.0-1

Task Status
example-cpp-minimal-build-static GitHub Actions
example-cpp-minimal-build-static-system-dependency GitHub Actions
example-cpp-tutorial GitHub Actions
example-python-minimal-build-fedora-conda GitHub Actions
example-python-minimal-build-ubuntu-venv GitHub Actions
test-alpine-linux-cpp GitHub Actions
test-build-cpp-fuzz GitHub Actions
test-build-vcpkg-win GitHub Actions
test-conda-cpp GitHub Actions
test-conda-cpp-valgrind GitHub Actions
test-conda-python-3.10 GitHub Actions
test-conda-python-3.10-cython2 GitHub Actions
test-conda-python-3.10-hdfs-2.9.2 GitHub Actions
test-conda-python-3.10-hdfs-3.2.1 GitHub Actions
test-conda-python-3.10-pandas-latest-numpy-latest GitHub Actions
test-conda-python-3.10-substrait GitHub Actions
test-conda-python-3.11 GitHub Actions
test-conda-python-3.11-dask-latest GitHub Actions
test-conda-python-3.11-dask-upstream_devel GitHub Actions
test-conda-python-3.11-hypothesis GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-1.26 GitHub Actions
test-conda-python-3.11-pandas-latest-numpy-latest GitHub Actions
test-conda-python-3.11-pandas-nightly-numpy-nightly GitHub Actions
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly GitHub Actions
test-conda-python-3.11-spark-master GitHub Actions
test-conda-python-3.12 GitHub Actions
test-conda-python-3.12-cpython-debug GitHub Actions
test-conda-python-3.13 GitHub Actions
test-conda-python-3.9 GitHub Actions
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5 GitHub Actions
test-conda-python-emscripten GitHub Actions
test-cuda-cpp-ubuntu-20.04-cuda-11.2.2 GitHub Actions
test-cuda-cpp-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-cuda-python-ubuntu-22.04-cuda-11.7.1 GitHub Actions
test-debian-12-cpp-amd64 GitHub Actions
test-debian-12-cpp-i386 GitHub Actions
test-debian-12-docs GitHub Actions
test-debian-12-python-3-amd64 GitHub Actions
test-debian-12-python-3-i386 GitHub Actions
test-debian-c-glib GitHub Actions
test-debian-ruby GitHub Actions
test-fedora-39-cpp GitHub Actions
test-fedora-39-python-3 GitHub Actions
test-r-arrow-backwards-compatibility GitHub Actions
test-r-clang-sanitizer GitHub Actions
test-r-depsource-bundled Azure
test-r-depsource-system GitHub Actions
test-r-dev-duckdb GitHub Actions
test-r-devdocs GitHub Actions
test-r-extra-packages GitHub Actions
test-r-gcc-11 GitHub Actions
test-r-gcc-12 GitHub Actions
test-r-install-local GitHub Actions
test-r-install-local-minsizerel GitHub Actions
test-r-linux-as-cran GitHub Actions
test-r-linux-rchk GitHub Actions
test-r-linux-valgrind GitHub Actions
test-r-macos-as-cran GitHub Actions
test-r-minimal-build Azure
test-r-offline-maximal GitHub Actions
test-r-offline-minimal Azure
test-r-rhub-debian-gcc-devel-lto-latest Azure
test-r-rhub-debian-gcc-release-custom-ccache Azure
test-r-rhub-ubuntu-release-latest Azure
test-r-rocker-r-ver-latest Azure
test-r-rstudio-r-base-4.1-opensuse155 Azure
test-r-rstudio-r-base-4.2-focal Azure
test-r-ubuntu-22.04 GitHub Actions
test-r-versions GitHub Actions
test-skyhook-integration GitHub Actions
test-ubuntu-20.04-cpp GitHub Actions
test-ubuntu-20.04-cpp-bundled GitHub Actions
test-ubuntu-22.04-cpp GitHub Actions
test-ubuntu-22.04-cpp-20 GitHub Actions
test-ubuntu-22.04-cpp-emscripten GitHub Actions
test-ubuntu-22.04-cpp-no-threading GitHub Actions
test-ubuntu-22.04-python-3 GitHub Actions
test-ubuntu-22.04-python-313-freethreading GitHub Actions
test-ubuntu-24.04-cpp GitHub Actions
test-ubuntu-24.04-cpp-gcc-13-bundled GitHub Actions
test-ubuntu-24.04-cpp-gcc-14 GitHub Actions
test-ubuntu-24.04-cpp-minimal-with-formats GitHub Actions
test-ubuntu-24.04-cpp-thread-sanitizer GitHub Actions
test-ubuntu-24.04-python-3 GitHub Actions
test-ubuntu-c-glib GitHub Actions
test-ubuntu-r-sanitizer GitHub Actions
test-ubuntu-ruby GitHub Actions

@raulcd

This comment was marked as outdated.

@raulcd

This comment was marked as outdated.

### Rationale for this change

jemalloc may have a problem on ARM.
See also: #44342

### What changes are included in this PR?

* Disable jemalloc by default on ARM.
* Disable jemalloc for manylinux wheel for ARM.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.

* GitHub Issue: #44342

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
@raulcd

This comment was marked as outdated.

…rification script (#44407)

### Rationale for this change

We can remove these pins now that conda-forge/glib-feedstock#191 has been closed. I've tested on my own crossbow and will test here momentarily. This effectively reverts #44270 and #44268.

### What changes are included in this PR?

Tweaks to release release verification script, specifically removing recently-added pins.

### Are these changes tested?

Yes: https://github.com/amoeba/crossbow/actions/runs/11335070956/job/31522511285.

### Are there any user-facing changes?

No.

Closes #44415.
* GitHub Issue: #44415

Authored-by: Bryce Mecum <[email protected]>
Signed-off-by: Bryce Mecum <[email protected]>
@raulcd
Copy link
Member Author

raulcd commented Oct 15, 2024

Revision: 70a2245

Submitted crossbow builds: ursacomputing/crossbow @ packaging-maint-18.0.0-2

Task Status
almalinux-8-amd64 GitHub Actions
almalinux-8-arm64 GitHub Actions
almalinux-9-amd64 GitHub Actions
almalinux-9-arm64 GitHub Actions
amazon-linux-2023-amd64 GitHub Actions
amazon-linux-2023-arm64 GitHub Actions
centos-7-amd64 GitHub Actions
centos-8-stream-amd64 GitHub Actions
centos-8-stream-arm64 GitHub Actions
centos-9-stream-amd64 GitHub Actions
centos-9-stream-arm64 GitHub Actions
conan-maximum GitHub Actions
conan-minimum GitHub Actions
conda-clean Azure
conda-osx-x64-cpu-py3 Azure
debian-bookworm-amd64 GitHub Actions
debian-bookworm-arm64 GitHub Actions
debian-trixie-amd64 GitHub Actions
debian-trixie-arm64 GitHub Actions
homebrew-cpp GitHub Actions
java-jars GitHub Actions
nuget GitHub Actions
python-sdist GitHub Actions
r-binary-packages GitHub Actions
ubuntu-focal-amd64 GitHub Actions
ubuntu-focal-arm64 GitHub Actions
ubuntu-jammy-amd64 GitHub Actions
ubuntu-jammy-arm64 GitHub Actions
ubuntu-noble-amd64 GitHub Actions
ubuntu-noble-arm64 GitHub Actions
wheel-macos-monterey-cp310-cp310-amd64 GitHub Actions
wheel-macos-monterey-cp310-cp310-arm64 GitHub Actions
wheel-macos-monterey-cp311-cp311-amd64 GitHub Actions
wheel-macos-monterey-cp311-cp311-arm64 GitHub Actions
wheel-macos-monterey-cp312-cp312-amd64 GitHub Actions
wheel-macos-monterey-cp312-cp312-arm64 GitHub Actions
wheel-macos-monterey-cp313-cp313-amd64 GitHub Actions
wheel-macos-monterey-cp313-cp313-arm64 GitHub Actions
wheel-macos-monterey-cp313-cp313t-amd64 GitHub Actions
wheel-macos-monterey-cp313-cp313t-arm64 GitHub Actions
wheel-macos-monterey-cp39-cp39-amd64 GitHub Actions
wheel-macos-monterey-cp39-cp39-arm64 GitHub Actions
wheel-manylinux-2-28-cp310-cp310-amd64 GitHub Actions
wheel-manylinux-2-28-cp310-cp310-arm64 GitHub Actions
wheel-manylinux-2-28-cp311-cp311-amd64 GitHub Actions
wheel-manylinux-2-28-cp311-cp311-arm64 GitHub Actions
wheel-manylinux-2-28-cp312-cp312-amd64 GitHub Actions
wheel-manylinux-2-28-cp312-cp312-arm64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313-amd64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313-arm64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-amd64 GitHub Actions
wheel-manylinux-2-28-cp313-cp313t-arm64 GitHub Actions
wheel-manylinux-2-28-cp39-cp39-amd64 GitHub Actions
wheel-manylinux-2-28-cp39-cp39-arm64 GitHub Actions
wheel-manylinux-2014-cp310-cp310-amd64 GitHub Actions
wheel-manylinux-2014-cp310-cp310-arm64 GitHub Actions
wheel-manylinux-2014-cp311-cp311-amd64 GitHub Actions
wheel-manylinux-2014-cp311-cp311-arm64 GitHub Actions
wheel-manylinux-2014-cp312-cp312-amd64 GitHub Actions
wheel-manylinux-2014-cp312-cp312-arm64 GitHub Actions
wheel-manylinux-2014-cp313-cp313-amd64 GitHub Actions
wheel-manylinux-2014-cp313-cp313-arm64 GitHub Actions
wheel-manylinux-2014-cp313-cp313t-amd64 GitHub Actions
wheel-manylinux-2014-cp313-cp313t-arm64 GitHub Actions
wheel-manylinux-2014-cp39-cp39-amd64 GitHub Actions
wheel-manylinux-2014-cp39-cp39-arm64 GitHub Actions
wheel-windows-cp310-amd64 GitHub Actions
wheel-windows-cp311-amd64 GitHub Actions
wheel-windows-cp312-amd64 GitHub Actions
wheel-windows-cp313-amd64 GitHub Actions
wheel-windows-cp39-amd64 GitHub Actions

@raulcd
Copy link
Member Author

raulcd commented Oct 15, 2024

Revision: 70a2245

Submitted crossbow builds: ursacomputing/crossbow @ release-maint-18.0.0-2

Task Status
verify-rc-source-cpp-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-cpp-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-cpp-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-cpp-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-cpp-macos-amd64 GitHub Actions
verify-rc-source-cpp-macos-arm64 GitHub Actions
verify-rc-source-cpp-macos-conda-amd64 GitHub Actions
verify-rc-source-csharp-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-csharp-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-csharp-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-csharp-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-csharp-macos-amd64 GitHub Actions
verify-rc-source-csharp-macos-arm64 GitHub Actions
verify-rc-source-integration-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-integration-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-integration-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-integration-macos-amd64 GitHub Actions
verify-rc-source-integration-macos-arm64 GitHub Actions
verify-rc-source-integration-macos-conda-amd64 GitHub Actions
verify-rc-source-java-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-java-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-java-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-java-macos-amd64 GitHub Actions
verify-rc-source-js-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-js-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-js-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-js-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-js-macos-amd64 GitHub Actions
verify-rc-source-js-macos-arm64 GitHub Actions
verify-rc-source-python-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-python-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-python-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-python-macos-amd64 GitHub Actions
verify-rc-source-python-macos-arm64 GitHub Actions
verify-rc-source-python-macos-conda-amd64 GitHub Actions
verify-rc-source-ruby-linux-almalinux-8-amd64 GitHub Actions
verify-rc-source-ruby-linux-conda-latest-amd64 GitHub Actions
verify-rc-source-ruby-linux-ubuntu-20.04-amd64 GitHub Actions
verify-rc-source-ruby-linux-ubuntu-22.04-amd64 GitHub Actions
verify-rc-source-ruby-macos-amd64 GitHub Actions
verify-rc-source-ruby-macos-arm64 GitHub Actions
verify-rc-source-windows GitHub Actions

…44425)

### Rationale for this change

It seems that the current `quay.io/centos/centos:stream9` for `linux/aarch64` is broken.

We use it for building and testing but testing is only broken for now. Because we use cached (a bit old) `quay.io/centos/centos:stream9` for now. Building will be failed when the cache is expired.

### What changes are included in this PR?

Change upload phase to after the build from after the test. Test is still failing but we can proceed our release process by this.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

No.
* GitHub Issue: #44422

Authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Raúl Cumplido <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants