Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Fix required java versions for spark 3.5 #48411

Open
wants to merge 2 commits into
base: branch-3.5
Choose a base branch
from

Conversation

dvorst
Copy link

@dvorst dvorst commented Oct 10, 2024

The original description "PySpark requires Java 8 or later" is incorrect since 3.5 does not support java 8 anymore and the latest supported version is 17, the downloading page however, does correctly state this. I thus corrected the mentioned java versions.

Edit: the downloading page i am referring to: https://spark.apache.org/docs/3.5.3/#downloading

The original description "PySpark requires Java 8 or later" is incorrect since the latest supported version is 17, the downloading page actually correctly states this. Furthermore, as the downloading page states, support for Java 8 is dropped from spark 3.5.0 and later. So I changed the required java versions to either 11 or 17 as indicated on the downloads page
Copy link
Contributor

@LuciferYang LuciferYang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean that Apache Spark 3.5 no longer supports Java 8 in PySpark? Are there any specific cases? If that's really the case, we should fix it to support Java 8 instead of revising the documentation.

@dvorst
Copy link
Author

dvorst commented Oct 11, 2024

The following is mentioned on the downloadings page (https://spark.apache.org/docs/3.5.3/#downloading)

Java 8 prior to version 8u371 support is deprecated as of Spark 3.5.0

For some reason, I misread this and was under the impression that support for Java 8 was deprecated in its entirety as of 3.5.0.

I added a new commit where I updated the documentation and included Java 8 back in, with an exception for versions prior to 8u371.

@LuciferYang
Copy link
Contributor

LuciferYang commented Oct 12, 2024

Current changes are fine to me, but please do not disrupt the template for the [r description, every item must be filled out, and the title of the pr should also adhere to the standard format(can refer to https://spark.apache.org/contributing.html). And are there any other cases where the description of Java 8 versions is not accurate?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants