From 3dd9105da9da77bbf9c748d2ab313f3ee4fc4b4f Mon Sep 17 00:00:00 2001 From: Maciej Date: Sat, 11 Jan 2020 13:40:54 +0100 Subject: [PATCH] Move coverage info to doc (#286) --- README.rst | 206 ++----------------------------------------- doc/api-coverage.rst | 196 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 203 insertions(+), 199 deletions(-) create mode 100644 doc/api-coverage.rst diff --git a/README.rst b/README.rst index a82f9c7d..4f0e7820 100644 --- a/README.rst +++ b/README.rst @@ -80,215 +80,23 @@ or `Pytype `__. This package is tested against MyPy development branch and in rare cases (primarily important upstrean bugfixes), is not compatible with the preceding MyPy release. PySpark Version Compatibility ---------------------- +----------------------------- Package versions follow PySpark versions with exception to maintenance releases - i.e. `pyspark-stubs==2.3.0` should be compatible with `pyspark>=2.3.0,<2.4.0`. Maintenance releases (`post1`, `post2`, ..., `postN`) are reserved for internal annotations updates. +API Coverage: +------------- + +As of release 2.4.0 most of the public API is covered. For details please check `API coverage document `__. + See also ------------- +-------- - `SPARK-17333 `__ - *Make pyspark interface friendly with static analysis*. - `PySpark typing hints `__ and `Revisiting PySpark type annotations `__ on `Apache Spark Developers List `__. -API Coverage ------------- - -+------------------------------------------------+---------------------+--------------------+------------+ -| Module | Dynamically typed | Statically typed | Notes | -+================================================+=====================+====================+============+ -| pyspark | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.accumulators | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.broadcast | ✔ | ✔ | Mixed | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.cloudpickle | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.conf | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.context | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.daemon | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.files | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.find\_spark\_home | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.heapq3 | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.java\_gateway | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.join | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.base | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.classification | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.clustering | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.common | ✔ | ✔ | Mixed | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.evaluation | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.feature | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.fpm | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.image | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.linalg | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.param | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.param.\_shared\_params\_code\_gen | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.param.shared | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.pipeline | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.recommendation | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.regression | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.stat | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.tests | ✘ | ✘ | Tests | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.tree | x | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.tuning | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.util | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.ml.wrapper | ✔ | ✔ | Mixed | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.classification | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.clustering | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.common | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.evaluation | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.feature | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.fpm | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.linalg | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.linalg.distributed | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.random | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.recommendation | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.regression | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.stat | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.stat.KernelDensity | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.stat.\_statistics | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.stat.distribution | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.stat.test | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.tests | ✘ | ✘ | Tests | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.tree | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.mllib.util | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.profiler | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.resourceinformation | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.rdd | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.rddsampler | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.resultiterable | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.serializers | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.shell | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.shuffle | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.catalog | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.cogroup | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.column | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.conf | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.context | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.dataframe | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.functions | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.group | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.readwriter | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.session | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.streaming | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.tests | ✘ | ✘ | Tests | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.types | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.udf | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.utils | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.sql.window | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.statcounter | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.status | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.storagelevel | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.context | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.dstream | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.kinesis | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.listener | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.tests | ✘ | ✘ | Tests | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.streaming.util | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.taskcontext | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.tests | ✘ | ✘ | Tests | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.traceback\_utils | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.util | ✔ | ✘ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.version | ✘ | ✔ | | -+------------------------------------------------+---------------------+--------------------+------------+ -| pyspark.worker | ✘ | ✘ | Internal | -+------------------------------------------------+---------------------+--------------------+------------+ - Disclaimer ---------- diff --git a/doc/api-coverage.rst b/doc/api-coverage.rst new file mode 100644 index 00000000..5e0504bc --- /dev/null +++ b/doc/api-coverage.rst @@ -0,0 +1,196 @@ +API Coverage +============= + ++------------------------------------------------+---------------------+--------------------+------------+ +| Module | Dynamically typed | Statically typed | Notes | ++================================================+=====================+====================+============+ +| pyspark | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.accumulators | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.broadcast | ✔ | ✔ | Mixed | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.cloudpickle | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.conf | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.context | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.daemon | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.files | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.find\_spark\_home | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.heapq3 | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.java\_gateway | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.join | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.base | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.classification | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.clustering | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.common | ✔ | ✔ | Mixed | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.evaluation | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.feature | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.fpm | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.image | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.linalg | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.param | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.param.\_shared\_params\_code\_gen | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.param.shared | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.pipeline | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.recommendation | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.regression | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.stat | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.tests | ✘ | ✘ | Tests | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.tree | x | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.tuning | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.util | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.ml.wrapper | ✔ | ✔ | Mixed | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.classification | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.clustering | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.common | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.evaluation | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.feature | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.fpm | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.linalg | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.linalg.distributed | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.random | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.recommendation | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.regression | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.stat | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.stat.KernelDensity | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.stat.\_statistics | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.stat.distribution | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.stat.test | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.tests | ✘ | ✘ | Tests | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.tree | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.mllib.util | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.profiler | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.resourceinformation | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.rdd | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.rddsampler | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.resultiterable | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.serializers | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.shell | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.shuffle | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.catalog | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.cogroup | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.column | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.conf | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.context | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.dataframe | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.functions | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.group | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.readwriter | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.session | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.streaming | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.tests | ✘ | ✘ | Tests | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.types | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.udf | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.utils | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.sql.window | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.statcounter | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.status | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.storagelevel | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.context | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.dstream | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.kinesis | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.listener | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.tests | ✘ | ✘ | Tests | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.streaming.util | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.taskcontext | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.tests | ✘ | ✘ | Tests | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.traceback\_utils | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.util | ✔ | ✘ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.version | ✘ | ✔ | | ++------------------------------------------------+---------------------+--------------------+------------+ +| pyspark.worker | ✘ | ✘ | Internal | ++------------------------------------------------+---------------------+--------------------+------------+