Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-48766][PYTHON] Document the behavior difference of `extraction…
…` between `element_at` and `try_element_at` ### What changes were proposed in this pull request? Document the behavior difference of `extraction` between `element_at` and `try_element_at` ### Why are the changes needed? when the function `try_element_at` was introduced in 3.5, its `extraction` handling was unintentionally not consistent with the `element_at`, which causes confusion. This PR document this behavior difference (I don't think we can fix it since it will be a breaking change). ``` In [1]: from pyspark.sql import functions as sf In [2]: df = spark.createDataFrame([({"a": 1.0, "b": 2.0}, "a")], ['data', 'b']) In [3]: df.select(sf.try_element_at(df.data, 'b')).show() +-----------------------+ |try_element_at(data, b)| +-----------------------+ | 1.0| +-----------------------+ In [4]: df.select(sf.element_at(df.data, 'b')).show() +-------------------+ |element_at(data, b)| +-------------------+ | 2.0| +-------------------+ ``` ### Does this PR introduce _any_ user-facing change? doc changes ### How was this patch tested? ci, added doctests ### Was this patch authored or co-authored using generative AI tooling? no Closes apache#47161 from zhengruifeng/doc_element_at_extraction. Authored-by: Ruifeng Zheng <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>
- Loading branch information