From 7515061ec237b0393e2fcc064a309fae29502dff Mon Sep 17 00:00:00 2001 From: Mathew Jacob Date: Mon, 7 Aug 2023 10:57:02 +0800 Subject: [PATCH] [SPARK-44264][PYTHON][ML][TESTS][FOLLOWUP] Adding Deepspeed To The Test Dockerfile ### What changes were proposed in this pull request? Added tests to the Dockerfile for tests in OSS Spark CI. ### Why are the changes needed? They'll skip the deepspeed tests otherwise. ### Does this PR introduce _any_ user-facing change? Nope, testing infra. ### How was this patch tested? Running the tests on machine. Closes #42347 from mathewjacob1002/testing_infra. Authored-by: Mathew Jacob Signed-off-by: Ruifeng Zheng --- dev/infra/Dockerfile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile index 9d7b29e25b49b..b69e682f239c8 100644 --- a/dev/infra/Dockerfile +++ b/dev/infra/Dockerfile @@ -73,3 +73,5 @@ RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos grpcio-sta # Add torch as a testing dependency for TorchDistributor RUN python3.9 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu RUN python3.9 -m pip install torcheval +# Add Deepspeed as a testing dependency for DeepspeedTorchDistributor +RUN python3.9 -m pip install deepspeed