[SPARK-44264][PYTHON][ML][TESTS][FOLLOWUP] Adding Deepspeed To The Te…

…st Dockerfile ### What changes were proposed in this pull request? Added tests to the Dockerfile for tests in OSS Spark CI. ### Why are the changes needed? They'll skip the deepspeed tests otherwise. ### Does this PR introduce _any_ user-facing change? Nope, testing infra. ### How was this patch tested? Running the tests on machine. Closes #42347 from mathewjacob1002/testing_infra. Authored-by: Mathew Jacob <[email protected]> Signed-off-by: Ruifeng Zheng <[email protected]>
apache · Aug 7, 2023 · 7515061 · 7515061
1 parent a98b112
commit 7515061
Showing 1 changed file with 2 additions and 0 deletions.
diff --git a/dev/infra/Dockerfile b/dev/infra/Dockerfile
@@ -73,3 +73,5 @@ RUN python3.9 -m pip install grpcio protobuf googleapis-common-protos grpcio-sta
 # Add torch as a testing dependency for TorchDistributor
 RUN python3.9 -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
 RUN python3.9 -m pip install torcheval
+# Add Deepspeed as a testing dependency for DeepspeedTorchDistributor
+RUN python3.9 -m pip install deepspeed