You have to make sure that there is mounted Azure File Storage on the machine you are runnning this code.
-
df -h
to check mounted AFS -
AFS has to be mounted to the /mnt/shipfs folder and have enough space
-
Make sure yout
config.py
is correct:K8S_PROXY
- check if you do not have a connection to kubernetes (kubectl get jobs
)HOST_OUTPUT_DIRECTORY
- set the output folder for your experimentsDOCKER_OUTPUT_DIRECTORY
,DOCKER_SAMPLE_DIRECTORY
- set these parameters according to your docker imageTIMEOUT
- set your timeout for every JobJOB_SPEC
- check this -
Launch -
python run_kubernetes.py
-
Every Kubernetes Job create 1 Kubernetes Pod. Monitor your Jobs with
kubectl
utility:
<pod_name> = <job_name>-[:5]
kubectl --server=127.0.0.1:8002 get jobs
kubectl --server=127.0.0.1:8002 get pods
kubectl --server=127.0.0.1:8002 describe pod <pod_name>
kubectl --server=127.0.0.1:8002 logs <pod_name>
scripts to run fs