Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ezaf 4067 | ray tune/train reports depricated fix #139

Open
wants to merge 44 commits into
base: release/fy24-q1
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
9074194
Banking+App examples (#90)
skandtandon Nov 10, 2023
7767fb7
EZAF-3535 | Ray Serve | FastApi | RaySyncHandles (#87)
sip-aravind-g Nov 10, 2023
247f695
Remove mentions of Seldon from Bike Sharing example (#91)
umka1332 Nov 21, 2023
cc46e91
Added access-control (#93)
Bhargavjd Nov 22, 2023
7913725
EZAF-4067 | migrate fibonacci example
sip-aravind-g Jan 8, 2024
facb5a9
EZAF-4067 | migrate fibonacci example
sip-aravind-g Jan 8, 2024
ab9413c
EZAF-4067 | migrate fibonacci example
sip-aravind-g Jan 8, 2024
cf775b0
EZAF-4067 | migrate ray cpu example
sip-aravind-g Jan 8, 2024
09d9cc2
EZAF-4067 | remote ip and add local svc details
sip-aravind-g Jan 8, 2024
d346371
EZAF-4067 | migrate workaround example
sip-aravind-g Jan 8, 2024
9b85fca
update mlflow examples (#111)
ask664 Jan 10, 2024
9b560bc
Fix trailing \n in bike-sharing tutorial (#113)
akanshasajimon Jan 11, 2024
786207f
EZAF-4067 | Hyperparameter example
sip-aravind-g Jan 11, 2024
c965ceb
migrated Ray S3 example to latest standards
Manish-2023 Jan 11, 2024
8725c13
EZAF-4067 | Hyperparameter Fix
sip-aravind-g Jan 11, 2024
de380a9
Updated Spark Examples (#117)
skandtandon Jan 12, 2024
b66357f
add seaborn to mlflow tutorial (#119)
akanshasajimon Jan 12, 2024
c78fa7c
EZAF-4067 | entrypoint update
sip-aravind-g Jan 12, 2024
abf9f1d
updated Ray-whylogs example to the latest standard
Manish-2004 Jan 12, 2024
0028999
Update Kubeflow examples for fy24-q1 (#114)
AyushSinha5588 Jan 12, 2024
77b0b43
EZAF-4126 | upgrading ray serve examples
sip-aravind-g Jan 12, 2024
641d22c
EZAF-4126 | upgrading ray serve examples
sip-aravind-g Jan 12, 2024
6ba5ac1
Update kubeflow example (#121)
AyushSinha5588 Jan 16, 2024
3ad2ade
Updated the readme files (#124)
skandtandon Jan 16, 2024
10462f0
update image tag (#120)
ask664 Jan 17, 2024
36cf4a5
Fix notebook for training mnist via kale and kfp on gpu (#127)
akravacyber Jan 18, 2024
b9c1cb5
Enhance Spark jobs for Airflow (#128)
akravacyber Jan 18, 2024
0abb919
Fix conflicts staging/fy24 q1 into release/fy24 q1 (#130)
akravacyber Jan 18, 2024
88034cf
Fix/merge release branch (#131)
akravacyber Jan 18, 2024
5a1a066
fix conflict
Jan 18, 2024
7a28e3a
EZAF-4067 | Upgrading RAY Examples (#109)
sip-aravind-g Jan 18, 2024
686a8af
Merge branch 'release/fy24-q1' into staging/fy24-q1
ask664 Jan 18, 2024
8a3412b
EZAF-3639 | RAY GPU EXAMPLE
sip-aravind-g Jan 22, 2024
58142b1
EZAF-3639 | RAY GPU EXAMPLE
sip-aravind-g Jan 22, 2024
c2fbc41
updated entrypoint_cpu_num value
Manish-2004 Jan 22, 2024
4b93de8
EZAF-4067 | conflict resolved
sip-aravind-g Jan 22, 2024
2c7ac97
updated entrypoint_cpu_num value
Manish-2004 Jan 22, 2024
ed7e4a7
conflict resolved
Manish-2004 Jan 22, 2024
7a33b74
Updated the Spark whylogs image (#135)
skandtandon Jan 23, 2024
2573385
Ezaf 4067 | EZAF-3639 | RAY GPU EXAMPLE | RAY 2.7.0 (#134)
sip-aravind-g Jan 23, 2024
570227c
resolved conflicts
Manish-2004 Jan 23, 2024
2b8f102
EZAF-4067 | ray tune/train reports depricated fix
sip-aravind-g Jan 24, 2024
172afbf
Merge branch 'staging/fy24-q1' into EZAF-4067
sip-aravind-g Jan 24, 2024
dedcef6
EZAF-4067 | ray tune/train reports depricated fix
sip-aravind-g Jan 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion Data-Science/Ray-CPU/run_cpu.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,10 @@
"get_job_logs = client.get_job_logs(job_id)\n",
"get_job_info = client.get_job_info(job_id)\n",
"async for lines in client.tail_job_logs(job_id):\n",
" print(lines, end=\"\") "
" print(lines, end=\"\") \n",
"\n",
"# Disconnect from the Ray cluster\n",
"ray.shutdown()"
]
},
{
Expand Down
24 changes: 24 additions & 0 deletions Data-Science/Ray-GPU/core.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
import tensorflow as tf

def main():
for iter in range(1, 5):
# Confirm that TensorFlow is using the GPU.
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

# If GPU is available and accessible for TF, resulting list should not be empty
tf.config.list_physical_devices('GPU')

# Enabling logging device placement to find out which devices TF operations and tensors are assigned to
tf.debugging.set_log_device_placement(True)

# Create some tensors
a = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])
c = tf.matmul(a, b)

# The cell above should print an indication that the MatMul op was executed on GPU:0
print("RAY FRAMEWORK: GPU RESULT", c)

if __name__ == "__main__":
main()

164 changes: 0 additions & 164 deletions Data-Science/Ray-GPU/ray_gpu_sample.ipynb

This file was deleted.

26 changes: 26 additions & 0 deletions Data-Science/Ray-GPU/readme copy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# RAY Fibonacci Example

## Prerequisites
- Python 3.7 or higher
- The Current Release RAY framework installed (version 2.7.0)
- Additional dependencies listed in requirements.txt

## Validation Steps
1. **Setup Environment**: Set up the required environment variables. Please make sure you have updated right head node hostport Ip and port.
ray_head_ip = "10.224.226.40" or
ray_head_ip = "kuberay-head-svc.kuberay.svc.cluster.local"
ray_head_port = 8265

2. **Validation Script**: Run the validation script.
With defined custom NAS/network storage path
To complete the tutorial follow simple steps below:
1. Login to you EzAF cluster.
2. Create a new notebook server using the `jupyter-data-science` image or any desired image, choice the kernel type "RAY".
3. Clone the repository locally.
4. Launch the `run.ipynb` notebook file.
5. keep the `core.py` along with the above nortebook (same location).
5. This example will executes a log on top of RAY cluster.

## Issues and Contributions

If you encounter any issues during the validation process or would like to contribute to the validation scripts, please open an issue or submit a pull request to EzUA Dev Team.
7 changes: 7 additions & 0 deletions Data-Science/Ray-GPU/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
ray[client]==2.7.0
ray[default]==2.7.0
time
warnings
tensorrt
tensorflow
nvidia-tensorrt==99.0.0
Loading