Merge branch 'main' into alvarovm-patch-1

docs/ai-testbed/cerebras/job-queuing-and-submission.md

@@ -13,6 +13,7 @@ NAME                          AGE  DURATION  PHASE    SYSTEMS     USER     LABEL
 wsjob-thjj8zticwsylhppkbmjqe  13s  1s        RUNNING  cer-cs2-01  username name=unet_pt  https://grafana.cerebras1.lab.alcf.anl.gov/d/WebHNShVz/wsjob-dashboard?orgId=1&var-wsjob=wsjob-thjj8zticwsylhppkbmjqe&from=1691705374000&to=now
 (venv_pt) $
 ```
+To view the grafana databoard for a job, follow the instructions at [Grafana WsJob Dashboard for Cerebras jobs](./miscellaneous.md#grafana-wsjob-dashboard-for-cerebras-jobs)
 
 Jobs can be canceled as shown:
 

docs/ai-testbed/cerebras/miscellaneous.md

@@ -5,6 +5,48 @@
 Cerebras documentation for porting code to run on a Cerebras CS-2 system:<br>
 [Ways to port your model](https://docs.cerebras.net/en/latest/wsc/port/index.html)
 
+## Grafana WsJob Dashboard for Cerebras jobs
+A Grafana dashboard provides support for visualizing, querying, and exploring the CS2 system’s metrics and enables to access system logs and traces.
+See the Cerebras documentation for the [Job Information Dashboard](https://docs.cerebras.net/en/latest/wsc/getting-started/grafana.html#wsjob-dashboard)
+
+Here is a summary (tested to work on Ubuntu and MacOS)<br>
+
+On your work machine with a web browser, e.g. your laptop,<br>
+edit /etc/hosts, using your editor of choice
+```console
+sudo nano /etc/hosts
+```
+Add this line
+```console
+127.0.0.1	grafana.cerebras1.lab.alcf.anl.gov
+```
+Save, and exit the editor
+
+Download the Grafana certificate present on the Cerebras node at /opt/cerebras/certs/grafana_tls.crt to your local machine. To add this certificate to your browser keychain, 
+
+1. On chrome, go to Settings->Privacy and security->Security->Manage device certificates
+2. Select System under "System Keychains" on the left hand side of your screen. Also select the "Certificate" tab.
+3. Drag and drop the downloaded certificate. Once it is added, it is visible as "lab.alcf.anl.gov"
+   ![Cerebras Wafer-Scale Cluster connection diagram](files/grafana_ctl.png)
+4. Select the certificate, and ensure that the "Trust" section is set to "Always Trust"
+   ![Cerebras Wafer-Scale Cluster connection diagram](files/Trust_ctl.png)
+
+
+On your work machine with a web browser, e.g. your laptop,<br>
+tunnel the grafana https port on the cerebras grafana host through to localhost
+```
+ssh -L 8443:grafana.cerebras1.lab.alcf.anl.gov:443 [email protected]
+```
+
+Point a browser at grafana. (Tested with Firefox and Chrome/Brave)<br>
+Open browser to a job grafana url shown in csctl get jobs, adding :8443 to hostname, e.g.<br>
+```console
+https://grafana.cerebras1.lab.alcf.anl.gov:8443/d/WebHNShVz/wsjob-dashboard?orgId=1&var-wsjob=wsjob-49b7uuojdelvtrcxu3cwbw&from=1684859330000&to=noww
+```
+
+Login to the dashboard with user admin, and password prom-operator
+
+
 <!---
 ## Determining the CS-2 version
 

docs/ai-testbed/getting-started.md

@@ -13,22 +13,22 @@ The AI accelerators complement the ALCF's current and next-generation supercompu
 The platforms are equipped with architectural features that support AI and data-centric workloads, making them well suited for research tasks involving the growing deluge of scientific data produced by powerful tools, such as supercomputers, light sources, telescopes, particle accelerators, and sensors. In addition, the testbed will allow researchers to explore novel workflows that combine AI methods with simulation and experimental science to accelerate the pace of discovery.
 
 ## How to Get Access
-Researchers interested in using the AI Testbed’s `Cerebras CS-2` and `SambaNova DataScale` platforms can now submit project proposals via the [ALCF’s Director’s Discretionary program](https://www.alcf.anl.gov/science/directors-discretionary-allocation-program). Access to additional testbed resources, including `Graphcore`, `Groq`, and `Habana` accelerators, will be announced at a later date. 
+Researchers interested in using the AI Testbed’s `Cerebras CS-2`, `SambaNova DataScale SN30`  and `Graphcore Bow Pod64` platforms can now submit project proposals via the [ALCF’s Director’s Discretionary program](https://www.alcf.anl.gov/science/directors-discretionary-allocation-program). Access to additional testbed resources, including `Groq`, and `Habana` accelerators, will be announced at a later date. 
 
 Submit your proposal requests at: [Allocation Request Page](https://accounts.alcf.anl.gov/allocationRequests){:target="_blank"}
 
 ## Getting Started
-1. Request a Director's Discretionary project on SambaNova/Cerebras.
+1. Request a Director's Discretionary project on SambaNova/Cerebras/Graphcore.
 
-2. Apply for an ALCF account after the project request is approved. Choose the SambaNova/Cerebras project that your PI has created at ALCF. If you have an active ALCF account, request to [join the project](https://accounts.alcf.anl.gov/joinProject){:target="_blank"} after your project is approved.
+2. Apply for an ALCF account after the project request is approved. Choose the SambaNova/Cerebras/Graphcore project that your PI has created at ALCF. If you have an active ALCF account, request to [join the project](https://accounts.alcf.anl.gov/joinProject){:target="_blank"} after your project is approved.
 
 3. Transfer data to ALCF using Globus after your account has been created.
 
     a. The endpoint for your data in ALCF is ``` alcf#ai_testbed_projects ``` with the path to your project being  ``` /<project name> ```. 
 
     b. The endpoint for your home directory on the AI Testbeds in ALCF is ``` alcf#ai_testbed_home ```.
 
-4. Add/invite team members to your ALCF project on SambaNova/Cerebras. 
+4. Add/invite team members to your ALCF project on SambaNova/Cerebras/Graphcore. 
 
 ## How to Contribute to Documentation
 The documentation is based on [MkDocs](https://www.mkdocs.org/){:target="_blank"} and source files are

docs/ai-testbed/graphcore/documentation.md

@@ -1,7 +1,7 @@
 # Documentation links
 
  [Poplar SDK](https://docs.graphcore.ai/projects/sdk-overview/en/latest/overview.html?highlight=poptorch#introduction)<br>
- [PyTorch for the IPU: User Guide](https://docs.graphcore.ai/projects/poptorch-user-guide/en/3.2.0/index.html)<br>
+ [PyTorch for the IPU: User Guide](https://docs.graphcore.ai/projects/poptorch-user-guide/en/latest/index.html)<br>
  [Targetting the IPU from Tensorflow 2](https://docs.graphcore.ai/projects/tensorflow-user-guide/en/latest/index.html) <br>
  [IPU programming guide](https://docs.graphcore.ai/projects/ipu-programmers-guide/en/latest/index.html) <br>
  [Examples](https://docs.graphcore.ai/projects/tutorials/en/latest/intro.html) <br>