Added general troubleshooting page

.gitattributes

@@ -0,0 +1,2 @@
+* text=auto
+

.gitignore

@@ -2,3 +2,4 @@ __pycache__
 site
 spack-ucdavis
 venv
+*.DS_Store

docs/admin/configuration.md

@@ -1,3 +1,6 @@
-# Configuration Management
+---
+template: admin.html
+title: Configuration Management 
+---
 
 ie: puppet

docs/admin/index.md

@@ -0,0 +1,13 @@
+---
+title: About
+---
+
+This section is for HPCCF admins to document our internal infrastructure, processes, and
+architectures.
+Although the information may be of _interest_ to end users, it is not designed or maintained for
+their consumption; nothing written here should be confused as an offering of service.
+For example, although we describe our [Virtual Machine](vms.md) infrastructure, which we used for
+hosting a variety of production-essential services for our clusters, we do **not** offer VM hosting
+for end users.
+
+

docs/admin/monitoring.md

@@ -1 +1,4 @@
-# Monitoring
+---
+template: admin.html
+title: Monitoring
+---

docs/admin/netbox.md

@@ -1 +1,60 @@
-# Netbox
+---
+template: admin.html
+title: Netbox
+---
+
+[HPCCF's Netbox Site](https://netbox.hpc.ucdavis.edu/dcim/sites/) is our source of truth for our
+rack layouts, network addressing, and other infrastructure. NetBox is an infrastructure resource modeling (IRM) application designed to empower network automation. NetBox was developed specifically to address the needs of network and infrastructure engineers.
+
+## Netbox Features
+
+- Comprehensive Data Model
+- Focused Development
+- Extensible and Customizable
+- Flexible Permissions
+- Custom Validation & Protection Rules
+- Device Configuration Rendering
+- Custom Scripts
+- Automated Events
+- Comprehensive Change Logging
+
+## Netbox Administration
+
+This section will give an overview of how HPCCF admins utilize and administer Netbox.
+
+### How to add assets into Netbox
+
+1. Navigate to HPCCF's Netbox instance here: [HPCCF's Netbox Site](https://netbox.hpc.ucdavis.edu/dcim/sites/) ![netbox1](../img/netbox1.jpeg)
+
+2. Select the site to which you will be adding an asset too. In this example I have chosen Campus DC: ![netbox2](../img/netbox2.jpeg)
+
+3. Scroll down to the bottom of this page and select which of the locations you will add your asset too, here I chose the Storage Cabinet: ![netbox3](../img/netbox3.jpeg) ![netbox4](../img/netbox4.jpeg)
+
+4. On this page scroll to the bottom and select Add a Device: ![netbox5](../img/netbox5.jpeg)
+
+5. After you have selected Add a Device you should see a page like this: ![netbox6](../img/netbox6.jpeg)
+
+6. Fill out this page with specifics of the asset, some fields are not required but try to fill out this section as much as possible with the fields available, here is an example of a created asset and how it should look: ![netbox7](../img/netbox7.jpeg)![netbox8](../img/netbox8.jpeg)![netbox9](../img/netbox9.jpeg)![netbox10](../img/netbox10.jpeg)![netbox11](../img/netbox11.jpeg)![netbox12](../img/netbox12.jpeg)![netbox13](../img/netbox13.jpeg)
+
+7. Ensure to click on Save to have the device added.
+
+### How to add components to an Asset
+
+1. On the asset page select the + Add Components dropdown and select the component you wish to add, for this I have chosen a Console Port: ![netbox14](../img/netbox14.jpeg)
+
+2. Here again you will fill out the dropdowns as thoroughly as possible, the example here is of an interface that has already been added: ![netbox15](../img/netbox15.jpeg)![netbox16](../img/netbox16.jpeg)![netbox17](../img/netbox17.jpeg)![netbox18](../img/netbox18.jpeg)
+
+3. Again make sure to click Save to ensure the component has been added.
+
+4. This process can be used to add all of the following componentes to a device:![netbox19](../img/netbox19.jpeg)
+
+### How to connect components
+
+1. After a component has been created such as an interface, power port or any other type of component you will want to connect it to something. For any component the process is similar within Netbox. In this example it will show how to connect an Infiniban port on a device to a port on an Infiniban switch. First navigate to the device you wish to work with and select the appropriate tab, in this case it will be Interfaces and you will see a page like this: ![netbox20](../img/netbox20.jpeg)
+
+2. Here we will connect ib1 to an infiniban switch by clicking the green dropdown off to the right of ib1 and we will be connecting to another interface on the infiniban switch so we will choose interface as shown here: ![netbox21](../img/netbox21.jpeg)
+
+3. Once selected you will come to a screen that looks like this: ![netbox22](../img/netbox22.jpeg)![netbox23](../img/netbox23.jpeg)![netbox24](../img/netbox24.jpeg)
+
+4. Once all filled out with the required information to complete the connection (and any additional information that can be provided) at the bottom make sure to create the connection, your screen should look something like this: ![netbox25](../img/netbox25.jpeg)![netbox26](../img/netbox26.jpeg)![netbox27](../img/netbox27.jpeg)
+

docs/admin/network.md

@@ -1 +1,4 @@
-# Network Architecture
+---
+template: admin.html
+title: Network
+---

docs/admin/provisioning.md

@@ -1,3 +1,6 @@
-# Provisioning
+---
+template: admin.html
+title: Provisioning
+---
 
 (cobbler, etc)

docs/admin/software.md

@@ -1,4 +1,7 @@
-# Software
+---
+template: admin.html
+title: Software Deployment
+---
 
 ## Spack
 

docs/admin/vms.md

@@ -1 +1,4 @@
-# Virtual Machines
+---
+template: admin.html
+title: Virtual Machines
+---

docs/data-transfer.md

@@ -1 +1,3 @@
-# Data Transfer
+---
+title: Data Transfer
+---

docs/farm/index.md

@@ -0,0 +1,81 @@
+
+# Farm
+
+![CBS unit signature](../assets/UCDavis_CAES_logo_RGB_vector.svg){ width="400" align="right" }
+
+
+
+Farm is a Linux-based supercomputing cluster for the [College of Agricultural and Environmental Sciences](https://caes.ucdavis.edu/) at UC Davis. 
+Designed for both research and teaching, it is a significant campus resource primarily for CPU and RAM-based 
+computing, with a wide selection of centrally-managed software available for research in genetics, proteomics, and 
+related bioinformatics pipelines, weather and environmental modeling, fluid and particle simulations, geographic 
+information system (GIS) software, and more. 
+
+For buying in resources in Farm cluster, contact CEAS IT director Adam 
+Getchell - <[email protected]>
+
+## Farm Hardware
+
+Farm is an evolving cluster that changes and grows to meet the current needs of researchers, and has undergone three 
+phases, with Farm III as the most recent evolution. 
+
+Farm III consists of 32 parallel nodes with up to 64 CPUs and 
+256GB RAM each in low2/med2/high2, plus 17 “bigmem” nodes with up to 96 CPUs and 1TB RAM each in the bml/bmm/bmh 
+queue. All Farm III bigmem and newer parallel nodes and storage are on EDR/100Gbit interconnects. Older parallel nodes 
+and storage are on FDR/55Gbit. 
+
+Farm II consists of 95 parallel nodes with 24 CPUs and 64GB RAM each in low/med/high, 
+plus 9 “bigmem” nodes with 64 CPUs and 512GB RAM each in the bigmeml/bigmemm/bigmemh queues, and 1 additional node 
+with 96 CPUs and 1TB RAM. Farm II nodes are on QDR/32Gbit interconnects. 
+
+Hardware from both Farm II and Farm III are 
+still in service; Farm I has been decommissioned as of 2014. 
+
+Farm also has multiple file servers with over 5.3PB of storage space in total.
+
+## Access to Farm
+
+All researchers in CA&ES are entitled to free access to:
+
+  -  8 nodes with 24 CPUs and 64GB RAM each (up to a maximum of 192 CPUs and 512GB RAM) in Farm II’s low, medium, and high-priority batch queues, 
+
+  -  4 nodes with 352 CPUs and 768GB RAM each 
+in Farm III's low2, med2, and high2-priority batch queues. 
+
+- The bml (bigmem, low priority/requeue) partition, which has 
+24 nodes with a combined 60 TB of RAM. 
+
+In addition to this, each new user is allocated a 20GB home directory. If you 
+want to use the CA&ES free tier, select “CA&ES free tier" from the list of sponsors [here](https://hippo.ucdavis.edu/Farm/myaccount). 
+
+Additional usage and access 
+may be purchased by contributing to Farm III by through the node and/or storage rates or by purchasing equipment and 
+contributing through the rack fee rate. 
+
+Contributors always receive priority access to the resources that they have 
+purchased within one minute with the “one-minute guarantee.” Users can also request additional unused resources on a 
+“fair share” basis in the medium or low partitions.
+
+## Farm Administration
+
+Farm hardware and software are administrated by the [HPC Core Facility Team](https://hpc.ucdavis.edu/people).
+
+### Current Rates
+
+As of October 2023, the rates for Farm III: 
+
+Node and Storage Rates (each buy-in guarantees access for 5 years): -
+
+- **Parallel (CPU) node:** $13,500 (512 GB RAM, 128 cores/256 threads, 2 TB /scratch) 
+- **Bigmem node:** 128 cores/256 threads, 
+2TB RAM (bml, bmm, bmh partitions) - $25,000 
+- **GPU:** $19,500 1/4 of a GPU node (A100 with 80GB GPU RAM, 16 CPU cores / 
+32 threads, 256GB system RAM) 
+- **Storage:** $100/TB with compression for 5 years (does not include backups)
+
+### Bring your own equipment:
+
+Equipment may be purchased directly by researchers based on actual cost. Equipment quote available upon request. 
+
+- Equipment purchases not using above rates - $375/year per rack unit for five years For more information about buying 
+into Farm, contact [Adam Getchell]([email protected]) or the [Helpdesk]([email protected]).

docs/farm/resources.md

@@ -0,0 +1,27 @@
+## The Farm cluster Resources
+
+Sponsor - CAES
+Information about Partitions - what is low, med and high and what are available GPUs on Farm?
+
+Free tier access to 20GB capped storage.
+
+Low partition - Internittent access to idle resources abbove limit
+
+Medium Partition - Shared use of idle resources above permitted limit
+
+High Partition - Dedicated use of invested resource.
+
+CPU threads - 15,384
+
+GPU count - 29
+
+Aggregated RAM - 66 TB
+
+Maximum RAM per node - 2TB
+
+Node Count - 202
+
+Inter-Connect - 200Gbps
+
+Total number of Users -    726/328
+

docs/franklin/resources.md

@@ -0,0 +1,3 @@
+---
+title: Resources
+---

docs/franklin/scheduling.md

@@ -1,3 +1,6 @@
+---
+title: Job Scheduling
+---
 
 ## Partitions
 

docs/franklin/software/cryoem.md

@@ -18,7 +18,7 @@ If you are completely unfamiliar with Relion, you should start with the [tutoria
 
 !!! Note
     Because Relion is GUI driven, you need to `ssh` to Franklin with X11 forwarding enabled.
-    Instructions for enabling X11 forwarding can be found in the [Access](../general/access.md#x11-forwarding) section.
+    Instructions for enabling X11 forwarding can be found in the [Access](../../general/access.md#x11-forwarding) section.
 
 ### Launching Relion
 
@@ -34,7 +34,7 @@ Currently Loaded Modules Matching: relion
 Change your working directory your Relion project directory and type `relion`.
 The Relion GUI should then pop up locally.
 There will be a bit of latency when using it, especially if you are off campus.
-You may be able to reduce latency by [enabling SSH compression](../general/access.md#x11-forwarding).
+You may be able to reduce latency by [enabling SSH compression](../../general/access.md#x11-forwarding).
 
 <figure markdown>
   ![The Relion start screen.](../../img/relion_start_gui.png)
@@ -84,7 +84,7 @@ The default GUI fields serve their original purposes:
 
 - **Number of MPI procs**: This will fill the Slurm `--ntasks` parameter. These tasks may be distributed across multiple nodes, depending on the number of **Threads** requested. For GPU runs, this should be the number of GPUs **+ 1**.
 - **Number of Threads**: The will fill the Slurm `--cpus-per-task` parameter, which means it is the *number of threads per MPI proc*. Some job types do not expose this field, as they can only be run with a single-thread per MPI proc.
-- **Queue name**: The Slurm partition to submit to, filling the `--partition` parameter. More information on partitions can be found in the [**Queueing**](../../scheduler/queues.md) section.
+- **Queue name**: The Slurm partition to submit to, filling the `--partition` parameter. More information on partitions can be found in the [**Queueing**](../resources.md) section.
 - **Standard submission script**: The location of the Slurm job script template that will be used. This field will be filled with the appropriate template for the loaded Relion module by default, and should not be changed.*For advanced users only:* if you are familiar with Relion and want to further fine-tune your Slurm scripts, you can write your own based on the provided templates found in `/share/apps/spack/templates/hpccf/franklin` or [in our spack GitHub repo](https://github.com/ucdavis/spack-ucdavis/tree/main/templates/hpccf/franklin).
 - **Minimum dedicated cores per node**: Unused on our system.
 

docs/franklin/storage.md

@@ -1,3 +1,7 @@
+---
+title: Storage
+---
+
 ## Home Directories
 
 All users are allocated 20GB of storage for their home directory.

docs/general/access.md

@@ -1,3 +1,37 @@
+In order to access your HPC account, you may need to generate an SSH key pair for authorization. You generate a pair of keys: a public key and a private key. 
+The private key is kept securely on your computer or device.
+The public key is uploaded to the servers or systems that you want to access securely via SSH.
+
+## How do I generate an SSH key pair?
+
+### Windows Operating System
+
+We recommend MobaXterm as the most straightforward SSH client. You can download its free home edition (Installer Edition) from https://mobaxterm.mobatek.net/ 
+The Mobaxterm Portable Edition is not recommended as it deletes the sessions. Once you install the stable version of MobaXterm, open its terminal and enter this command:
+
+
+`ssh-keygen`
+
+This command will create a private key and a public key. Do not share your private key; we recommend giving it a passphrase for security.
+To view the .ssh directory and to read the public key, enter these commands:
+
+```
+ls -al ~/.ssh
+more ~/.ssh/*.pub
+```
+
+### MACOS:
+
+Use a terminal to create an SSH key pair using the command:
+
+`ssh-keygen`
+
+To view the .ssh directory and to read the public key, enter these commands:
+
+```
+ls -al ~/.ssh
+more ~/.ssh/*.pub
+``` 
 
 ## X11 Forwarding
 
@@ -11,15 +45,19 @@ If you are SSHing from a Linux distribution, you likely already have an X11 serv
 If you are on campus, you can use the `-Y` flag to enable it, like:
 
 ```bash
-$ ssh -Y [USER]@franklin.hpc.ucdavis.edu
+$ ssh -Y [USER]@[CLUSTER].hpc.ucdavis.edu
 ```
 
 If you are off campus on a slower internet connection, you may get better performance by enabling compression with:
 
 ```bash
-$ ssh -Y [USER]@franklin.hpc.ucdavis.edu
+$ ssh -Y [USER]@[CLUSTER].hpc.ucdavis.edu
 ```
+If you have multiple SSH key pairs, and you want to use a specific private key to connect to the clusters, use the otpion `-i` to specify path to the private key with SSH:-
 
+```bash
+$ ssh -i /path/to/private/key [USER]@[CLUSTER].hpc.ucdavis.edu
+```
 ### MacOS
 
 MacOS does not come with an X11 implementation out of the box.

docs/general/account-requests.md

@@ -1,13 +1,29 @@
 # Requesting an Account
 
-## Choosing a Cluster
+HPC accounts are provisioned on a per-cluster basis and granted with the permission of their principal investigator. Accounts that are provisioned under each PI will have access to that PI's purchased resources and their own separate home directory.
 
-Information on departments and uses for the major clusters
+Access to HPC clusters is granted via the use of SSH keys. An SSH public key is required to generate an account. For information on creating SSH keys, please visit the [access](https://docs.hpc.ucdavis.edu/general/access/) documentation page.
 
-## HiPPO
+### HiPPO
 
-How to hippo for farm, franklin, peloton
+The High-Performance Personnel Onboarding (HiPPO) portal can provision resources for the Farm, Franklin, and Peloton HPC clusters. Users can request an account on [HiPPO](https://hippo.ucdavis.edu) by logging in with UC Davis CAS and selecting their PI. 
 
-## Other
+Users who do not have a PI and are interested in sponsored tiers for Farm can request an account by selecting the IT director for CAES, Adam Getchell, as their PI.
 
-For non-HiPPO clusters, please see their cluster-specific accounts section.
+Users who do not have a PI and who are affiliated with the College of Letters and Science can request an sponsored account on Peloton by selecting the IT director for CLAS, Jeremy Phillips as their PI. 
+
+
+### HPC1 and HPC2
+
+Users who are associated with PI's in the College of Engineering can request accounts on [HPC1](https://wiki.cse.ucdavis.edu/cgi-bin/engr.pl) and [HPC2](https://hpc.ucdavis.edu/form/account-request-form) by going to the appropriate web form.
+
+### LSSC0 (Barbera) 
+Users who want access to resources on LSSC0 can request an account within the Genome Center Computing [Portal](https://computing.genomecenter.ucdavis.edu/) and selecting 'Request an Account' with their PI.
+
+### Atomate
+
+Atomate accounts can be requested [here](https://wiki.cse.ucdavis.edu/cgi-bin/atomate.pl).
+
+### Cardio, Demon, Impact
+
+Accounts on these systems can be requested [here](https://wiki.cse.ucdavis.edu/cgi-bin/index2.pl).