Skip to content

Commit

Permalink
Merge branch 'argonne-lcf:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
kaushikvelusamy authored Nov 5, 2024
2 parents 2f4bb02 + fb19888 commit 95ddead
Show file tree
Hide file tree
Showing 50 changed files with 1,122 additions and 521 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
## I am unable to sign in to the Accounts website. What do I do?
Only users with active ALCF accounts can sign in to the Account and Project Management website. If you have an active account, verify that you are using the correct ALCF username. Note that username is case-sensitive. If you forgot your username, contact [email protected]. For passcode token issues, please review the troubleshooting information on this page: [Passcode Tokens](https://docs.alcf.anl.gov/account-project-management/accounts-and-access/alcf-passcode-tokens/).

If your account is inactive, please submit a reactivation request here: [https://accounts.alcf.anl.gov/#/accountReactivate](https://accounts.alcf.anl.gov/#/accountReactivate).
If your account is inactive, please submit a reactivation request here: [https://my.alcf.anl.gov/accounts/#/accountReactivate](https://my.alcf.anl.gov/accounts/#/accountReactivate).

If you never had an ALCF account, please apply for one here: [https://accounts.alcf.anl.gov/#/accountRequest](https://accounts.alcf.anl.gov/#/accountRequest). Note that all ALCF accounts must be associated with a project with an active allocation.
If you never had an ALCF account, please apply for one here: [https://my.alcf.anl.gov/accounts/#/accountRequest](https://my.alcf.anl.gov/accounts/#/accountRequest). Note that all ALCF accounts must be associated with a project with an active allocation.

## How do I request a new project/allocation?
There are 3 allocation opportunities at ALCF. Please see How to Get an Allocation on how to get time on our systems.
Expand All @@ -19,7 +19,7 @@ To request an extension of your existing discretionary allocation or to request
- How many additional hours you are requesting?

## How do I join a project?
To join a project, please go to [https://accounts.alcf.anl.gov](http://accounts.alcf.anl.gov/), then click "join a project". Once there, scroll down to the project you want to join and click on it. At the bottom of the next page, please click on the "Request Membership" button. Once we receive approval from the PI regarding your membership request, we will provide you with access to the necessary resources.
To join a project, please go to [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/), then click "join a project". Once there, scroll down to the project you want to join and click on it. At the bottom of the next page, please click on the "Request Membership" button. Once we receive approval from the PI regarding your membership request, we will provide you with access to the necessary resources.

## How do I request a reservation?
Reservation requests must include information detailed here:
Expand All @@ -29,8 +29,8 @@ Reservation requests must include information detailed here:
## How do I apply for a new account?
**Note:** All ALCF accounts must be associated with an allocated project.

- Request a new account: [https://www.alcf.anl.gov/support-center/get-started/request-account](https://www.alcf.anl.gov/support-center/get-started/request-account)
- ALCF Accounts: [https://accounts.alcf.anl.gov/](https://accounts.alcf.anl.gov/)
- Request a new account: [https://my.alcf.anl.gov/accounts/#/accountRequest](https://my.alcf.anl.gov/accounts/#/accountRequest)
- ALCF Accounts: [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/)

## What do I do when my ALCF account expires?
Please forward your account expiry email to your Sponsor. As soon as we receive an approval email from your Sponsor, we'll proceed with your account renewal process as needed.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

All computing carried out on the ALCF systems is associated with a user "account." This account is used to log onto the login servers and run jobs on the resources. If someone has a user account, then he or she has a login name that is recorded in the user database. This web page describes the process that users will need to understand to manage account details, including policies and procedures.

If you need an account, visit the Accounts and Project Management website: [Request an account](https://accounts.alcf.anl.gov/#!/accountRequest)
If you need an account, visit the Accounts and Project Management website: [Request an account](https://my.alcf.anl.gov/accounts/#/accountRequest)

If you want to learn how to get started, visit the Get Started Guide: [Get Started Guide](https://www.alcf.anl.gov/support-center/get-started)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ While requesting an allocation, users can choose from:
[Pullback Policy](../../policies/queue-scheduling/pullback-policy.md)

## Requesting Additional Allocation Hours
If you are a PI of a Director's Discretionary project that has an active allocation, you can request additional time or an extension using the [allocation request form](https://accounts.alcf.anl.gov/allocationRequests).
If you are a PI of a Director's Discretionary project that has an active allocation, you can request additional time or an extension using the [allocation request form](https://my.alcf.anl.gov/accounts/#/allocationRequests).

<figure markdown>
![Project Management](files/request-allocation.png){ width="700" }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,9 +25,9 @@ Before your project begins, you will receive an email with the following project
- **Allocation End Date**: The end date of your award.

## Account Setup
**If you do not have an ALCF account:** You will need to request one at [https://accounts.alcf.anl.gov/accountRequest](https://accounts.alcf.anl.gov/accountRequest). When prompted for project name, please select the project short name you were given in your award email from [[email protected]](mailto:[email protected]).
**If you do not have an ALCF account:** You will need to request one at [https://my.alcf.anl.gov/accounts/#/accountRequest](https://my.alcf.anl.gov/accounts/#/accountRequest). When prompted for project name, please select the project short name you were given in your award email from [[email protected]](mailto:[email protected]).

**If you have an active ALCF account:** Submit a request to join the newly awarded project at [https://accounts.alcf.anl.gov/#!/joinProject](https://accounts.alcf.anl.gov/#!/joinProject).
**If you have an active ALCF account:** Submit a request to join the newly awarded project at [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/).

### Information for Foreign National Access
The U.S. Department of Energy has guidelines and requirements for foreign nationals who access its facilities and sites. This guidance is issued in DOE Order 142.3, which is part of Argonne's contract; therefore, all foreign nationals (non-U.S. Citizens) must obtain authorization prior to using ALCF resources.
Expand Down Expand Up @@ -76,11 +76,11 @@ PI/proxies with active ALCF accounts can also approve new account requests, proj
### Accounts and Access for your Project Members
All project members will need an ALCF user account to access project data and to run jobs on ALCF systems.

Members that **do not have an ALCF account** should request one at: [https://accounts.alcf.anl.gov/#/accountRequest](https://accounts.alcf.anl.gov/#/accountRequest). When prompted for project name, they should select your project short name.
Members that **do not have an ALCF account** should request one at: [https://my.alcf.anl.gov/accounts/#/accountRequest](https://my.alcf.anl.gov/accounts/#/accountRequest). When prompted for project name, they should select your project short name.

Members with **ALCF accounts that are no longer active** should submit a reactivation request here: [https://accounts.alcf.anl.gov/#/accountReactivate](https://accounts.alcf.anl.gov/#/accountReactivate). When prompted for project name, they should select your project short name.
Members with **ALCF accounts that are no longer active** should submit a reactivation request here: [https://my.alcf.anl.gov/accounts/#/accountReactivate](https://my.alcf.anl.gov/accounts/#/accountReactivate). When prompted for project name, they should select your project short name.

Members with **active ALCF accounts but have not been added to your project** should submit a request to join your project by going to this page: [https://accounts.alcf.anl.gov/#!/joinProject](https://accounts.alcf.anl.gov/#!/joinProject). They should search for your project and click the "Request Membership" button for that project.
Members with **active ALCF accounts but have not been added to your project** should submit a request to join your project by going to this page: [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/). They should search for your project and click the "Request Membership" button for that project.

## Moving Your Data
We encourage you to use Globus to move your project data to your ALCF project directory before your allocation begins. For details, see [Using Globus](../../data-management/data-transfer/using-globus.md).
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Managing Your Team Members

The PI or Proxy must approve each member of the team to gain access and to run project jobs on the ALCF's resources. If you have an active ALCF account, you can manage your project team by logging into the ALCF account and project management website and navigating to [https://accounts.alcf.anl.gov/#/manageProjects](https://accounts.alcf.anl.gov/#/manageProjects)
The PI or Proxy must approve each member of the team to gain access and to run project jobs on the ALCF's resources. If you have an active ALCF account, you can manage your project team by logging into the ALCF account and project management website and navigating to [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/)

Project members will need to have an active ALCF user account to access project data and to run jobs on ALCF systems. See [Accounts and Access for your Project Members]{https://docs.alcf.anl.gov/account-project-management/project-management/starting-alcf-award/#accounts-and-access-for-your-project-members) for information on how team members can get an account, reactivate an account, or request to join your project.

## Accessing your project(s)
1. Log in at [https://accounts.alcf.anl.gov/#/manageProjects](https://accounts.alcf.anl.gov/#/manageProjects) using your credentials (ALCF username and Physical/Mobile token passcode one-time passcode).
1. Log in at [https://my.alcf.anl.gov/](https://my.alcf.anl.gov/) using your credentials (ALCF username and Physical/Mobile token passcode one-time passcode).
2. You will see a list of projects of which you are the Primary Investigator (PI).
4. Click on the desired project to view information and management options for the selected project.

Expand Down
4 changes: 2 additions & 2 deletions docs/ai-testbed/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,12 +15,12 @@ The platforms are equipped with architectural features that support AI and data-
## How to Get Access
Researchers interested in using the AI Testbed’s `Cerebras CS-2`, `SambaNova DataScale SN30`, `Graphcore Bow Pod64` and `GroqRack` platforms can now submit project proposals via the [ALCF’s Director’s Discretionary program](https://www.alcf.anl.gov/science/directors-discretionary-allocation-program). Access to additional testbed resources, including `Habana` accelerators, will be announced at a later date.

Submit your proposal requests at: [Allocation Request Page](https://accounts.alcf.anl.gov/allocationRequests){:target="_blank"}
Submit your proposal requests at: [Allocation Request Page](https://my.alcf.anl.gov/accounts/#/allocationRequests){:target="_blank"}

## Getting Started
1. Request a Director's Discretionary project on SambaNova/Cerebras/Graphcore/Groq.

2. Apply for an ALCF account after the project request is approved. Choose the SambaNova/Cerebras/Graphcore/Groq project that your PI has created at ALCF. If you have an active ALCF account, request to [join the project](https://accounts.alcf.anl.gov/joinProject){:target="_blank"} after your project is approved.
2. Apply for an ALCF account after the project request is approved. Choose the SambaNova/Cerebras/Graphcore/Groq project that your PI has created at ALCF. If you have an active ALCF account, request to [join the project](https://my.alcf.anl.gov/){:target="_blank"} after your project is approved.

3. Transfer data to ALCF using Globus after your account has been created.

Expand Down
4 changes: 2 additions & 2 deletions docs/ai-testbed/groq/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@
## Allocations

If you do not already have an allocation, you will need to request one here:
[Discretionary Allocation Request (New & Renewal)](https://accounts.alcf.anl.gov/#/allocationRequests)
[Discretionary Allocation Request (New & Renewal)](https://my.alcf.anl.gov/accounts/#/allocationRequests)

## Accounts

If you do not have an ALCF account (but have an allocation), request one here: [ALCF Account and Project Management](https://accounts.alcf.anl.gov/#/home)
If you do not have an ALCF account (but have an allocation), request one here: [ALCF Account and Project Management](https://my.alcf.anl.gov/)

## Setup

Expand Down
20 changes: 16 additions & 4 deletions docs/aurora/data-management/moving_data_to_aurora/globus.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,25 @@
### Transfering files through Globus
# Transfering files through Globus

Currently only Globus personal is supported
Currently, only Globus personal is supported on Aurora.
Perform the following steps to transfer data to/from the Aurora login nodes.

1. On a fresh connection to the login nodes, ensuring no proxies are being set (which may require commenting the proxy settings in the `~/.bashrc` or `~/.bash_profile` files), execute
```bash
/soft/tools/proxychains/bin/proxychains4 -f /soft/tools/proxychains/etc/proxychains.conf /soft/tools/globusconnect/globusconnect -setup --no-gui
```
and follow the instruction to setup personal endpoint.

2. Paste the link provided by the above command in a browser and follow the instructions to setup a personal endpoint
* When requested, input your ALCF username and one-time password from your CRYPTOCard/MobilePASS+ token
* Select the Allow button
* Enter the authentication code generated back into the terminal
* Enter a name for the endpoint (e.g., `aurora_login_uan11`)

3. On the same terminal, execute
```bash
/soft/tools/proxychains/bin/proxychains4 -f /soft/tools/proxychains/etc/proxychains.conf /soft/tools/globusconnect/globusconnect -start &
```
You can also add -restrict-paths /lus/flare/projects/YOURPROJECT to access folders outside of your home
* By default, the command only gives access to your home directory
* You can add `-restrict-paths /lus/flare/projects/YOURPROJECT` to access your project directory

4. Open the [Globus web app](https://app.globus.org/file-manager?destination_id=05d2c76a-e867-4f67-aa57-76edeb0beda0) and search for the endpoint name defined above. You will now see your home directory (and project directory, if requested) on Aurora and can initiate transfers with other endpoints (e.g., the Eagle file system on Polaris at `alcf#dtn_eagle`)

29 changes: 22 additions & 7 deletions docs/aurora/data-management/moving_data_to_aurora/scp.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,41 @@

## Data Transfer

Currently, scp and SFTP are the only ways to transfer data to/from Aurora.
Use scp and SFTP to transfer data to/from Aurora.

### Transferring files from Aurora (Flare) to Sunspot (Gila)
From an Aurora login-node, you can transfer files to Sunspot's `gila` filesystem using the `scp` command. But first, you need to create an SSH keypair on Aurora and copy the public key (`*.pub`) to the `~/.ssh/authorized_keys` file on Sunspot.
1. Create SSH keys on the laptop/desktop/remote machine. See "Creating SSH Keys" section on [this page](https://help.cels.anl.gov/docs/linux/ssh/).
2. Copy the public key (*.pub) from ~/.ssh folder on Aurora to ~/.ssh/authorized_keys file on Sunspot (sunspot.alcf.anl.gov)
3. Run the scp command on Aurora to transfer files to Sunspot
2. Copy the contents of the public key file (`*.pub`) located in the `~/.ssh/` folder on Aurora, and append it to the `~/.ssh/authorized_keys` file on Sunspot (`sunspot.alcf.anl.gov`)
3. Run the `scp` command on Aurora to transfer files to Sunspot
```
username@aurora-uan-0010:~> scp test_file [email protected]://lus/gila/projects/Aurora_deployment/username
...
username@uan-0001:/gila/Aurora_deployment/username> cat test_file
this is a test file
```

### Transferring files to Aurora
### Transferring files to Aurora (Flare)

With the bastion pass-through nodes currently used to access both Sunspot and Aurora, users will find it helpful to modify their `.ssh/config` files on Aurora appropriately to facilitate transfers to Aurora from other ALCF systems. These changes are similar to what Sunspot users may have already implemented. From an Aurora login-node, this readily enables one to transfer files from Sunspot's `gila` filesystem or one of the production filesystems at ALCF (`home` and `eagle`) mounted on an ALCF system's login node. With the use of `ProxyJump` below, entering the MobilePass+ or Cryptocard passcode twice will be needed (once for bastion and once for the other resource). A simple example shows the `.ssh/config` entries for Polaris and the `scp` command for transferring from Polaris:

```
$ cat .ssh/config
knight@aurora-uan-0009:~> cat .ssh/config
username@aurora-uan-0009:~> cat .ssh/config
Host bastion.alcf.anl.gov
User knight
User username
Host polaris.alcf.anl.gov
ProxyJump bastion.alcf.anl.gov
DynamicForward 3142
user knight
user username
```

```
knight@aurora-uan-0009:~> scp knight@polaris.alcf.anl.gov:/eagle/catalyst/proj-shared/knight/test.txt ./
username@aurora-uan-0009:~> scp username@polaris.alcf.anl.gov:/eagle/catalyst/proj-shared/username/test.txt ./
---------------------------------------------------------------------------
Notice to Users
...
Expand All @@ -30,7 +45,7 @@ knight@aurora-uan-0009:~> scp [email protected]:/eagle/catalyst/proj-s
Notice to Users
...
[Password:
knight@aurora-uan-0009:~> cat test.txt
username@aurora-uan-0009:~> cat test.txt
from_polaris eagle
```

Expand Down
Loading

0 comments on commit 95ddead

Please sign in to comment.