Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SYCL] updates to oneapi compiler documentation #263

Merged
merged 5 commits into from
Sep 9, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 18 additions & 20 deletions docs/polaris/compiling-and-linking/oneapi-compiler.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,34 @@
# OneAPI Compilers and Support
# oneAPI Compilers and Support

The Intel OneAPI compiler and Codeplay plugins for Nvidia GPUs are available on Polaris.
The Intel oneAPI compiler and Codeplay plugins for Nvidia GPUs are available on Polaris.
The oneAPI compilers are not enabled under the Cray Programming Environment system but can be used separately.
Two oneAPI variants are provided, the first being a "release" version based on Intel's officially released oneAPI toolkit.
[Intel Release Notes](https://www.intel.com/content/www/us/en/developer/articles/release-notes/intel-oneapi-toolkit-release-notes.html)

!!! note

The 2023.1 release of oneAPI Toolkit does not support oneMKL or oneDPL on Nvidia.
The 2023.2.0 release of oneAPI Toolkit does not yet support oneMKL or oneDPL on Nvidia devices.

The other variant being a build of main from the open source. This variant will be more up-to-date at the risk of bugs and breakages based on code that has not undergone a full release cycle.
The documentation is located on the [SYCL](../programming-models/sycl-polaris.md) page.
The other variant being a build from the open-source. This variant will be more up-to-date at the risk of bugs and breakages based on code that has not undergone a full release cycle.
The documentation is located on the [SYCL](../programming-models/sycl-polaris.md) page. Most notable differences being, `icx/icpx` are the names of C/C++ compilers respectively when using the release version of the module where as `clang/clang++` are for open-source variant.


## Compile and Link
OneAPI uses the clang (or icx/icpx wrapper) for compiling and linking for the Nvidia A100 SM80 architecture.
oneAPI uses the clang (or icx/icpx wrapper) for compiling and linking for the Nvidia A100 SM80 architecture.


```
module load PrgEnv-nvhpc
module use /soft/compilers/oneapi/release/modulefiles
module load compiler
icpx -std=c++17 -fsycl -fsycl-targets=nvptx64-nvidia-cuda -Xsycl-target-backend --cuda-gpu-arch=sm_80 test.cpp
module load oneapi/release
icpx -std=c++17 -fsycl -fsycl-targets=nvptx64-nvidia-cuda -Xsycl-target-backend --cuda-gpu-arch=sm_80 test.cpp
```

```
harms@polaris-login-04:~/working/polaris/oneapi> icpx --version
Intel(R) oneAPI DPC++/C++ Compiler 2023.1.0 (2023.1.0.20230320)
Intel(R) oneAPI DPC++/C++ Compiler 2023.2.0 (2023.2.0.20230721)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /soft/compilers/oneapi/release/2023.1/compiler/2023.1.0/linux/bin-llvm
Configuration file: /soft/compilers/oneapi/release/2023.1/compiler/2023.1.0/linux/bin-llvm/../bin/icpx.cfg
InstalledDir: /soft/compilers/oneapi/release/2023.2/compiler/2023.2.1/linux/bin-llvm
Configuration file: /soft/compilers/oneapi/release/2023.2/compiler/2023.2.1/linux/bin-llvm/../bin/icpx.cfg
```

## Running
Expand All @@ -48,13 +46,13 @@ Expected output of sycl-ls and which platforms are available.

```
harms@x3004c0s7b0n0:~> which sycl-ls
/soft/compilers/oneapi/release/2023.1/compiler/2023.1.0/linux/bin/sycl-ls
/soft/compilers/oneapi/release/2023.2/compiler/2023.2.1/linux/bin/sycl-ls

harms@x3004c0s7b0n0:~> sycl-ls
[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.15.3.0.20_160000]
[opencl:cpu:1] Intel(R) OpenCL, AMD EPYC 7543P 32-Core Processor 3.0 [2023.15.3.0.20_160000]
[ext_oneapi_cuda:gpu:0] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 0.0 [CUDA 11.4]
[ext_oneapi_cuda:gpu:1] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 0.0 [CUDA 11.4]
[ext_oneapi_cuda:gpu:2] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 0.0 [CUDA 11.4]
[ext_oneapi_cuda:gpu:3] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 0.0 [CUDA 11.4]
[opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.16.7.0.21_160000]
[opencl:cpu:1] Intel(R) OpenCL, AMD EPYC 7543P 32-Core Processor 3.0 [2023.16.7.0.21_160000]
[ext_oneapi_cuda:gpu:0] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 8.8 [CUDA 11.4]
[ext_oneapi_cuda:gpu:1] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 8.8 [CUDA 11.4]
[ext_oneapi_cuda:gpu:2] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 8.8 [CUDA 11.4]
[ext_oneapi_cuda:gpu:3] NVIDIA CUDA BACKEND, NVIDIA A100-SXM4-40GB 8.8 [CUDA 11.4]
```
19 changes: 16 additions & 3 deletions docs/polaris/programming-models/sycl-polaris.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,14 +10,26 @@
module load oneapi
```

:warning: This module (compilers, libraries) gets built periodically from the latest open-source rather than releases. As such, these compilers will get new features and updates quickly that may break on occasion.
!!! note

This module (compilers, libraries) gets built periodically from the latest open-source rather than releases. For more details on the release version of compiler, please find the details [here](../compiling-and-linking/oneapi-compiler.md). As such, these compilers will get new features and updates quickly that may break on occasion. Please submit any issues at the respective github repositories for the compilers and libraries.

## Components
- These are the list of components associated with this module

| User Application | Component |
|-------------------|---------------------------------------------------------|
| Compilers | [DPC++](https://docs.nvidia.com/cuda/cublas/index.html) |
| oneMKL Interfaces | [oneMKL](https://github.com/oneapi-src/oneMKL) |
| oneDPL | [oneDPL](https://github.com/oneapi-src/onedpl) |
| SYCLomatic/DPCT | [dpct](https://github.com/oneapi-src/syclomatic) |

## Dependencies
- SYCL programming model is supported through `oneapi` compilers that were built from source-code
- Loading this module switches the default programming environment to GNU and with the following dependencies
- PrgEnv-gnu
- cudatoolkit-standalone
- Environment Variable set: `SYCL_DEVICE_SELECTOR=ext_oneapi_cuda:gpu`
- Environment variable is set when loading the module: `ONEAPI_DEVICE_SELECTOR=ext_oneapi_cuda:gpu`

## Example (memory intilization)

Expand Down Expand Up @@ -139,13 +151,13 @@ For further details regarding the arguments passed to `mpiexec` command shown ab
[oneMKL Interfaces](https://github.com/oneapi-src/oneMKL) is an open-source implementation of the oneMKL Data Parallel C++ (DPC++) interface according to the [oneMKL specification](https://spec.oneapi.io/versions/latest/elements/oneMKL/source/index.html). It works with multiple devices (backends) using device-specific libraries underneath.

oneMKL is part of oneAPI. Various backend supported are shown below. More Information [here](https://github.com/oneapi-src/oneMKL#supported-configurations).

| User Application | Third-Party Library |
|------------------|--------------------------------------------------------------|
| | [cuBLAS](https://docs.nvidia.com/cuda/cublas/index.html) |
| oneMKL interface | [cuSOLVER](https://docs.nvidia.com/cuda/cusolver/index.html) |
| | [cuRAND](https://docs.nvidia.com/cuda/curand/index.html) |


## Example (using onemkl::gemm)
The following snippet shows how to compile and run a SYCL code with oneMKL library. For instance, a GPU-based GEMM is performed using `mkl::gemm` API and the results are compared to a CPU-based GEMM performed using the traditional blas (e.g., AOCL-BLIS) library.
```c++
Expand Down Expand Up @@ -272,6 +284,7 @@ int main() {
```

Compile and Run

The user would need to provide paths the math-libraris as shown below. Also please provide AOCL library for CPU GEMM by `module load aocl`.
Environment variables `MKLROOT` is defined with `oneapi` module & `AOCL_ROOT` is defined with `aocl` module.
Note: Please pay attention to the linker options for AOCL & oneMKL libraries.
Expand Down