Building and Running ABACUS

Requirements

Please refer to installation guide. Below, we will discuss each required components:

• Compiler, including MPI support
• Math libs, including FFTW
• ELPA
• Cereal

Intel oneAPI toolkit

The Intel® oneAPI toolkit provides a complete toolchain. The Intel® oneAPI Base Toolkit contains Intel® oneAPI Math Kernel Library (aka MKL), providing a fast BLAS library. The Intel® oneAPI HPC Toolkit contains compilers with MPI Library.

Pros: you don't need to compile various math libraries by yourself.

Cons: large disk usage; potential backfire on non-Intel platform.

Below are minimum required components by ABACUS:

apt-get install -y  \
	intel-oneapi-compiler-dpcpp-cpp-and-cpp-classic \
	intel-oneapi-compiler-fortran \
	intel-oneapi-mkl-devel \
	intel-oneapi-mpi-devel

Alternatively, install the whole Toolkit by the links above (recommended), or by the commands below. Be sure to check disk free space before:

wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18236/l_BaseKit_p_2021.4.0.3422_offline.sh
sudo bash l_BaseKit_p_2021.4.0.3422_offline.sh
wget https://registrationcenter-download.intel.com/akdlm/irc_nas/18211/l_HPCKit_p_2021.4.0.3347_offline.sh
sudo bash l_HPCKit_p_2021.4.0.3347_offline.sh

After installing, configure your system. To work at a Command Line Interface (CLI), you may configure the components of the oneAPI toolkits using environment variable script.

source /opt/intel/oneapi/setvars.sh

DO NOT forget to set env vars each time you create a new shell window! Most error happens here. To configure environment variables to be set up automatically, add the command above to your ~/.bashrc or ~/.zshrc, respectively. Check correctness by mpiicc --version, and it should show something like:

icc (ICC) 2021.1 Beta 20201112 Copyright (C) 1985-2020 Intel Corporation. All rights reserved.

The compilers we use are: mpiicc, mpiicpc and mpiifort, being counterparts to mpicc, mpicxx and gfortran.

Now, go to the ELPA part after completing the installation. Continue to read if you decide not using Intel oneAPI kit.

Compilers

gcc

ABACUS requires a minimum gcc version of 5.1 . Check by gcc --version. We recommend using the latest gcc. If the gcc is not qualified,

• Install from apt/yum: apt-get install gcc
• Install Red Hat Developer Toolset

gfortran

ELPA requires a fortran compiler. apt-get install gfortran helps.

MPI

Abacus relies on MPI to scale up. This requires compilers with MPI wrapper, e.g. mpicc. We recommend using MPICH:

apt-get install mpich libmpich-dev

Math libs

BLAS

We use OpenBLAS.

git clone https://github.com/xianyi/OpenBLAS.git
cd OpenBLAS 
make -j9 FC=gfortran
make install

ScaLAPACK

git clone https://github.com/darelbeida/scalapack.git -b v2.0.2-openblas --single-branch --depth=1 \
cd scalapack && make lib
cp libscalapack.a /usr/local/lib/ 

FFTW

The support to FFTW2 is out-dated in ABACUS; FFTW here only reference to FFTW3. Simply installing FFTW by apt-get will not install the MPI support part, i.e. a serial version of FFTW. To get FFTW with parallelism support, build from source:

wget http://www.fftw.org/fftw-3.3.9.tar.gz
tar zxvf fftw-3.3.9.tar.gz
cd fftw-3.3.9
./configure --enable-mpi-fortran --enable-orterun-prefix-by-default FC=gfortran
make -j9
make PREFIX=/usr/local install

ELPA

We use ELPA to solve eigenvalue.

wget https://elpa.mpcdf.mpg.de/software/tarball-archive/Releases/2021.05.002/elpa-2021.05.002.tar.gz
tar xzf elpa-2021.05.002.tar.gz
cd elpa-2021.05.002 
mkdir build 
cd build 

The configure process varies between compilers.

For gcc compiler, use the configure command below:

../configure --enable-openmp CFLAGS="-O3 -march=native -funsafe-loop-optimizations -funsafe-math-optimizations -ftree-vect-loop-version -ftree-vectorize" \
    FCFLAGS="-O2 -mavx" --disable-avx512 

Here's another one for Intel oneAPI(don't forget to run setvars.sh before!):

CC=mpiicc CXX=mpiicpc FC=mpiifort ../configure --enable-openmp FCFLAGS="-qmkl=cluster"

Set the ELPA install directory by appending --prefix=/absolute_path/elpa (replacing with your directory here) as a parameter to ../configure. ELPA by default install to /usr/local, which may requires system privilege.

Build and install.

make -j9 
make install
ln -s /usr/local/include/elpa-2021.05.002/elpa /usr/local/include/ 

Please note that ln -s /usr/local/include/elpa-2021.05.002/elpa /usr/local/include/ is necessary, alternatively you could edit the cmake/FindELPA.cmake by adding the include/elpa-2021.05.002 into find_path.

find_path(ELPA_INCLUDE_DIR
    elpa/elpa.h
    HINTS ${ELPA_DIR}
    PATH_SUFFIXES "include" "include/elpa" "include/elpa-2021.05.002"
    )

Cereal

We use Cereal to serialize data. It is a header-only library.

git clone https://github.com/USCiLab/cereal.git \
cp -r cereal/include /usr/local

LibXC

Libxc is a library of exchange-correlation and kinetic energy functionals for density-functional theory.

wget https://gitlab.com/libxc/libxc/-/archive/6.2.0/libxc-6.2.0.tar.gz \
    && tar xzf libxc-6.2.0.tar.gz \
    && cd libxc-6.2.0 \
    && mkdir build \
    && cmake -B build -DCMAKE_INSTALL_PREFIX=~/libxc -DBUILD_TESTING=OFF \
    && cmake --build build \
    && cmake --install build