Fluidity PyOP2 on CX1

To install Fluidity/PyOP2 on CX1:

1 - Edit your bash.rc file and add the following line

module load python/2.7.3

2 - Be sure to execute the following steps before compiling fluidity:

module load python/2.7.3
export PYTHON_VERSION=2.7

3 - execute the script provided by the AMCG website at the following link, slightly modified as follows:

#PBS -N comp101binmod
#PBS -l walltime=1:00:00
#PBS -l select=1:ncpus=1:mem=1800mb


module load intel-suite/11.1
module load mpi
module load vtk/5.8.0
module load netcdf
module load udunits
module load parmetis
module load petsc/3.3-p3-intel-11
module load zoltan
module load libscotch

#module load python/2.7.3

export FLUIDITY_DIR=/path/to/fluidity

export LIBS="-L${ZOLTAN_HOME}/lib -L${VTK_LIBS} -L${NETCDF_HOME}/lib -L${UDUNITS_HOME}/lib -L${PARMETIS_HOME}/lib -L/usr/X11R6/lib64/ -L${MKL_HOME}/lib/64/ -lmkl_intel_lp64 -lmkl_sequential -lmkl_core"
export CPPFLAGS="-I${ZOLTAN_HOME}/include -I${VTK_INCLUDE} -I${NETCDF_HOME}/include -I${UDUNITS_HOME}/include -I${PARMETIS_HOME}/include"
export ZOLTAN_DEPS="-L${ZOLTAN_HOME}/lib -L${LIBSCOTCH_HOME}/lib -lpthread -lptscotch -lpatoh -lptscotcherr -lptscotcherrexit -lscotchmetis -lscotch"

export PYTHONPATH=${FLUIDITY_DIR}/python:$PYTHONPATH

cd ${FLUIDITY_DIR}

export PYTHON_VERSION=2.7
export LDFLAGS=-L/apps/python/2.7.3/lib

./configure --disable-petsc-fortran-modules --without-adjoint --enable-2d-adaptivity > compile.log 2>&1 && make clean >> compile.log 2>&1 && make >> compile.log 2>&1 && make fltools >> compile.log 2>&1

To use Fluidity on top of PyOP2:

We assume you have already checked out PyOP2 in your home directory

cd
mkdir local

To install.. 4a) ..decorator

wget http://pypi.python.org/packages/source/d/decorator/decorator-3.4.0.tar.gz
tar xzf decorator-3.4.0.tar.gz
cd decorator-3.4.0
python setup.py install --prefix=~/local

4b) ..yaml

wget http://pyyaml.org/download/pyyaml/PyYAML-3.10.tar.gz
tar xzf PyYAML-3.10.tar.gz
cd PyYAML-3.10
python setup.py install --prefix=~/local

4c) ..instant

wget https://launchpad.net/instant/1.0.x/1.0.0/+download/instant-1.0.0.tar.gz
tar xzf instant-1.0.0.tar.gz
cd instant-1.0.0
python setup.py install --prefix=~/local

4d) ..ScientificPython

wget https://sourcesup.renater.fr/frs/download.php/2309/ScientificPython-2.8.tar.gz
tar xzf ScientificPython-2.8.tar.gz
cd ScientificPython-2.8
python setup.py install --prefix=~/local

5. Inside your PyOP2 clone, create a file cx1.env with the following content:

module load cython

export OP2_DIR=/path/to/op2/repo/op2
export PYTHONPATH=~/local:$PYTHONPATH
export PYTHONPATH=`pwd`:$PYTHONPATH

Execute it and install PyOP2 in the usual way

6. For using the FFC interface, you also need to install the following components

The FFC interface depends on various FEniCS components:

UFL > 1.0~2 FFC pyop2 branch FIAT (depends on ScientificPython) UFC

From now on, for simplicity, we assume these components are installed on your home directory.

7. (TEMPORARY PATCH) In order to enable just-in-time compilation, we need to instruct icc on where to get the python2.7 library. So, be sure to add the following string

    /apps/python/2.7.3/lib

to each instant/numpy call. For example, if your beckend is the sequential one, you will invoke inline_with_numpy in the following way:

_fun = inline_with_numpy(.....,
                         library_dirs=[OP2_LIB, "/apps/python/2.7.3/lib"],
                         ......)

8. Finally, to run a fluidity test case, you can modify this template script:

 # !/bin/bash

 #PBS -l mem=4gb
 #PBS -l ncpus=1
 #PBS -l ngpus=0
 #PBS -W block=true

PBS_O_WORKDIR=/where/the/fluidity/flml_file/is/located
PROJECT=project_name
EXECUTABLE=filename.flml

cd $PBS_O_WORKDIR

module load intel-suite/2013
module load gmsh
module load mpi
module load vtk/5.8.0
module load valgrind
module load libgmp
module load matplotlib

 # Directory with your built Fluidity sourcetree in (edit this to your own path!)
export FLUIDITY_DIR=

export PYTHONPATH=${FLUIDITY_DIR}/python:${FLUIDITY_DIR}/lib/python2.7/site-packages:~/ufl:~/ffc:~/PyOP2:~/local/lib64/python2.6/site-packages:~/local/lib/python2.7/site-packages:~/local/lib/python2.6/site-packages:$PYTHONPATH

export OP2_DIR=/path/to/op2/repo/op2

 #cp the fluidity executable and project files on tmp
EXECDIR=$TMPDIR/$PROJECT

mkdir $EXECDIR -p

cp path/to/fluidity/binaries/fluidity $EXECDIR
cp * $EXECDIR

cd $EXECDIR

./fluidity $EXECUTABLE

 #cp results back in my directory, if any
cp your_file_containing_results