Channelflow 2.0 MPI Requirements

Hey all,

Is it possible to have the specific MPI related packages listed in INSTALL.md?

I can’t seem to get the installation
to work with MPI enabled and would like to figure out if its user error or a missing component.

Specifically,

Fails 91% of tests:
cmake <path_to_source> -DCMAKE_BUILD_TYPE=release -DWITH_HDF5CXX=ON -DCMAKE_CXX_COMPILER=mpicxx

Passes all tests:
cmake <path_to_source> -DCMAKE_BUILD_TYPE=release -DWITH_HDF5CXX=ON -DCMAKE_CXX_COMPILER=c++ -DUSE_MPI=OFF

Trying to compile with MPI enabled using g++, gcc, c++ throws an error for me.

Thanks!

-Matt Gudorf

Dear Matt,

thank you very much for your feedback. In general, we are collecting the individual issues to improve the documentation in the INSTALL.md. However, please understand that we cannot consider all potential issues due to system specific settings.

About your problem related to MPI: I do not understand if your compilation fails or if you get runtime errors when you run the Channelflow tests (make test). Can you please provide us with the terminal outputs (from cmake and/or the tests) attached as files?

This will help us a lot to help you. Thanks,
Florian

I think my problems can be fixed with information of which mpi library to use: openmpi? mpich? mpich2? intel mpi?

I will provide output if it doesn’t work when I try again tomorrow.

-Matt

We typically use openmpi but I think it does not matter which library you use. What matters though is what compiler is behind your mpicxx-wrapper, the default should be gcc. Depending on the compilers you use (gnu, intel, clang), the libraries fftw, netcdf must be compiled with the chosen compiler.
To find out
mpicxx -show

Hmm I think my problems are all stemming from a lack of permissions on my side; sorry to be confused enough
to bring this to you, I try to avoid installing packages by compiling from source and so I’m not that experienced with it.

-Matt

Matt: I can compile fine on openSUSE 15.0 using

g++-7.3.1
openmpi3-devel-3.0.0
fftw3-mpi-devel-3.3.6
netcdf-devel-4.6.1
hdf5-devel-1.10.1

Here’s my successful command-line call to cmake.

gibson@sophist$ cmake ~/gitsource/channelflow -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=~/channelflow-2.0 -DWITH_HDF5CXX=on

What operating system are you using? What package versions? Have you had success since you last posted? Please post any difficulties you’re having --that’s exactly what this forum is for. It’s very helpful for both the developers and other users to have these problems and their solutions posted online.

I never got it to install properly with MPI enabled. I am running Linux Mint 17.3 Rosa on a virtual machine for installation testing purposes, as this matches the OS that the terminal at GATech is running.

With the following command line argument:

make ~/channelflow/ -DCMAKE_BUILD_TYPE=release -DCMAKE_INSTALL_PREFIX=~/channelflow-2.0 -DWITH_HDF5CXX=on -DCMAKE_CXX_COMPILER=g++

With output:

– The C compiler identification is GNU 4.8.4
– The CXX compiler identification is GNU 4.8.4
– Check for working C compiler: /usr/bin/cc
– Check for working C compiler: /usr/bin/cc – works
– Detecting C compiler ABI info
– Detecting C compiler ABI info - done
– Detecting C compile features
– Detecting C compile features - done
– Check for working CXX compiler: /usr/bin/g++
– Check for working CXX compiler: /usr/bin/g++ – works
– Detecting CXX compiler ABI info
– Detecting CXX compiler ABI info - done
– Detecting CXX compile features
– Detecting CXX compile features - done

– configuring channelflow version 2.0.2
– Using GNU compiler options

– Starting system introspection
– Looking for drand48
– Looking for drand48 - found
– Looking for include file wordexp.h
– Looking for include file wordexp.h - found
– Found Eigen3: /usr/include/eigen3 (Required is at least version “2.91.0”)
– Found FFTW
– FFTW_LIBRARY=/usr/lib/x86_64-linux-gnu/libfftw3.so
– FFTW_OMP_LIBRARY=/usr/lib/x86_64-linux-gnu/libfftw3_omp.so
– FFTW_MPI_LIBRARY=/usr/lib/x86_64-linux-gnu/libfftw3_mpi.so
– Looking for include file fftw3.h
– Looking for include file fftw3.h - found
– Found MPI_C: /usr/lib/openmpi/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
– Found MPI_CXX: /usr/lib/openmpi/lib/libmpi_cxx.so;/usr/lib/openmpi/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
– MPI libs: /usr/lib/openmpi/lib/libmpi_cxx.so;/usr/lib/openmpi/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
– FFTW_MPI library: /usr/lib/x86_64-linux-gnu/libfftw3_mpi.so
CMake Error at CMakeLists.txt:120 (message):
Option USE_MPI is set and MPI is found, but the compiler (/usr/bin/g++)
failed to compile MPI code. To set another compiler (such as
/usr/bin/mpicxx), use cmake option -DCMAKE_CXX_COMPILER=<compiler> or set
environment variable CXX. If you do not want to use MPI, clean up and
rerun cmake with -DUSE_MPI=off.

– Configuring incomplete, errors occurred!
See also “/home/matt/build/CMakeFiles/CMakeOutput.log”.

I tried to update the version of g++ that came with the distribution but I think it doesn’t know of a repository other than the one with g++=4.8.4. Maybe I need to install new version of gcc/g++ from source? I try to install using executables to avoid this because I don’t know linux too well, but I don’t know why it doesn’t update to a newer version. Sorry for never getting back to this its just got lost.

-Matt

You just need to change that to -DCMAKE_CXX_COMPILER=/path/to/mpic++.

MPI compilation requires a special mpi-enabled wrapper for the compiler. This should be part of your open MPI package.

writing on phone, will check and edit as needed in the morning.

I got it working; thank you both for the help.

I had previously done just as John had said: using mpic++ as the compiler, but it didn’t work. The problem for me was that without,

-DCMAKE_INSTALL_PREFIX=/some/path/that/is/local/

it was trying to install to a place that I do not have permission to write to.

Cheers,

Matt

1 Like