Continued Issues with Intel2020-2
We now appear to have a highly reproducible test case for Intel 2020 Update 2.
This uses a 34 Mpc volume (2x the volume of the EAGLE-25) over two nodes of COSMA-7.
Example that hangs:
/cosma6/data/dp004/dc-borr1/XL_oldfe_wave_2/runs/HangExample_Run_2
Note that all of the 32 simulations hang on the step after writing a snapshot.
Code at: /cosma6/data/dp004/dc-borr1/XL_wave_1/swiftsim
with build build_c7
.
This is revision 02d319da, with very minor hotfixes (as these are production runs all designed to use the exact same version of the code).
Modules used:
module load intel_comp/2020-update2
module load intel_mpi/2020-update2
module load ucx/1.8.1
module load parmetis/4.0.3-64bit
module load parallel_hdf5/1.10.6
module load fftw/3.3.8cosma7 # On cosma 5 or 6 use fftw/3.3.8 On cosma 8, use fftw/3.3.8epyc
module load gsl/2.5
module load llvm/10.0.1 # Only necessary if wanting to use the code formatting tool
module load python/3.6.5
Configuration:
../configure --with-hydro=sphenix --with-kernel=quartic-spline --with-subgrid=EAGLE-XL --with-tbbmalloc --enable-ipo --disable-hand-vec
The exact same revision works perfectly with:
module load intel_comp/2018
module load intel_mpi/2018
module load ucx/1.8.1
module load parmetis/4.0.3-64bit
module load parallel_hdf5/1.10.3
module load fftw/3.3.8cosma7 # On cosma 5 or 6 use fftw/3.3.8 On cosma 8, use fftw/3.3.8epyc
module load gsl/2.5
module load llvm/10.0.1 # Only necessary if wanting to use the code formatting tool
module load python/3.6.5
Unfortunately with the whole 'thesis' thing I don't have the time to investigate this but worth keeping in mind.