EAGLE_25x3 redistribute is very slow
When running the Dirac benchmark it seems that we spend a lot of time in engine_do_redistribute
(10 - 24 minutes and still increasing when test stopped). This may mean we need to increase the
buffer size for distributing particles or there is another issue, why is the time increasing
for instance. Using module:
swift/c5/intel/intelmpi/5.1.3
and configure:
./configure --with-metis
on COSMA6 running on 8 nodes, with non-standard parameters:
InitialConditions:
replicate: 3
Scheduler:
max_top_level_cells: 30
and submission script (SLURM):
#!/bin/bash -l
#
# Batch script for bash users
#
#SBATCH --ntasks-per-node 1
#SBATCH --nodes 8
#SBATCH -J SWIFT-mpi-test
#SBATCH -o job.%J.dump
#SBATCH -e job.%J.err
#SBATCH -p cosma6
#SBATCH -A dp004
#SBATCH --exclusive
##SBATCH -t 1440
#SBATCH -t 600
module purge
module load swift
module load swift/c5/intel/intelmpi/5.1.3
unset I_MPI_HYDRA_BOOTSTRAP
mpirun -np 8 ../swift_mpi -a -t 16 -s eagle_25.yml
echo "Job done, info follows..."
sacct -j $SLURM_JOBID --format=JobID,JobName,Partition,MaxRSS,Elapsed,ExitCode
exit