Only repartition when the previous step processed some large fraction of all the
particles, and then only when the loads between the ranks are out of balance.
This is for several reasons:
Repartitioning is expensive, so should only be done when necessary.
Frequent repartitioning with multi-dt is not necessary (for the EAGLE volumes anyway).
It is more representative to check the load balance when all tasks have been ran.
The load balance is determined from the user CPU time per step (including
the CPU time from all threads). We exclude the system time as that is not
down to processing and tends to even out the ranks artificially, much as
elapsed time does (since we wait for all the MPI tasks to come together).
The load imbalance allowed is determined by the parameter DomainDecomposition:trigger,
this can also be set to a number greater than one, in which case the old
repartitioning scheme of every 'trigger' steps will be used (previously trigger
was always 100).
Peter W. Draper authored