Skip to content

WIP: Use ParMETIS to reduce the amount of particle movement

Peter W. Draper requested to merge parmetis into master

Uses ParMETIS to share repartitioning the cell graph across all the MPI nodes, that should scale better. ParMETIS also has methods that will refine an existing solution, not just create a new one. This should reduce the amount of particle movement.

As part of this work we are removing the use of the serial version of METIS and reducing the number of repartitioning techniques offered (these tend to give marginally worse solutions in tests).

Still to do:

  • see if it is possible to work out when a refinement gives a worse solution than a new partition. When that is true use a stochastic search to find the best one (see branch metis-rand). Old ivs code could give an estimate of similarity, based on cells or particles.

  • lots of testing, is the actual balance still good.

  • Merge groups of MPI calls to avoid overheads.

  • Think about making use of asynchronous MPI calls.

  • look at using the threadpool to speed up critical loops.

  • see if the initial partition can be better matched to the positions of particles (old ivs code).

Edited by Peter W. Draper

Merge request reports