Fix various issues with MPI and partitioning

These are two bugs and one feature in the current handling of repartitioning and MPI tasks.

Extracted from current developments so they do not get lost.

Also needed for the best performance so far are:

• an increase the number of top-level cells.

• MPI tuning to improve the latency. For Intel MPI defining

  I_MPI_EAGER_THRESHOLD=4194304

  shows good improvements.

mentioned in issue #366

Added 12 commits:

• 83aa4241...635e7c31 - 11 commits from branch master
• d7a5efc0 - Merge remote-tracking branch 'origin/master' into mpi-fixes

Title changed from WIP: Fix various issues with MPI and partitioning to Fix various issues with MPI and partitioning

Added MPI enhancement labels

Milestone changed to Improved MPI scaling

Reassigned to @matthieu

These code changes are definitely useful, so let's merge them into master and start looking at improvements to that base.

Agreed! Thanks.

Status changed to merged

mentioned in commit dcb99d38

mentioned in merge request !441 (merged)