Add a synchronous redistribute option

Adds an option to perform the redistribute phase using synchronous MPI calls.

The redistribute phase potentially sends a lot of particle data between nodes and has been seen to fail due to memory limits for the whole machine, not just SWIFT. This because of the large cache usage by the transport layers. Switching this option on can help in these instances, but at the cost of a substantial slowdown.

Another one for a rainy day. This is the code that @jwillis is using with large FOF tests.

added 371 commits

• c3fa7c05...2cf116fc - 370 commits from branch master
• 55294df4 - Merge branch 'master' into redist-sync

Compare with previous version

mentioned in merge request !925 (merged)

added 85 commits

• 55294df4...bc5a6188 - 83 commits from branch master
• b52e6c84 - Merge branch 'master' into redist-sync
• 2ec982df - Move the updated synchronised redistribute function to the new engine_redistribute.c file.

Compare with previous version

I have now fixed the merge conflict that followed from the code refactoring.

Thanks.

mentioned in merge request !930 (merged)

That all ran fine for me. Happy for it to be merged.

Thanks, please go ahead.

merged

mentioned in commit 2cf7c82c