Draft: Add host weights to METIS partitioning

assigned to @pdraper

mentioned in issue #856

added 1 commit

37032af6 - Match the short of the names when reading a host weight file. Should be OK and...

Compare with previous version

added 1 commit

f541c900 - Back to hiding METIS data types until needed. Keeps MPI simple.

Compare with previous version

Passing simple tests. Needs more work, but we can wait for better access to COSMA8 Rome for that.

Things to do, document, free all the allocated memory.

Longer term, can we derive weights from some runtime benchmarks?

Actually it seems that the ParMETIS version isn't stable and the host weights are not always respected. Suspect this is down to the similarity permutation as that will not respect the ParMETIS ordering.

added 1 commit

1b281584 - Don't allow ParMETIS solution permutation when using non-uniform host weights....

Compare with previous version

added 1 commit

7d4f9436 - Also correct the task timings, need to put an unbiased set of weights into...

Compare with previous version

Also needed to correct the task timings so that the imprint of the different nodes is removed. Not clear that this is done well as the improvements are still not great. Balance on my test seems stuck at 7%, a few percent better than we see for the single host type runs.

Some tidying up still probably needed (normalization is probably early now we use the corrections as given on the task times).

Seems that 7%, with a scale factor or 1.122, is about the best this technique can do. Having a single scaling factor is clearly too limiting and doesn't account for any non-scaling parts, such as MPI communication and probably anything that adds to the deadtime.