Add new initial partition schemes and extend repartition ones.

Add new schemes for the initial partition of cells for MPI.

The initial cell distribution can now be based on a METIS partition of the cell graph using particle counts per cell and a simple unweighted version. The distribution can also be based on a vectorised selection of seed cells. This is guaranteed to work for all cases when the number of MPI nodes is less than the number of cells (the grid and METIS based ones do not guarantee this), so is the fallback method when the others fail.

Repartition is also extended to offer particle count based weights, as well the existing task times weights for edges and vertices, which is extended to allow weighting by edges only and a hybrid particle count vertices and times for edges.

A failure to repartition is handled by continuing with the existing partition.

The new options are handled by flags:

• -R - reparition type: 'n', 'b', 'v', 'e', 'x'
• -P - initial Partition type: 'g', 'm', 'w', 'v'

note the old '-g' flag is removed, use '-P g' or '-P g n n n' now.

The repartition types are:

• 'n' None, for completeness.
• 'b' both, task time weighted vertices and edges (as as current).
• 'v' vertex particle count weighted vertices, no edge weights.
• 'e' task time weighted edges, no vertex weights.
• 'x' task time weighted edges, particle count weighted vertices.

And the initial partition types:

• 'g' geometric grid partition (as now)
• 'm' METIS partition without any weights
• 'w' METIS partition with particle count weighted vertices
• 'v' vectorised cell array positions used as seeds for partition

Not giving any options just gives the existing behaviour, i.e. grid initial partition and task time weighted redistribution.

Title changed from WIP: add new initial partition schemes and extend repartition ones. to Add new initial partition schemes and extend repartition ones.

Reassigned to @nnrw56

Pedro initially assigning to you, please pass it to @matthieu if you're happy.

mentioned in issue #64 (closed)

Sorry for taking so long to look at this. The only thing that worries me a bit is that engine_repartition is a huge function and changing things in there may have unclear side effects. Would it be a lot of work to break out the different splitting approaches into separate functions, or at least the unique parts of each approach?

If it's too much work, then we can just leave it for some later stage. Reassigning to @matthieu, as I'm otherwise happy with it!

Reassigned to @matthieu

Title changed from Add new initial partition schemes and extend repartition ones. to WIP: Add new initial partition schemes and extend repartition ones.

I'll have a look at refactoring all the partitioning code into either more engine functions, or if I can, just get engine to call out for this service. The latter would be best I think.

WIP as there are now merge conflicts and no urgency so I may as well complete this.

Added 200 commits:

• 908355e5...e537eb76 - 197 commits from branch master
• 56fcc973 - Merge branch 'master' into initial_partitions
• 9b262cc3 - Handle drift and kick tasks in partition weights
• 93d2fa21 - Document parameter change in engine_split()

These commits merge in the multistep changes to master. Needed to change the command-line flags as there was a conflict with -e.

From your last two comments on that thread, are you now happy for me to merge it in or is it still WIP ?

Still WIP, I'm looking at what can refactored to tidy up the repartition function. I'll remove WIP when I think that is ready.

Added 79 commits:

• 93d2fa21...3b2b2be6 - 77 commits from branch master
• 13ee83ea - Refactor METIS cell graph initialisation code into one function.
• 92b5e9cf - Merge branch 'master' into initial_partitions

Added 1 commit:

• 24db049f - Reconfigure engine_repartition() into several functions so that the logic

Added 1 commit:

• b0e4a2a0 - Move all repartitioning code out of engine and into partition

Added 1 commit:

• 38e72a9c - Documentation fixes

Added 1 commit:

• 0666afd8 - Move initial partition code out of engine and into partition

Added 1 commit:

• 548403d9 - Documentation tweaks

Title changed from WIP: Add new initial partition schemes and extend repartition ones. to Add new initial partition schemes and extend repartition ones.

I've had a go at refactoring much of the partition code into a number of clearer functions and moved most of it out of engine.c into partition.c. I avoided that first time as I didn't want any MPI, but it seems impossible to avoid.

@nnrw56 you'd better have one last look at all this.