Delayed foreign allocation

Fixes #456 (closed).

We don't blindly allocate memory for foreign particles based on the top-level cell content any more. The new strategy is to recurse once down the cells once all the tasks (including the local-foreign pairs and MPI comms) have been constructed. In this first pass we only go down to the level where we reach tasks (so it's rather quick). We use this to count the number of particles that we will need to allocate. We then do that and finally link the particle arrays to their cells in the same way as before, just starting from the super level instead of the top level.

This saves quite a bit of memory and is also marginally faster as we only recurse into the parts of the (foreign) tree that we need to.

Future improvements possibly include:

• Do the same for the stars once we a have a better star-over-MPI strategy.
• Use the threadpool to parallelize the recursion since it is embarassingly parallel. (But it's also fast so do we care?)

mentioned in issue #456 (closed)

Sorry to overfill your time @pdraper... I'll slow down a bit in the next few weeks.

added 1 commit

• 71c42df2 - Correct calculation of the memory allocated for foreign particles.

Compare with previous version

marked as a Work In Progress

changed the description

That should be WIP as there is a much better way of doing this.

added 2 commits

• c103f7e8 - Do not count the particles in the leaves when looking for the number of foreign…
• f80c65cf - Base the logic upon the MPI recv tasks instead of the hydro pair tasks.

Compare with previous version

unmarked as a Work In Progress

That is now much better and works as planned.

added 34 commits

• f80c65cf...aa863097 - 33 commits from branch master
• 2eb69dff - Merge branch 'master' into delayed_foreign_allocation

Compare with previous version

Thanks, this all looks to be working so will accept. You happy?

Yes, all good to go. Thanks!

merged

mentioned in commit 0d8f1852