Drift all particles before repartitioning

Drifting them afterwards can result in some cells having inconsistent particle lists, so this must be done first. This fixes the crashes I was seeing in MPI runs of the SodShock with 4x12 cores.

@pedro. Does this sense to you? Not quite sure I understand why we occasionally have particle lists using previously freed memory at this stage.

Shouldn't we drift before rebuilding as well as before repartitioning ? In non-MPI mode we don't repartition but we still need to drift all particles to the current time before rebuilding the mesh, no ?

Title changed from Drift all particles before repartitioning to WEP: Drift all particles before repartitioning

Title changed from WEP: Drift all particles before repartitioning to WIP: Drift all particles before repartitioning

Just saw a couple of line of debugging slipped through, WIP until they are removed.

Added 1 commit:

• b7322b75 - Revert debugging hacks

Added 1 commit:

• cc6a8d95 - Re-instate drifting before rebuilding as that is needed when not drifting all

Hmm, OK, so we need to drift before rebuilding (when not drifting all), but cannot drift after repartitioning before rebuilding. Nice.

This new version re-instates the drift before rebuild, but only if needed, so not after a repartition, not when the policy is drift_all and not before time has begun. Hopefully that works and passes the sniff test.

We need to drift before a rebuild or before a repartition.

When not doing MPI, we don't have repartitions so we only need to take care of rebuilds. With MPI, a repartition triggers a rebuild. Can we have a rebuild without a repartition ?

Yes, in non-repartition steps the repartition type is set to REPART_NONE, so the drift before the rebuild should be triggered as you had it, i.e. every step with a rebuild.

I was considering wrapping all of the drifting into a function engine_drift() which could time itself when running with -v 1 in the same way that we time engine_prepare() and engine_marktasks(). What do you think ?

Apart from this, is it still WIP ?

Title changed from WIP: Drift all particles before repartitioning to Drift all particles before repartitioning

All seems to be doing what is needed now, so no longer WIP. Yes, it would be good to see timings as before, so why not.

Added 1 commit:

• 4d965cd1 - Add engine_drift() function so that we can time drifts

OK, done that as well.

Thanks, you beat me to it...

Added 1 commit:

• e41ae208 - Added a few timers that get printed to the screen when running verbosly

I have added a few extra timers to help us single out the bottlenecks. Feel free to remove them if you think it's too much.

Other than that I think this is good to go.

Added 44 commits:

• e41ae208...2ec778b8 - 43 commits from branch master
• 05cf40a0 - Merge branch 'master' into drift-fixes

Ok, fixed the conflict.