Allow star formation when running over MPI
This extends the current version of the code by allowing star formation when running over MPI. This means:
- Creating space for the spare particles only for local + foreign cells and not all top-level cells,
- Sorting the spare particles only for local cells,
- Discarding the spare particles when redistributing.
- Fixing #560 (closed).
- Fix a memory leak in the freeing of stars' sort array that follows the creation of a star,
- Move the star formation task to the top-level since it does restructure the
spart
array in the whole cell, - Add time-step communication tasks for each particle type rather than one single bundled operation.
The final step (coming soon in a separate branch) is to allow both Star formation and feedback at the same time over MPI.
Merge request reports
Activity
Ah yes, sorry. The cooling needs to be on as well.
So:
./configure --with-subgrid=EAGLE --disable-hand-vec cd examples/EAGLE_low_z/EAGLE_6/ ../../Cooling/getEagleCoolingTable.sh mpirun -np 4 ../../swift_mpi --cosmology --hydro --self-gravity --stars --star-formation --cooling --threads=4 eagle_6.yml
added 1 commit
- 992f4f92 - Use a better value (the default one) for the gas fraction assumed by the EAGLE…
added 1 commit
- c59e68ee - Update the md5sum of the EAGLE low-z examples now that stars have been added to the ICs.
Thanks, thought I was missing a step, not all those... The bad news is that I see an issue with the debugging checks enabled:
7 1.276163e-02 0.9091649 0.0999104 1.635742e-07 43 43 24 24 0 76.118 0 8 1.276179e-02 0.9091755 0.0998976 1.635753e-07 43 46 2498 24160 17521 18323.580 0 [0000] [00102.4] engine_drift_all: Drifting all to a=9.091861e-01 [0003] [00102.5] drift.h:drift_spart():132: s-particle has not been drifted to the current time sp->ti_drift=140737488355328, c->ti_old=17592186044416, ti_current=158329674399744 application called MPI_Abort(MPI_COMM_WORLD, -1) - process 3
Same place twice in a row, so consistent.
Seems like you are a victim of #560 (closed). That's good as I was struggling to find a run where it happens below 1000s of steps... Now I can go and fix it more easily. That has nothing to do with MPI though.
With EAGLE_12 it runs for 13 steps:
12 1.276098e-02 0.9091226 0.0999616 4.089259e-08 41 43 3164 3166 2 222.398 0 13 1.276102e-02 0.9091253 0.0999584 4.089259e-08 41 41 22 22 0 89.392 0 [0003] [00114.0] drift.h:drift_spart():132: s-particle has not been drifted to the current time sp->ti_drift=52776558133248, c->ti_old=4398046511104, ti_current=61572651155456 application called MPI_Abort(MPI_COMM_WORLD, -1) - process 3
Think I'll wait for a fix before trying anything else.
added 1 commit
- b5206f3b - When forming a star in a cell that never had a star before, set the ti_old_part…
added 1 commit
- 55871447 - Prevent users from running with star-formation but without stars.
added 1 commit
- 86fbab05 - Undo unwanted change to the EAGLE-12 parameter file.
added 1 commit
- c57d8f39 - Allow the creation of hydro self-tasks in a cell that has only stars as they…
Ok. I think I am all done now with this one. I have added a fix for #560 (closed).
added 1 commit
- 87b84844 - Release the star formation lock before exiting early with a warning