Split the star formation task into two parts

Split the star formation task into two parts:

• The star formation bit
• The re-sorting bit.

The re-sorting bit is only relevant for the local stars-related tasks. So by doing so, we can move the communication of the star counts earlier into the task graph. This may help with #584.

I will work on improving the re-sorting bit next in a separate branch as we do not really need to re-sort everything. This will also help speed things up.

assigned to @pdraper

added 1 commit

• d4868df0 - Documented the new task.

Compare with previous version

changed the description

added 1 commit

• c0d792d8 - Update the timers for the new stars re-sorting task.

Compare with previous version

I'm running a job for these updates and it is looking hopeful so far -- star formation and sorts are running and the total times are smaller than before, but it is going to take me a while longer to get to the interesting parts and make the task plots. Hopefully tomorrow.

Ah, excellent news. Something to look forward to eventually.

Well, it works and here are some plots to consider as to why it may not have helped that much (I have the task plots as well, but you cannot compare directly to the previous run, so not very useful, anyway...):

Number of star_formation tasks per step:

Similar for the two runs, old is master, new this branch. Now minimum time in feedback task:

Better, now maxtime:

Wow! But we have a new task type, resort, so if we add the maximum time for that as well:

Oh dear, so the time is smaller in resort, but not by much as we evolve. Runs were for full physics (as in #584) and ran for most of a day.

So I guess we accept this?

Not quite sure I understand the plots but if it's faster then I like it. Not sure what you mean by feedback here. It should be the star formation task that has changed.

I have a follow up project to split the new task into 8 smaller pieces that should help with scalability.

Sorry, one too many checks running. Just checked and this is star_formation! In fact it is no faster than before, if you sum the two tasks you get much the same time, no surprise. So the gain is probably some decrease in dead time, but if we could repeat things exactly I expect it would be a small effect.

OK, going to accept this. Ready for you as well?

Yes, all good, thanks.

merged

mentioned in commit 027223b6

Failed to merge this earlier as the squash is dirty. Sadly it was quiet about that so I assumed it had worked. Something to watch out for in the future.

mentioned in merge request !860 (closed)