Skip to content

Draft: Refactoring of the time-step communication tasks

Matthieu Schaller requested to merge fewer_timestep_comms into master

Significant re-factoring of the way the time-step sizes are being exchanged.


  • A new top-level task collects the time-step sizes from the super level to the top-level. This was formerly done by making engine_collect_end_of_step() recurse.
  • The timestep, timestep_limiter, and timestep_sync tasks all unlock that top-level task.
  • engine_collect_end_of_step() now only loops (via threadpool) over the local top-level cells. No recursion any more.
  • For each pair of top-level cells in the proxies we construct a pair of send/recv comm tasks.
  • That comm task packs up the dt of the whole hierarchy sends it and unpacks the time-step sizes.
  • The top-level time-step collection task unlocks the send.
  • The individual per-species tend communication tasks that used to live at the super level are removed.
  • The second call to engine_launch() done every step to deal with the timestep limiter effect is removed (as it is now properly dealt with by the top-level task dependency)

This should help speed up the smallest steps by reducing the level of the plateau we usually see in the "main sequence" plots.

Edited by Matthieu Schaller

Merge request reports