List of local top-level cells
Thought about this when looking at the output of the very large runs. In a few places in the code we loop over the top-level cells and then recurse in them to gather some data or activate things. The typical example for this is cell_unskip().
When we run large simulations, we need a large number of top-level cells for better domain decomposition flexibility and good communications (see #366). This comes at the cost of spending more time looping over more top-level cells. For instance on 200 nodes with 80^3 top-level cells, you need around 100ms for the unskipping; this is unacceptably high.
Now, we do not need to loop over all top-level cells. In practice, we only need to loop over the top-level cells local to a given node or potentially to the ones that are in the direct neighbourhood of our local top-level cells. Referring to the case above, we only need to look at ~1/200 of the top-level cells, potentially speeding-up things massively.
Could we construct, at the time of redistribute, a list of all the local cells and then use this in the loop over cells ? That adds a level of indirection but it might help.