[WIP] Speed-up cell_unskip_tasks

So this is a bit embarassing... Turns out that in cell_activate_subcell_tasks for pairs we were potentially recursing all the way down the tree before checking if anything is active at all.

Moved the check for activity to the top of the recursion, as it is done for the subcell-task interaction functions, this is already a bit faster on my laptop without breaking anything obvious. Should also improve parallel performance since this part of the code scales badly on the small timesteps.

@jwillis could you run this against master on you special node?

added 4 commits

• b4e55401...73a09c7a - 3 commits from branch master
• e8898f01 - Merge branch 'master' into task_activation

Compare with previous version

Tested and approved thus far. Yields correct results. Waiting for cosma to reappear to see whether it is beneficial on the cluster.

Now that cosma is back, @jwillis could you try this and the latest master on your node?

added 8 commits

• e8898f01...04146b23 - 7 commits from branch master
• 1ee48f09 - Merge branch 'master' into task_activation

Compare with previous version

Does not look like it makes any difference, I am afraid.

Wow, that's odd... Just to be sure, could you make the threadpool-task plots for the first 1000 steps on 16 cores with and without the change? Thanks!

Will do

here are the plots: http://icc.dur.ac.uk/~jlvc76/SWIFT/Task_activation/master_plots/ vs. http://icc.dur.ac.uk/~jlvc76/SWIFT/Task_activation/task_activation_plots/

closed

reopened

(track keeping: @jwillis is going to try this again on the sable node)

marked as a Work In Progress

changed the description

I merged the latest master into this branch and ran a scaling test.

There's no difference with master.

Cool, thanks! Can you also make the threadpool mapper plots, just to make sure it's doing what it should?

You should be able to view them here:

http://icc.dur.ac.uk/~rsrd54/task_activation/threadpool_plots/threadpool-index.html

using 16 threads.

Cheers! Do you have a recent master run to compare it to?

http://icc.dur.ac.uk/~rsrd54/master/threadpool_plots_master/threadpool-index.html

That should be the master as of yesterday

Thanks! But weird, the numbers in both plots are identical, to all digits... Can you double-check that they are indeed what you think they are?

Oh sorry, I copied the wrong file this should be correct now:

http://icc.dur.ac.uk/~rsrd54/master/threadpool_plots_master/threadpool-index.html

OK, this is still somehow messed-up... For any of the timesteps, you can check the total time for the runner_do_unskip mapper, and in this branch they're 1.5-2x faster, which is expected, but every other mapper type is slower.

If I have time I'll sync this branch to head to see if I've missed anything, as it's been a while since I filed this merge request.

mentioned in merge request !443 (merged)

added 127 commits

• 1ee48f09...856b9ea5 - 126 commits from branch master
• 1e2a25f0 - Merge branch 'master' into task_activation

Compare with previous version

I've ran master and task_activation again with only -Y 1.

Master:

http://icc.dur.ac.uk/~rsrd54/master/threadpool_only_plots_master/threadpool-index.html

Task_activation:

http://icc.dur.ac.uk/~rsrd54/task_activation/threadpool_only_plots/threadpool-index.html

One thing to keep in mind when looking at the plots is that they don't necessarily have the same x-axis range for both runs.

OK, I really don't get it... Now it's better in some cases and worse in others. @jwillis, can you run this through VTune and look at cell_activate_tasks and sub-functions to see where we win/lose? Thanks!

Allternatively, could you run the same branch 5 times and make independent sets of plots so that we can get an idea of the level of noise?

Or, could you run on one single socket to minimize NUMA effects?

Is it cell_activate_subcell_tasks?

master

task_activation

Yup, that's the one! So it did effectively get faster, by almost 2x, but why does everything else get slower?

Why is everything completely different?

How so?

Well the time spent in each of the functions I can see on the vTune outputs is different. Furthermore, they are not even in the same order. So it is not just an offset, it seems that we are stuck with some noise.

Because the function is faster though it moves down the profile so will be surrounded by different functions. And yeah, the order does change due to noise also.

So do we have any idea where the differences come from?

No clue at this stage. But I did not look at it in details since the unskip changes quite a bit in the gravity branch.

This will need serious reviewing following the merging of the gravity unskipping into master.

I keep this merge request open to remind us of the required work.

closed