Subcell recursion

Do you have a recommendation regarding how to test this? If there is a subtle typo in one of the sid vectors, it's going to be a nightmare to find.

Yes, although I admit I hadn't given this much thought before submitting the merge request...

I could turn on task debugging and run a few steps with/without this change, and then diff the list of active tasks. They should be identical.

This change affects task splitting, sub-cell interactions, and sub-cell task activation both in the initial step and subsequent steps. The above-mentioned test checks that the task splitting and activation is ok, but not the interactions themselves. However, since these functions now all use the same code, any bug that would affect the interactions should show up in the splitting and activation.

Oh, and any problem in the interactions will probably immediately affect the number of neighbours per particle and cause either the ghost tasks to fail, or the simulation results to differ significantly.

That did not break anything obvious in my tests. I'll let @pdraper have a look first though.

changed milestone to %Paris

added 455 commits

cd1eeeeb...ba95e9a1 - 454 commits from branch master
bb654d2b - Merge remote-tracking branch 'origin/master' into subcell_recursion

Compare with previous version

Seems to be OK, the only thing I worried about turned out to be true for master as well. It seems that the number of tasks we use for pair/grav and grav_mm varies by a small number, even in steps that interact all the particles. All the other tasks report the same numbers. @matthieu, that expected?

I do not understand the question, sorry.

OK, I'll try again. To check that all is well I ran with -y 1 and compared the numbers of tasks reported per step. I also later forced each step to interact all particles (fixed dt). The only counts of tasks that seemed different were pair/grav and grav_mm. For instance:

# task ntasks min max sum mean percent fixed_cost
           sort/none               64         0.0143        30.4741       172.8134         2.7002        11.7504      27002
           self/grav               56         0.0935        33.4274       404.1911         7.2177        27.4828      72176
           pair/density            36         0.0005         0.0503         0.1973         0.0055         0.0134         54
           pair/force              36         0.0048        26.7916        35.6932         0.9915         2.4269       9914
           pair/grav             8145         0.0004        25.9322       273.3548         0.0336        18.5866        335
       sub_self/density            37         0.0838        15.2475        27.7715         0.7506         1.8883       7505
       sub_self/force              37         0.0836        11.4077        29.2635         0.7909         1.9898       7909
       sub_pair/density           787         0.0002         0.4860         4.0865         0.0052         0.2779         51
       sub_pair/force             787         0.0002         1.8268        10.9916         0.0140         0.7474        139
      init_grav/none               33         0.0098         0.7697         5.5388         0.1678         0.3766       1678
          ghost/none               55         0.0067        16.1934        26.9452         0.4899         1.8321       4899
     drift_part/none              105         0.0205        37.1171       345.3972         3.2895        23.4851      32894
    drift_gpart/none               52         0.0115        25.0653        63.7225         1.2254         4.3328      12254
end_hydro_force/none               34         0.0032         0.0759         0.4190         0.0123         0.0285        123
          kick1/none               33         0.0022         0.1675         0.8643         0.0262         0.0588        261
          kick2/none               33         0.0048         0.2120         1.1770         0.0357         0.0800        356
       timestep/none               33         0.0115         0.2534         1.3342         0.0404         0.0907        404
grav_long_range/none               33         0.0876         0.3544         4.5742         0.1386         0.3110       1386
        grav_mm/none             3811         0.0004        24.0226        60.0118         0.0157         4.0805        157
      grav_down/none               33         0.0042         0.1199         0.6770         0.0205         0.0460        205
      grav_mesh/none               33         0.0057         0.2954         1.5164         0.0459         0.1031        459
 grav_end_force/none               33         0.0010         0.0256         0.1663         0.0050         0.0113         50

and

# task ntasks min max sum mean percent fixed_cost
           sort/none               64         0.0115        20.9215       137.4385         2.1475         9.1452      21474
           self/grav               56         0.0914        64.2106       478.5341         8.5453        31.8417      85452
           pair/density            36         0.0006         0.0299         0.1686         0.0047         0.0112         46
           pair/force              36         0.0038        12.9637        20.5768         0.5716         1.3692       5715
           pair/grav             8148         0.0004        25.1240       259.9774         0.0319        17.2989        319
       sub_self/density            37         0.0879        16.3348        28.9623         0.7828         1.9272       7827
       sub_self/force              37         0.1040         8.9915        27.5224         0.7438         1.8313       7438
       sub_pair/density           787         0.0002         0.3249         3.7855         0.0048         0.2519         48
       sub_pair/force             787         0.0002        15.0484        26.6731         0.0339         1.7748        338
      init_grav/none               33         0.0236         0.7775         7.1139         0.2156         0.4734       2155
          ghost/none               55         0.0148        20.9466        58.0642         1.0557         3.8636      10557
     drift_part/none              105         0.0230        54.5471       352.9905         3.3618        23.4880      33618
    drift_gpart/none               52         0.0140        10.4476        28.8557         0.5549         1.9201       5549
end_hydro_force/none               34         0.0036         0.0583         0.3804         0.0112         0.0253        111
          kick1/none               33         0.0028         0.1277         0.8125         0.0246         0.0541        246
          kick2/none               33         0.0053         0.1670         1.0947         0.0332         0.0728        331
       timestep/none               33         0.0109         0.2050         1.3093         0.0397         0.0871        396
grav_long_range/none               33         0.1115        16.3728        24.2555         0.7350         1.6140       7350
        grav_mm/none             3808         0.0004        16.0196        41.8377         0.0110         2.7839        109
      grav_down/none               33         0.0063         0.1322         0.7465         0.0226         0.0497        226
      grav_mesh/none               33         0.0057         0.2749         1.5847         0.0480         0.1054        480
 grav_end_force/none               33         0.0012         0.0283         0.1674         0.0051         0.0111         50

are from master and this branch. Wondered about the differences, but it turned out these are actually different between runs on either branch as well (and nothing to do with the changes here, since not subcell related). Just being paranoid!

Thanks. :)

Ok, so that's good for this branch. But bad in general... Is that after many steps?

It is usually obvious after one step, on different runs.

Thanks. The gravity task business relies more on floating-point results than the hydro tasks. I can see how some pair/grav get turned into grav_mm in some situations but on the next run because of rounding you don't. The result of the calculation should be almost identical though as the pair/grav will call the M-M calculation.

I see, yes the sum of these two tasks is the same.

OK, think this is all looking good now. Any reason not to merge?

Not that I can see.

Actually, we don't seem to make use of the sid item in the structure. Is there a reason to keep it?

I found a use at this line!

And here.

Did wonder about that myself and found these.

My bad. Looks like I searched the web-page with these files collapsed.

BTW, no need to enable task debugging to get these tasks files I compared. Since we keep the tic/toc values now they are always available in the form above, you just need to configure task debugging if you want the full dumps for processing into plots etc.

               scheduler_splittask_hydro(
                   scheduler_addtask(s, task_type_pair, t->subtype, 9, 0,
                                     ci->progeny[5], cj->progeny[6]),
                   s);
               scheduler_splittask_hydro(
                   scheduler_addtask(s, task_type_pair, t->subtype, 12, 0,
                                     ci->progeny[7], cj->progeny[6]),
                   s);
               break;
           } /* switch(sid) */
           /* Loop over the sub-cell pairs for the current sid and add new tasks
            * for them. */
           struct cell_split_pair *csp = &cell_split_pairs[sid];
           t->ci = ci->progeny[csp->pairs[0].pid];
           t->cj = cj->progeny[csp->pairs[0].pjd];
           t->flags = csp->pairs[0].sid;

                   scheduler_addtask(s, task_type_pair, t->subtype, 12, 0,
                                     ci->progeny[7], cj->progeny[6]),
                   s);
               break;
           } /* switch(sid) */
           /* Loop over the sub-cell pairs for the current sid and add new tasks
            * for them. */
           struct cell_split_pair *csp = &cell_split_pairs[sid];
           t->ci = ci->progeny[csp->pairs[0].pid];
           t->cj = cj->progeny[csp->pairs[0].pjd];
           t->flags = csp->pairs[0].sid;
           for (int k = 1; k < csp->count; k++) {
             scheduler_splittask_hydro(
                 scheduler_addtask(s, task_type_pair, t->subtype,
                                   csp->pairs[k].sid, 0,
                                   ci->progeny[csp->pairs[k].pid],

Subcell recursion

Merge request reports

Activity