Negative wait
Here is a bug/feature encountered today while running with different values of space_splittask. When using a value of 250000. I get a negative wait for one task in the 0th time-step.
Test case: EAGLE_25
Parameter file is the default one with the following change:
# Parameters for the task scheduling
Scheduler:
cell_sub_size: 250000 # Command line is: swift -s -t 16 eagle_25.yml -n 4097 -v 1
I get the following output:
Welcome to the cosmological hydrodynamical code
______ _________________
/ ___/ | / / _/ ___/_ __/
\__ \| | /| / // // /_ / /
___/ /| |/ |/ // // __/ / /
/____/ |__/|__/___/_/ /_/
SPH With Inter-dependent Fine-grained Tasking
Version : 0.4.0
Revision: v0.4.0-62-g3306ffd4, Branch: master
Webpage : www.swiftsim.com
Compiler: GCC, Version: 4.8.1
HDF5 library version: 1.8.9
[00000.0] main: CPU frequency used for tick conversion: 2599981246 Hz
[00000.0] main: sizeof(struct part) is 128 bytes.
[00000.0] main: sizeof(struct xpart) is 32 bytes.
[00000.0] main: sizeof(struct gpart) is 96 bytes.
[00000.0] main: sizeof(struct task) is 64 bytes.
[00000.0] main: sizeof(struct cell) is 544 bytes.
[00000.0] main: Reading runtime parameters from file 'eagle_25_250k.yml'
[00000.0] main: Internal unit system: U_M = 1.989000e+43 g.
[00000.0] main: Internal unit system: U_L = 3.085678e+24 cm.
[00000.0] main: Internal unit system: U_t = 3.085678e+19 s.
[00000.0] main: Internal unit system: U_I = 1.000000e+00 A.
[00000.0] main: Internal unit system: U_T = 1.000000e+00 K.
[00000.0] phys_const_print: Gravitational constant = 4.302051e+01
[00000.0] phys_const_print: Speed of light = 2.997925e+05
[00000.0] phys_const_print: Planck constant = 9.787529e-02
[00000.0] phys_const_print: Boltzmann constant = 6.941420e-70
[00000.0] phys_const_print: Thomson cross-section = 6.986843e-74
[00000.0] phys_const_print: Electron-Volt = 8.055187e-66
[00000.0] phys_const_print: Year = 1.022690e-12
[00000.0] phys_const_print: Astronomical Unit = 4.848136e-12
[00000.0] phys_const_print: Parsec = 9.999999e-07
[00000.0] phys_const_print: Solar mass = 9.997486e-11
[00000.0] main: Reading ICs from file './EAGLE_ICs_25.hdf5'
[00000.0] read_ic_single: IC and internal units match. No conversion needed.
[00112.4] read_ic_single: Particle Type 4 not yet supported. Particles ignored
[00112.4] read_ic_single: Particle Type 5 not yet supported. Particles ignored
[00113.8] main: Reading initial conditions took 113829.171 ms.
[00114.4] main: Read 50726350 gas particles and 0 gparts from the ICs.
[00114.4] space_init: max_size set to 8000000, sub_size set to 250000, split_size set to 400
[00116.6] space_regrid: h_max is 6.610e-01 (cell_max=1.000e-01).
[00116.6] space_regrid: set cell dimensions to [ 12 12 12 ].
[00116.6] space_regrid: took 393.290 ms.
[00116.6] main: space_init took 2146.504 ms.
[00116.6] main: space dimensions are [ 16.942 16.942 16.942 ].
[00116.6] main: space is periodic.
[00116.6] main: highest-level cell dimensions are [ 12 12 12 ].
[00116.6] main: 50726350 parts in 1728 cells.
[00116.6] main: 0 gparts in 1728 cells.
[00116.6] main: maximum depth is 0.
[00116.6] main: map_cellcheck picked up 0 parts.
[00116.6] main: nr of cells at depth 0 is 1728.
[00116.6] engine_init: no processor affinity used
[00116.6] engine_policy: engine policies are [ steal keep numa_affinity hydro ]
[00116.6] hydro_props_print: Equation of state: Ideal gas.
[00116.6] hydro_props_print: Adiabatic index gamma: 1.666667.
[00116.6] hydro_props_print: Hydrodynamic scheme: Gadget-2 version of SPH (Springel 2005) in 3D.
[00116.6] hydro_props_print: Hydrodynamic kernel: Cubic spline (M4) with 48.00 +/- 0.10 neighbours (eta=1.234800).
[00116.6] hydro_props_print: Hydrodynamic integration: CFL parameter: 0.1000.
[00116.6] hydro_props_print: Hydrodynamic integration: Max change of volume: 2.00 (max|dlog(h)/dt|=0.231049).
[00116.6] engine_init: Absolute minimal timestep size: 3.725290e-11
[00116.6] engine_init: Minimal timestep size (on time-line): 7.450580e-11
[00116.6] engine_init: Maximal timestep size (on time-line): 7.812500e-05
[00116.6] engine_compute_next_snapshot_time: Next output time set to t=9.999999e-04.
[00116.6] engine_init: runner 0 using qid=0 no cpuid.
[00116.6] engine_init: runner 1 using qid=1 no cpuid.
[00116.6] engine_init: runner 2 using qid=2 no cpuid.
[00116.6] engine_init: runner 3 using qid=3 no cpuid.
[00116.6] engine_init: runner 4 using qid=4 no cpuid.
[00116.6] engine_init: runner 5 using qid=5 no cpuid.
[00116.6] engine_init: runner 6 using qid=6 no cpuid.
[00116.6] engine_init: runner 7 using qid=7 no cpuid.
[00116.6] engine_init: runner 8 using qid=8 no cpuid.
[00116.6] engine_init: runner 9 using qid=9 no cpuid.
[00116.6] engine_init: runner 10 using qid=10 no cpuid.
[00116.6] engine_init: runner 11 using qid=11 no cpuid.
[00116.6] engine_init: runner 12 using qid=12 no cpuid.
[00116.6] engine_init: runner 13 using qid=13 no cpuid.
[00116.6] engine_init: runner 14 using qid=14 no cpuid.
[00116.6] engine_init: runner 15 using qid=15 no cpuid.
[00116.7] main: engine_init took 102.826 ms.
[00116.7] main: Running on 50726350 gas particles and 0 DM particles from t=0.000e+00 until t=1.000e-02 with 16 threads and 16 queues (dt_min=1.000e-10, dt_max=1.000e-04)..\
.
[00116.7] engine_init_particles: Running initialisation fake time-step.
[00116.7] space_regrid: h_max is 4.979e-02 (cell_max=0.000e+00).
[00116.7] space_regrid: took 0.351 ms.
[00120.7] space_parts_sort: took 2766.690 ms.
[00120.7] space_gparts_sort: took 0.572 ms.
[00122.3] space_split: took 1581.249 ms.
[00122.3] space_rebuild: took 5595.663 ms.
[00127.3] engine_maketasks: took 5043.299 ms.
[00127.4] engine_marktasks: took 52.172 ms.
[00127.4] engine_print_task_counts: task counts are [ none=0 sort=431187 self=710360 pair=5971964 sub_self=25490 sub_pair=89204 init=25822 ghost=25822 extra_ghost=0 kick=25\
822 kick_fixdt=0 send=0 recv=0 grav_gather_m=0 grav_fft=0 grav_mm=0 grav_up=0 grav_external=0 skipped=0 ]
[00127.4] engine_print_task_counts: nr_parts = 50726350.
[00127.4] engine_print_task_counts: nr_gparts = 0.
[00127.4] engine_rebuild: took 10729.500 ms.
[00127.4] engine_prepare: took 10730.248 ms.
[00127.5] engine_marktasks: took 51.900 ms.
[00127.6] scheduler.c:scheduler_done():1209: Negative wait!