Error running converted Gadget ICs
Chris is trying to run Gadget converted ICs, a dark matter only simulation and is running into some issues.
ICs file: ics_gadget.hdf5
Parameter file: cpower_cosmo.yml
Cmd line: ../swift -c -G -t 6 cpower_cosmo.yml
Config:
Version : 0.7.0
Revision: v0.7.0-302-gfa0354ba, Branch: master, Date: 2018-05-02 22:23:58 +0200
Webpage : www.swiftsim.com
Config. options: '--disable-doxygen-doc --enable-debug --enable-sanitizer'
Compiler: GCC, Version: 7.2.0
CFLAGS : '-g -O0 -gdwarf -fvar-tracking-assignments -O3 -fomit-frame-pointer -malign-double -fstrict-aliasing -ffast-math -funroll-loops -march=skylake -mavx2 -fsanitize=address -fno-omit-frame-pointer -pthread -Wall -Wextra -Wno-unused-parameter -Wshadow -Werror'
HDF5 library version: 1.10.1
FFTW library version: 3.x (details not available)
GSL library version: 2.4when I try with Chris's parameter file I get:
[00000.3] engine_init_particles: Setting particles to a valid state...
[00000.3] engine_init_particles: Computing initial gas densities.
[00000.4] engine.c:engine_addlink():131: Link table overflow.
[00000.4] engine.c:engine_addlink():131: Link table overflow.
[00000.4] engine.c:engine_addlink():131: Link table overflow.
[00000.4] engine.c:engine_addlink():131: Link table overflow.
Aborted (core dumped)I can get further by using a modified EAGLE_6 parameter file: eagle_6.yml
Cmd line: ../swift -c -G -t 1 eagle_6.yml
But it seg faults in the gravity (multipole_to_mesh_CIC):
[00000.0] main: CPU frequency used for tick conversion: 2400022702 Hz
[00000.0] main: Running on: James-Skylake-Platform
[00000.0] main: sizeof(part) is 128 bytes.
[00000.0] main: sizeof(xpart) is 64 bytes.
[00000.0] main: sizeof(spart) is 64 bytes.
[00000.0] main: sizeof(gpart) is 96 bytes.
[00000.0] main: sizeof(multipole) is 236 bytes.
[00000.0] main: sizeof(grav_tensor) is 228 bytes.
[00000.0] main: sizeof(task) is 64 bytes.
[00000.0] main: sizeof(cell) is 672 bytes.
[00000.0] main: Reading runtime parameters from file 'eagle_6.yml'
[00000.3] cosmology_print: Density parameters: [O_m, O_l, O_b, O_k, O_r] = [0.276000, 0.724000, 0.045500, 0.000000, 0.000000]
[00000.3] cosmology_print: Dark energy equation of state: w_0=-1.000000 w_a=0.000000
[00000.3] cosmology_print: Hubble constant: h = 0.703000, H_0 = 7.030001e+01 U_t^(-1)
[00000.3] cosmology_print: Hubble time: 1/H0 = 1.422475e-02 U_t
[00000.3] cosmology_print: Universe age at present day: 1.403547e-02 U_t
[00000.3] main: Reading ICs from file './ics_gadget.hdf5'
[00000.3] main: Cleaning up h-factors (h=0.703000)
[00000.3] io_read_unit_system: Reading IC units from ICs.
[00000.3] read_ic_single: Conversion needed from:
[00000.3] read_ic_single: (ICs) Unit system: U_M = 1.989000e+43 g.
[00000.3] read_ic_single: (ICs) Unit system: U_L = 3.085600e+24 cm.
[00000.3] read_ic_single: (ICs) Unit system: U_t = 1.000000e+05 s.
[00000.3] read_ic_single: (ICs) Unit system: U_I = 1.000000e+00 A.
[00000.3] read_ic_single: (ICs) Unit system: U_T = 1.000000e+00 K.
[00000.3] read_ic_single: to:
[00000.3] read_ic_single: (internal) Unit system: U_M = 1.989000e+43 g.
[00000.3] read_ic_single: (internal) Unit system: U_L = 3.085678e+24 cm.
[00000.3] read_ic_single: (internal) Unit system: U_t = 3.085678e+19 s.
[00000.3] read_ic_single: (internal) Unit system: U_I = 1.000000e+00 A.
[00000.3] read_ic_single: (internal) Unit system: U_T = 1.000000e+00 K.
[00000.3] main: Reading initial conditions took 50.207 ms.
[00000.3] main: Read 0 gas particles, 0 star particles and 262144 gparts from the ICs.
[00000.3] main: space_init took 2.719 ms.
[00000.3] main: space dimensions are [ 142.244 142.244 142.244 ].
[00000.3] main: space is periodic.
[00000.3] main: highest-level cell dimensions are [ 8 8 8 ].
[00000.3] main: 0 parts in 512 cells.
[00000.3] main: 262144 gparts in 512 cells.
[00000.3] main: 0 sparts in 512 cells.
[00000.3] main: maximum depth is 0.
[00000.3] potential_print_backend: External potential is 'No external potential'.
[00000.3] cooling_print_backend: Cooling function is 'No cooling'.
[00000.3] chemistry_print_backend: Chemistry function is 'No chemistry'.
[00000.3] engine_config: no processor affinity used
[00000.3] engine_policy: engine policies are [ 'steal' 'keep' 'numa affinity' 'self gravity' 'cosmological integration' ]
[00000.3] gravity_props_print: Self-gravity scheme: FMM-MM with m-poles of order 5
[00000.3] gravity_props_print: Self-gravity time integration: eta=0.0250
[00000.3] gravity_props_print: Self-gravity opening angle: theta=0.8500
[00000.3] gravity_props_print: Self-gravity comoving softening: epsilon=0.0081 (Plummer equivalent: 0.0027)
[00000.3] gravity_props_print: Self-gravity maximal physical softening: epsilon=0.0021 (Plummer equivalent: 0.0007)
[00000.3] gravity_props_print: Self-gravity mesh smoothing-scale: a_smooth=1.250000
[00000.3] gravity_props_print: Self-gravity tree cut-off: r_cut_max=4.500000
[00000.3] gravity_props_print: Self-gravity truncation cut-off: r_cut_min=0.100000
[00000.3] engine_config: Absolute minimal timestep size: 2.728254e-17
[00000.3] engine_config: Minimal timestep size (on time-line): 5.210156e-11
[00000.3] engine_config: Maximal timestep size (on time-line): 5.463244e-05
[00000.3] engine_config: Restarts will be dumped every 6.000000 hours
[00000.3] main: engine_init took 0.632 ms.
[00000.3] main: Running on 0 gas particles, 0 star particles and 262144 DM particles (262144 gravity particles)
[00000.3] main: from t=4.956e-05 until t=1.404e-02 with 1 threads and 1 queues (dt_min=1.000e-10, dt_max=1.000e-04)...
[00000.3] engine_init_particles: Setting particles to a valid state...
[00000.3] engine_init_particles: Computing initial gas densities.
[00000.4] engine_init_particles: Converting internal energy variable.
[00000.4] engine_init_particles: Running initial fake time-step.
# Step Time Scale-factor Redshift Time-step Time-bins Updates g-Updates s-Updates Wall-clock time [ms] Props
0 4.956107e-05 1.960780e-02 50.00011 0.000000e+00 1 56 0 262144 0 2578.564 1
=================================================================
==29825==ERROR: AddressSanitizer: heap-use-after-free on address 0x621000010cb0 at pc 0x00000055e56a bp 0x7f4904c30b10 sp 0x7f4904c30b00
READ of size 8 at 0x621000010cb0 thread T1
#0 0x55e569 in multipole_to_mesh_CIC /home/james/Work/SWIFT/master/swiftsim/src/runner_doiact_fft.c:93
#1 0x55e569 in runner_do_grav_fft /home/james/Work/SWIFT/master/swiftsim/src/runner_doiact_fft.c:215
#2 0x550aec in runner_main /home/james/Work/SWIFT/master/swiftsim/src/runner.c:2113
#3 0x7f490adcf6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
#4 0x7f490ab05b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)
0x621000010cb0 is located 1968 bytes inside of 4704-byte region [0x621000010500,0x621000011760)
freed by thread T1 here:
#0 0x7f490c97d7f8 in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xde7f8)
#1 0x5a2df5 in fftw_kernel_free (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2df5)
#2 0x5a2cee in fftw_ifree (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2cee)
#3 0x5a2d10 in fftw_ifree0 (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2d10)
#4 0x5a3098 in sgrow (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a3098)
#5 0x5a30e0 in register_solver (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a30e0)
#6 0x5a5bba in fftw_solver_register (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a5bba)
#7 0x621aaa in fftw_rdft_rank0_register (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x621aaa)
#8 0x740d56 in fftw_solvtab_exec (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x740d56)
#9 0x61a25a in fftw_rdft_conf_standard (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x61a25a)
#10 0x5a690d in fftw_configure_planner (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a690d)
#11 0x5a2c1f in fftw_the_planner (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2c1f)
#12 0x5a1e79 in fftw_mkapiplan (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a1e79)
#13 0x5a2acd in fftw_plan_many_dft_r2c (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2acd)
#14 0x5a280e in fftw_plan_dft_r2c (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a280e)
#15 0x5a27bc in fftw_plan_dft_r2c_3d (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a27bc)
#16 0x55c9da in runner_do_grav_fft /home/james/Work/SWIFT/master/swiftsim/src/runner_doiact_fft.c:207
#17 0x550aec in runner_main /home/james/Work/SWIFT/master/swiftsim/src/runner.c:2113
#18 0x7f490adcf6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
previously allocated by thread T1 here:
#0 0x7f490c97db90 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xdeb90)
#1 0x5a2dd3 in fftw_kernel_malloc (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2dd3)
#2 0x5a2cad in fftw_malloc_plain (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2cad)
#3 0x5a300f in sgrow (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a300f)
#4 0x5a30e0 in register_solver (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a30e0)
#5 0x5a5bba in fftw_solver_register (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a5bba)
#6 0x7482e1 in regone (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x7482e1)
#7 0x748388 in fftw_regsolver_ct_directw (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x748388)
#8 0x7453f3 in fftw_kdft_dit_register (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x7453f3)
#9 0x5e5e93 in fftw_codelet_t1_20 (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5e5e93)
#10 0x740d56 in fftw_solvtab_exec (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x740d56)
#11 0x5a81f2 in fftw_dft_conf_standard (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a81f2)
#12 0x5a6901 in fftw_configure_planner (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a6901)
#13 0x5a2c1f in fftw_the_planner (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2c1f)
#14 0x5a1e79 in fftw_mkapiplan (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a1e79)
#15 0x5a2acd in fftw_plan_many_dft_r2c (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a2acd)
#16 0x5a280e in fftw_plan_dft_r2c (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a280e)
#17 0x5a27bc in fftw_plan_dft_r2c_3d (/home/james/Work/SWIFT/master/swiftsim/examples/swift+0x5a27bc)
#18 0x55c9da in runner_do_grav_fft /home/james/Work/SWIFT/master/swiftsim/src/runner_doiact_fft.c:207
#19 0x550aec in runner_main /home/james/Work/SWIFT/master/swiftsim/src/runner.c:2113
#20 0x7f490adcf6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
Thread T1 created by T0 here:
#0 0x7f490c8d6d6f in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37d6f)
#1 0x4797c0 in engine_config /home/james/Work/SWIFT/master/swiftsim/src/engine.c:5822
#2 0x406ab9 in main /home/james/Work/SWIFT/master/swiftsim/examples/main.c:805
#3 0x7f490aa1f82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
SUMMARY: AddressSanitizer: heap-use-after-free /home/james/Work/SWIFT/master/swiftsim/src/runner_doiact_fft.c:93 in multipole_to_mesh_CIC
Shadow bytes around the buggy address:
0x0c427fffa140: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa150: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa160: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa170: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c427fffa190: fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd fd fd
0x0c427fffa1a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa1b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa1c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa1d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c427fffa1e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==29825==ABORTINGIt seems like rho on line 93 of runner_doiact_fft.c is freed before it should be, causing a "heap-use-after-free"
seg fault.
I will try and look into this further, but does anything look blatantly wrong?