Logger fix intel

changed milestone to %Continuous Simulation Data Stream

added bug label

added 1 commit

4b22e641 - Format

It's not clear to me at all what this function does, and it looks like the documentation is not complete.

Can you expand the comment to explain what is going on here?

I have a hunch that the parameter flag should be of type enum logger_special_flags, but since I don't know what this function is doing, I cannot be sure. It may seem like a hassle to restrict the types, but it makes catching bugs related to API misunderstanding a lot easier.

The flag is not supposed to be an enum. We are unpacking the data from the logfile and are trying to extract the enum from the data. I am updating the documentation.

changed this line in version 3 of the diff

What is reader->log.log.map's type? Making it char * would make most of these changes superfluous.

It is a void *.

Can you change it to a char *? Or will this break other parts of the code?

I have changed it and it seems fine.

Cool, thanks! Then you shouldn't need the (char *) conversion here and in the rest of this MR.

changed this line in version 5 of the diff

added 1 commit

24fb517d - Update doc

Compare with previous version

added 1 commit

bd3c1989 - Change datatype mmap

Compare with previous version

Could you try running with GCC's undefined behaviour sanitizer switched on to check there are no other UBs in there? Configure with --enable-undefined-sanitizer.

You just need to compile, right? Everything seems fine, I just need to remove some (char *).

You need to compile and then run. This will check things at runtime.

added 1 commit

6394d483 - void -> char

Compare with previous version

added 1 commit

c9eb78e6 - missing void -> char

Compare with previous version

added 1 commit

7c5aa16f - Still missing void

Compare with previous version

Everything seems fine now. It compiles with intel and I was able to do 10 steps of SmallCosmoVolume_DM with the sanitizer.

Nice, LGTM!

While you are at it, can you run the same thing with just --enable-sanitizer all the way to the end please?

I have run with -n 10, it should do the same job, no?

That's the memory sanitizer this time. And no, please, do a full run and not an abrupt end. This is to test for leaks.

I am getting some leaks, but I think my logger is fine:

=================================================================
==192840==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 10240000 byte(s) in 10 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #7 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #8 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #9 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #10 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #11 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #12 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 9216000 byte(s) in 9 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #7 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #8 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #9 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #10 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #11 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #12 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 7168000 byte(s) in 7 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #7 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

Indirect leak of 4160000 byte(s) in 10 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #7 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #8 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #9 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #10 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #11 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #12 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 4096000 byte(s) in 4 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #6 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #7 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #8 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #9 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #10 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #11 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 3744000 byte(s) in 9 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #7 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #8 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #9 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #10 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #11 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #12 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 3072000 byte(s) in 3 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #6 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #7 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #8 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #9 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #10 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #11 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 3072000 byte(s) in 3 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #7 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

Indirect leak of 2912000 byte(s) in 7 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #7 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

Indirect leak of 1664000 byte(s) in 4 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #6 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #7 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #8 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #9 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #10 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #11 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 1248000 byte(s) in 3 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f08ae in threadpool_map (/media/data/Linux/data/swift_quick2/examples/swift+0x4f08ae)
    #6 0x53c1e4 in space_split (/media/data/Linux/data/swift_quick2/examples/swift+0x53c1e4)
    #7 0x5296b7 in space_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x5296b7)
    #8 0x443c19 in engine_rebuild (/media/data/Linux/data/swift_quick2/examples/swift+0x443c19)
    #9 0x449121 in engine_init_particles (/media/data/Linux/data/swift_quick2/examples/swift+0x449121)
    #10 0x410570 in main (/media/data/Linux/data/swift_quick2/examples/swift+0x410570)
    #11 0x7fd6867a90b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)

Indirect leak of 1248000 byte(s) in 3 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x530f8d in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x530f8d)
    #4 0x53c097 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c097)
    #5 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #6 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #7 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

Indirect leak of 1024000 byte(s) in 1 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x418135 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x418135)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #6 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

Indirect leak of 416000 byte(s) in 1 object(s) allocated from:
    #0 0x7fd688b7af15 in __interceptor_posix_memalign /home/loikki/hdd/tmp/gcc-9.2.0-bo6tpsymozxrcsr2zwnwc5zkt7q4tpup/spack-src/libsanitizer/asan/asan_malloc_linux.cc:217
    #1 0x4183a3 in space_getcells (/media/data/Linux/data/swift_quick2/examples/swift+0x4183a3)
    #2 0x53012b in space_split_recursive (/media/data/Linux/data/swift_quick2/examples/swift+0x53012b)
    #3 0x53c0e1 in space_split_mapper (/media/data/Linux/data/swift_quick2/examples/swift+0x53c0e1)
    #4 0x4f00b6 in threadpool_chomp (/media/data/Linux/data/swift_quick2/examples/swift+0x4f00b6)
    #5 0x4f022a in threadpool_runner (/media/data/Linux/data/swift_quick2/examples/swift+0x4f022a)
    #6 0x7fd688223608 in start_thread /build/glibc-eX1tMB/glibc-2.31/nptl/pthread_create.c:477

SUMMARY: AddressSanitizer: 53280000 byte(s) leaked in 74 allocation(s).

 INLINE static enum logger_special_flags logger_unpack_flags_and_data(
     uint32_t flag, int *data) {
   *data = flag & 0xFFFFFF;
   return flag >> (3 * 8);
   return (enum logger_special_flags)(flag >> (3 * 8));

   /* Get a few pointers. */
   const struct header *h = &reader->log.header;
   void *map = reader->log.log.map;
   char *map = (char *)reader->log.log.map;

Logger fix intel

Merge request reports

Activity