WIP: Invoke HBT halo finder on the fly
This merge request allows running the HBT halo finder/merger tree builder on the fly in Swift simulations. It requires a modified version of HBT which provides a library interface similar to the one used for VELOCIraptor - see https://github.com/jchelly/HBTplus/tree/on-the-fly.
HBT must be compiled using cmake to generate the library interface:
cd HBTplus mkdir build cd build cmake .. -DHBT_INT8=ON -DHBT_REAL8=ON -DHBT_INPUT_INT8=ON make
This produces libHBT.so and libHBT.h in the build directory. To compile Swift:
../configure \ --enable-fof \ --with-hbt=/path/to/hbt/HBTplus/build/
HBT requires FoF groups as input, so Swift must be compiled with FoF enabled.
To run with HBT, the Swift .yml file must contain FoF and HBT sections:
FOF: basename: fof_output # Filename for the FOF outputs. min_group_size: 20 # The minimum no. of particles required for a group. linking_length_ratio: 0.2 # Linking length in units of the main inter-particle separation. seed_black_holes_enabled: 0 # Enable seeding of black holes in FoF groups HBT: config_file_name: ../HBT_SmallCosmoVol.conf # HBT config file basename: ./hbt/ # Where to write output scale_factor_first: 0.1 # Scale-factor of the first output delta_time: 1.1 # Delta log-a between outputs output_list_on: 1 # Whether to use output list file output_list: hbt_output_list.txt # Filename for output list keep_subhalos_in_memory: 0 # Whether to keep subhalos in memory between invocations # (0=off to save memory, 1=on to reduce I/O) Snapshots: invoke_hbt: 1 ...
and Swift must be run with the --fof and --hbt command line flags.
Several parameters in the HBT input file are ignored because the correct value is passed in from Swift:
- omega_m0, omega_lambda0
- Units: MassInMsunh, LengthInMpch, VelInKmS
Currently the softening is NOT passed in. In fact, we can only specify a single, fixed comoving softening for all particles in the input file. Would be better to set this automatically.
HBT needs the particle subhalo membership information from the previous snapshot to process the next one. Fortunately, it can read its own output so the outputs serve as restart files. Restarting a Swift+HBT run works as long as we don't delete the last HBT output before the restart file.
A possible enhancement would be to have HBT able to serialize the subhalo data to Swift's restart files. We could generate the halo catalogue and merger trees without ever writing any particle data to disk, except for the contents of the restart files. This would be useful if we want merger trees with good temporal resolution but we don't have much disk space for snapshots. Only works if we're keeping the subhalo data in memory between HBT calls.
If we don't want to keep the subhalo data in memory and we also don't want to keep all the snapshots on disk, then I think we would need to keep the most recent HBT output and the last output prior to each set of restart files.