Sinks : Invalid particle position X/Y/Z for pi
In merge request !2192 (merged) we discovered that running the examples/SinkParticles/HomogeneousBox with master branch produces the following error:
[34120.8] runner_doiact_functions_stars.h:runner_do_sym_pair_stars_feedback():506: Invalid particle position in Y for pi (piy=5.813438e-01 ci->width[1]=2.906504e-01)This was a non-MPI test.
How to reproduce
Configure:
./configure --with-chemistry=GEAR_10 --with-cooling=grackle_0 --with-stars=GEAR --with-star-formation=GEAR --with-feedback=GEAR --with-sink=GEAR --with-kernel=wendland-C2 --with-grackle=$GRACKLE_ROOT --with-metis=$METIS_ROOT --with-parmetis=$PARMETIS_ROOT --enable-mpi-mesh-gravity --enable-debugging-checks --enable-debug Note: Probably that metis, parmetis and --enable-mpi-mesh-gravity are not useful to trigger the bug. To be checked.
Compile the code...
Run the examples/SinkParticles/HomogeneousBox example in debug mode (debug=1 ./run.sh) with master. The example uses the sinks and SF from sinks.
Sbatch script
#!/bin/bash
#SBATCH --job-name=HomoBox
#SBATCH --ntasks=1
#SBATCH --nodes=1 #-N
#SBATCH --cpus-per-task=128 #-c
#SBATCH --mail-user=darwin.roduit@epfl.ch
#SBATCH --mail-type=ALL
srun ~/program/swiftsim_sink_mpi/swift --hydro --self-gravity --sinks --stars --cooling --feedback --sync --limiter --threads=${SLURM_CPUS_PER_TASK} params.ymlParameter file
# Define the system of units to use internally.
InternalUnitSystem:
UnitMass_in_cgs: 1.9891E43 # 10^10 solar masses
UnitLength_in_cgs: 3.08567758E21 # 1 kpc
UnitVelocity_in_cgs: 1E5 # km/s
UnitCurrent_in_cgs: 1 # Amperes
UnitTemp_in_cgs: 1 # Kelvin
# Parameters for the self-gravity scheme
Gravity:
MAC: adaptive # Choice of mulitpole acceptance criterion: 'adaptive' OR 'geometric'.
epsilon_fmm: 0.001 # Tolerance parameter for the adaptive multipole acceptance criterion.
theta_cr: 0.7 # Opening angle for the purely gemoetric criterion.
eta: 0.025 # Constant dimensionless multiplier for time integration.
theta: 0.7 # Opening angle (Multipole acceptance criterion).
max_physical_baryon_softening: 0.01 # Physical softening length (in internal units)
mesh_side_length: 32
# Parameters governing the time integration (Set dt_min and dt_max to the same value for a fixed time-step run.)
TimeIntegration:
time_begin: 0. # The starting time of the simulation (in internal units).
time_end: 1.0 # The end time of the simulation (in internal units).
dt_min: 1e-12 # The minimal time-step size of the simulation (in internal units).
dt_max: 1e-2 # The maximal time-step size of the simulation (in internal units).
# Parameters governing the snapshots
Snapshots:
subdir: snap
basename: snapshot # Common part of the name of output files
time_first: 0. # (Optional) Time of the first output if non-cosmological time-integration (in internal units)
delta_time: 0.01 # Time difference between consecutive outputs (in internal units)
Restarts:
enable: 1
delta_hours: 6
Scheduler:
cell_extra_gparts: 10000 # (Optional) Number of spare sparts per top-level allocated at rebuild time for on-the-fly creation.
cell_extra_sinks: 10000 # (Optional) Number of spare sparts per top-level allocated at rebuild time for on-the-fly creation.
cell_extra_sparts: 10000 # (Optional) Number of spare sparts per top-level allocated at rebuild time for on-the-fly creation.
max_top_level_cells: 3 #
# dependency_graph_frequency: 1 # (Optional) Dumping frequency of the dependency graph. By default, writes only at the first step.
# dependency_graph_cell: 4063249 # hydro super # 4128783 hydro super #327696 grav super #1981349908 #393237 # 4128783 # grav super 393237
links_per_tasks: 30
# Parameters governing domain decomposition
DomainDecomposition:
initial_type: memory # (Optional) The initial decomposition strategy: "grid",
# "region", "memory", or "vectorized".
#initial_grid: [10,10,10] # (Optional) Grid sizes if the "grid" strategy is chosen.
synchronous: 0 # (Optional) Use synchronous MPI requests to redistribute, uses less system memory, but slower.
repartition_type: fullcosts # (Optional) The re-decomposition strategy, one of:
# "none", "fullcosts", "edgecosts", "memory" or
# "timecosts".
trigger: 0.05 # (Optional) Fractional (<1) CPU time difference between MPI ranks required to trigger a
# new decomposition, or number of steps (>1) between decompositions
minfrac: 0.9 # (Optional) Fractional of all particles that should be updated in previous step when
# using CPU time trigger
usemetis: 1 # Use serial METIS when ParMETIS is also available.
adaptive: 1 # Use adaptive repartition when ParMETIS is available, otherwise simple refinement.
itr: 100 # When adaptive defines the ratio of inter node communication time to data redistribution time, in the range 0.00001 to 10000000.0.
# Lower values give less data movement during redistributions, at the cost of global balance which may require more communication.
use_fixed_costs: 0 # If 1 then use any compiled in fixed costs for
# task weights in first repartition, if 0 only use task timings, if > 1 only use fixed costs, unless none are available.
# Parameters governing the conserved quantities statistics
Statistics:
delta_time: 1e-1 # Time between statistics output
time_first: 0. # (Optional) Time of the first stats output if non-cosmological time-integration (in internal units)
# Parameters related to the initial conditions
InitialConditions:
file_name: snapshot_0003restart.hdf5 #ICs_homogeneous_box.hdf5 # The file to read
periodic: 1 # Are we running with periodic ICs?
# Parameters for the hydrodynamics scheme
SPH:
resolution_eta: 1.2348 # Target smoothing length in units of the mean inter-particle separation (1.2348 == 48Ngbs with the cubic spline kernel).
CFL_condition: 0.1 # Courant-Friedrich-Levy condition for time integration.
#h_max: 0.2
minimal_temperature: 1
# Cooling with Grackle 3.0
GrackleCooling:
cloudy_table: CloudyData_UVB=HM2012_high_density.h5 # Name of the Cloudy Table (available on the grackle bitbucket repository)
with_UV_background: 0 # Enable or not the UV background
redshift: -1 # Redshift to use (-1 means time based redshift)
with_metal_cooling: 1 # Enable or not the metal cooling
provide_volumetric_heating_rates: 0 # User provide volumetric heating rates
provide_specific_heating_rates: 0 # User provide specific heating rates
self_shielding_method: -1 # Grackle (<= 3) or Gear self shielding method
self_shielding_threshold_atom_per_cm3: 0.007 # Required only with GEAR's self shielding. Density threshold of the self shielding
max_steps: 1000
convergence_limit: 1e-2
thermal_time_myr: 5
maximal_density_Hpcm3: 1e10
GEARChemistry:
initial_metallicity: -5
GEARFeedback:
supernovae_energy_erg: 1e51 # supernovae energy, used only for SNIa
supernovae_efficiency: 0.1 # supernovae energy efficiency, used for both SNIa and SNII
yields_table: POPIIsw.h5
yields_table_first_stars: POPIIsw.h5 #Put POPIIIsw.h5 when not testing
discrete_yields: 1
imf_transition_metallicity: -5 # Maximal metallicity ([Fe/H]) for a first star (0 to deactivate).ppp
#metallicity_max_first_stars: 1.766037e-8 # Maximal metallicity (in mass fraction) for a first star (-1 to deactivate). (10**-5)*0.0017660372 == XFe_sol* 10**[Fe/H]
elements: [Fe, Mg, O, C, Al, Ca, Ba, Zn, Eu] # Elements to read in the yields table. The number of element should be one less than the number of elements (N) requested during the configuration (--with-chemistry=GEAR_N).
GEARSink:
use_fixed_cut_off_radius: 1
cut_off_radius: 10e-3 # Cut off radius of all the sinks in internal units.
f_acc: 0.1
temperature_threshold_K: 3e4 # Max temperature (in K) for forming a sink when density_threshold_Hpcm3 <= density <= maximal_density_threshold_Hpcm3.
density_threshold_Hpcm3: 1e1 # Minimum gas density (in g/cm3) required to form a sink particle
maximal_density_threshold_Hpcm3: 1e2 # If the gas density exceeds this value (in g/cm3), a sink forms regardless of temperature if all other criteria are passed
stellar_particle_mass_Msun: 60 # Mass of the stellar particle representing the low mass stars, in solar mass
minimal_discrete_mass_Msun: 8 # Minimal mass of stars represented by discrete particles, in solar mass
stellar_particle_mass_first_stars_Msun: 60 # Mass of the stellar particle representing the low mass stars, in solar mass
minimal_discrete_mass_first_stars_Msun: 8 # Minimal mass of stars represented by discrete particles, in solar mass
star_spawning_sigma_factor: 0.5 # Factor to rescale the velocity dispersion of the stars when they are spawned. (Default: 0.2)
sink_formation_contracting_gas_criterion: 0 # (Optional) Activate the contracting gas check for sink formation. (Default: 1)
sink_formation_smoothing_length_criterion: 0 # (Optional) Activate the smoothing length check for sink formation. (Default: 1)
sink_formation_jeans_instability_criterion: 0 # (Optional) Activate the two Jeans instability checks for sink formation. (Default: 1)
sink_formation_bound_state_criterion: 0 # (Optional) Activate the bound state check for sink formation. (Default: 1)
sink_formation_overlapping_sink_criterion: 0 # (Optional) Activate the overlapping sink check for sink formation. (Default: 1)
disable_sink_formation: 0 # (Optional) Disable sink formation. (Default: 0)
disable_star_formation: 1
# Timesteps parameters
CFL_condition: 0.8 # Courant-Friedrich-Levy condition for time integration.
timestep_age_threshold_unlimited_Myr: 100. # (Optional) Age above which sinks have no time-step restriction any more (in Mega-years). Defaults to 0.
timestep_age_threshold_Myr: 25. # (Optional) Age at which sink switch from young to old for time-stepping purposes (in Mega-years). Defaults to FLT_MAX.
max_timestep_young_Myr: 2. # (Optional) Maximal time-step length of young sinks (in Mega-years). Defaults to FLT_MAX.
max_timestep_old_Myr: 5. # (Optional) Maximal time-step length of old sinks (in Mega-years). Defaults to FLT_MAX.
# n_IMF: 2. # (Optional) Number of times the IMF mass can be swallowed in a single timestep. (Default: FLTM_MAX)
GEARStarFormation:
star_formation_efficiency: 0.01 # star formation efficiency (c_*)
maximal_temperature_K: 3e4 # Upper limit to the temperature of a star forming particle
density_threshold_Hpcm3: 10 # Density threshold in Hydrogen atoms/cm3
n_stars_per_particle: 4
min_mass_frac: 0.5
GEARPressureFloor:
jeans_factor: 10
Stars:
timestep_age_threshold_unlimited_Myr: 30. # (Optional) Age above which sinks have no time-step restriction any more (in Mega-years). Defaults to 0.
timestep_age_threshold_Myr: 10. # (Optional) Age at which sink switch from young to old for time-stepping purposes (in Mega-years). Defaults to FLT_MAX.
max_timestep_young_Myr: 1 # (Optional) Maximal time-step length of young sinks (in Mega-years). Defaults to FLT_MAX.
max_timestep_old_Myr: 5 # (Optional) Maximal time-step length of old sinks (in Mega-years). Defaults to FLT_MAX.