diff --git a/doc/RTD/source/ParameterFiles/output_selection.rst b/doc/RTD/source/ParameterFiles/output_selection.rst index 3ec69485d175e9c295e9883125e920e687c66453..8100c97abdc8f2839cffa29c9683bcf68bea00f0 100644 --- a/doc/RTD/source/ParameterFiles/output_selection.rst +++ b/doc/RTD/source/ParameterFiles/output_selection.rst @@ -49,14 +49,32 @@ final time is specified in the list. Output Selection ~~~~~~~~~~~~~~~~ -With SWIFT, you can select the particle fields to output in snapshot -using the parameter file. In section ``SelectOutput``, you can remove -a field by adding a parameter formatted in the following way -``field_parttype`` where ``field`` is the name of the field that you -want to remove (e.g. ``Masses``) and ``parttype`` is the type of -particles that contains this field (e.g. ``Gas``, ``DM`` or ``Star``). -For a parameter, the only values accepted are 0 (skip this field when -writing) or 1 (default, do not skip this field when writing). By +Users can generate a ``yaml`` file containing all the possible fields +available for a given configuration of SWIFT by running +``./swift --output-params output.yml`` or equivalently ``./swift -o +output.yml``. The file generated contains the list of fields that a +simulation running with this config would output in each snapshot. It +also lists the description string of each field and the unit +conversion string to go from internal comoving units to physical +CGS. Entries in the file look like: + +.. code:: YAML + + SelectOutput: + # Particle Type Gas + Coordinates_Gas: 1 # Co-moving positions of the particles. ::: Conversion to physical CGS: a U_L [ cm ] + Velocities_Gas: 1 # Peculiar velocities of the stars. This is (a * dx/dt) where x is the co-moving positions of the particles. ::: Conversion to physical CGS: U_L U_t^-1 [ cm s^-1 ] + Masses_Gas: 1 # Masses of the particles. ::: Conversion to physical CGS: U_M [ g ] + SmoothingLengths_Gas: 1 # Co-moving smoothing lengths (FWHM of the kernel) of the particles. ::: Conversion to physical CGS: a U_L [ cm ] + +Users can select the particle fields to output in snapshot using the +YAML parameter file. In section ``SelectOutput``, users can demand to +remove a field by adding a parameter formatted in the following way +``field_parttype`` where ``field`` is the name of the field that is to +be removed (e.g. ``Masses``) and ``parttype`` is the type of particles +that contains this field (``Gas``, ``DM``, ``Stars`` or ``BH``). For +a parameter, the only values accepted are ``0`` (skip this field when +writing) or ``1`` (default, do not skip this field when writing). By default all fields are written. This field is mostly used to remove unnecessary output by listing them @@ -64,10 +82,12 @@ with 0's. A classic use-case for this feature is a DM-only simulation (pure n-body) where all particles have the same mass. Outputting the mass field in the snapshots results in extra i/o time and unnecessary waste of disk space. The corresponding section of the ``yaml`` -parameter file would look like this:: +parameter file would look like: +.. code:: YAML + SelectOutput: Masses_DM: 0 -You can generate a ``yaml`` file containing all the possible fields -available for a given configuration of SWIFT by running ``./swift --output-params output.yml``. +Entries can simply be copied from the ``output.yml`` generated by the +``-o`` runtime flag. diff --git a/doc/RTD/source/ParameterFiles/parameter_description.rst b/doc/RTD/source/ParameterFiles/parameter_description.rst index 93b85311b0d07d432ffdd7338bf68b7b3afe8cbe..08543367b0e6ba4361d025c3c2cfe93c9c7fbe9c 100644 --- a/doc/RTD/source/ParameterFiles/parameter_description.rst +++ b/doc/RTD/source/ParameterFiles/parameter_description.rst @@ -692,10 +692,12 @@ using the parameter: The default level of ``0`` implies no compression and values have to be in the range :math:`[0-9]`. This integer is passed to the i/o library and used for the -loss-less GZIP compression algorithm. Higher values imply higher compression but -also more time spent deflating and inflating the data. Note that up until HDF5 -1.10.x this option is not available when using the MPI-parallel version of the -i/o routines. +loss-less GZIP compression algorithm. The compression is applied to *all* the +fields in the snapshots. Higher values imply higher compression but also more +time spent deflating and inflating the data. When compression is switched on +the SHUFFLE filter is also applied to get higher compression rates. Note that up +until HDF5 1.10.x this option is not available when using the MPI-parallel +version of the i/o routines. Finally, it is possible to specify a different system of units for the snapshots than the one that was used internally by SWIFT. The format is identical to the diff --git a/doc/RTD/source/Snapshots/index.rst b/doc/RTD/source/Snapshots/index.rst index f1466eec18bf2625d1dfc3aff75f1e40a96b5ebc..3510e2e1369af067c99c3935fd622cdca65bc834 100644 --- a/doc/RTD/source/Snapshots/index.rst +++ b/doc/RTD/source/Snapshots/index.rst @@ -165,7 +165,20 @@ The last column in the table gives the ``enum`` value from ``part_type.h`` corresponding to a given entry in the files. Each group contains a series of arrays corresponding to each field of the -particles stored in the snapshots. +particles stored in the snapshots. The exact list of fields depends on what +compile time options were used and what module was activated. A full list can be +obtained by running SWIFT with the ``-o`` runtime option (See +:ref:`Output_selection_label` for details). Each field contains a short +description attribute giving a brief summary of what the quantity represents. + +All the individual arrays created by SWIFT have had the Fletcher 32 check-sum +filter applied by the HDF5 library when writing them. This means that any +eventual data corruption on the disks will be detected and reported by the +library when attempting to read the data. + +Additionally, some compression filter may have been applied to the fields. See +the :ref:`Parameters_snapshots` section of the parameter file description for +more details. Unit information for individual fields -------------------------------------- diff --git a/examples/parameter_example.yml b/examples/parameter_example.yml index e9963a631a8f703d6574bd76336a4adf18f87a30..ddec176922c7f6449acb571cb91be2d0893f7a5d 100644 --- a/examples/parameter_example.yml +++ b/examples/parameter_example.yml @@ -132,7 +132,7 @@ Snapshots: time_first: 0. # (Optional) Time of the first output if non-cosmological time-integration (in internal units) delta_time: 0.01 # Time difference between consecutive outputs (in internal units) invoke_stf: 0 # (Optional) Call VELOCIraptor every time a snapshot is written irrespective of the VELOCIraptor output strategy. - compression: 0 # (Optional) Set the level of compression of the HDF5 datasets [0-9]. 0 does no compression. + compression: 0 # (Optional) Set the level of GZIP compression of the HDF5 datasets [0-9]. 0 does no compression. The lossless compression is applied to *all* the fields. distributed: 0 # (Optional) When running over MPI, should each rank write a partial snapshot or do we want a single file? 1 implies one file per MPI rank. int_time_label_on: 0 # (Optional) Enable to label the snapshots using the time rounded to an integer (in internal units) UnitMass_in_cgs: 1 # (Optional) Unit system for the outputs (Grams)