Clarifications in the documentation about compression and output lists.

doc/RTD/source/ParameterFiles/output_selection.rst

@@ -49,14 +49,32 @@ final time is specified in the list.
 Output Selection
 ~~~~~~~~~~~~~~~~
 
-With SWIFT, you can select the particle fields to output in snapshot
-using the parameter file.  In section ``SelectOutput``, you can remove
-a field by adding a parameter formatted in the following way
-``field_parttype`` where ``field`` is the name of the field that you
-want to remove (e.g. ``Masses``) and ``parttype`` is the type of
-particles that contains this field (e.g. ``Gas``, ``DM`` or ``Star``).
-For a parameter, the only values accepted are 0 (skip this field when
-writing) or 1 (default, do not skip this field when writing). By
+Users can generate a ``yaml`` file containing all the possible fields
+available for a given configuration of SWIFT by running
+``./swift --output-params output.yml`` or equivalently ``./swift -o
+output.yml``. The file generated contains the list of fields that a
+simulation running with this config would output in each snapshot. It
+also lists the description string of each field and the unit
+conversion string to go from internal comoving units to physical
+CGS. Entries in the file look like:
+
+.. code:: YAML
+
+   SelectOutput:
+     # Particle Type Gas
+     Coordinates_Gas:      1  # Co-moving positions of the particles. ::: Conversion to physical CGS: a U_L  [ cm ]
+     Velocities_Gas:       1  # Peculiar velocities of the stars. This is (a * dx/dt) where x is the co-moving positions of the particles. ::: Conversion to physical CGS: U_L U_t^-1  [ cm s^-1 ]
+     Masses_Gas:           1  # Masses of the particles. ::: Conversion to physical CGS: U_M  [ g ]
+     SmoothingLengths_Gas: 1  # Co-moving smoothing lengths (FWHM of the kernel) of the particles. ::: Conversion to physical CGS: a U_L  [ cm ]
+
+Users can select the particle fields to output in snapshot using the
+YAML parameter file.  In section ``SelectOutput``, users can demand to
+remove a field by adding a parameter formatted in the following way
+``field_parttype`` where ``field`` is the name of the field that is to
+be removed (e.g. ``Masses``) and ``parttype`` is the type of particles
+that contains this field (``Gas``, ``DM``, ``Stars`` or ``BH``).  For
+a parameter, the only values accepted are ``0`` (skip this field when
+writing) or ``1`` (default, do not skip this field when writing). By
 default all fields are written.
 
 This field is mostly used to remove unnecessary output by listing them
@@ -64,10 +82,12 @@ with 0's. A classic use-case for this feature is a DM-only simulation
 (pure n-body) where all particles have the same mass. Outputting the
 mass field in the snapshots results in extra i/o time and unnecessary
 waste of disk space. The corresponding section of the ``yaml``
-parameter file would look like this::
+parameter file would look like:
 
+.. code:: YAML
+	  
   SelectOutput:
     Masses_DM:   0
 
-You can generate a ``yaml`` file containing all the possible fields
-available for a given configuration of SWIFT by running ``./swift --output-params output.yml``.
+Entries can simply be copied from the ``output.yml`` generated by the
+``-o`` runtime flag. 

doc/RTD/source/ParameterFiles/parameter_description.rst

@@ -692,10 +692,12 @@ using the parameter:
 
 The default level of ``0`` implies no compression and values have to be in the
 range :math:`[0-9]`. This integer is passed to the i/o library and used for the
-loss-less GZIP compression algorithm. Higher values imply higher compression but
-also more time spent deflating and inflating the data. Note that up until HDF5
-1.10.x this option is not available when using the MPI-parallel version of the
-i/o routines.
+loss-less GZIP compression algorithm. The compression is applied to *all* the
+fields in the snapshots. Higher values imply higher compression but also more
+time spent deflating and inflating the data.  When compression is switched on
+the SHUFFLE filter is also applied to get higher compression rates. Note that up
+until HDF5 1.10.x this option is not available when using the MPI-parallel
+version of the i/o routines.
 
 Finally, it is possible to specify a different system of units for the snapshots
 than the one that was used internally by SWIFT. The format is identical to the

doc/RTD/source/Snapshots/index.rst

@@ -165,7 +165,20 @@ The last column in the table gives the ``enum`` value from ``part_type.h``
 corresponding to a given entry in the files.
 
 Each group contains a series of arrays corresponding to each field of the
-particles stored in the snapshots.
+particles stored in the snapshots. The exact list of fields depends on what
+compile time options were used and what module was activated. A full list can be
+obtained by running SWIFT with the ``-o`` runtime option (See
+:ref:`Output_selection_label` for details). Each field contains a short
+description attribute giving a brief summary of what the quantity represents.
+
+All the individual arrays created by SWIFT have had the Fletcher 32 check-sum
+filter applied by the HDF5 library when writing them. This means that any
+eventual data corruption on the disks will be detected and reported by the
+library when attempting to read the data.
+
+Additionally, some compression filter may have been applied to the fields. See
+the :ref:`Parameters_snapshots` section of the parameter file description for
+more details.
 
 Unit information for individual fields
 --------------------------------------

examples/parameter_example.yml

@@ -132,7 +132,7 @@ Snapshots:
   time_first: 0.          # (Optional) Time of the first output if non-cosmological time-integration (in internal units)
   delta_time: 0.01        # Time difference between consecutive outputs (in internal units)
   invoke_stf: 0           # (Optional) Call VELOCIraptor every time a snapshot is written irrespective of the VELOCIraptor output strategy.
-  compression: 0          # (Optional) Set the level of compression of the HDF5 datasets [0-9]. 0 does no compression.
+  compression: 0          # (Optional) Set the level of GZIP compression of the HDF5 datasets [0-9]. 0 does no compression. The lossless compression is applied to *all* the fields.
   distributed: 0          # (Optional) When running over MPI, should each rank write a partial snapshot or do we want a single file? 1 implies one file per MPI rank.
   int_time_label_on:   0  # (Optional) Enable to label the snapshots using the time rounded to an integer (in internal units)
   UnitMass_in_cgs:     1  # (Optional) Unit system for the outputs (Grams)