Commit 0ed9010c authored by Matthieu Schaller's avatar Matthieu Schaller
Browse files

Clarifications in the documentation about compression and output lists.

parent ef62b521
......@@ -49,14 +49,32 @@ final time is specified in the list.
Output Selection
~~~~~~~~~~~~~~~~
With SWIFT, you can select the particle fields to output in snapshot
using the parameter file. In section ``SelectOutput``, you can remove
a field by adding a parameter formatted in the following way
``field_parttype`` where ``field`` is the name of the field that you
want to remove (e.g. ``Masses``) and ``parttype`` is the type of
particles that contains this field (e.g. ``Gas``, ``DM`` or ``Star``).
For a parameter, the only values accepted are 0 (skip this field when
writing) or 1 (default, do not skip this field when writing). By
Users can generate a ``yaml`` file containing all the possible fields
available for a given configuration of SWIFT by running
``./swift --output-params output.yml`` or equivalently ``./swift -o
output.yml``. The file generated contains the list of fields that a
simulation running with this config would output in each snapshot. It
also lists the description string of each field and the unit
conversion string to go from internal comoving units to physical
CGS. Entries in the file look like:
.. code:: YAML
SelectOutput:
# Particle Type Gas
Coordinates_Gas: 1 # Co-moving positions of the particles. ::: Conversion to physical CGS: a U_L [ cm ]
Velocities_Gas: 1 # Peculiar velocities of the stars. This is (a * dx/dt) where x is the co-moving positions of the particles. ::: Conversion to physical CGS: U_L U_t^-1 [ cm s^-1 ]
Masses_Gas: 1 # Masses of the particles. ::: Conversion to physical CGS: U_M [ g ]
SmoothingLengths_Gas: 1 # Co-moving smoothing lengths (FWHM of the kernel) of the particles. ::: Conversion to physical CGS: a U_L [ cm ]
Users can select the particle fields to output in snapshot using the
YAML parameter file. In section ``SelectOutput``, users can demand to
remove a field by adding a parameter formatted in the following way
``field_parttype`` where ``field`` is the name of the field that is to
be removed (e.g. ``Masses``) and ``parttype`` is the type of particles
that contains this field (``Gas``, ``DM``, ``Stars`` or ``BH``). For
a parameter, the only values accepted are ``0`` (skip this field when
writing) or ``1`` (default, do not skip this field when writing). By
default all fields are written.
This field is mostly used to remove unnecessary output by listing them
......@@ -64,10 +82,12 @@ with 0's. A classic use-case for this feature is a DM-only simulation
(pure n-body) where all particles have the same mass. Outputting the
mass field in the snapshots results in extra i/o time and unnecessary
waste of disk space. The corresponding section of the ``yaml``
parameter file would look like this::
parameter file would look like:
.. code:: YAML
SelectOutput:
Masses_DM: 0
You can generate a ``yaml`` file containing all the possible fields
available for a given configuration of SWIFT by running ``./swift --output-params output.yml``.
Entries can simply be copied from the ``output.yml`` generated by the
``-o`` runtime flag.
......@@ -692,10 +692,12 @@ using the parameter:
The default level of ``0`` implies no compression and values have to be in the
range :math:`[0-9]`. This integer is passed to the i/o library and used for the
loss-less GZIP compression algorithm. Higher values imply higher compression but
also more time spent deflating and inflating the data. Note that up until HDF5
1.10.x this option is not available when using the MPI-parallel version of the
i/o routines.
loss-less GZIP compression algorithm. The compression is applied to *all* the
fields in the snapshots. Higher values imply higher compression but also more
time spent deflating and inflating the data. When compression is switched on
the SHUFFLE filter is also applied to get higher compression rates. Note that up
until HDF5 1.10.x this option is not available when using the MPI-parallel
version of the i/o routines.
Finally, it is possible to specify a different system of units for the snapshots
than the one that was used internally by SWIFT. The format is identical to the
......
......@@ -165,7 +165,20 @@ The last column in the table gives the ``enum`` value from ``part_type.h``
corresponding to a given entry in the files.
Each group contains a series of arrays corresponding to each field of the
particles stored in the snapshots.
particles stored in the snapshots. The exact list of fields depends on what
compile time options were used and what module was activated. A full list can be
obtained by running SWIFT with the ``-o`` runtime option (See
:ref:`Output_selection_label` for details). Each field contains a short
description attribute giving a brief summary of what the quantity represents.
All the individual arrays created by SWIFT have had the Fletcher 32 check-sum
filter applied by the HDF5 library when writing them. This means that any
eventual data corruption on the disks will be detected and reported by the
library when attempting to read the data.
Additionally, some compression filter may have been applied to the fields. See
the :ref:`Parameters_snapshots` section of the parameter file description for
more details.
Unit information for individual fields
--------------------------------------
......
......@@ -132,7 +132,7 @@ Snapshots:
time_first: 0. # (Optional) Time of the first output if non-cosmological time-integration (in internal units)
delta_time: 0.01 # Time difference between consecutive outputs (in internal units)
invoke_stf: 0 # (Optional) Call VELOCIraptor every time a snapshot is written irrespective of the VELOCIraptor output strategy.
compression: 0 # (Optional) Set the level of compression of the HDF5 datasets [0-9]. 0 does no compression.
compression: 0 # (Optional) Set the level of GZIP compression of the HDF5 datasets [0-9]. 0 does no compression. The lossless compression is applied to *all* the fields.
distributed: 0 # (Optional) When running over MPI, should each rank write a partial snapshot or do we want a single file? 1 implies one file per MPI rank.
int_time_label_on: 0 # (Optional) Enable to label the snapshots using the time rounded to an integer (in internal units)
UnitMass_in_cgs: 1 # (Optional) Unit system for the outputs (Grams)
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment