Io selection changes

Current status of the i/o selection changes. Related to #651 (closed).

Adds the ability to add a file select_output.yml where you can specify different output selection properties per snapshot, and the ability to then specify which snapshots should have which choice of SelectOutput in output_list.txt.

I also changed the name of outputlist.{c,h} to output_list.{c,h} to be consistent with the rest of SWIFT's naming scheme.

TODO:

• Extend ability from single_io.c to other I/O schemes
• Narrative documentation
• Fix the SelectOutput checking function to check whether this is valid metadata
• Fix the SelectOutput writer to use the new string "on" v.s. "off", instead of 0 and 1
• Resolve merge conflicts.

Closes #651 (closed), #670 (closed), #671 (closed)

doc/RTD/source/ParameterFiles/output_selection.rst

@@ -60,34 +60,89 @@ CGS. Entries in the file look like:
 
 .. code:: YAML
 
-   SelectOutput:
-     # Particle Type Gas
-     Coordinates_Gas:      1  # Co-moving positions of the particles. ::: Conversion to physical CGS: a U_L  [ cm ]
-     Velocities_Gas:       1  # Peculiar velocities of the stars. This is (a * dx/dt) where x is the co-moving positions of the particles. ::: Conversion to physical CGS: U_L U_t^-1  [ cm s^-1 ]
-     Masses_Gas:           1  # Masses of the particles. ::: Conversion to physical CGS: U_M  [ g ]
-     SmoothingLengths_Gas: 1  # Co-moving smoothing lengths (FWHM of the kernel) of the particles. ::: Conversion to physical CGS: a U_L  [ cm ]
-
-Users can select the particle fields to output in snapshot using the
-YAML parameter file.  In section ``SelectOutput``, users can demand to
-remove a field by adding a parameter formatted in the following way
-``field_parttype`` where ``field`` is the name of the field that is to
-be removed (e.g. ``Masses``) and ``parttype`` is the type of particles
-that contains this field (``Gas``, ``DM``, ``Stars`` or ``BH``).  For
-a parameter, the only values accepted are ``0`` (skip this field when
-writing) or ``1`` (default, do not skip this field when writing). By
-default all fields are written.
-
-This field is mostly used to remove unnecessary output by listing them
-with 0's. A classic use-case for this feature is a DM-only simulation
-(pure n-body) where all particles have the same mass. Outputting the
-mass field in the snapshots results in extra i/o time and unnecessary
-waste of disk space. The corresponding section of the ``yaml``
-parameter file would look like:
+  Default:
+    # Particle Type Gas
+    Coordinates_Gas: off  # Co-moving positions of the particles : a U_L  [ cm ]
+    Velocities_Gas: on  # Peculiar velocities of the stars. This is (a * dx/dt) where x is the co-moving positions of the particles : U_L U_t^-1  [ cm s^-1 ]
+    Masses_Gas: on  # Masses of the particles : U_M  [ g ]
+    SmoothingLengths_Gas: on  # Co-moving smoothing lengths (FWHM of the kernel) of the particles : a U_L  [ cm ]
+    ...
+
+Users can select the particle fields to output in snapshot using a (separate)
+YAML parameter file. By default, you can define a section `Default` at the
+top level of this file (in the exact same way as the file dumped by using the
+`-o` option in SWIFT). By default, all fields are written, but by using the
+"off" string, you can force the code to skip over unwanted outputs.
+
+You must then, in the regular SWIFT parameter file, select the following
+options:
 
 .. code:: YAML
-	  
-  SelectOutput:
-    Masses_DM:   0
+
+  Snapshots:
+    select_output_on: 1
+    select_output: your_select_output_yaml.yml
+
+This field is mostly used to remove unnecessary output by listing them with
+0's. A classic use-case for this feature is a DM-only simulation (pure
+n-body) where all particles have the same mass. Outputting the mass field in
+the snapshots results in extra i/o time and unnecessary waste of disk space.
+The corresponding section of the YAML file would look like:
+
+.. code:: YAML
+
+  Default:
+    Masses_DM: off
 
 Entries can simply be copied from the ``output.yml`` generated by the
 ``-o`` runtime flag. 
+
+
+Combining Output Lists and Output Selection
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It is possible to combine the behaviour of the output list and the select
+output file. To do so, you will need to enable both the ``select_output`` and
+``output_list`` options in your main ``parameter_file.yml`` as follows:
+
+.. code:: YAML
+
+  Snapshots:
+    output_list_on: 1
+    output_list: "output_list.txt"
+    select_output_on: 1
+    select_output: "select_output.yml"
+
+A typical use case for such a scenario is the dumping of 'snapshots' and
+so-called 'snipshots', containing less information than their full snapshot
+cousins. To do this, we will define two top-level sections in our
+``select_output.yml`` file as follows:
+
+.. code:: YAML
+
+  # Only turn off DM masses in snapshots, everything else is turned on
+  Snapshot:
+    Masses_DM: off
+
+  # Turn off lots of stuff in snipshots!
+  Snipshot:
+    Metal_Mass_Fractions_Gas: off
+    Element_Mass_Fractions_Gas: off
+    ...
+
+To then select which outputs are 'snapshots' and which are 'snipshots', you
+will need to add the ``Select Output`` column to the ``output_list.txt`` as
+follows::
+
+  # Redshift, Select Output
+  100.0, Snapshot
+  90.0, Snipshot
+  80.0, Snipshot
+  70.0, Snipshot
+  60.0, Snapshot
+  ...
+
+This will enable your simulation to perform partial dumps only at the outputs
+labelled as ``Snipshot``. The name of the output selection that corresponds
+to your choice in the output list will be written to the snapshot header as
+``Header/SelectOutput``.