Add option for particle type level default output behaviour to new output list scheme
This small update adds the possibility to define a default output policy for each particle type. This is done through the (fake) field 'Standard' in the output selection YAML file (e.g. Standard_DM: off
). This is exactly equivalent to switching off each individual particle field for this type, but (a) involves less work, and (b) is robust to the addition of new output fields after an output selection YAML file has been created.
@jborrow -- Could you please also have a look at this, and check whether it looks ok?
Implements #680 (closed).
Merge request reports
Activity
We had planned to implement something similar,
All_X
, to prevent the writing of the particle group completely: #680 (closed). Let me think about whether this is preferrable - it does lead to, sometimes, incorrect values ofNumPart_ThisFile
and the dumping of unecessary cell metadata (which can be quite large relative to e.g. the black holes).This or #680 (closed) are quite similar in spirit.
I would worry here that the
Default
in the name is similar to theDefault
used for the case where no output selection is used.Also, indeed, it would be good to adjust the meta-data to save up a bit of space but also prevent problems with tools reading the data. For instance, the number of particles in the header should reflect the fact that a whole type of particles was cancelled.
Ah, I hadn't seen #680 (closed)... A few thoughts:
(i) I have no strong feelings whatsoever about the naming. Agreed with Matthieu that the double-use of
Default
may not be the most optimal choice.(ii) I think one difference to #680 (closed) is that this explicitly does not override individual field selectors. It is meant only as a default setting; this would also make it easy to, say, only write gas particle coordinates (set
Default_Gas: off
,Coordinates_Gas: on
).(iii) Agreed,
NumPart_ThisFile
should be adjusted and the cell metadata reduced appropriately whenever all output fields are cancelled. I guess the question is how do we know that (in this approach) there is no additionalxxx_Gas: on
or that (in the current master) no field name has been omitted?Edited by Yannick BahéI see, this is not a full overwrite but changes the default assumed behaviour from "on" to "off". Users can nevertheless re-activate a specific field. That has the drawback of working out whether anything of a given particle type is written a lot harder.
So, the question is whether this is a feature we want or is the simpler full-field on/off of #680 (closed) enough for normal use cases?
Thanks Josh! A couple of consistency questions:
(i) Are the entries from the new output selection YAML file tested for consistency somewhere?
(ii) In
io_check_output_fields()
, the code currently raises an error if there is aSelectOutput:
section in the (main) YAML file; however the comment (and subsequentcontinue;
) suggest that this should maybe just have been a warning...?Edited by Yannick BahéSorry, ignore point (i) above. Obviously the whole point of this function is now to check this YAML file, rather than the standard YAML file... I guess this also means in (ii) above, the error should be an error (so no need for
continue;
afterwards).I would think it is then the most straightforward option to just integrate the test for 'particle type is completely disabled' directly into
io_check_output_fields()
. My suggestion would be to make this function add one new parameter per section to theoutput_options->select_output
structure, e.g.Snapshot:WriteAnyFields_Gas
, set to1
if there is at least one field to be written, and0
if it is completely disabled. We could then quickly check at the beginning of theptype
-specific section in the output functions whether the particle type is fully disabled, and skip it (analogous to what happens already now if there are zero particles of this type).The problem with this is that it requires different choices based on whether we are using an output list or not.
So in the case of no output list, we know there is only one type of output selection.
This means we could have
output_options->should_write_part_type
or something as a list of bools that tell us whether or not that class of particles should be written ever.In the case where we have an output list, then we need an array of these options, so something like
output_options->should_write[output_type][part_type]
Which is more messy, and duplicates the behaviour.This is added to by the fact that
output_options
andoutput_list
do not know about each other. I suppose what we could do is have a pointer to theoutput_options
in theoutput_list
, and store the information about the particle selections inoutput_options
instead ofoutput_list
.That's correct. There is a buffer, initially filled with
Default
, that gets overwritten if the output list is used. See e.g. https://gitlab.cosma.dur.ac.uk/swift/swiftsim/blob/master/src/single_io.c#L820