CSDS Move the index files into the reader.
Keeping in memory the information required for the index files during the simulation is costing far too much in term of memory but also performances. To reduce this, I have moved all the logic into the reader where we will not have as much information to track.
Some other changes that are worth to mention:
- In the CSDS yaml file, I am writing the initial number of particles. This value is used to initialize the arrays when generating the index files. The arrays can still growth if needed.
- When all the particles are written just before the first step, they are now flagged as being created.
- The data within the special flag contains now the particle. As the type was found from the index file, now we need a way to get it directly from the logfile. We still use 4 bytes but now we have 1 byte for the particle type, 2 bytes for any information (e.g. MPI rank for particles leaving/entering a rank) and 1 byte for the type of event (e.g. particle leaving/entering a rank, star formation, deletion, creation, ...).
- Initialize the time step counter to 0. There is no need to write the particles quickly at the start of the simulation as we are already manually writing them.
Edited by Matthieu Schaller