Added functionality for a new parameter that allows you to set the location/name for SFR.txt

Added functionality for a new parameter that allows you to set the location and name of the SFR.txt file, usefull when running multiple runs from the same directory.

added 1 commit

• 0b18e600 - fixed typo in parameter_example.yml

Compare with previous version

assigned to @matthieu

added feature request label

Great, thanks. Have you tried running the code, stopping it and using restart files to see whether this works?

Also, it could be good to document the new parameter in doc/RTD/... such that it appears on the online documentation (http://swift.dur.ac.uk/docs/ParameterFiles/parameter_description.html). Although I appreciate that we haven't documented the other parameters of that section... But that would be a way to start.

added 3 commits

• d6700bc7 - Added the parameter to the documentation
• b739f4c7 - Added the parameter to the documentation
• 7ccd5962 - Merge branch 'sfr_txt_option' of https://gitlab.cosma.dur.ac.uk/swift/swiftsim into sfr_txt_option

Compare with previous version

I added the documentation and tested it with restart, after a restart it keeps writing to the correct file. It also defaults back to just using SFR.txt if you don't put it in the parameter file.

@jborrow is that something fun for the pipeline?

@folkert any thoughts on this since you wrote the logger?

added 1 commit

• 1eb4aa1c - Update to parameter_example.yml to use default instead of standard

Compare with previous version

resolved all threads

This is not going to work well with the pipeline, at all, unfortunately.

Can we write the name in the header? I don't like that we can now store snapshots and SFR.txt in different directories, either. That's concerning from a user-confusion perspective.

That was the exact problem of why I added this, when you use subdir for the snapshots the SFR still get's put in the place you run swift from. For my personal use this will always be set to the snapshot subdir/SFR.txt

Shouldn't we then make that change? There are other output files, like SNIa.txt, used_parameters.yml, etc. that maybe should just be put in that directory?

@matthieu, initially I also added that the SFR.txt file name is an option, but I think you removed it because you wanted it to be always the same file. I also thought it was a thing of swift that you should not run multiple simulations in the same folder. If in the case of @rkugel use, can we not add a option to store everything in the same file?

I am fine with the change.

It's a general thing that you should not run many things in the same directory for sure. Not just SWIFT...

I think the argument already exists in Snapshots:subdir, we could use this to place the files in the correct directories?

@jborrow do you mean have all the relevant output files be moved to the same directory as the snapshots?

Yes, that would be my idea. Have an output data directory. Otherwise this will just lead to mix-ups.

The problem with the pipeline is specifically that we assume that the data is all stored together (aside from the catalogues, that can be in a different directory), and that the additional files (timesteps_*.txt, SNIa.txt, SFR.txt) all have consistent filenames. I'd worry about people breaking that or introducing what I would say is unnecessary code complexity on the analysis side by introducing this change. However, if we changed it so that you could specify e.g.

MetaData:
  data_directory: /hello/world

and that put all outputs in that place, sure. Then you're going to end up with filesystem problems, though, like does this directory exist, how to deal with batch files and their outputs - presumably you'd want them in the same directory that may or may not already exist because the files are created before SWIFT is instigated...