WIP: snipes example test information

@matthieu I guess you know best who was responsible for creating the various tests. Could you get them involved in this too?

Most of them would be me... :/

So, I'd keep all the EAGLE ones out of that. We should make some sub-directories with the different categories of tests.

CosmoVolume and friends can be discarded.
The uniform box in various dimensions were useful to check that nothing happens. Also easy to setup a really big test for MPI-test purposes.
The main purpose of the perturbed box is to test that the perturbation gets damped (not sure how to make this into a pass/fail test though) and they can be used to make glass files of arbitrary size.
MultiTypes is my go-to test to check that the i/o code can read arbitrary sets of particle types.
KeplerianRing is @jborrow's test. Does not work with any of the current scheme. Maybe in MFM? Should work with Pressure-Energy and a decent viscosity switch.
SmoothedMetallicity is used by @lhausammann and in the future by @aborissov to make sure the metal smoothing in GEAR and EAGLE is computed correctly.
CoolingBox and friends need improving to be used with the various cooling models. Should work with the const-lambda model.
Moon and Uranus impact are for @jkeger. The ICs are being downloaded from a getIC script. We should move these tests to a separate sub-category. To be configured with the planetary EoS and the MinimalMultiMat scheme.

The tests of the different potentials should work if compiled with the corresponding external potential. Might be worth also creating a sub-directory for them.

My 2 cents on the information you provide:

Is it worth adding the default values to the .yml files? If more options appear (and some will) we will have to edit all the yaml files to add them. For instance when star formation appears in master, none of these tests will make use of it. Should we then really have to edit all these files?
For the READMEs, it would be nice to include a reference to a paper describing the test if such a paper exists. We should also indicate in there that some schemes do not necessarily give a correct answer.

Are the default values ever going to change? Will new flags be added with default values that change the behaviour of the code? If both of these will never happen, then adding default values is indeed not necessary. But in any other case it sounds like a good idea to make sure someone actively thinks about what happens to each test in these cases.

On a related note: we should probably make a distinction between configuration flags that change the physics (with-adiabatic-index) and configuration flags the change the code (e.g. with-hydro).

Nice idea! I will take care of the CoolingBox and Smoothed Metallicity examples after my exams. Should I use your branch or a new one?

Just a heads up to the maintainers of the physics tests: it's probably best off waiting for a few weeks until me and Bert have this system up-and-running.

Another heads up: snipes will be python3.5 or higher only.

@matthieu, @bvandenbroucke. The way that I envisage the "default values" system working is as follows:

A the top level, there are the built-in SWIFT defaults (those provided in parameter_example.yml).
If on a test-by-test basis we want to change these (such as dt_min), then they will be provided in that test's parameter file in the appropriate directory.
If on a run-by-run basis we want to change the parameters, such as, idk, the softening length for self-gravity, then this will be handled through an extra system. Only these parameters will be changed in the parameterfile, so it will look functionally like those that we have already for the 'regular' tests that we so far modify by hand.

In terms of priority, we have:

run-by-run > test-by-test > default.

Only the things in run-by-run and test-by-test will be included in test_configuration.yml, and the rest will be left to the defaults currently in SWIFT.

This will mean we won't need to edit any of the configuration files once star formation, or other physics, becomes a thing, as they should be switched off by default.

Maybe we should add some subdirectories? For example, one for all the hydro tests, one for the gravity, one for the cooling, ...

I think it would help to understand the tests

We have a plan to "tag" tests like that, as some test more than one thing so putting them in boxes is somewhat difficult.

You would then be able to run all of the e.g. hydro tests by invoking some command like

snipes --run-tests=hydro

@bvandenbroucke you would hope that new flags will not affect the behaviour of the code unless they are switched on. But the default value might change. We will probably want a better hydro scheme to be the default. Possibly something like anarchy-pressure-energy I suppose.

Changing the default hydro scheme would/should not affect the tests directly (except maybe the planetary tests?). But changing the default value of the adiabatic index would. That's why we should probably make a distinction between physics specific configuration options and other options, something we will need to do for snipes anyway.

And maybe we will need some meta options as well? I can imagine the CoolingBox should work with different cooling implementations, as long as there is one. So in that case the configuration.yml could contain something like with-cooling: yes or with-cooling: default instead of an actual cooling implementation.

At this stage, everything is still up for discussion, and I'm mostly interested in gathering information. I like @jborrow's idea for the default values. Having tags would make snipes more powerful than using different folders, as we could also add run time specific tags (e.g. run all hydro tests that fit on 2 cores and have an expected run time of less than half an hour). But it might be a good idea to separate the purely technical tests (like the EAGLEs) from the physics and features tests by putting them in different folders.

Feel free to use this branch to add information, and let me know in the discussion when a test should be recategorised in the list. I will try to keep the list on the top of the page up to date, so that we have a nice overview.

This all sounds great. I'm still testing and debugging the planetary code itself, so if it's okay I'll leave the two examples as they are until everything's working, then update them with the new system.

They are also just illustrative examples rather than real "tests" with known solutions. So feel free to ignore MoonFormingImpact and UranusImpact for now.

I have just accepted @lhausammann's upgrade to the i/o system that allows users to specify fields they do not want to write in the snapshots (#410 (closed) !520 (merged)). That may be of interest for the development discussed here as it allows to reduce the amount of unnecessary data being written.

Here is a suggestion for an additional test:

Uniform box in terms of mass, positions and h. However, the particles have distinct values for their internal energy. In 1D, you could have every second particle with u=u_1 and the other ones with u=u_2. In principle, nothing should happen, at least in Pressure-Energy. In ANARCHY-PU, nothing will happen but the energy of all the particles should slowly equalize to a constant. That should show that the diffusion term work and be an example of why it is needed.

I have removed BigCosmoVolume, CosmoVolume and CosmologicalBox and will replace them with one small cosmology box that runs in less than 1hr.

FWIW, I hope to work on SNIPES at the DurHack hackathon.

Do we need to keep this open as a merge request?

I guess it doesn't hurt to close it, as long as the information stays available and we keep the branch (for now).

closed

To be re-opened when/if this get revived.

WIP: snipes example test information

List of tests

`DONE`

`UNSURE`

`INCOMPLETE`

`REMOVE?`

Activity

WIP: snipes example test information

List of tests

DONE

UNSURE

INCOMPLETE

REMOVE?

Merge request reports

Activity

`DONE`

`UNSURE`

`INCOMPLETE`

`REMOVE?`