Make initial offset into OSTs random when dumping restarts.

Make initial offset into OSTs random so we do not reuse the same OSTs for all restarts and runs. This can be an issue when the ranks is less than the number of OSTs and to a lesser extent when the number of OSTs isn't an exact multiple of the number of ranks.

Restart version of !1706 (merged).

doc/RTD/source/ParameterFiles/parameter_description.rst

@@ -862,9 +862,9 @@ On Lustre filesystems [#f4]_ it is important to properly stripe files to achieve
 a good writing speed. If the parameter ``lustre_OST_count`` is set to the number
 of OSTs present on the system, then SWIFT will set the `stripe count` of each
 distributed file to `1` and set each file's `stripe index` to the MPI rank
-generating it modulo the OST count. If the parameter is not set then the files
-will be created with the default system policy (or whatever was set for the
-directory where the files are written). This parameter has no effect on
+generating it modulo the OST count [#f5]_. If the parameter is not set then the
+files will be created with the default system policy (or whatever was set for
+the directory where the files are written). This parameter has no effect on
 non-Lustre file systems and no effect if distributed snapshots are not used.
 
 * The number of Lustre OSTs to distribute the single-striped distributed
@@ -1394,9 +1394,9 @@ On Lustre filesystems [#f4]_ it is important to properly stripe files to achieve
 a good writing and reading speed. If the parameter ``lustre_OST_count`` is set
 to the number of OSTs present on the system, then SWIFT will set the `stripe
 count` of each restart file to `1` and set each file's `stripe index` to the MPI
-rank generating it modulo the OST count. If the parameter is not set then the
-files will be created with the default system policy (or whatever was set for
-the directory where the files are written). This parameter has no effect on
+rank generating it modulo the OST count [#f5]_. If the parameter is not set then
+the files will be created with the default system policy (or whatever was set
+for the directory where the files are written). This parameter has no effect on
 non-Lustre file systems.
 
 * The number of Lustre OSTs to distribute the single-striped restart files over:
@@ -1914,3 +1914,8 @@ A complete specification of the model looks like
 	 matter, 3 --> sinks, 4 --> stars, 5 --> black holes, 6 --> neutrinos.
 
 .. [#f4] https://wiki.lustre.org/Main_Page
+
+.. [#f5] We add a per-output random integer to the OST value such that we don't
+	 generate a bias towards low OSTs. This averages the load over all OSTs
+	 over the course of a run even if the number of OSTs does not divide the
+	 number of files and vice-versa.