This package is a standalone part of SWIFT that aims to roughly simulate the MPI interactions that taking a single step of a simulation makes.

The interactions are captured from a run of SWIFT when configured using the configure option --enable-mpiuse-reports. When this is enabled each step of the simulation produces logs for each rank which record when the MPI interaction was started and when it completed. Other information such as the ranks involved, the size of the data exchanged, the MPI tags used and which SWIFT task types were used are also recorded.

To use the program swiftmpistepsim you need to select the step of interest (for instance one whose run-time seems dominated by the MPI tasks) and then concatenate all the logs for that step into a single file. You can then run using:

   mpirun -np <nranks> swiftmpistepsim <step-log> <output-log>

which will output timings for the various MPI calls and record a log for the reproduction in the file <output-log>.

To simulate SWIFT we use three threads, which run simultaneously, one that injects the MPI commands, i.e. initiates the interaction using calls to MPI_Isend and MPI_Irecv, and two other threads that poll the MPI library using MPI_Test to discover when the exchanges have been completed.

SWIFT itself uses more threads than this for the injection and polling phases, but it is not thought to make a large difference. A later development could explore that...

Peter W. Draper 18 Sep 2019.