From 708b853fd47f89501fee9a39a2b39e6853b6c4f0 Mon Sep 17 00:00:00 2001
From: "Peter W. Draper" <p.w.draper@durham.ac.uk>
Date: Thu, 28 Mar 2019 11:42:21 +0000
Subject: [PATCH] Add some documentation about memory reports

---
 doc/RTD/source/AnalysisTools/index.rst | 43 ++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/doc/RTD/source/AnalysisTools/index.rst b/doc/RTD/source/AnalysisTools/index.rst
index f68fa77536..b157baed90 100644
--- a/doc/RTD/source/AnalysisTools/index.rst
+++ b/doc/RTD/source/AnalysisTools/index.rst
@@ -1,5 +1,6 @@
 .. AnalysisTools
    Loic Hausammann 20th March 2019
+   Peter W. Draper 28th March 2019
 
 .. _analysistools:
 
@@ -33,3 +34,45 @@ or install ``npm`` and then run the following commands
    http-server .
 
 Now you can open the web page ``http://localhost:8080/cell_hierarchy.html``.
+
+Memory usage reports
+--------------------
+
+When SWIFT is configured using the ``--enable-memuse-reports`` flag it will
+log any calls to allocate or free memory that make use of the
+``swift_memalign()``, ``swift_malloc()``, ``swift_calloc()`` and
+``swift_free()`` functions and will generate a report at the end of each
+step. It will also attempt to dump the current memory use when SWIFT is
+aborted by calling the ``error()`` function. Failed memory allocations will be
+reported in these logs.
+
+These functions should be used by developers when allocating significant
+amounts of memory -- so don't use these for high frequency small allocations.
+Each call to the ``swift_`` functions differs to the standard calls by the
+inclusion of a "label", this should match between allocations and frees and
+ideally should be a short label that describes the use of the memory, i.e.
+"parts", "gparts", "hydro.sort" etc.
+
+The output files are called ``memuse_report-step<n>.dat`` or
+``memuse_report-rank<m>-step<n>.dat`` if running using MPI. These have a line
+for each allocation or free that records the time, memory address, step,
+whether an allocation or free, the label and when an allocation, the amount of
+memory. The comments in this file also record the actual memory use of the
+process (including threads) as reported by the operating system at the end of
+the step.
+
+To post process these files into a memory used timeline and get a report of
+the peak memory use, as well as the memory still in use at the end of the step
+there is an basic analysis script ``analyse_memuse_logs.py`` and two wrappers
+that process a directory of logs, these are ``./process_memuse_logs`` and
+``./process_memuse_logs_MPI`` for non-MPI and MPI runs respectively.
+
+Note that the process scripts process each step individually and also process
+all the logs as a single sequence. When interpreting these some care should be
+taken as they are not all the memory allocated, just important allocations in
+SWIFT and when looking at a single step the context of any previous steps is
+not used, so you only see allocations made in that step and the effect of any
+matching frees (so allocations made in previous steps that are freed in this
+step will not be understood and will be ignored, you need the global analysis
+to understand that).
+
-- 
GitLab