Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
SWIFTsim
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Deploy
Releases
Model registry
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
SWIFT
SWIFTsim
Commits
708b853f
Commit
708b853f
authored
6 years ago
by
Peter W. Draper
Browse files
Options
Downloads
Patches
Plain Diff
Add some documentation about memory reports
parent
bb2cf105
Branches
Branches containing commit
Tags
Tags containing commit
1 merge request
!757
Memory allocations logger
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
doc/RTD/source/AnalysisTools/index.rst
+43
-0
43 additions, 0 deletions
doc/RTD/source/AnalysisTools/index.rst
with
43 additions
and
0 deletions
doc/RTD/source/AnalysisTools/index.rst
+
43
−
0
View file @
708b853f
.. AnalysisTools
Loic Hausammann 20th March 2019
Peter W. Draper 28th March 2019
.. _analysistools:
...
...
@@ -33,3 +34,45 @@ or install ``npm`` and then run the following commands
http-server .
Now you can open the web page ``http://localhost:8080/cell_hierarchy.html``.
Memory usage reports
--------------------
When SWIFT is configured using the ``--enable-memuse-reports`` flag it will
log any calls to allocate or free memory that make use of the
``swift_memalign()``, ``swift_malloc()``, ``swift_calloc()`` and
``swift_free()`` functions and will generate a report at the end of each
step. It will also attempt to dump the current memory use when SWIFT is
aborted by calling the ``error()`` function. Failed memory allocations will be
reported in these logs.
These functions should be used by developers when allocating significant
amounts of memory -- so don't use these for high frequency small allocations.
Each call to the ``swift_`` functions differs to the standard calls by the
inclusion of a "label", this should match between allocations and frees and
ideally should be a short label that describes the use of the memory, i.e.
"parts", "gparts", "hydro.sort" etc.
The output files are called ``memuse_report-step<n>.dat`` or
``memuse_report-rank<m>-step<n>.dat`` if running using MPI. These have a line
for each allocation or free that records the time, memory address, step,
whether an allocation or free, the label and when an allocation, the amount of
memory. The comments in this file also record the actual memory use of the
process (including threads) as reported by the operating system at the end of
the step.
To post process these files into a memory used timeline and get a report of
the peak memory use, as well as the memory still in use at the end of the step
there is an basic analysis script ``analyse_memuse_logs.py`` and two wrappers
that process a directory of logs, these are ``./process_memuse_logs`` and
``./process_memuse_logs_MPI`` for non-MPI and MPI runs respectively.
Note that the process scripts process each step individually and also process
all the logs as a single sequence. When interpreting these some care should be
taken as they are not all the memory allocated, just important allocations in
SWIFT and when looking at a single step the context of any previous steps is
not used, so you only see allocations made in that step and the effect of any
matching frees (so allocations made in previous steps that are freed in this
step will not be understood and will be ignored, you need the global analysis
to understand that).
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment