Leaner gparts and cell structure
A bunch of updates to help with the memory foot-print, especially in cases with relatively simple physics:
- Remove the padding of the
gpartstructure. (We were not taking advantage of it anyway) - Add a compile-time option to not compute (and hence store in RAM) potentials as they are not always needed.
- Split the cell structure into sub-structures for each particle type.
- Hide all the non-strictly necessary fields of these sub-structures into an anonymous union when compiling without star/BH/sink.
- Add an empty hydro scheme to use the same trick as above also for the gas so that we have the leanest possible cell structure in gravity-only runs.
The 4th item in the list is quite a dirty trick I would think. Alternatives require massive code restructuring and probably runtime costs. This allows the code to compile as all the variables are defined. Though if one attempts to use them (e.g. because you have stars but compiled without) then you are in for a treat.
Merge request reports
Activity
added memory usage label
added enhancement label
assigned to @pdraper
added 39 commits
-
cf19ceff...80d75425 - 19 commits from branch
master - c7e0d943 - Remove the alignment constraint of gparts. Add the option to not compute the...
- 7db9e151 - Do not compile all the stars, black holes and sink cell properties when...
- 1b2d4d66 - Temporary state. Do not create hydro communication task links if there are no star or BH model
- da4d70c9 - Make more cell variables available
- aaf2b5da - Only collect h_max when you can
- b7852eeb - Re-organise things a little in the cell structure
- 8d451789 - Applied code formatting
- 58b0b8af - Fix also the case of the task unskipping
- bd6b90b5 - Use the unaligned memswap for gparts now that their size is not a multiple of...
- ade90f5b - Move more unnecessary variables in the unions
- 6bc50f99 - Clean-up small memory leak when ending a simulation. Was a false positive but...
- 5a4a2c5e - Log a few more memory calls
- 0a08a424 - Log a few more allocations internal to the hydro tasks to hunt down the leak
- efdcf07f - Revert "Log a few more allocations internal to the hydro tasks to hunt down the leak"
- 173940ca - Added empty implementation of a hydro scheme to save memory in DMO runs
- 11183ffc - Moved the hydro-related cell fields to a separate structure
- 466699d3 - Added licensing in the new files
- 3e31efb7 - Fix FPE in the repartition trigger
- 0312f5b3 - Applied code formatting tool
- 5d189d8c - Post-rebase fixes
Toggle commit list-
cf19ceff...80d75425 - 19 commits from branch
added 1 commit
- 16b91987 - Add the memreport's main array to the list of memory allocations we report
429 429 if (abs_trigger > 1.f) abs_trigger = 0.f; /* Not relevant. */ 430 430 fprintf(timelog, 431 431 "# %d balance: %f, expected: %f (sys: %f, total: %f)\n", 432 e->step, balance, abs_trigger, (smaxtime - smintime) / smean, 432 e->step, balance, abs_trigger, 433 (smaxtime - smintime) / (smean + FLT_MAX), changed this line in version 4 of the diff
added 1 commit
- c4efd09e - Fix silly typo in the reporting of CPU balance
added 1 commit
- 7b14951e - Guard against when more than MEMUSE_INITLOG new allocations are logged while...
added 1 commit
- f9e95760 - Add report about reallocations of foreign particles
added 1 commit
- 09189bdc - Also report the RES memory usage when running on a single node
added 1 commit
- cdb932c7 - Added counter to monitor all the hydro.sorts allocations
added 104 commits
-
d963b67f...503a2191 - 76 commits from branch
master - 10c2f4c9 - Remove the alignment constraint of gparts. Add the option to not compute the...
- 52fd9115 - Do not compile all the stars, black holes and sink cell properties when...
- d66dc48b - Temporary state. Do not create hydro communication task links if there are no star or BH model
- 75418a9b - Make more cell variables available
- 6f6c3126 - Only collect h_max when you can
- 2811711c - Re-organise things a little in the cell structure
- cd183dd7 - Applied code formatting
- 222341d5 - Fix also the case of the task unskipping
- 37dbfce5 - Use the unaligned memswap for gparts now that their size is not a multiple of...
- 02acbc5f - Move more unnecessary variables in the unions
- 18eb70b8 - Clean-up small memory leak when ending a simulation. Was a false positive but...
- 7f21fff2 - Log a few more memory calls
- 43b3b5e6 - Log a few more allocations internal to the hydro tasks to hunt down the leak
- a5add36d - Revert "Log a few more allocations internal to the hydro tasks to hunt down the leak"
- 115bfc28 - Added empty implementation of a hydro scheme to save memory in DMO runs
- 9ca2d0fe - Moved the hydro-related cell fields to a separate structure
- 45a4597d - Added licensing in the new files
- a3821db5 - Fix FPE in the repartition trigger
- ef9b8cdf - Applied code formatting tool
- 377b6ef5 - Post-rebase fixes
- 105c9378 - Add the memreport's main array to the list of memory allocations we report
- e2ebc616 - Fix silly typo in the reporting of CPU balance
- df807b91 - Also report the RES memory usage when running on a single node
- 6b5927e9 - Applied code formatting tool
- 16ee2c17 - Added counter to monitor all the hydro.sorts allocations
- b966dc14 - Be verbose about reallocating the queue and unlock arrays
- 61fd990d - Collect the max memory allocated at any time in the ghost or hydro functions
- 0e8926a5 - Better types in messages
Toggle commit list-
d963b67f...503a2191 - 76 commits from branch
I'll close this until !1162 (merged) is merged and we'll then revisit the additional changes that this brings.
Please register or sign in to reply