Skip to content
Snippets Groups Projects

Parallel rebuild2

Merged Matthieu Schaller requested to merge parallel_rebuild2 into master
All threads resolved!

Fixes #458 (closed). Also renders the discussion in #483 (closed) void.

Edited by Matthieu Schaller

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • So it's super-weird to me that this is so slow...

    My only guess is that the spin-locks used around the MPI_Waitany (currently swift_lock_type) generate too much background noise. Can you try replacing the swift_lock_type with a pthread_mutex and initialize it with PTHREAD_MUTEX_INITIALIZER? The mutex has a higher unlock latency than the spin-lock, but doesn't burn any CPU while waiting.

    As for having different threads look at different chunks of the array, that can cause load imbalances, say if you have two threads and ten entries, there may be zero available in the first chunk, but all available in the second, and the first thread will wait idly while the second has too much work.

  • mentioned in issue #458 (closed)

  • FYI, I have been working on an orthogonal solution to this. We currently construct much too many proxies compared to what we will need later on when constructing the tasks. That speeds up the whole exchange of cells a lot but I want to make sure the new decision making does not lead to forgotten corner cases where we actually need the much more conservative approach currently in use.

  • Here is a suggestion before some of this drifts too much out of date: Let's merge in the good parts and keep the parallelization of the cell unpacking for later.

    Also, the reduction of the number of proxies has a dramatic effect on this. I'll make a separate merge request for these changes.

  • Sounds like a good plan to me. I think you already have a branch that does this?

    I'm really curious to see why this is a slowdown. It's in tracking down weird things like this that we've found interesting bugs and edge-cases, so I'm actually even a bit excited to see where this goes :)

  • Matthieu Schaller added 73 commits

    added 73 commits

    • 750ccb14...e1bdccb3 - 71 commits from branch master
    • 6be06310 - Do not use the threadpool parallelization for the waiting and unpacking of cell proxies.
    • 35314bbc - Merge branch 'master' into parallel_rebuild2

    Compare with previous version

  • Matthieu Schaller resolved all discussions

    resolved all discussions

  • added 1 commit

    • 97320153 - Document the mapper function to remember that it is currently unused.

    Compare with previous version

  • Ok. I have reverted the threadpool call to the waiting and un-packing of cells.

    Once merged, we can resume work by reverting commit 6be06310.

  • Should be ready to go.

  • Agreed, accepting.

  • Peter W. Draper mentioned in commit bae9515b

    mentioned in commit bae9515b

  • Left the branch for your further tests, you'll need a new MR for those.

  • Perfect, thanks!

  • Please register or sign in to reply
    Loading