Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • SWIFTsim SWIFTsim
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 60
    • Issues 60
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 20
    • Merge requests 20
  • Deployments
    • Deployments
    • Releases
  • Packages and registries
    • Packages and registries
    • Model experiments
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
Collapse sidebar
  • SWIFT
  • SWIFTsimSWIFTsim
  • Issues
  • #93

Multi-time step MPI problems.

Tests of the following batch script:

#!/bin/bash -l
#
# Batch script for bash users
#
#BSUB -L /bin/bash
#BSUB -n 4
#BSUB -J SWIFT-mpi-test
#BSUB -oo job.dump
#BSUB -eo job.err
#BSUB -q bench1
#BSUB -P durham
#BSUB -R span[ptile=1]
#BSUB -x
#BSUB -W 00:30

NTHREADS=12

module purge
module load swift
module load swift/c4/intel/intelmpi/5.1.2

mpirun -np 4 ../swift_mpi -t $NTHREADS -f SodShock/sodShock.hdf5 -m 0.01 -w 5000 -c 1. -d 1e-7 -e 0.01

Give rise to a deadlock unless the affinity code is completely disabled (we seem to have many threads sharing the same cores). Once the affinity code is removed, the job will run but occasions of negative time steps are seen, for instance:

42 0.005371 0.000122 1024127 490.381
43 0.004639 -0.000732 1024128 134.555
44 0.004883 0.000244 1024128 121.384
45 0.005127 0.000244 1024128 112.810
46 0.005371 0.000244 1024128 117.759

and the job eventually fails when h_max requires a regrid of the top-level cells. This never happened for the SodShock with fixed dt.

For reference the SHA at this time was: 33caebf8

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking