... | ... | @@ -3,6 +3,7 @@ Problems (28/03/2020): |
|
|
- Not possible to access the outside world from the machine (tables,...),
|
|
|
|
|
|
------------------------------------------------
|
|
|
# Compilation
|
|
|
|
|
|
Start on a different machine (say cosma):
|
|
|
```bash
|
... | ... | @@ -53,11 +54,47 @@ cd swift-0.8.5 |
|
|
```
|
|
|
|
|
|
To do:
|
|
|
- Work out how to use metis
|
|
|
- Work out how to use tbbmalloc
|
|
|
- Work out how to transfer data efficiently
|
|
|
|
|
|
-------------------------------------------------------------------
|
|
|
# Submission scripts
|
|
|
|
|
|
Single node job:
|
|
|
```
|
|
|
#!/bin/bash
|
|
|
#PBS -N SWIFT-EAGLE-12
|
|
|
#PBS -l select=1:node_type=rome:mpiprocs=1:ompthreads=64
|
|
|
#PBS -l walltime=24:00:00
|
|
|
#PBS -o output.log
|
|
|
#PBS -e error.log
|
|
|
#PBS -m abe
|
|
|
#PBS -M youremail@yourprovider.com
|
|
|
|
|
|
# Change to the directory that the job was submitted from
|
|
|
cd $PBS_O_WORKDIR
|
|
|
|
|
|
# Unload the defaults
|
|
|
module unload mpt
|
|
|
module unload gcc/9.2.0
|
|
|
|
|
|
# Load the modules we need
|
|
|
module load intel/19.1.0
|
|
|
module load mpt/2.22
|
|
|
module load parmetis/4.0.3-int64
|
|
|
module load fftw/3.3.8
|
|
|
module load hdf5/1.10.5
|
|
|
module load tbb/19.1.0
|
|
|
module load gsl
|
|
|
module load sundials/5.1.0-int64
|
|
|
|
|
|
export OMP_NUM_THREADS=64
|
|
|
|
|
|
# Run SWIFT
|
|
|
./swift --cosmology --eagle --threads=64 --pin eagle_12.yml -v 1
|
|
|
```
|
|
|
|
|
|
-------------------------------------------------------------------
|
|
|
# Python Tooling
|
|
|
|
|
|
As HAWK has no connection to the outside world, you need to use a tunnel to transfer pip data. They also do not seem to correctly support python environments so we will user install everything.
|
... | ... | |