Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
S
swiftmpistepsim
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Build
Pipelines
Jobs
Pipeline schedules
Artifacts
Deploy
Releases
Container registry
Model registry
Operate
Environments
Monitor
Incidents
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
SWIFT
swiftmpistepsim
Commits
f261af99
Commit
f261af99
authored
5 years ago
by
Peter W. Draper
Browse files
Options
Downloads
Patches
Plain Diff
Add script to process output log and match sends and recvs combining into a single row
parent
3f28aa34
No related branches found
No related tags found
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
post-process.py
+111
-0
111 additions, 0 deletions
post-process.py
with
111 additions
and
0 deletions
post-process.py
0 → 100755
+
111
−
0
View file @
f261af99
#!/usr/bin/env python
"""
Usage:
post-process.py [options] output-log
Match the sends and recvs across the ranks and output a unified log.
This file is part of SWIFT.
Copyright (C) 2019 Peter W. Draper (p.w.draper@durham.ac.uk)
All Rights Reserved.
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU Lesser General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
"""
import
sys
import
argparse
# Handle the command line.
parser
=
argparse
.
ArgumentParser
(
description
=
"
Match MPI reports
"
)
parser
.
add_argument
(
"
input
"
,
help
=
"
Output log from simulator
"
)
parser
.
add_argument
(
"
-v
"
,
"
--verbose
"
,
dest
=
"
verbose
"
,
help
=
"
Verbose output
"
,
default
=
False
,
action
=
"
store_true
"
,
)
args
=
parser
.
parse_args
()
infile
=
args
.
input
# Indices for words in a line.
logticcol
=
0
logcol
=
1
injcol
=
2
endcol
=
3
dticcol
=
4
stepcol
=
5
rankcol
=
6
otherrankcol
=
7
itypecol
=
8
isubtypecol
=
9
tagcol
=
10
sizecol
=
11
ntestcol
=
12
sumcol
=
13
mincol
=
14
maxcol
=
15
# Keyed lines.
sends
=
{}
recvs
=
{}
# Gather keys from the input log. We created dicts with matchable keys
# for when sends start and recvs end. Other pairings are possible...
# Note size of completion recv is negative.
with
open
(
infile
,
"
r
"
)
as
fp
:
for
line
in
fp
:
if
line
[
0
]
==
'
#
'
:
continue
words
=
line
.
split
()
if
words
[
itypecol
]
==
"
22
"
:
key
=
words
[
otherrankcol
]
+
"
/
"
+
\
words
[
rankcol
]
+
"
/
"
+
\
words
[
isubtypecol
]
+
"
/
"
+
\
words
[
tagcol
]
+
"
/
"
+
\
words
[
sizecol
]
if
not
key
in
sends
:
sends
[
key
]
=
[
line
[:
-
1
]]
else
:
sends
[
key
].
append
(
line
[:
-
1
])
elif
words
[
itypecol
]
==
"
23
"
:
key
=
words
[
rankcol
]
+
"
/
"
+
\
words
[
otherrankcol
]
+
"
/
"
+
\
words
[
isubtypecol
]
+
"
/
"
+
\
words
[
tagcol
]
+
"
/
"
+
\
words
[
sizecol
]
if
not
key
in
recvs
:
recvs
[
key
]
=
[
line
[:
-
1
]]
else
:
recvs
[
key
].
append
(
line
[:
-
1
])
# Now output. Note we could have unmatched recv keys, we don't check for that.
print
"
# send_logticin send_logtic send_injtic send_endtic send_dtic send_step send_rank send_otherrank send_itype send_isubtype send_tag send_size send_nr_tests send_tsum send_tmin send_tmax recv_logticin recv_logtic recv_injtic recv_endtic recv_dtic recv_step recv_rank recv_otherrank recv_itype recv_isubtype recv_tag recv_size recv_nr_tests recv_tsum recv_tmin recv_tmax
"
for
key
in
sends
:
if
key
in
recvs
:
if
len
(
sends
[
key
])
==
1
and
len
(
recvs
[
key
])
==
1
:
print
sends
[
key
][
0
],
recvs
[
key
][
0
]
else
:
print
"
# ERROR: found
"
,
len
(
sends
[
key
]),
"
/
"
,
len
(
recvs
[
key
]),
"
matches for key:
"
,
key
,
"
should be 1/1
"
else
:
print
"
# ERROR: missing recv key:
"
,
key
sys
.
exit
(
0
)
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment