Replaced make_cell_hierarchy.sh with a more flexible Python version.

doc/RTD/source/AnalysisTools/index.rst

@@ -42,30 +42,28 @@ It defines how many steps are done in between two task level output dumps.
 Cell graph
 ----------
 
-An interactive graph of the cells is available with the configuration option ``--enable-cell-graph``.
-During a run, SWIFT will generate a ``cell_hierarchy_*.csv`` file per MPI rank at the frequency given by the parameter ``--cell-dumps=n``.
-The command ``tools/make_cell_hierarchy.sh cell_hierarchy_0000_*.csv`` merges the files at time step 0 together and generates the file ``cell_hierarchy.html``
-that contains the graph and can be read with your favorite web browser.
-
-With most web browsers, you cannot access the files directly.
-If it is the case, the cells will never appear (but everything else should be fine).
-To solve this problem, you will need to either access them through an existing server (e.g. public http provided by your university)
-or install ``npm`` and then run the following commands
-
-.. code-block:: bash
-
-   npm install http-server -g
-   http-server .
-
-Now you can open the web page ``http://localhost:8080/cell_hierarchy.html``.
-When running a large simulation, the data loading may take a while (a few seconds for EAGLE_6).
-Your browser should not be hanging, but will seems to be idle.
-
-If you wish to add some information to the graph, you can do it by modifying the files ``src/space.c`` and ``tools/data/cell_hierarchy.html``.
-In the first one, you will need to modify the calls to ``fprintf`` in the functions ``space_write_cell_hierarchy`` and ``space_write_cell``.
-Here the code is simply writing CSV files containing all the required information about the cells.
-In the second one, you will need to find the function ``mouseover`` and add the field that you have created.
-You can also increase the size of the bubble through the style parameter ``height``.
+An interactive graph of the cells is available with the configuration option ``--enable-cell-graph``. During a
+run, SWIFT will generate a ``cell_hierarchy_*.csv`` file per MPI rank at the frequency given by the parameter
+``--cell-dumps=n``. The script ``tools/make_cell_hierarchy.py`` can be used to collate the files produced by
+different MPI ranks and convert them into a web page that shows an interactive cell hierarchy. The script
+takes the names of all the files you want to include as input, and requires an output prefix that will be used
+to name the output files ``prefix.csv`` and ``prefix.html``. If the prefix path contains directories that do
+not exist, the script will create those.
+
+The output files cannot be directly viewed from a browser, because they require a server connection to
+interactively load the data. You can either copy them over to a server, or set up a local server yourself. The
+latter can also be done directly by the script by using the optional parameter ``--serve``.
+
+When running a large simulation, the data loading may take a while (a few seconds for EAGLE_6). Your browser
+should not be hanging, but will appear to be idle. For really large simulations, the browser will give up and
+will probably display an error message.
+
+If you wish to add some information to the graph, you can do it by modifying the files ``src/space.c`` and
+``tools/data/cell_hierarchy.html``. In the first one, you will need to modify the calls to ``fprintf`` in the
+functions ``space_write_cell_hierarchy`` and ``space_write_cell``. Here the code is simply writing CSV files
+containing all the required information about the cells. In the second file, you will need to find the
+function ``mouseover`` and add the field that you have created. You can also increase the size of the bubble
+through the style parameter ``height``.
 
 Memory usage reports
 --------------------

tools/make_cell_hierarchy.py

+#!/usr/bin/env python3
+"""
+Usage:
+    make_cell_hierarchy.py --input <LIST OF FILES> --output <OUTPUT PREFIX>
+                           [--serve]
+
+Where <LIST OF FILES> is a list of input files (or file pattern matching that
+list) corresponding to the cell hierarchy on different MPI ranks for a single
+time step.
+<OUTPUT PREFIX> is the name of the output .html and .csv files that will be
+created. If the prefix path contains a directory that does not exist, this
+directory will be created.
+If the optional argument --serve is provided, the script will start a local
+server that can run the generated .html page and opens it in a browser.
+
+Based on a bash script written by Loic Hausammann.
+
+This file is part of SWIFT.
+
+Copyright (C) 2021 Bert Vandenbroucke (bert.vandenbroucke@gmail.com)
+All Rights Reserved.
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Lesser General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
+
+You should have received a copy of the GNU Lesser General Public License
+along with this program.  If not, see <http://www.gnu.org/licenses/>.
+"""
+
+import argparse
+import os
+import sys
+
+# Get the script folder, we need to retrieve the html template from it
+scriptfolder = os.path.dirname(sys.argv[0])
+
+# Handle the command line.
+parser = argparse.ArgumentParser(description="Create cell hierarchy web page.")
+
+parser.add_argument("input", help="Input file(s)", nargs="+")
+parser.add_argument("output", help="Output file prefix")
+parser.add_argument(
+    "--serve",
+    help="Run a local web server and display the generated web page?",
+    action="store_true",
+)
+args = parser.parse_args()
+
+# First check if the output prefix contains a folder and create it to make
+# sure it exists
+folder, fileprefix = os.path.split(args.output)
+if len(folder) > 0:
+    os.makedirs(folder, exist_ok=True)
+else:
+    folder = None
+
+# Accumulate all input files into a single file
+with open("{0}.csv".format(args.output), "w") as ofile:
+    for fname in sorted(args.input):
+        if not os.path.exists(fname):
+            print('Error: "{0}" does not exist!'.format(fname))
+            exit(1)
+        with open(fname, "r") as ifile:
+            ofile.write(ifile.read())
+
+# Read the html template
+with open("{0}/data/cell_hierarchy.html".format(scriptfolder), "r") as ifile:
+    html = ifile.read()
+
+# Replace the old csv file with the actual csv file
+csvname = "{0}.csv".format(fileprefix)
+html = html.replace("cell_hierarchy.csv", csvname)
+
+# Write out the new html file
+with open("{0}.html".format(args.output), "w") as ofile:
+    ofile.write(html)
+
+if args.serve:
+    import http.server
+    import socketserver
+    import webbrowser
+
+    class Handler(http.server.SimpleHTTPRequestHandler):
+        def __init__(self, *args, **kwargs):
+            super().__init__(*args, directory=folder, **kwargs)
+
+    print("Running a local server. Use CTRL+C to stop it.")
+    PORT = 8000
+    found_port = False
+    while not found_port:
+        try:
+            with socketserver.TCPServer(("", PORT), Handler) as httpd:
+                found_port = True
+                print("serving at port", PORT)
+                webbrowser.open(
+                    "http://localhost:{0}/{1}.html".format(PORT, fileprefix)
+                )
+                httpd.serve_forever()
+        except:
+            PORT += 1

tools/make_cell_hierarchy.sh

-#!/bin/bash
-
-output=cell_hierarchy.html
-
-# merge all mpi ranks together
-csv_output=cell_hierarchy.csv
-if [ -f $csv_output ]
-then
-   rm $csv_output
-fi
-
-for filename in $@;
-do
-    cat $filename >> cell_hierarchy.csv
-done
-
-# copy HTML page to the repository
-DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
-cp $DIR/data/cell_hierarchy.html $output
-
-echo $output has been generated