diff --git a/doc/RTD/source/Skeleton/engine.rst b/doc/RTD/source/Skeleton/engine.rst
index 1fe05cf4f7960ead32bd1e43e6a108c166ebe8b7..9bae46a306c260d00df28bf10594455417cdb40e 100644
--- a/doc/RTD/source/Skeleton/engine.rst
+++ b/doc/RTD/source/Skeleton/engine.rst
@@ -1,4 +1,5 @@
.. Engine
+ November 2018
Mladen Ivkovic
@@ -23,19 +24,51 @@ Calling sequence in ``swiftsim/examples/main.c``
- Define ``struct engine e`` that contains all the stuff the engine needs to advance the simulation in time, like the number of threads on which to run on, the space with wich the runner is associated, it's threads and policy, a task scheduler, starting and ending times, minimal and maximal timesteps, the MPI rank...
- :term:`pin the main thread <pinning a thread>`: Pins an engine (for each MPI rank) to a processor using pthreads black magic.
-- If restarting: The previously used engine struct will be read in again, then configured. (The configuration in detail is a nightmare, so unless you're explicitly tinkering with the engine, just skip it.)
+- If restarting: The previously used engine struct will be read in again, then configured. (The configuration in detail is a nightmare, so unless you're explicitly tinkering with the engine, don't open this Pandora's box.)
- Else:
- - Construct engine policy, initialize it and then construct the engine itself.
- - (Later) split the domain
+ - Construct :ref:`engine policy <engine_policy>`, initialize it and then construct the engine itself.
+ - :ref:`Split the domain <domain_decomposition>`:
- - **engine_split**:
- - Do initial partition
- - create :term:`proxies <proxy>`
- - re-allocate and re-:term:`link <links>` particles to add buffer space in the arrays
+ - engine_split:
+ - Do initial partition
+ - create :term:`proxies <proxy>`
+ - re-allocate and re-:term:`link <links>` particles to add buffer space in the arrays
- - **engine_redistribute**:
- -
+ - engine_redistribute:
+ - Figure out how many particles of which :term:`type <particle types>` each node needs to send where
+ - Communicate this information to other nodes.
+ - Exchange particles (asynchronously)
+ - re-link the particle types
+ - initialize particles:
+ - set particles to a valid state by calling ``engine_first_init_particles``
+ - test
+ - construct all cells and tasks, get ready to start running by calling ``engine_rebuild``
+
+ - todo
+
+ - compute densities? ``engine_skip_force_and_kick``
+ - initialize particle data by calling ``space_init_(g/s)parts``
+ - launch engine
+ - prepare all tasks for a new round, then call ``engine_skip_drift``
+ - initialize particle data by calling ``space_init_(g/s)parts``
+ - recover the integer end of the next time step: ``engine_collect_end_of_step``
+ - do some checks on particles
+
+
+
+
+
+
+
+
+
+.. _engine_policy:
+
+Engine Policy
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+text
diff --git a/doc/RTD/source/Skeleton/glossary.rst b/doc/RTD/source/Skeleton/glossary.rst
index bae312c3dabd431c93b5a00d9c4f14e9b014b3e9..4e8ed151f31055f298e45df75de63bbc2072ba18 100644
--- a/doc/RTD/source/Skeleton/glossary.rst
+++ b/doc/RTD/source/Skeleton/glossary.rst
@@ -1,4 +1,5 @@
-.. Glossary
+.. Glossary for code skeleton
+ November 2018
Mladen Ivkovic
@@ -13,6 +14,9 @@ Glossary
By default, when running multiple tasks on multicore architectures, tasks are allowed to run on any core, and also switch on which core they are run on.
To boost performance, instead you can pin a task on a specific core, where the data the task will require is stored most closely in memory, thus avoiding unnecessary data transfers from other cores.
+ inhibited particle
+ TODO
+
links
Each particle is split into "2 pieces": ``part`` + ``gpart``. (For stars, it's ``gpart`` + ``spart``. For dark matter only, no hydro/stars are needed, so it's a standalone ``gpart``). This is done so you can split hydro/gravity calculations without working on the same part of memory while computing them simultaneously. However, the particle parts need to be linked properly, which is what is meant by a link.
@@ -24,11 +28,14 @@ Glossary
Assigning a thread to a specific core to run on.
See :term:`Affinity`
- runner
- the threads. Structure that has the main properties of threads like ID, type, etc.
-
proxy
structs that represent other MPI ranks; They contain the data on what information needs to be sent to other ranks
+ particle types
+ multiple particle types are employed in SWIFT. This allows tasks to compute multiple kinds of interactions on particle simultaneously. There are "normal" SPH particles, star particles and gravity particles. For more details, refer to the :ref:`particles` section.
+
+ runner
+ the threads. Structure that has the main properties of threads like ID, type, etc.
+
stop file
Undocumented feature at the moment. A way to stop the simulation in a clean way: SWIFT will periodically check whether such a stop file exists, and stop the code cleanly when it does such that it can be restarted cleanly.
diff --git a/doc/RTD/source/Skeleton/index.rst b/doc/RTD/source/Skeleton/index.rst
index 01ce5891a53824bb1beed41904447599e4a57ee4..9269ce1b361f992760a27339ffcdae75a7cab24b 100644
--- a/doc/RTD/source/Skeleton/index.rst
+++ b/doc/RTD/source/Skeleton/index.rst
@@ -1,4 +1,5 @@
.. Code Skeleton
+ November 2018
Mladen Ivkovic
Code Skeleton
@@ -15,6 +16,7 @@ for some of the main routines of SWIFT.
main
engine
interaction
+ particles
misc
glossary
todo
diff --git a/doc/RTD/source/Skeleton/interaction.rst b/doc/RTD/source/Skeleton/interaction.rst
index c73f639c811af81395885e54dd9c44811a65b3b4..8c4f3c665b0ef0eeb8c3c6a44dfdc9ea2994f81a 100644
--- a/doc/RTD/source/Skeleton/interaction.rst
+++ b/doc/RTD/source/Skeleton/interaction.rst
@@ -1,4 +1,5 @@
-.. error.h
+.. How to print messages; error.h
+ November 2018
Mladen Ivkovic
diff --git a/doc/RTD/source/Skeleton/main.rst b/doc/RTD/source/Skeleton/main.rst
index e5bc451e21095435259bbdedeff24f55ab4ba844..44f8b5260c14fb733958a474b976910aa984f204 100644
--- a/doc/RTD/source/Skeleton/main.rst
+++ b/doc/RTD/source/Skeleton/main.rst
@@ -1,4 +1,5 @@
.. main.c
+ November 2018
Mladen Ivkovic
diff --git a/doc/RTD/source/Skeleton/misc.rst b/doc/RTD/source/Skeleton/misc.rst
index 40b7a863f4ef43041169022bc01b09692c196cd3..6e8368447e957f272c763b5ebebc1ec68a447c56 100644
--- a/doc/RTD/source/Skeleton/misc.rst
+++ b/doc/RTD/source/Skeleton/misc.rst
@@ -1,10 +1,11 @@
-.. Misc stuff that I don't know where else to pu
+.. Misc stuff that I don't know where else to put
+ November 2018
Mladen Ivkovic
-.. _misc_glossary:
+.. _misc:
-Miscellaneous Stuff and Glossary
---------------------------------------
+Miscellaneous Stuff
+-----------------------
Here's a list of stuff that I didn't know where else to put, but seem to be important concepts.
diff --git a/doc/RTD/source/Skeleton/particles.rst b/doc/RTD/source/Skeleton/particles.rst
new file mode 100644
index 0000000000000000000000000000000000000000..bb9fc57ca672fb0dd4b08fec81ca425f44e66983
--- /dev/null
+++ b/doc/RTD/source/Skeleton/particles.rst
@@ -0,0 +1,134 @@
+.. particle initialisation
+ November 2018
+ Mladen Ivkovic
+
+
+
+
+.. _particles:
+
+Particles
+-----------------------------
+
+
+General Information
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+
+**Particle Types**
+
+
+SWIFT utilizes multiple particle types, defined as different structs. The exact definition used depends on the methods you chose to use. The following are descriptions of the defaut choice: 'minimal' hydro and 'default' gravity.
+For details on others, have a look at the files themselves. The particle types are and they contain: (NOTE: 'ALFS' is an abbreviation for 'At Last Full Step')
+
+- ``xpart``: a particle type containing particle fields not needed during the SPH loops over neighbours.
+
+ .. code :: text
+
+ offset between current position and position at last tree requild
+ offset between the current position and position at the last sort
+ velocity ALFS
+ gravitational acceleration ALFS
+ internal energy ALFS
+ additional data for cooling information
+
+- ``part``: hydro (SPH) particles
+
+ .. code-block :: none
+
+ particle ID
+ pointer to corresponding gravity particle
+ position
+ velocity
+ acceleration
+ mass
+ smoothing length
+ internal energy
+ time derivative of internal energy
+ density
+ force and density related quantities in a struct, stored as an union
+ chemistry information in a chemistry_part_data struct [if no chemistry is selected, this will use no memory]
+ time bin / time step length of particle
+
+- ``gpart``: gravity particles
+
+ .. code-block :: none
+
+ particle ID or array index of hydro particle with which this gpart is linked
+ position
+ velocity
+ acceleration
+ mass
+ time bin/time step length of particle
+ type of gravity particle (dark matter, gas, star...)
+
+- ``spart``: star particles
+
+ .. code-block :: none
+
+ particle ID
+ pointer to corresponding gravity particle
+ particle position
+ offset between current position and position at last tree rebuild
+ particle velocity
+ mass
+ cutoff radius
+ time bin/time step length of particle
+ struct with density related quantities
+
+
+The multiple particle types are used to separate various interactions and enables the tasks to compute the interactions independently of each other simultaneously on the same particle.
+Hydro and star particles have a corresponding gravity particle, respectively, which are linked by pointers.
+As you can see, some particle properties are present multiple times in different particle types. While requiring more memory than absolutely necessary, this allows to boost performance by keeping memory local to each CPU as well as easier MPI communications because particles of each type encapsulate the data they need in their own structs.
+
+
+
+
+
+
+
+
+
+
+.. _particle_init:
+
+Particle Initialisation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This section describes how particles are stored and handled during initialisation, and what state they're left in before the main simulation loop starts, in case the simulation is started from scratch or restarted.
+
+
+General Information
+~~~~~~~~~~~~~~~~~~~~~
+
+
+- the pointers to the ``gparts``, ``parts`` and ``sparts`` arrays are defined at the beginning of ``main.c`` to make sure that they're always in scope. They are pointers to arrays of the respective types and are initialized to ``NULL``.
+-
+
+
+
+
+Fresh Start
+~~~~~~~~~~~~~~~~~~~~
+- the particles are read in in ``main.c`` after all necessary stuff has been set up. Depending on what libraries you use, they will be read in slightly differently:
+
+ - If you have parallel HDF5: ``read_ic_parallel`` from ``parallel_io.c`` is called
+
+ - Every MPI task reads in the total number of particles of each type
+ - Particles are assigned to a MPI task by evenly dividing up contiguous sections of every particle type among all MPI tasks. Every MPI task now knows that it will initially contain ``N[particle_type]`` particles of type ``particle type``, which will be a fraction of ``N_total[particle_type]``.
+ - particle arrays for ``parts``, ``gparts`` and ``sparts`` are allocated with size ``N[particle_type]`` and set to zero.
+ - prepare which particle fields to read from the IC files by calling ``hydro_read_particles``, ``darkmatter_read_particles`` and ``stars_read_particles``
+ - every MPI task reads in it's own share of particle data using HFD5 magic.
+
+ TODO: left off at parallel_io.c line 881. Check how exactly arrays are populated.
+
+ - If you don't have parallel HDF5, but use MPI:
+
+ - If you only use non-parallel HDF5:
+
+
+
+
+
+Restart
+~~~~~~~~~~~~~~~~~~~~
diff --git a/doc/RTD/source/Skeleton/todo.rst b/doc/RTD/source/Skeleton/todo.rst
index a61df6397e2460f67f937416608e79528fa7672c..f696b8b772b6bc032988eb8c9a4bdb8e4f3a342e 100644
--- a/doc/RTD/source/Skeleton/todo.rst
+++ b/doc/RTD/source/Skeleton/todo.rst
@@ -1,4 +1,5 @@
-.. main.c
+.. Todo
+ November 2018
Mladen Ivkovic
@@ -7,6 +8,14 @@ TODO
------------------
+Questions
+~~~~~~~~~~~~~~
+
+inhibited particles (engine.c/547)
+
+
+
+
.. _domain_decomposition:
Domain Decomposition