Support for XMF file descriptors with multiple particle types.

This fixes #127 (closed) and adds support for all particle types within the XMF files. Both in single node and MPI modes.

Title changed from Multi types xmf to Support for XMF file descriptors with multiple particle types.

Not quite correct. The closing tag for the gas gridtype doesn't encompass the attributes, that is:

<Grid Name="Gas" GridType="Uniform">
<Topology TopologyType="Polyvertex" Dimensions="1024128"/>
<Geometry GeometryType="XYZ">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
</Geometry>
 <!-- Done geometry for Gas, start of particle fields list -->
</Grid> <!-- End of meta-data for parttype=Gas -->
<Attribute Name="Coordinates" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
</Attribute>
<Attribute Name="Velocities" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Velocities</DataItem>
</Attribute>
<Attribute Name="Masses" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Masses</DataItem>
</Attribute>
<Attribute Name="SmoothingLength" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/SmoothingLength</DataItem>
</Attribute>
<Attribute Name="InternalEnergy" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/InternalEnergy</DataItem>
</Attribute>
<Attribute Name="ParticleIDs" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/ParticleIDs</DataItem>
</Attribute>
<Attribute Name="Acceleration" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Acceleration</DataItem>
</Attribute>
<Attribute Name="Density" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Density</DataItem>
</Attribute>

should be organised like:

<Grid Name="Gas" GridType="Uniform">
<Topology TopologyType="Polyvertex" Dimensions="1024128"/>
<Geometry GeometryType="XYZ">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
</Geometry>
 <!-- Done geometry for Gas, start of particle fields list -->
<Attribute Name="Coordinates" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
</Attribute>
<Attribute Name="Velocities" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Velocities</DataItem>
</Attribute>
<Attribute Name="Masses" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Masses</DataItem>
</Attribute>
<Attribute Name="SmoothingLength" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/SmoothingLength</DataItem>
</Attribute>
<Attribute Name="InternalEnergy" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/InternalEnergy</DataItem>
</Attribute>
<Attribute Name="ParticleIDs" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/ParticleIDs</DataItem>
</Attribute>
<Attribute Name="Acceleration" AttributeType="Vector" Center="Node">
<DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Acceleration</DataItem>
</Attribute>
<Attribute Name="Density" AttributeType="Scalar" Center="Node">
<DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Density</DataItem>
</Attribute>

</Grid> <!-- End of meta-data for parttype=Gas -->

Was this with or without MPI ?

With MPI.

Thanks.

Added 1 commit:

• 39498fc7 - Don't close the group tags on the rank 0

Ok, that's now correct. The tag was being closed on rank 0 before the arrays were actually being written.

Parallel MPI will not compile:

parallel_io.c(272): error #167: argument of type "long long" is incompatible with parameter of type "char *"
    if (mpi_rank == 0) writeXMFline(xmfFile, fileName, name, N_total, dim, type);
                                                             ^

parallel_io.c(272): error #165: too few arguments in function call
    if (mpi_rank == 0) writeXMFline(xmfFile, fileName, name, N_total, dim, type);
                                                                               ^

In file included from hydro_io.h(28),
                 from parallel_io.c(338):
./hydro/Gadget2/hydro_io.h(78): error #55: too many arguments in invocation of macro "writeArray"
               N, 3, parts, N_total, mpi_rank, offset, x, us, UNIT_CONV_LENGTH);
                                                              ^

In file included from hydro_io.h(28),
                 from parallel_io.c(338):
./hydro/Gadget2/hydro_io.h(77): error: expression must have pointer-to-object type
    writeArray(h_grp, fileName, xmfFile, partTypeGroupName, "Coordinates", DOUBLE,
    ^

In file included from hydro_io.h(28),
                 from parallel_io.c(338):
./hydro/Gadget2/hydro_io.h(77): error: identifier "x" is undefined
    writeArray(h_grp, fileName, xmfFile, partTypeGroupName, "Coordinates", DOUBLE,
.
.
.
.

Oh, sorry, I forgot to mention that I had dropped support for that version until we have a better i/o strategy. Tests for this are on-going.

Should I remove this version from the configure scripts and the main() for now ?

Yes, please. If we don't support it make that clear, however, if it really is coming back then modify the configure script to issue an error instead.

Added 1 commit:

• 1f9b0dbe - Extended the multi-types i/o to the parallel-HDF5 version

Added 1 commit:

• 2d3e60b5 - Code formatting

I have updated the code such that the parallel-HDF5 version also works.

I'll keep this possibility alive for a little longer. At the moment it is still the best i/o we have.

Thanks, all seems to be working for me now. Accepting.

mentioned in commit c4d1b553

Status changed to merged