Skip to content
Snippets Groups Projects

Support for XMF file descriptors with multiple particle types.

Merged Matthieu Schaller requested to merge multi_types_xmf into master

This fixes #127 (closed) and adds support for all particle types within the XMF files. Both in single node and MPI modes.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Matthieu Schaller Title changed from Multi types xmf to Support for XMF file descriptors with multiple particle types.

    Title changed from Multi types xmf to Support for XMF file descriptors with multiple particle types.

  • Not quite correct. The closing tag for the gas gridtype doesn't encompass the attributes, that is:

    <Grid Name="Gas" GridType="Uniform">
    <Topology TopologyType="Polyvertex" Dimensions="1024128"/>
    <Geometry GeometryType="XYZ">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
    </Geometry>
     <!-- Done geometry for Gas, start of particle fields list -->
    </Grid> <!-- End of meta-data for parttype=Gas -->
    <Attribute Name="Coordinates" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
    </Attribute>
    <Attribute Name="Velocities" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Velocities</DataItem>
    </Attribute>
    <Attribute Name="Masses" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Masses</DataItem>
    </Attribute>
    <Attribute Name="SmoothingLength" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/SmoothingLength</DataItem>
    </Attribute>
    <Attribute Name="InternalEnergy" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/InternalEnergy</DataItem>
    </Attribute>
    <Attribute Name="ParticleIDs" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/ParticleIDs</DataItem>
    </Attribute>
    <Attribute Name="Acceleration" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Acceleration</DataItem>
    </Attribute>
    <Attribute Name="Density" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Density</DataItem>
    </Attribute>
    

    should be organised like:

    <Grid Name="Gas" GridType="Uniform">
    <Topology TopologyType="Polyvertex" Dimensions="1024128"/>
    <Geometry GeometryType="XYZ">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
    </Geometry>
     <!-- Done geometry for Gas, start of particle fields list -->
    <Attribute Name="Coordinates" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/Coordinates</DataItem>
    </Attribute>
    <Attribute Name="Velocities" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Velocities</DataItem>
    </Attribute>
    <Attribute Name="Masses" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Masses</DataItem>
    </Attribute>
    <Attribute Name="SmoothingLength" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/SmoothingLength</DataItem>
    </Attribute>
    <Attribute Name="InternalEnergy" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/InternalEnergy</DataItem>
    </Attribute>
    <Attribute Name="ParticleIDs" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="8" Format="HDF">output_019.hdf5:/PartType0/ParticleIDs</DataItem>
    </Attribute>
    <Attribute Name="Acceleration" AttributeType="Vector" Center="Node">
    <DataItem Dimensions="1024128 3" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Acceleration</DataItem>
    </Attribute>
    <Attribute Name="Density" AttributeType="Scalar" Center="Node">
    <DataItem Dimensions="1024128" NumberType="Double" Precision="4" Format="HDF">output_019.hdf5:/PartType0/Density</DataItem>
    </Attribute>
    
    </Grid> <!-- End of meta-data for parttype=Gas -->
    
  • Was this with or without MPI ?

  • Added 1 commit:

    • 39498fc7 - Don't close the group tags on the rank 0
  • Ok, that's now correct. The tag was being closed on rank 0 before the arrays were actually being written.

  • Parallel MPI will not compile:

    parallel_io.c(272): error #167: argument of type "long long" is incompatible with parameter of type "char *"
        if (mpi_rank == 0) writeXMFline(xmfFile, fileName, name, N_total, dim, type);
                                                                 ^
    
    parallel_io.c(272): error #165: too few arguments in function call
        if (mpi_rank == 0) writeXMFline(xmfFile, fileName, name, N_total, dim, type);
                                                                                   ^
    
    In file included from hydro_io.h(28),
                     from parallel_io.c(338):
    ./hydro/Gadget2/hydro_io.h(78): error #55: too many arguments in invocation of macro "writeArray"
                   N, 3, parts, N_total, mpi_rank, offset, x, us, UNIT_CONV_LENGTH);
                                                                  ^
    
    In file included from hydro_io.h(28),
                     from parallel_io.c(338):
    ./hydro/Gadget2/hydro_io.h(77): error: expression must have pointer-to-object type
        writeArray(h_grp, fileName, xmfFile, partTypeGroupName, "Coordinates", DOUBLE,
        ^
    
    In file included from hydro_io.h(28),
                     from parallel_io.c(338):
    ./hydro/Gadget2/hydro_io.h(77): error: identifier "x" is undefined
        writeArray(h_grp, fileName, xmfFile, partTypeGroupName, "Coordinates", DOUBLE,
    .
    .
    .
    .
  • Oh, sorry, I forgot to mention that I had dropped support for that version until we have a better i/o strategy. Tests for this are on-going.

  • Should I remove this version from the configure scripts and the main() for now ?

  • Yes, please. If we don't support it make that clear, however, if it really is coming back then modify the configure script to issue an error instead.

  • Added 1 commit:

    • 1f9b0dbe - Extended the multi-types i/o to the parallel-HDF5 version
  • Added 1 commit:

  • I have updated the code such that the parallel-HDF5 version also works.

  • I'll keep this possibility alive for a little longer. At the moment it is still the best i/o we have.

  • Thanks, all seems to be working for me now. Accepting.

  • Peter W. Draper mentioned in commit c4d1b553

    mentioned in commit c4d1b553

  • Peter W. Draper Status changed to merged

    Status changed to merged

Please register or sign in to reply
Loading