View Issue Details [ Jump to Notes ] [ Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0006698ParaView(No Category)public2008-04-01 15:272011-01-13 17:00
ReporterBerk Geveci 
Assigned ToUtkarsh Ayachit 
PrioritynormalSeverityminorReproducibilityalways
StatusclosedResolutionunable to reproduce 
PlatformOSOS Version
Product Version 
Target Version3.6Fixed in Version 
Summary0006698: Multi-block writer does not work in parallel
DescriptionvtkXMLCompositeDataWriter does not work in parallel. Each processor writes its blocks using a serial writer. If multiple processors share the same block, they all write to the same file, causing corruption. Maybe we should use parallel writers?
TagsNo tags attached.
Project
Topic Name
Type
Attached Files

 Relationships
related to 0006779closedUtkarsh Ayachit Cannot save Exodus as anything 

  Notes
(0011020)
Berk Geveci (administrator)
2008-04-01 16:20

 Instead of using parallel writer (which is difficult due to the possibility of null or empty nodes), we could have the composite reader/writer handle this case. Every piece would make sure to assign a different filename and when more than one node have a block for the same node, the file would like the following:
<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0" byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
  <vtkMultiBlockDataSet>
    <DataSet index="0" file="mb/mb_0.vtp"/>
    <Block index="1">
      <DataSet index="0" file="mb/mb_1_0.vtp"/>
      <DataSet index="0" file="mb/mb_1_1.vtp"/>
      <DataSet index="1" file="mb/mb_2.vtp"/>
    </Block>
  </vtkMultiBlockDataSet>
</VTKFile>

The reader would deal with appending datasets with the same index. Only datasets that can be appended would be supported (i.e. no structured datasets)
(0013225)
Utkarsh Ayachit (administrator)
2008-08-29 08:35

 vtkXMLCompositeDataWriter now works in parallel. Each
process now always writes out files with process number encoded into the
filename thus assuring that they are unique. Then when writing out the meta-file
the root process locates all leaf nodes that have non-null datasets on more than
1 process and inserts a multi-piece at that location.

eg. writing out can.ex2 on 3 processes results in following XML

<?xml version="1.0"?>
<VTKFile type="vtkMultiBlockDataSet" version="1.0" byte_order="LittleEndian" compressor="vtkZLibDataCompressor">
  <vtkMultiBlockDataSet>
    <Block index="0">
      <Piece index="0">
        <DataSet index="0" file="can/can_0_0.vtu"/>
        <DataSet index="1" file="can/can_1_0.vtu"/>
        <DataSet index="2" file="can/can_2_0.vtu"/>
      </Piece>
      <Piece index="1">
        <DataSet index="0" file="can/can_0_1.vtu"/>
        <DataSet index="1" file="can/can_1_1.vtu"/>
        <DataSet index="2" file="can/can_2_1.vtu"/>
      </Piece>
    </Block>
    <Block index="1"/>
    <Block index="2"/>
    <Block index="3"/>
    <Block index="4"/>
    <Block index="5"/>
    <Block index="6"/>
    <Block index="7"/>
  </vtkMultiBlockDataSet>
</VTKFile>


/cvsroot/ParaView3/ParaView3/VTK/IO/vtkXMLCompositeDataWriter.cxx,v <-- VTK/IO/vtkXMLCompositeDataWriter.cxx
new revision: 1.3; previous revision: 1.2
/cvsroot/ParaView3/ParaView3/VTK/IO/vtkXMLCompositeDataWriter.h,v <-- VTK/IO/vtkXMLCompositeDataWriter.h
new revision: 1.2; previous revision: 1.1
/cvsroot/ParaView3/ParaView3/VTK/Parallel/vtkXMLPMultiBlockDataWriter.cxx,v <-- VTK/Parallel/vtkXMLPMultiBlockDataWriter.cxx
new revision: 1.2; previous revision: 1.1
/cvsroot/ParaView3/ParaView3/VTK/Parallel/vtkXMLPMultiBlockDataWriter.h,v <-- VTK/Parallel/vtkXMLPMultiBlockDataWriter.h
new revision: 1.2; previous revision: 1.1
(0013365)
Alan Scott (manager)
2008-09-08 17:08

 Pushed back for feedback. I sent comments in a separate e-mail, along with an example bad dataset.
(0015251)
Utkarsh Ayachit (administrator)
2009-02-20 10:46

 Alan,

Can you retest this? I cannot reproduce the problem anymore.

Thanks.
(0015502)
Alan Scott (manager)
2009-03-02 21:34

 Tested client/ remote server.

 Issue History
Date Modified Username Field Change
2008-04-01 15:27 Berk Geveci New Issue
2008-04-01 16:20 Berk Geveci Note Added: 0011020
2008-04-10 17:59 Ken Moreland Status backlog => tabled
2008-04-10 17:59 Ken Moreland Assigned To => Berk Geveci
2008-06-03 11:06 Utkarsh Ayachit Assigned To Berk Geveci => Burlen
2008-08-07 13:17 Utkarsh Ayachit Assigned To Burlen => Berk Geveci
2008-08-19 15:18 Utkarsh Ayachit Relationship added related to 0006779
2008-08-27 15:37 Berk Geveci Assigned To Berk Geveci => Utkarsh Ayachit
2008-08-29 08:35 Utkarsh Ayachit Status tabled => @80@
2008-08-29 08:35 Utkarsh Ayachit Resolution open => fixed
2008-08-29 08:35 Utkarsh Ayachit Note Added: 0013225
2008-09-08 17:08 Alan Scott Status @80@ => @20@
2008-09-08 17:08 Alan Scott Resolution fixed => reopened
2008-09-08 17:08 Alan Scott Note Added: 0013365
2009-02-17 14:09 Ken Moreland Category 3.4 => 3.6
2009-02-20 10:46 Utkarsh Ayachit Note Added: 0015251
2009-02-20 10:46 Utkarsh Ayachit Status @20@ => @80@
2009-02-20 10:46 Utkarsh Ayachit Resolution reopened => unable to reproduce
2009-03-02 21:34 Alan Scott Note Added: 0015502
2009-03-02 21:34 Alan Scott Status @80@ => closed
2009-05-13 13:46 Utkarsh Ayachit Target Version Multi-block support => 3.6
2011-01-13 17:00 Source_changeset_attached => VTK master a2bd8391
2011-01-13 17:00 Source_changeset_attached => VTK master 020ef709
2011-06-16 13:10 Zack Galbreath Category => (No Category)

Copyright © 2000 - 2018 MantisBT Team