[Paraview] Python interpreter not being created for coprocessing?

[Vanmoer, Mark W]

Andy, ah, the problem was programmer error ;) So I now have my fortran image program that's similar to the python wiki example working and I think I'm ready to go back to the actual application and get working on it.

While I was figuring this out, though, I came up with a question that's maybe better for the VTK list. When the vtkImageData extents get set, they overlap. E.g., in the python wiki example there's this line:

imageData.SetExtent(mpi.rank*2, (mpi.rank+1)*2, 0, 5, 0, 5)

if np == 2, then there'll be partitions with extents (0, 2, 0, 5, 0, 5) and (2, 4, 0, 5, 0, 5). This isn't thread safe, right? Because the plane X == 2 will be written to by both ranks? Do the extents actually need to touch? I'm thinking of this as a discrete space so (0, 2, 0, 5, 0,5) and (3, 4, 0, 5, 0, 5) would cover all the nodal points.

Mark

From: Vanmoer, Mark W
Sent: Friday, October 12, 2012 5:05 PM
To: 'Andy Bauer'
Cc: paraview at paraview.org
Subject: RE: [Paraview] Python interpreter not being created for coprocessing?

Hi Andy,

Thanks for asking. Had to put this aside for a while, but where I am now is I decided I wanted to alter the C++ wiki example to work with a partitioned vtkImageData set, similar to the Python example. Then translate that to Fortran, so I would have a better idea of what I needed to do for the actual application.

C++ works fine, but in the Fortran version, with mpirun -np 2 I'm getting the following error:
ERROR: In /home/mvanmoer/builds/ParaView/repo/ParaView/VTK/Common/ExecutionModel/vtkExecutive.cxx, line 753
vtkCompositeDataPipeline (0x3cf5130): Algorithm vtkXMLImageDataWriter(0x3cf4d20) returned failure for request: vtkInformation (0x3cf70c0)
  Debug: Off
  Modified Time: 114245
  Reference Count: 1
  Registered Events: (none)
  Request: REQUEST_DATA
  FROM_OUTPUT_PORT: -1
  FORWARD_DIRECTION: 0
  ALGORITHM_AFTER_FORWARD: 1

I don't get that error for any other np, 1, 3, 4, 5, 6, 8, 16, etc. Wrong values in the output .vtis though (except for np 1). My guess is I'm not sending the Fortran array to the coprocessor correctly. I'm trying to use a 3D fortran array and I think I might need to switch to a 1D.

Mark


From: Andy Bauer [mailto:andy.bauer at kitware.com]
Sent: Friday, October 12, 2012 4:23 PM
To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Hi Mark,

I just wanted to check in with you to see how far you got. I'm interested in seeing if you got it to build with all of the options you were looking for.

Andy
On Thu, Sep 27, 2012 at 10:40 AM, Andy Bauer <andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>> wrote:
Yeah, for structured grids you'll need to set the extent on each process as well as use vtkCPInputDataDescription::SetWholeExtent() to specify the global grid extent. The partitioned arrays in addfields() for single component tuples needs to be the same length as the number of points/cells on the local, partitioned grid.

The extent support for coprocessing is a git dodgy as we're trying to do things in a way that the pipeline doesn't like. One solution to that is to use a vtkMultiPieceDataSet but I haven't tried that enough yet to see how well it works.

Andy

On Tue, Sep 25, 2012 at 6:12 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Ah, apparently I'm not setting the extents with SetExtent(), vtkCPInputDataDescription::SetWholeExtent() as mentioned on slide 79 of the SC12 slides.

I've been working off of the Phasta example. So if I understand the general idea right, I need to be sending the extent information from the application to the coprocessor call, then
I need to make sure that those also get passed to the image data set creator, and I need to also send the partitioned arrays to the addfields_() call?

Mark


From: Vanmoer, Mark W
Sent: Tuesday, September 25, 2012 3:28 PM

To: 'Andy Bauer'
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: RE: [Paraview] Python interpreter not being created for coprocessing?

Right, single process. I'm working with one code that's all CPU and another that's mixed CPU/GPU. Coprocessing works well for both. There's also MPI versions of the codes that I'm trying to instrument.

The computational mesh is completely regular so I've been using vtkImageData. The MPI versions split the mesh over the ranks. When I run an MPI version with  -np > 1, then I get the following for each process:
ERROR: In /home/mvanmoer/builds/ParaView/ParaView-3.14.1-Source/ParaViewCore/VTKExtensions/vtkKdTreeGenerator.cxx, line 168
vtkKdTreeGenerator (0x5ac6430): RegionIDs cannot be 0.

It doesn't crash, but it gives an incorrect image. I'll look through your updated tutorials, I'd been looking through the SC10 slides.

Mark

From: Andy Bauer [mailto:andy.bauer at kitware.com]<mailto:[mailto:andy.bauer at kitware.com]>
Sent: Tuesday, September 25, 2012 2:15 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?


On Tue, Sep 25, 2012 at 12:21 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
I pulled it directly from stage/13460_CP_FortranAdaptor.

Great, then it was me after all that fixed it!


I'm now at the point where it runs great on a single node with GPUs, I'm trying to extend it to multiple nodes, which I think involves setting up vtkMultiBlockDataSets? I was going to look through the NPICAdaptor for hints on that.

By single node I'm assuming that you mean a single process, correct? What's the type of grid that you're planning on using? The PhastaAdaptor is an example of dealing with an unstructured grid. I also just updated the coprocessing tutorials page (now at http://paraview.org/Wiki/Coprocessing_Tutorials) to include the tutorial I'll be doing at SC12. It should have some useful information for doing this. You could just do a polydata input, just set the center of the sphere source in your example based on the process rank (e.g. Sphere->SetCenter(10*rank, 0, 0); ) so that it's easy to tell the difference between each processes input part/domain partition.

Andy


Mark


From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Tuesday, September 25, 2012 10:13 AM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Did you get my changes from ParaView's stage branch? My changes didn't make it into ParaView master yet so I'm not sure what changed that caused things to work now. Well, if it's working I'm glad -- I'm not sure I can personally take credit for it though :)

Let me know how it goes with the GPU stuff.

Andy
On Mon, Sep 24, 2012 at 5:48 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Hi Andy,

I've tested this and it works with both gcc/mpich2 and pgi/openmpi.

Thanks again!
Mark

From: Vanmoer, Mark W
Sent: Wednesday, September 19, 2012 1:30 PM
To: Andy Bauer
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: RE: [Paraview] Python interpreter not being created for coprocessing?

Hi Andy, that sounds like some deep debugging, thanks for your effort. I'll test this and let you know. It's on my to do list to learn CMake, I've had the book on my desk for a month now.

Also, I made another Fortran example that's closer in spirit to the wiki example, which uses the DataGenerator class as the C++ does, if you're interested.

Thanks,
Mark
________________________________
From: Andy Bauer
Sent: 9/19/2012 1:12 PM
To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?
Hi Mark,

I finally got around to fixing this. It's in the stage branch (remotes/stage/13460_CP_FortranAdaptor) and hopefully will go into the master branch on the next gatekeeper review. There were 2 issues fixed in this:
1) for using newer GCC compilers and other compilers that hide symbols in the library (http://gcc.gnu.org/wiki/Visibility) the FortranAdaptor API methods weren't available to other libraries and executables trying to link to them.
2) Since 3.14.1 I changed around the way ParaView gets initialized through the coprocessing library (now through vtkCPProcessor::Initialize() as it should be instead of vtkCPPythonScriptPipeline::Initialize()). I forgot to make the change in the FortranAdaptorAPI.cxx which caused the python interpretor to not be initialized properly, as you noticed.

Thanks for the help in finding this problem. Sorry for the slow response on it. I created a CMakeLists.txt file to help build as I've been using CMake too long and can't remember how to work with Makefiles anymore! I'm attaching it in case you'd like to use it. I didn't test it with an install of ParaView as I really only work directly with ParaView builds.

Let me know if this solves your problems. I seem to remember there being more as far as using CUDA and PGI but hopefully this gets you close.

By the way, I'll probably use what you sent me to create a Fortran test or example to help with making sure it doesn't get broken in the future.

Andy
On Wed, Sep 12, 2012 at 4:42 PM, Vanmoer, Mark W <mvanmoer at illinois.edu<mailto:mvanmoer at illinois.edu>> wrote:
Tarball attached, it's not exactly the wiki example because I didn't want to bother with passing a DataGenerator object back and forth from C++ to Fortran, so I just make a sphere. The pipeline script is a shrink filter generated by pvclient.

I checked CMake and it looks like it's finding the same version of python.
Mark

From: Andy Bauer [mailto:andy.bauer at kitware.com<mailto:andy.bauer at kitware.com>]
Sent: Wednesday, September 12, 2012 1:02 PM

To: Vanmoer, Mark W
Cc: paraview at paraview.org<mailto:paraview at paraview.org>
Subject: Re: [Paraview] Python interpreter not being created for coprocessing?

Can you send me your Fortran version of the wiki example?