MantisBT - ParaView
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0013794ParaView(No Category)public2013-01-16 16:482015-01-11 08:19
ReporterAlan Scott 
Assigned ToDavid C. Lonie 
PrioritynormalSeverityminorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOSOS Version
Product Versiongit-master 
Target Version4.2Fixed in Version4.2 
ProjectSandia
Topic Name13794_fix_mp4py
Typeincorrect functionality
Summary0013794: Remote server, find max error
DescriptionWhen running remote server, find data/ max fails with a python error.

Linux, remote server, master.
Open can.exo. All vars on. Apply.
Edit/ Find Data. Change GlobalId to EQPS. Change "is" to "is max".
Run Selection Query.

It will hang for a while, then the following gets written out:
Traceback (most recent call last):
  File "<string>", line 5, in <module>
  File "/projects/viz/paraview/ParaView/3.98.1-Mesa/Linux-redsky-x86_64/lib/paraview-3.98/site-packages/paraview/extract_selection.py", line 147, in Exec
    ds = ExecData(self, iterCD.GetCurrentDataObject(), selection)
  File "/projects/viz/paraview/ParaView/3.98.1-Mesa/Linux-redsky-x86_64/lib/paraview-3.98/site-packages/paraview/extract_selection.py", line 122, in ExecData
    mask = eval(selection_node.GetQueryString(), globals(), new_locals)
  File "<string>", line 1, in <module>
  File "/projects/viz/paraview/ParaView/3.98.1-Mesa/Linux-redsky-x86_64/lib/paraview-3.98/site-packages/paraview/vtk/algorithms.py", line 304, in global_max
    MPI.COMM_WORLD.Allreduce(None, [M, MPI.DOUBLE], MPI.MAX)
  File "Comm.pyx", line 544, in mpi4py.MPI.Comm.Allreduce (/projects/viz/paraview/src/ParaView/3.98.1/ParaView3/Utilities/mpi4py/src/mpi4py.MPI.c:63736)
  File "message.pxi", line 585, in mpi4py.MPI._p_msg_cco.for_allreduce (/projects/viz/paraview/src/ParaView/3.98.1/ParaView3/Utilities/mpi4py/src/mpi4py.MPI.c:23520)
  File "message.pxi", line 523, in mpi4py.MPI._p_msg_cco.for_cro_send (/projects/viz/paraview/src/ParaView/3.98.1/ParaView3/Utilities/mpi4py/src/mpi4py.MPI.c:23007)
  File "message.pxi", line 106, in mpi4py.MPI.message_simple (/projects/viz/paraview/src/ParaView/3.98.1/ParaView3/Utilities/mpi4py/src/mpi4py.MPI.c:19218)
TypeError: message: expecting buffer or list/tuple

This is a bug.
Steps To Reproduce
Additional Information
TagsNo tags attached.
Relationships
related to 0010958closed Utkarsh Ayachit Edit/Find data command - find and calculate min and max variable for each time step of an animation 
parent of 0014565closed David C. Lonie Python Calculator and Extract Selection broken for composite datasets 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2013-01-16 16:48Alan ScottNew Issue
2013-01-16 17:41Utkarsh AyachitNote Added: 0030153
2013-01-16 17:51Utkarsh AyachitNote Added: 0030154
2013-01-17 17:10Utkarsh AyachitNote Added: 0030178
2013-01-24 22:34Utkarsh AyachitTarget Version => 4.0
2013-02-11 18:32Utkarsh AyachitAssigned To => Utkarsh Ayachit
2013-02-13 12:51Alan ScottRelationship addedrelated to 0010958
2014-01-16 13:47Utkarsh AyachitAssigned ToUtkarsh Ayachit => David C. Lonie
2014-01-16 13:58Utkarsh AyachitNote Added: 0032182
2014-01-16 16:59Utkarsh AyachitTarget Version4.0 => 4.2
2014-02-11 12:26David C. LonieRelationship addedparent of 0014565
2014-03-03 14:42David C. LonieTopic Name => 13794_fix_mp4py
2014-03-03 14:42David C. LonieNote Added: 0032444
2014-03-03 14:42David C. LonieStatusbacklog => gatekeeper review
2014-03-03 14:42David C. LonieResolutionopen => fixed
2014-03-09 13:24Utkarsh AyachitFixed in Version => git-master
2014-03-09 13:24Utkarsh AyachitStatusgatekeeper review => customer review
2014-03-09 13:24Utkarsh AyachitNote Added: 0032462
2014-03-21 16:26Alan ScottNote Added: 0032529
2014-03-21 16:28Utkarsh AyachitStatuscustomer review => todo
2014-03-21 20:09Alan ScottNote Added: 0032531
2014-03-21 20:09Alan ScottStatustodo => closed
2014-08-29 11:00Utkarsh AyachitFixed in Versiongit-master => 4.2
2015-01-11 08:19Utkarsh AyachitSource_changeset_attached => ParaView master 392223ec
2015-01-11 08:19David C. LonieSource_changeset_attached => ParaView master 10d058a4

Notes
(0030153)
Utkarsh Ayachit   
2013-01-16 17:41   
commit e1114c112528cfea3cdc0e62e4afc14382c38635
Author: Utkarsh Ayachit <utkarsh.ayachit@kitware.com>
Date: Wed Jan 16 17:39:46 2013 -0500

    BUG 0013794: Fix mpi4py API change issue.
    
    Since we recently updated to mpi4py 1.3, looks like MPI.Reduceall(..) with empty
    send buffer no longer works. We need to create a send and receive buffer to use.
    Updated the algorithms.py file to reflect the API change.
    
    Change-Id: I9a58d698dd3df3678d97ae6f969807e65522d0d7
(0030154)
Utkarsh Ayachit   
2013-01-16 17:51   
DEVELOPER NOTE:

The next bug to fix to address this issue is that vtkPythonExtractSelection doesn't correctly handle composite datasets that may have non-aligned null blocks between processes e.g. when can.ex2 is opened in parallel.
(0030178)
Utkarsh Ayachit   
2013-01-17 17:10   
DEVELOPER NOTE:

We have the following issues with min/max.

1) Currently, the min/max/mean are computing per-block. That's not what users would expect.

To fix this, we need to update the algorithms.py file in ParaView to handle composite datasets better. The user does foo(array), we should pass array as a custom array which itself comprises of arrays from all the blocks in the composite dataset.

2) Min/Max/Mean across processes has issues when any process has no data.

The collective MPI calls deadlock. We'll need to address that by ensuring that all processes participate in the collective operations even if they don't have the requested arrays. That may require us to share information about arrays on all processes with everyone.
(0032182)
Utkarsh Ayachit   
2014-01-16 13:58   
DEVELOPER NOTE:

"max" is currently not available in the UI. To reproduce this issue you can use the following "Query" instead:

EQPS == global_max(EQPS)
(0032444)
David C. Lonie   
2014-03-03 14:42   
Applying Utkarsh's patch to change the mpi4py API fixes the issue.
(0032462)
Utkarsh Ayachit   
2014-03-09 13:24   
SUMMARY
---------------------------------------------
Topics merged into master:
        13794_fix_mp4py
        14572-search-series-parameter-widget
        14620_FixRulerCrash
        disabled-multiblock-pieces
        fix_example_qt5
        optimize_amrconnectivity
        point-sprite-plugin-crash
        pvweb-visualizer
        remove-extra-resources
(VTK) update-vtk-20140303


-------------------------------------------
(0032529)
Alan Scott   
2014-03-21 16:26   
The query above is hanging (not crashing) remote server. Be sure to check before closing this bug.
(0032531)
Alan Scott   
2014-03-21 20:09   
"is max" no longer exists, since it was incorrect in parallel. Thus, I am closing this bug. The meat of this bug (give us the ability to find min and max) will be fixed in one of the related bugs off this one, currently under development. I will also test it then.