ParaView and VTK on Xeon Phi (KNC)

From KitwarePublic
Jump to: navigation, search

Cross-compiling VTK/ParaView for Xeon Phi Co-processor (KNC)

VTK and Paraview can be cross-compiled and run on the Intel Xeon Phi co-processors (MIC) by using CMake’s cross-compiling support. Following is an overview of the steps:

  1. Cross-compile OSMesa library for Xeon Phi.
  2. Compile VTK/ParaView for the host system. This step is only needed once for each version.
  3. Create CMake configuration files for cross-compiling (needed just once for each target architecture).
  4. Cross-compile VTK/Paraview for Xeon Phi.

These steps are described in detail in the following sections.

Building OSMesa for Xeon Phi:

Since there is no graphics acceleration or X server available for Xeon Phi, VTK and ParaView need to be configured and built for offscreen rendering using the OSMesa library. The latest version of Mesa that is known to work on the MIC without any code changes is 10.1.1. To simplify the build process, configure Mesa with the minimal options needed for OSMesa. It is recommended to use gallium-llvm instead of classic mesa for better performance and features. The commands to build OSMesa for Xeon Phi are shown below. The options, --enable-texture-float and DEFAULT_SOFTWARE_DEPTH_BITS=31 are recommended for VTK. [1]

export MESA_SRC_DIR=
export MESA_BUILD_DIR=
export MESA_INSTALL_DIR=
 
cd $MESA_SRC_DIR
autoreconf -fi
 
cd $MESA_BUILD_DIR
$MESA_SRC_DIR/configure \
 CXX="icpc -mmic" CC="icc -mmic" F77="ifort -mmic" FC="ifort -mmic" \
 CXXFLAGS="-O2 -D__NOT_HAVE_DRM_H -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \
 CFLAGS="-O2 -D__NOT_HAVE_DRM_H -DDEFAULT_SOFTWARE_DEPTH_BITS=31" \
 --disable-xvmc --disable-glx --disable-dri \
 --with-dri-drivers="" --with-gallium-drivers="swrast" \
 --enable-texture-float \
 --disable-shared-glapi --disable-egl \
 --with-egl-platforms="" \
 --enable-gallium-osmesa --enable-gallium-llvm=yes \
 --prefix=$MESA_INSTALL_DIR \
 --host=x86_64-k1om-linux
make
make install


Build VTK/ParaView for the Host System:

Some of the executables built during the VTK/ParaView build process are also executed during the build process. Since Xeon Phi executables cannot be run on the host, the required executables need to be pre-built for the host. These executables are called Compile Tools. If a full host build is not needed, its possible to build just the compile tools by running the following commands:

For VTK:

export VTK_SRC_DIR=
export VTK_HOST_BUILD_DIR=
 
cd $VTK_HOST_BUILD_DIR
cmake $VTK_SRC_DIR
make vtkCompileTools

For ParaView:

export PARAVIEW_SRC_DIR=
export PARAVIEW_HOST_BUILD_DIR=
 
cd $PARAVIEW_HOST_BUILD_DIR
cmake $PARAVIEW_SRC_DIR
make pvCompileTools


CMake Configuration Files for Cross-Compiling:

CMake needs two types of configuration files for cross-compiling VTK/ParaView.

The first is the tool-chain file that tells CMake the details about the cross compiling environment. Following is an example tool-chain file for cross-compiling for Xeon Phi.

   1.    SET(CMAKE_SYSTEM_NAME Linux)
   2.    SET(CMAKE_SYSTEM_PROCESSOR k1om)
   3.    SET(CMAKE_SYSTEM_VERSION 1)
   4.    
   5.    # specify the cross compiler
   6.    SET(CMAKE_C_COMPILER   icc -mmic)
   7.    SET(CMAKE_CXX_COMPILER icpc -mmic)
   8.    SET(_CMAKE_TOOLCHAIN_PREFIX  x86_64-k1om-linux-)
   9.    
  10.    # where is the target environment
  11.    SET(CMAKE_FIND_ROOT_PATH /opt/mpss/3.3/sysroots/k1om-mpss-linux)
  12.    
  13.    # search for programs in the build host directories
  14.    SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
  15.    # search for libraries and headers in the target directories
  16.    SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
  17.    SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)

Please refer to CMake Cross Compiling wiki page for details about the toolchain file.[2] Some of the values may need to be changed depending on how your host system is configured for cross-compiling. For example on some systems, the Xeon Phi version of certain third party libraries may not be installed under the CMAKE_FIND_ROOT_PATH folders. In such cases, lines 14-17 need to be commented out, otherwise those library will not be found by CMake. An example toolchain file for Xeon Phis on stampede.tacc.utexas.edu can be found in VTK/CMake/stampede-mic-toolchain.cmake.

The second configuration file is the TryRunResults.cmake file. For some types of system introspection, CMake needs to compile and run some test programs. Since Xeon Phi programs cannot be run on the host, they need to be run manually on the Phi and their results stored in a CMake cache file. To create this file, the first step is to configure the VTK build:

export MIC_TOOLCHAIN_FILE=
export VTK_MIC_BUILD_DIR=
 
cd $VTK_MIC_BUILD_DIR
cmake -DCMAKE_TOOLCHAIN_FILE=$MIC_TOOLCHAIN_FILE $VTK_SRC_DIR

The commands are similar for ParaView. The configure will fail, but it will generate a TryRunResults.cmake file. This file will have a list of SET() commands for some CMake cache variables. Each of these will be preceded by a verbose comment describing the program to execute and how to set its results to the variables. Execute these programs on the Xeon Phi and update the corresponding variables’ SET() command with their results.

The above steps only need to be performed once. Save the cache file in some other directory so that it doesn’t get overwritten. The same file can be used for future builds of both VTK and Paraview. A TryRunResults file for MICs on stampede.tacc.utexas.edu is provided at VTK/CMake/TryRunResults-Paraview-VTK-stampede-mic.cmake.

Build VTK/ParaView for Xeon Phi:

With the above files, VTK and Paraview can be configured and cross-compiled for Xeon Phi. Please note that modules that depend on system/third-party libraries, that do not have a Xeon Phi version, will have to be disabled. We have been able to build the default VTK with OSMesa and the default ParaView server (no gui) with offscreen rendering and MPI support. Following are the commands to perform the cross compilation:

VTK:

export MIC_TOOLCHAIN_FILE=
export VTK_MIC_BUILD_DIR=
 
cd $VTK_MIC_BUILD_DIR
cmake \
 -DCMAKE_TOOLCHAIN_FILE=$MIC_TOOLCHAIN_FILE \
 -DVTKCompileTools_DIR=$VTK_HOST_BUILD_DIR \
 -C$TRYRUN_RESULTS_FILE \
 -DVTK_USE_X=OFF -DVTK_OPENGL_HAS_OSMESA=ON -DVTK_USE_OFFSCREEN=ON \
 -DOSMESA_INCLUDE_DIR=$MESA_INSTALL_DIR/include \
 -DOSMESA_LIBRARY=$MESA_INSTALL_DIR/lib/libOSMesa.so \
 -DOPENGL_INCLUDE_DIR=$MESA_INSTALL_DIR/include \
 $VTK_SRC_DIR
make

ParaView:

export MIC_TOOLCHAIN_FILE=
export PARAVIEW_MIC_BUILD_DIR=
 
cd PARAVIEW_MIC_BUILD_DIR
cmake \
 -DCMAKE_TOOLCHAIN_FILE=$MIC_TOOLCHAIN_FILE \
 -DParaViewCompileTools_DIR=$PARAVIEW_HOST_BUILD_DIR \
 -C$TRYRUN_RESULTS_FILE \
 -DPARAVIEW_BUILD_QT_GUI=OFF \
 -DVTK_USE_X=OFF -DVTK_OPENGL_HAS_OSMESA=ON -DVTK_USE_OFFSCREEN=ON \
 -DOSMESA_INCLUDE_DIR=$MESA_INSTALL_DIR/include \
 -DOSMESA_LIBRARY=$MESA_INSTALL_DIR/lib/libOSMesa.so \
 -DOPENGL_INCLUDE_DIR=$MESA_INSTALL_DIR/include \
 $PARAVIEW_SRC_DIR
make

If the cross compile environment is not set up properly on the host system, CMake may find the host version of gl,glu and glut libraries which will cause problem during the build. Ensure that the following cache variable are set to NOTFOUND or null: OPENGL_gl_LIBRARY, OPENGL_glu_LIBRARY, GLUT_INCLUDE_DIR, GLUT_glut_LIBRARY. It is convenient to use ccmake instead of cmake as it allows to configure the builds by updating the cache variables interactively. It is recommended to use the system provided version of the HDF5 (set environment variable HDF5_ROOT) and Zlib libraries. The HDF5 library provided with VTK has a bug [3] in the build process which results in build failure in cross compilation mode.

Since ParaView is built for offscreen rendering, the ParaView GUI application will not be built. But the paraview server will be built and you can connect to the server from any ParaView GUI application. Multiple paraview server processes can also be lauched if Xeon Phi/MIC version of MPI library is available. We have tested with intel’s MPI library (impi).

NOTES

  1. ParaView/ParaView And Mesa 3D
  2. CMake Cross Compiling
  3. 0015139: The CMake build system for building the third part library - HDF5, does not work in cross-compile mode