ParaView 2.4.x User Testing
Overview
This document describes the ParaView User Testing plan.
Users
We have also identified as set of users who we have or will ask to participate in user testing. We specifically wish to test ParaView on SNL DART users in both SNL/CA and SNL/NM, including Bruce Kistler (CA), Dan Kletzli (CA), Nathan Spencer (CA), Jay Dike (CA), Michael Pasik (NM), Tre ` Shelton (NM), Michael Wong (NM), Erik Strack, Rebecca Brannon, and Allen Robinson.
Use Cases
In order to structure our user testing, we have identified a set of ParaView Use Cases submitted by some of our users. Based on this preliminary survey, we feel the following ParaView use cases would be a good subset for user testing:
- Color the mesh by scalar variables including derived variables
- Computer new variables from output variable
- Produce a plane clip through a 3D mesh
- Have the profile plots update automatically when you change time step
- Provide tech annotations including time annotations
- Provide a geometry display tool for displaying interactive flipbook animations on a desktop or multi-panel display wall
- Load all sidesets in a Genesis file
- Control the visibility of individual sidesets
- Discern the identity of sidesets
- Change the display of sidesets to surface, wireframe, and hidden line representations
- Probe coordinate and node/element numbers
- Report the closest node/element numbers to an arbitrary user-specified location
- Automate probing
- Given a node number, find out the elements that include it
- Visualize a vector field
- Create a derived variable
- Produce an XY plot
- Export extracted XY data in a CSV file
- Produce an image/screenshot
Test Plan (and Test Plan Planning)
The use cases above are somewhat vague, and as they are user-contributed use cases, not all of these desired features may be available in the current ParaView 2.4.x. Some time (completed in 1-2 weeks?) will be need by the ParaQ design team (specifically Matthew, with the help of others - Brian?) to formalize these use cases in a tutorial-like step-by-step tutorial-like test plan document. In other words, instead of simply including the direction "Color the mesh by temperature", we will formalize the test plan by documenting "Click on X, Click on Y, etc". Note that this high level of fidelity and formality will also help in the tester's (Matthew's) understanding of existing advanced ParaView functionality, which is a prerequisite of ParaView user testing.
Once we have formalized the ParaView Test plan, user testing may begin. We will perform Think-Aloud usability studies, which involves giving a user a task, such as "Color the mesh by temperature". We will NOT immediately give the user the detailed instructions of "Click on X, Click on Y, etc". While the user is performing the necessary functions, we will request that the user "thinks aloud", or tries his or her best to verbalize their current perception and cognitive activities. In exchange, we will be able to understand our typical users, and how we can improve ParaView and ParaQ to better suit the typical user's needs and expectations.
Testing will first occur in SNL/CA, and then in SNL/NM. User testing will be performed by Matthew. Testing will likely start in SNL/CA in mid April, and occur in SNL/NM in late April or early May. We will then write a detailed report on each user test.
Other usability testing techniques that may occur will include Cognitive Walkthroughs Heuristic Evaluations, both of which do NOT require live users.
Future Testing
We still plan to perform bi-monthly usability critiques, either in the form of the formal usability methods listed above, or more informal approaches. Unless the usability issue is of emergency urgency, most improvements will occur in ParaQ, and not ParaView. ParaView user testing should be completed right around the time of the first ParaQ release. At this time, we will want to perform similar user testing on ParaQ. User testing should be an iterative approach, we plan to cycle through similar user testing plans for each release phase of ParaQ.
