The Data Model (Intermediate)

4. Invalid Positions Component
Yet another handy component to know about is called "invalid positions". There is also an "invalid connections" component, but let's concentrate on "invalid positions". Very often in real-world data collection, something unexpected happens, usually involving the silly sensor breaking or running out of battery power or some similar annoying little item. Or, in computational science, you may hit some kind of boundary condition that you want to flag differently from "good" data. The outcome is that not all the data within an array is "good" data and you'd like some convenient way to mark the bad stuff so it's not used in calculations, statistics, or even shown in the image. "Invalid positions" to the rescue! This is an array equal in size to the "positions" array (in one of its two forms). The data type is byte, so the only two values that can occur in this array are byte(0) and byte(1). Byte(1) means "invalid". When DX encounters a Field with "positions" and "invalid positions" Components, it masks off all the "positions" that have a corresponding (dependent) "invalid positions" value of byte(1).

Small example:

X	Y	Data	Invalid Positions
0	0	2.4	00    // 00 or 01 indicate byte type data values
0	1	3.4	00
1	0	-999	01   // this value is "invalid"
1	1	5.1	00

As always, the "positions" (X and Y) would reside in one Component, the "data" in another, and the "invalid positions" in another. As always, the "data" and "invalid positions", being dependent on "positions", are listed within their own Component Arrays in the same order as the fully expanded "positions" list, thus preserving the one-to-one relationship that we see in the rows in the example.

This masking operation is all done under the covers for free. All you have to do is figure out how to get those byte(1)'s in the "invalid positions" array in the first place!

There are several ways to do this. The very simplest is to use Colormap and Color. Set the minimum value in the Colormap Editor to a value greater than the actual minimum in your data set (or set the max value less than the real max in the data). Any data that falls outside the range being colored by this colormap will be marked invalid and will not show up in the Image. Holey data, Batman!

 

Try it. Either open a sample that uses Colormap (like Color.net) or whip up a quick test net all by yourself, like:

Import "watermolecule.dx"
MapToPlane
Colormap // remember how to hook up Colormap and Color!
Color
Image

Execute, then go back and modify the Colormap Editor range values by typing in different values and watching the Image change. You should see parts of the MapToPlane disappear when you make the Colormap range smaller than the data range.

 

No amount of bending the HSVO (Hue, Saturation, Value, Opacity) curves will change the min-max range. If you aren't sure how to use features in the Colormap Editor, open the Editor, then use the Context-Sensitive Help cursor (question mark) to click on the Editor window itself.

 

Import and Export: Import; Realization: MapToPlane; Special: Colormap; Transformation: Color; Rendering: Image. Don't forget to data-drive both Color and Colormap with the output of MapToPlane. Use Execute on Change and keep the Image window visible while you make changes in the Colormap Editor for the greatest interactivity. You need not change any parameters in MapToPlane, but feel free to experiment with them. The default "plane" receives data that is very non-linearly distributed. Use the Colormap Editor Options menu to select Axis Display->Log(Histogram) to assist your explorations.

There are other ways to generate "invalid positions", but to use them, we must take a detour to discuss DX's crown jewel: Compute.