Example: Removing unwanted data

From time to time, a point does not report a plausible realistic value. For example, a value may get corrupted coming over the network. This erroneous value could be a very high or a negative number. Such data can render Px views and charts meaningless, especially if you are using aggregation or rollup, or if you are passing the data to another function. Typically, an organization may have to send the data outside of the system to cleanse it or manually modify the histories. This topic documents two algorithms. The first filters out corrupt data. The second not only filters the data out, but also changes the data to a valid value.

Demand Range Filter Algorithm

This algorithm determines if a value falls within a valid range.

Figure 19.   Demand range filter algorithm flowchart
Image

Wire sheet view

Figure 20.   Demand range filter used to cleanse erroneous data
Image