Raw data filter example
This filter prepares data for meaningful analysis. This example illustrates how the filter works and suggests what a meaningful
dataset would look like.
Here is a raw dataset representing power readings taken hourly beginning at 11 am for 10 hours:
| Timestamp | Value | Status |
|---|---|---|
| 15/12/21 11:00 | 20 | {} |
| 15/12/21 12:00 | 25 | {} |
| 15/12/21 13:00 | 15 | {} |
| 15/12/21 14:00 | -20 | {} |
| 15/12/21 15:00 | 85 | {} |
| 15/12/21 16:00 | 130 | {} |
| 15/12/21 17:00 | 60 | {} |
| 15/12/21 18:00 | 150 | {} |
| 15/12/21 19:00 | -15 | {} |
| 15/12/21 20:00 | 90 | {} |
Values higher than 100 and lower than zero are outliers that will inaccurately skew the resulting analysis. A data definition is configured as follows:
The raw data filter outputs these values:
| Timestamp | Value | Status |
|---|---|---|
| 15/12/21 11:00 | 20 | {} |
| 15/12/21 12:00 | 25 | {} |
| 15/12/21 13:00 | 15 | {} |
| 15/12/21 14:00 | removed by the filter | |
| 15/12/21 15:00 | 85 | {} |
| 15/12/21 16:00 | removed by the filter | |
| 15/12/21 17:00 | 60 | {} |
| 15/12/21 18:00 | removed by the filter | |
| 15/12/21 19:00 | removed by the filter | |
| 15/12/21 20:00 | 90 | {} |
Using linear interpolation the resulting dataset looks like this:
| Timestamp | Value | Status | Trend Flags |
|---|---|---|---|
| 15/12/21 11:00 | 20 | {} | {} |
| 15/12/21 12:00 | 25 | {} | {} |
| 15/12/21 13:00 | 15 | {} | {} |
| 15/12/21 14:00 | 50 | {} | {li} |
| 15/12/21 15:00 | 85 | {} | {} |
| 15/12/21 16:00 | 72.5 | {} | {li} |
| 15/12/21 17:00 | 60 | {} | {} |
| 15/12/21 18:00 | 70 | {} | {li} |
| 15/12/21 19:00 | 80 | {} | {li} |
| 15/12/21 20:00 | 90 | {} | {} |