High Stability Anomaly Detection in Random Environments

Abstract

We propose anomaly detection to refine input data for predictive machine learning systems. When training, if there are outliers such as spike noises mixed in the input data, the quality of the trained model is deteriorated. The removing such outliers would be expected the service quality of machine learning systems improves such as autonomous vehicles and ship navigation. Conventionally, anomaly detection methods generally require the support of domain experts, and they could not treat with unstable random environments well. We propose a new anomaly detection method, which is highly stable and is capable of treating with random environments without experts. The proposed methods focus on a pairwise correlation between two input time-series, change rates of them are calculated and summarized on a quadrant chart for further analysis. The experiment using an open time-series dataset shows that the proposed methods successfully detect anomalies, and the detected data points are easily illustrated in a human-interpretable way.