Hydrologic Time Series Anomaly Detection Based on Flink
Joint Authors
Ye, Feng
Liu, Zihao
Liu, Qinghua
Wang, Zhijian
Source
Mathematical Problems in Engineering
Issue
Vol. 2020, Issue 2020 (31 Dec. 2020), pp.1-12, 12 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2020-05-28
Country of Publication
Egypt
No. of Pages
12
Main Subjects
Abstract EN
The data mining and calculation of time series in critical application is still worth studying.
Currently, in the field of hydrological time series, most of the detection of outliers focus on improving the specificity.
To efficiently detect outliers in massive hydrologic sensor data, an anomaly detection method for hydrological time series based on Flink is proposed.
Firstly, the sliding window and the ARIMA model are used to forecast data stream.
Then, the confidence interval is calculated for the prediction result, and the results outside the interval range are judged as alternative anomaly data.
Finally, based on the historical batch data, the K-Means++ algorithm is used to cluster the batch data.
The state transition probability is calculated, and the anomaly data are evaluated in quality.
Taking the hydrological sensor data obtained from the Chu River as experimental data, experiments on the detection time and outlier detection performance are carried out, respectively.
The results show that when calculating the tens of millions of data, the time costed by two slaves is less than that by one slave, and the maximum reduction is 17.43%.
The sensitivity of the evaluation is increased from 72.91% to 92.98%.
In terms of delay, the average delay of different slaves is roughly the same, which is maintained within 20 ms.
It shows that, under big data platform, the proposed algorithm can effectively improve the computational efficiency of hydrologic time series detection for tens of millions of data and has a significant improvement in sensitivity.
American Psychological Association (APA)
Ye, Feng& Liu, Zihao& Liu, Qinghua& Wang, Zhijian. 2020. Hydrologic Time Series Anomaly Detection Based on Flink. Mathematical Problems in Engineering،Vol. 2020, no. 2020, pp.1-12.
https://search.emarefa.net/detail/BIM-1194261
Modern Language Association (MLA)
Ye, Feng…[et al.]. Hydrologic Time Series Anomaly Detection Based on Flink. Mathematical Problems in Engineering No. 2020 (2020), pp.1-12.
https://search.emarefa.net/detail/BIM-1194261
American Medical Association (AMA)
Ye, Feng& Liu, Zihao& Liu, Qinghua& Wang, Zhijian. Hydrologic Time Series Anomaly Detection Based on Flink. Mathematical Problems in Engineering. 2020. Vol. 2020, no. 2020, pp.1-12.
https://search.emarefa.net/detail/BIM-1194261
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1194261