Collaborative detection of cyber security threats in big data

Abstract EN

In the era of big data, it is a problem to be solved for promoting the healthy development of the Internet and the Internet+, protecting the information security of individuals, institutions and countries.

Hence, this paper constructs a collaborative detection system of cyber security threats in big data.

Firstly, it describes the log collection model of Flume, the data cache of Kafka, and the data process of Esper; then it designs one-to-many log collection, consistent data cache, Complex Event Processing (CEP) data process using event query and event pattern matching; finally, it tests on the datasets and analyzes the results from six aspects.

The results demonstrate that the system has good reliability, high efficiency and accurate detection results; moreover, the system has the advantages of low cost and flexible operation.