Detection of Trojaning Attack on Neural Networks via Cost of Sample Classification

Abstract EN

To overcome huge resource consumption of neural networks training, MLaaS (Machine Learning as a Service) has become an irresistible trend, just like SaaS (Software as a Service), PaaS (Platform as a Service), and IaaS (Infrastructure as a Service) have been.

But it comes with some security issues of untrustworthy third-party services.

Especially machine learning providers may deploy trojan backdoors in provided models for the pursuit of extra profit or other illegal purposes.

Against the redundant nodes-based trojaning attack on neural networks, we proposed a novel detecting method, which only requires the untrusted model to be tested and a small batch of legitimate dataset.

By comparing different processes of neural networks training, we found that the embedding of malicious nodes will make their parameter configuration abnormal.

Moreover, by analysing the cost distribution of test dataset on network nodes, we successfully detect the trojaned nodes in the neural networks.

As far as we know, the research on the defence against trojaning attack on neural networks is still in its infancy, and our research may shed light on the security of MLaaS in real-life scenarios.