![](/images/graphics-bg.png)
Deep Neural Networks with Multistate Activation Functions
Joint Authors
Cai, Chenghao
Xu, Yanyan
Ke, Dengfeng
Su, Kaile
Source
Computational Intelligence and Neuroscience
Issue
Vol. 2015, Issue 2015 (31 Dec. 2015), pp.1-10, 10 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2015-09-10
Country of Publication
Egypt
No. of Pages
10
Main Subjects
Abstract EN
We propose multistate activation functions (MSAFs) for deep neural networks (DNNs).
These MSAFs are new kinds of activation functions which are capable of representing more than two states, including the N-order MSAFs and the symmetrical MSAF.
DNNs with these MSAFs can be trained via conventional Stochastic Gradient Descent (SGD) as well as mean-normalised SGD.
We also discuss how these MSAFs perform when used to resolve classification problems.
Experimental results on the TIMIT corpus reveal that, on speech recognition tasks, DNNs with MSAFs perform better than the conventional DNNs, getting a relative improvement of 5.60% on phoneme error rates.
Further experiments also reveal that mean-normalised SGD facilitates the training processes of DNNs with MSAFs, especially when being with large training sets.
The models can also be directly trained without pretraining when the training set is sufficiently large, which results in a considerable relative improvement of 5.82% on word error rates.
American Psychological Association (APA)
Cai, Chenghao& Xu, Yanyan& Ke, Dengfeng& Su, Kaile. 2015. Deep Neural Networks with Multistate Activation Functions. Computational Intelligence and Neuroscience،Vol. 2015, no. 2015, pp.1-10.
https://search.emarefa.net/detail/BIM-1057744
Modern Language Association (MLA)
Cai, Chenghao…[et al.]. Deep Neural Networks with Multistate Activation Functions. Computational Intelligence and Neuroscience No. 2015 (2015), pp.1-10.
https://search.emarefa.net/detail/BIM-1057744
American Medical Association (AMA)
Cai, Chenghao& Xu, Yanyan& Ke, Dengfeng& Su, Kaile. Deep Neural Networks with Multistate Activation Functions. Computational Intelligence and Neuroscience. 2015. Vol. 2015, no. 2015, pp.1-10.
https://search.emarefa.net/detail/BIM-1057744
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1057744