A Sarsa(λ) Algorithm Based on Double-Layer Fuzzy Reasoning
Joint Authors
Liu, Quan
Mu, Xiang
Huang, Wei
Fu, Qiming
Zhang, Yonggang
Source
Mathematical Problems in Engineering
Issue
Vol. 2013, Issue 2013 (31 Dec. 2013), pp.1-9, 9 p.
Publisher
Hindawi Publishing Corporation
Publication Date
2013-12-05
Country of Publication
Egypt
No. of Pages
9
Main Subjects
Abstract EN
Solving reinforcement learning problems in continuous space with function approximation is currently a research hotspot of machine learning.
When dealing with the continuous space problems, the classic Q-iteration algorithms based on lookup table or function approximation converge slowly and are difficult to derive a continuous policy.
To overcome the above weaknesses, we propose an algorithm named DFR-Sarsa(λ) based on double-layer fuzzy reasoning and prove its convergence.
In this algorithm, the first reasoning layer uses fuzzy sets of state to compute continuous actions; the second reasoning layer uses fuzzy sets of action to compute the components of Q-value.
Then, these two fuzzy layers are combined to compute the Q-value function of continuous action space.
Besides, this algorithm utilizes the membership degrees of activation rules in the two fuzzy reasoning layers to update the eligibility traces.
Applying DFR-Sarsa(λ) to the Mountain Car and Cart-pole Balancing problems, experimental results show that the algorithm not only can be used to get a continuous action policy, but also has a better convergence performance.
American Psychological Association (APA)
Liu, Quan& Mu, Xiang& Huang, Wei& Fu, Qiming& Zhang, Yonggang. 2013. A Sarsa(λ) Algorithm Based on Double-Layer Fuzzy Reasoning. Mathematical Problems in Engineering،Vol. 2013, no. 2013, pp.1-9.
https://search.emarefa.net/detail/BIM-1009823
Modern Language Association (MLA)
Liu, Quan…[et al.]. A Sarsa(λ) Algorithm Based on Double-Layer Fuzzy Reasoning. Mathematical Problems in Engineering No. 2013 (2013), pp.1-9.
https://search.emarefa.net/detail/BIM-1009823
American Medical Association (AMA)
Liu, Quan& Mu, Xiang& Huang, Wei& Fu, Qiming& Zhang, Yonggang. A Sarsa(λ) Algorithm Based on Double-Layer Fuzzy Reasoning. Mathematical Problems in Engineering. 2013. Vol. 2013, no. 2013, pp.1-9.
https://search.emarefa.net/detail/BIM-1009823
Data Type
Journal Articles
Language
English
Notes
Includes bibliographical references
Record ID
BIM-1009823