Optimization of Echo State Networks by Covariance Matrix Adaption Evolutionary Strategy

Liu, K; Zhang, J

doi:10.23919/IConAC.2018.8749124

Optimization of Echo State Networks by Covariance Matrix Adaption Evolutionary Strategy

Lookup NU author(s): Dr Kai Liu, Dr Jie Zhang ORCiD

Downloads

Full text for this publication is not currently held within this repository. Alternative links are provided below where available.

Abstract

Echo state networks (ESNs) have been shown to be an effective alternative to conventional recurrent neuralnetworks due to its simple training process and good fitting performance of time series modelling tasks. In the primary ESN principle, the random setting of reservoir is considered to be the main advantage of ESN. However, because of the randomly generated connectivity and weight parameters, appropriate setting of the structural parameters which can significantly influence the modelling accuracy is considered a key issue in building ESN models. Evolutionary Strategy (ES) has been shown being a powerful stochastic global optimization method. Moreover, covariance matrix adaption evolutionary strategy (CMA-ES) is an artistically and parallel search method which transforms the searching covariance matrix to guide the best search direction. This paper proposes a CMA-ES-ESN method to optimize several structural parameters of an ESN such as reservoir size, leak rate and spectral radius factor. Finally, the results are compared with those from the original ESN and GA-ESN, ESN optimized by genetic algorithm.