Dynamic System Identification by Neural Network: A New, Fast Learning Method Based on Error Back Propagation

A theoretical formulation of a new fast learning method based on back propagation is presented in this paper. In contrast to the existing back propagation algorithm which is based solely on the modification of connecting weights between the units (i.e., neurons) of different layers of the neural network, the present method calculates the optimum slope of the sigmoid function at each unit together with the variation on the connecting weights. The effectiveness and versatility of the present method is verified by the system identification of (a) linear and (b) non-linear (Duffing and fluid-type) single degree of freedom mass-spring dynamic models. In all three cases, the present method excels in speed and accuracy compared to that of the existing method of fixed slope. The physics behind the faster convergence rate and better accuracy of the proposed variable slope method is illustrated and explained with the help of the effect of the slope of sigmoid function on the output level of each neuron. The enhancement of the degree of accuracy and the speed of learning of the variable slope method is due to the broadening of the range of output values of each unit by the op timization of slope of the sigmoid function. The greater the degree of non-linearity of the dynamic system, the more effective the proposed variable slope method than the fixed slope back propagation method. Despite the fact that the calculation of sensitivity of slope of sigmoid function for each unit requires some extra amount of computational effort in each cycle of training, it is numerically cost effective when its performance is judged from the degree of accuracy of learning and the total number of iterations required for satisfactory dynamic system identification.