Investigating the prediction capabilities of the simple recurrent neural network on real temporal sequences

The goal of this paper is to evaluate the prediction capabilities of the simple recurrent neural network (SRNN). The main focus is on the prediction of non-orthogonal vector components of real temporal sequences. A prediction problem is formulated in which the input is a component of a real sequence and the output is a prediction of the next component of the sequence. A method is developed to train a single SRNN to predict the components of sequences belonging to multiple classes. The selection of a distinguishing initial context vector for each class is proposed to improve the prediction performance of the SRNN. A systematic method to re-train the SRNN with noisy exemplars is developed to improve the prediction generalization of the network. Through the methods developed in the paper, it is demonstrated that: (a) a single SRNN can be trained to predict, contextually, the components of real temporal sequences belonging to different classes, (b) the prediction error of the SRNN can be decreased by using a distinguishing initial context vector for each class, and (c) the prediction generalization of the SRNN can be increased significantly by re-training the network with noisy exemplars.