On-line Sequential Extreme Learning Machine Based on Recursive Partial Least Squares

This paper proposes the online sequential extreme learning machine algorithm based on the recursive partial least-squares method (OS-ELM-RPLS). It is an improvement to the online sequential extreme learning machine based on recursive least-squares (OS-ELM-RLS) introduced in (Huang et al., 2005 [1]). Like in the batch extreme learning machine (ELM), in OS-ELM-RLS the input weights of a single-hidden layer feedforward neural network (SLFN) are randomly generated, however the output weights are obtained by a recursive least-squares (RLS) solution. However, due to multicollinearities in the columns of the hidden-layer output matrix caused by presence of redundant input variables or by the large number of hidden-layer neurons, the problem of estimation the output weights can become ill-conditioned. In order to circumvent or mitigate such ill-conditioning problem, it is proposed to replace the RLS method by the recursive partial least-squares (RPLS) method. OS-ELM-RPLS was applied and compared with three other methods over three real-world data sets. In all the experiments, the proposed method always exhibits the best prediction performance.