Least third-order cumulant method with adaptive regularization parameter selection for neural networks

摘要

This paper introduces an interesting property of the least third-order cumulant objective function. The property is that the solution is optimal when the gradients of Mean Squares error and third-order cumulant error are zero vectors. The optimal solutions are independent of the value of regularization parameter λ. Also, an adaptive regularization parameter selection method is derived to control the convergences of Mean Squares error and the cumulant error terms. The proposed selection method is able to tunnel through the sub-optimal solutions, of which the locations are controllable, via changing the value of the regularization parameter. Consequently, the least third-order cumulant method with the adaptive regularization parameter selection method is theoretically capable of estimating an optimal solution when it is applied to regression problems.