Properties of a Batch Training Algorithm for Feedforward Networks

摘要

We examine properties for a batch training algorithm known as the output weight optimization–hidden weight optimization (OWO–HWO). Using the concept of equivalent networks, we analyze the effect of input transformation on BP. We introduce new theory of affine invariance and partial affine invariance for neural networks and prove this property for OWO–HWO. Finally, we relate HWO to BP and show that every iteration of HWO is equivalent to BP applied to whitening transformed data. Experimental results validate the connection between OWO–HWO and OWO–BP.