Bayesian Statistics as an Alternative to Gradient Descent in Sequence Learning
Rainer Spiegel, Ludwig Maximilians UniversitÃ¤t
iJET Volume 2, Number 3, ISSN 1863-0383 Publisher: International Association of Online Engineering, Kassel, Germany
Recurrent neural networks are frequently applied to simulate sequence learning applications such as language processing, sensory-motor learning, etc. For this purpose, they often apply a truncated gradient descent (=error correcting) learning algorithm. In order to converge to a solution that is congruent with a target set of sequences, many iterations of sequence presentations and weight adjustments are typically needed. Moreover, there is no guarantee of finding the global minimum of error in a multidimensional error landscape resulting from the discrepancy between target values and the networkâ??s prediction. This paper presents a new approach of inferring the global error minimum right from the start. It further applies this information to reverse-engineer the weights. As a consequence, learning is speeded-up tremendously, whilst computationally-expensive iterative training trials can be skipped. Technology applications in established and emerging industries will be discussed.
Spiegel, R. (2007). Bayesian Statistics as an Alternative to Gradient Descent in Sequence Learning. International Journal of Emerging Technologies in Learning (iJET), 2(3),. Kassel, Germany: International Association of Online Engineering.