2024 Nobel Prize Winner's AI Breakthrough: Restricted Boltzmann Machines

Geoffrey Hinton, a 2024 Nobel Prize recipient in Physics, is celebrated for his pioneering work in AI, particularly with Restricted Boltzmann Machines (RBMs). RBMs are a type of unsupervised learning model where only inputs are used to learn features without output labels. They consist of visible and hidden binary neurons, with the goal to minimize the energy of joint configurations. The energy function, inspired by the Boltzmann distribution, includes interaction terms between layers and bias terms for each unit. Calculating the partition function Z, which normalizes the probability distribution, is computationally intensive due to the need to sum over all possible configurations. To bypass this, Gibbs Sampling, a Markov Chain Monte Carlo method, is employed for inference. Training involves minimizing the negative log-likelihood, using contrastive divergence to approximate the negative phase of the gradient. This method uses truncated Gibbs Sampling to estimate the gradient, typically with one iteration. Hinton’s work has provided a foundation for understanding and implementing RBMs, significantly advancing the field of machine learning.

Source: towardsdatascience.com