Contents
What is Gaussian process regression GPR?
Gaussian process regression (GPR) is a nonparametric, Bayesian approach to regression that is making waves in the area of machine learning. GPR has several benefits, working well on small datasets and having the ability to provide uncertainty measurements on the predictions.
What is a kernel in GPR?
A kernel (or covariance function) describes the covariance of the Gaussian process random variables. Together with the mean function the kernel completely defines a Gaussian process.
What is Gaussian process used for?
Gaussian processes are useful in statistical modelling, benefiting from properties inherited from the normal distribution. For example, if a random process is modelled as a Gaussian process, the distributions of various derived quantities can be obtained explicitly.
How is the Gaussian process used in hyperparameter optimization?
The Gaussian process is modeling the probability of our model’s performance ( f) as a function of the hyperparameters ( x ), and so at any value of x, it gives us a normal distribution for its prediction of the value of f:
Is there a tutorial on Bayesian hyperparameter optimization?
For more information, Brochu et al., 2010 is a great tutorial on Bayesian optimization, which includes an intro to Gaussian processes and info about several different types of acquisition functions. But enough math – on to the code! Bayesian optimization isn’t specific to finding hyperparameters – it lets you optimize any expensive function.
What is the error of the Gaussian process?
The gray dots show the error for hyperparameters which have been tried. Also shown is the Gaussian process’s estimate as to the error over all hyperparameter combinations (blue line) and its 1 sigma confidence interval (shaded area).
How does a Gaussian process model the dependent variable Y?
A Gaussian process models the dependent variable y (in our case, the cross-validated performance) as being drawn from a N -dimensional multivariate normal distribution: We’ll just normalize the data such that it has a mean of 0, and use m u = 0.