Contents
When do we use Gaussian process to model data?
We realize there is a problem if we use Gaussian Process regression to model the data: Gaussian Process regression models continuous functions f (x). We cannot require such a function to return only 1 or 0. A Gaussian Process regression model may return 1.1 at some x locations, and 0.3 at other locations. This problem is easy to solve.
Is the Gaussian process prior always multivariate?
As the name may suggest, the Gaussian Process prior is always a multivariate Gaussian distribution. It models the function at the top of the previous figure. To refresh your memory, take a look at Understanding Gaussian Process, the Socratic Way.
Can a Bayesian machine learn a non Gaussian distribution?
Modern Bayesian machine learning can comfortably handle non-Gaussian distributions. I will describe the Gaussian Process binary classification m o del. It uses a Gaussian prior but a Bernoulli likelihood.
How is the Bernoulli distribution related to the Gaussian process?
The Bernoulli distribution has a single parameter — the probability between 0 and 1, that describes how likely an event happens. To create a connection between this probability and our Gaussian Process function f (x), we define the function g (f (x)) that squashes the real-valued Gaussian Process function f (x) into a value in the range [0, 1].
What to do when things are not Gaussian?
Variational Gaussian Process — What To Do When Things Are Not Gaussian introduces variational inference to allow us to use a non-Gaussian likelihood in a Gaussian Process model. Sparse and Variational Gaussian Process — What To Do When Data is Large introduces inducing variables to allow us to scale a Gaussian Process model to large datasets.
When is a Gaussian assumption not the best assumption?
However, if say, your goal is to generate new data similar to some training data and that you know that the data you would like to generate should be strictly positive, then a Gaussian assumption may not be the best one.