Contents
Which is the predictive distribution of a Gaussian process?
): (1) So, the predictive distribution given the model parameters is Gaussian: p(yjx;w) = N(w>x;˙2 n ) (2) We can make two extensions for the linear regression model: (1) Regularization (Ridge Regression) We model w as a random variable and assume a prior distribution over parameters p(w).
What are the drawbacks of a Gaussian process?
Its main drawback is that it only allows a limited flexibility; if the relationship between input and output can- not reasonably be approximated by a linear function, the model will give poor predictions. In this section we first discuss the Bayesian treatment of the linear model.
How to derive kernel ridge regression from Gaussian distributions?
Homework: verify that using Eq.(6) for predictions is exactly kernel ridge regression as derived in Lecture 9. hint: use the fact that the mean of a Gaussian distribution is also it’s mode. A di\erent (maybe nicer) way to derive Gaussian process regression is to think about modeling fdirectly (instead of y)!
How to do a Gaussian process regression ( GPR )?
For a practical implementation of Gaussian process regression (gpr), we need a couple of ingredients: \let’s use a zero-mean gp !re-scale y i’s to have zero mean se the Cholesky decomposition of K+ ˙2 n
How are Gaussian processes used in machine learning?
If there are multiple test inputs X , then K = k(X ;X ). This is a Gaussian Process regression model we can use for machine learning. It extends the kernel ridge regression model with an entire predictive distribution giving us a principled way to model predictive uncertainty!
How to solve the 2 Gaussian processes problem?
2 Gaussian Processes Problem: fis a (in\\fnite-dimensional) function, but multivariate Gaussians are \\fnite-dimensional. Solution: Let’s extend multivariate Gaussians to in\\fnite dimensions!
What is the covariance of a multivariate Gaussian?
1 Multivariate Gaussian distributions The multivariate Gaussian can be defined in terms of its mean, µ, a p x 1 vector, and its covariance, Σ, p x p positive definite, symmetrical, invertible matrix. The covariance for a pair of components i and j: σij = E[xixj]−E[xi]E[xj] (1) The variance for a single ith component: σii = E[x2 i]−E[xi]2 (2)
How to control variance in a ridge regression?
To control variance, we might regularize the coefficients i.e., Might control how large the coefficients grow Might impose the ridge constraint: minimize Xn i=1 (yi−β⊤zi)2s.t. Xp j=1 β2 j≤t ⇔minimize (y −Zβ)⊤(y −Zβ) s.t. Xp j=1 β2 j≤t By convention (very important!):