Contents
Why is non-convex optimization hard?
The nonlinear missile dynamics, atmospheric dynamics, discrete time processes, etc result in a pretty nonlinear reaction to changes in the guidance algorithm, making the optimization hard to solve. The fact this cost function will be non-convex makes the fact it is time consuming to evaluate a big issue.
Which of the following is non convex set?
|x| = 5 is not a convex set as any two points from negative and positive x-axis if are joined will not lie in set.
How do you tell if a function is convex or concave?
To find out if it is concave or convex, look at the second derivative. If the result is positive, it is convex. If it is negative, then it is concave. To find the second derivative, we repeat the process using as our expression.
How do you determine if a function is convex or concave Hessian?
We can determine the concavity/convexity of a function by determining whether the Hessian is negative or positive semidefinite, as follows. if H(x) is positive definite for all x ∈ S then f is strictly convex.
Which is an example of a non convex problem?
Examples of non-convex problems •Matrix completion, principle component analysis •Low-rank models and tensor decomposition •Maximum likelihood estimation with hidden variables •Usually non-convex •The big one: deep neural networks Why are neural networks non-convex? •They’re often made of convex parts! •This by itself would be convex.
What makes the problem of non convex optimization hard?
Non-Convex Problems •Anything that’s not convex What makes non-convex optimization hard? •Potentially many local minima •Saddle points •Very flat regions •Widely varying curvature Source: https://commons.wikimedia.org/wiki/File:Saddle_point.svg But is it actually that hard? •Yes, non-convex optimization is at least NP-hard
Is the update rule the same for non convex functions?
•Local convergence to the global minimum •Global convergence to the global minimum Non-convex Stochastic Gradient Descent Stochastic Gradient Descent •The update rule is the same for non-convex functions •Same intuition of moving in a direction that lowers objective
What should I do in a non convex optimization lecture?
•Also, be sure to do at least: 1. Summarize the paper 2. Discuss the paper’s strengths andweaknesses 3. Discuss the paper’s impact. Non-Convex Optimization CS6787 Lecture 7—Fall 2017 Review —We’ve covered many methods •Stochastic gradient descent •Mini-batching •Momentum •Variance reduction •Nice convergence proofs that give us a rate