What is a non-convex optimization problem?
A non-convex optimization problem is any problem where the objective or any of the constraints are non-convex, as pictured below. Such a problem may have multiple feasible regions and multiple locally optimal points within each region.
Is NP non-convex optimization hard?
Nonconvex optimization is NP-hard, even the goal is to compute a local minimizer. In applied disciplines, however, nonconvex problems abound, and simple algorithms, such as gradient descent and alternating direction, are often surprisingly effective.
What is a non-convex function?
A non-convex function is wavy – has some ‘valleys’ (local minima) that aren’t as deep as the overall deepest ‘valley’ (global minimum). Optimization algorithms can get stuck in the local minimum, and it can be hard to tell when this happens.
What makes the problem of non convex optimization hard?
Non-Convex Problems •Anything that’s not convex What makes non-convex optimization hard? •Potentially many local minima •Saddle points •Very flat regions •Widely varying curvature Source: https://commons.wikimedia.org/wiki/File:Saddle_point.svg But is it actually that hard? •Yes, non-convex optimization is at least NP-hard
What should I do in a non convex optimization lecture?
•Also, be sure to do at least: 1. Summarize the paper 2. Discuss the paper’s strengths andweaknesses 3. Discuss the paper’s impact. Non-Convex Optimization CS6787 Lecture 7—Fall 2017 Review —We’ve covered many methods •Stochastic gradient descent •Mini-batching •Momentum •Variance reduction •Nice convergence proofs that give us a rate
Which is an example of a non convex problem?
Examples of non-convex problems •Matrix completion, principle component analysis •Low-rank models and tensor decomposition •Maximum likelihood estimation with hidden variables •Usually non-convex •The big one: deep neural networks Why are neural networks non-convex? •They’re often made of convex parts! •This by itself would be convex.
Is the update rule the same for non convex functions?
•Local convergence to the global minimum •Global convergence to the global minimum Non-convex Stochastic Gradient Descent Stochastic Gradient Descent •The update rule is the same for non-convex functions •Same intuition of moving in a direction that lowers objective