Skip to content
• Unconstrained optimization:
min f(x)

For unconstrianted optimization, a necessary condition is $\nabla f(x)=0$.

The interpretion is: if $\nabla f(x)\ne 0$, f(x) can be still reduced if x moves along $-\nabla f(x)$.

• Constrained optimization:
min f(x)
subject to h(x)=0

For unconstrianted optimization, a necessary condition is $\nabla f(x)=\lambda \nabla h(x)$.

The interpretion is: project $\nabla f(x)$ to the tangent plane of $h(x)=0$. Move x along with the projection, f(x) can be reduced.

Keep in mind that: the gradient $\nabla f(x)$ plays a very important role in optimization.

Advertisements
No comments yet