Read · 1 min
Suppose that we would like to find the extreme values of some function subject to some constraint .
To do so, let us draw level surfaces for the function : these are surfaces such that every point on the surface has the same value of .
The level surface might intersect the constraint surface in some region or curve. If this is the case, then changing the value of by a small amount will result in a new level curve, which also intersects the constraint surface in some region or curve. Thus, this level curve is not representative of an extreme value of . This is similar to the case of a single variable function with for some value . Since the values of can go either way, does not have a maximum or minimum at .
However, the other case is that the level surface is tangent to the constraint curve in a single point. In this case, changing the value for the level curve by even a small amount in one direction leads to no more intersection between the level surface and the constraint curve. Thus, achieves either a maximum or minimum at this point.
For the two surfaces to be tangent, their normal vectors must be parallel. Thus, one normal vector is the other vector multiplied by a scalar, known as a Lagrange multiplier.