Suppose you want to find the rate of change of a function f. This
rate of change should depend on *where you are* and *in
what direction* you're moving.

Think of standing on the side of a hill. The rate of change certainly depends on where on the hill you're standing. But even if you know where you're standing, the rate of change --- the slope of the hill --- depends on what direction you move in. If you move straight uphill, the rate of change is large and positive; if you move straight downhill, the rate of change is large and negative. If you move along the hill "sideways", the rate of change is 0, because your altitude doesn't change.

You can say "where you are" by giving a *point*; you
can say "what direction you're moving in" by giving a
*vector*.

Suppose, then, that you have a function f with multiple inputs but only one output. How rapidly is f changing at a point p in the direction of the vector v?

You can use the same procedure that you use to define the ordinary
derivative: Move a little bit, measure the average change, then take
the limit as the amount you move goes to 0. Here, then, is the
definition of the * directional derivative* of f
at p in the direction of v:

Notice that is just the amount you moved to get from p to .

I'll clean this up, and also obtain a formula which is better for computation. Let , so . As , I have , so

This formula is a little better. Notice that is a *unit* vector. This
makes sense, because the only function of is to "point out" a direction. You wouldn't
want a bigger to give a bigger rate of change.

However, you can use the Chain Rule to do better. Let

Notice that . Then

The formula is

You could also write the formula above as

Here is an important interpretation.

* Proposition.* (a) points in the direction of most rapid
increase of f, and points in the
direction of most rapid decrease of f.

(b) is the rate of most rapid increase of f, and is the rate of most rapid decrease of f.

(c) The gradient vector at a point is perpendicular to the level
curve (or level surface, or in general, the * level
set* ) of the function.

* Proof.* If is the angle between and , I have

( because is a unit vector.)

The last expression is largest when --- that is when . So the directional derivative is largest when points in the same direction as . And in that direction (since ), the value of the directional derivative is .

This establishes (a) and (b) for "most rapid increase", and similar reasoning gives the statements for "most rapid decrease".

For simplicity, I'll consider (c) in the case of a level curve. Suppose is a function of 2 variables. A level curve is a curve . Suppose I parametrize this curve by

Thus, .

Then differentiating this equation using the Chain Rule, I have

Since is the tangent vector to the curve, the last equation says that the gradient and the tangent vector are perpendicular. This is what it means for the gradient vector to be perpendicular to the curve.

In other words, the gradient gives the "biggest rate of change". If you want the rate of change in another direction, you scale the gradient down by taking the scalar component in the desired direction.

First of all, the formula above gives us a convenient way of computing the directional derivative.

* Example.* The picture below shows that graph of
. This surface is called the
* monkey saddle*. (Do you see why?)

Find the directional derivative of f at in the direction of .

The gradient of f is

Note that . So the directional derivative at in the direction of is

This is the rate of change at f at the point in the direction of .

Tou can think of the directional derivative as being obtained from the gradient by "scaling down" the gradient in the direction of . Here "scaling down" means that you take the scalar component of on .

The picture below illustrates this idea. Each segment in the picture below has been scaled so its length is the magnitude of the directional derivative in the direction in which the segment points.

Notice that the largest segments are in the directions and --- the directions of the gradient and its negative, respectively. The segment shrink in the perpendicular directions, because the rates of change perpendicular to the gradient --- i.e. along level curves --- are 0.

* Example.* Find the rate of change of at the point in the direction toward the origin. Is f
increasing or decreasing in this direction?

First, compute the gradient at the point:

Next, determine the direction vector. I'm at and I'm looking toward the origin . Therefore,

Make this into a unit vector by dividing by its length:

Finally, take the dot product of the unit vector with the gradient:

f is increasing in this direction, since the directional derivative is positive.

* Example.* For a differentiable function , it is known that the directional
derivative at in the direction of is , and the
directional derivative at in the direction of
the point is .

What is ?

Write

Then

First, the directional derivative in the direction of is , so

Likewise, the directional derivative in the direction of the point is . The vector from to is , so

From I get . Plug this into and solve for a:

Then . Thus, .

is actually not a *vector*, but a
*function* which produces a vector when you plug in a point.
For example, if , then

So , but , and so on. You get a different vector at each point.

A function which takes a point as input and produces a vector as
output is called a * vector field*. In this case,
you can think of as
"attaching" a vector to each point in the plane.

* Example.* Sketch the graph of together with its gradient field.

The gradient is . Here are the graph of with the gradient field "underneath":

Notice that the arrows in the field point in the steepest uphill direction for the surface above: The gradient points in the direction of most rapid increase.

Notice also that the *size* of an arrow in the picture varies
from point to point. The size of an arrow is the rate of most rapid
increase.

* Example.* Sketch the gradient field, the graph,
and the level curves for .

This is the gradient field for :

This is the graph of :

Here are the level curves for together with the gradient field:

At points "on the hill", the
gradient points *toward* the top of the hill. At points "in the valley", the gradient
points *away from* the bottom of the valley. In general, the
gradient points in the *direction most rapid increase*; that
is, in the *steepest uphill direction* at a point.

The reason for the quotes in the last paragraph --- and a common
source of confusion --- is that the graph (the surface) is a
3-dimensional object, while the gradient is a 2-dimensional object.
If the graph is like the surface of the earth, then the gradient
field is like a *map* of the surface.

Here is an application of the gradient to constructing tangent planes. It's useful when you can't easily express a surface either parametrically or as a graph of a function.

* Example.* Find the equation of the tangent
plane to

If you set , , , the equation is satisfied.
Hence, the point *is* on the surface.

Since it's hard to solve the equation for z, you can't use the normal vector formula

But you can get a normal using the gradient and a little trick.

Define

w is a function of 3 variables. If I set , I get the original equation. Thus, the original
surface is the * level surface* for the new function w. Therefore, will be perpendicular to the level surface
at a point.

The gradient is

Thus, is perpendicular to the level surface --- that is, to the original surface --- at .

Hence, the tangent plane is

Copyright 2018 by Bruce Ikenaga