Study guides · Introductory

Derivatives and gradients: read local change

Use explicit functions and numbers to understand rates of change, partial derivatives, gradients, and the chain rule.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Interpret a derivative as a local response
  2. Collect partial derivatives into a gradient
  3. Follow the chain rule through a computation
  4. Do not turn local information into a global guarantee
  5. Predict change along a specified displacement
  6. Ask Logeia about this articleCheck your understanding

Interpret a derivative as a local response

A derivative describes how a function responds to a sufficiently small change in its input. For f(x)=x squared, the derivative is 2x, so the slope at x=3 is 6. Increasing x by 0.01 predicts an output increase of about 0.06. The exact increase is 0.0601. The small discrepancy is useful: differentiation gives a local linear approximation, not an assertion that every finite change follows the tangent line exactly.

Collect partial derivatives into a gradient

For f(x,y)=x squared plus 3y squared, differentiating with respect to x while holding y fixed gives 2x. Holding x fixed and differentiating with respect to y gives 6y. The gradient is the vector(2x,6y), which becomes(2,12) at(1,2). A small displacement(dx,dy) changes the function by approximately 2dx plus 12dy. Each component therefore measures sensitivity along one coordinate direction under a specified choice of coordinates.

Follow the chain rule through a computation

For f(x)=(3x+1) squared, separate the inner operation u=3x+1 from the outer operation u squared. Multiply the outer derivative 2u by the inner derivative 3 to obtain 6(3x+1). A computation graph applies the same reasoning to many elementary operations. When a variable influences the output along several paths, add the contributions from those paths. This accumulation is essential; multiplying derivatives along only one path can omit part of the dependence.

Do not turn local information into a global guarantee

A zero gradient does not establish a minimum. It can also occur at a maximum, a saddle point, or a flat region. Nondifferentiable functions and discrete choices require additional treatment rather than an unqualified application of the usual derivative. Finite differences can check a computed gradient, but an extremely small step can amplify floating-point cancellation. Keep the mathematical derivative, its numerical approximation, and the assumptions of an optimization method separate when interpreting the result.

Predict change along a specified displacement

At a point with gradient(2,12), choose a displacement of 0.01 in x and minus 0.01 in y. The linear prediction is 2 times 0.01 plus 12 times minus 0.01, or minus 0.10. Contributions from both coordinates matter, so increasing one input does not necessarily increase the total output. When variables carry physical units, track the derivative units as well. Comparing component magnitudes without considering coordinate scale can misrepresent which change is practically significant. This interpretation depends on the chosen coordinates.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Prepare for AI systems · STEP 12

Choose a learning path