Start with the segment inequality
A set is convex when the segment between any two of its points stays in the set. A function on a convex domain is convex when its value along that segment is no greater than the corresponding weighted average of endpoint values. This is a statement about every pair of feasible points, not merely a graph that looks bowl-shaped. In higher dimensions, visual intuition is especially unreliable. Check the domain and objective separately: a convex objective over a nonconvex feasible set does not automatically produce the guarantees of a convex optimization problem.
Use the first-order condition
For a differentiable convex function, a first-order affine approximation is a global lower bound. If the gradient vanishes in an unconstrained problem, that lower bound shows that no other point has a smaller objective. Convexity turns a local differential condition into a global guarantee. It does not guarantee that a minimizer exists: exp(x) on the real line approaches its infimum zero without attaining it at a finite point. When implementing a gradient-based stopping rule, distinguish existence of a minimizer, near-stationarity, and the particular accuracy criterion being requested.
Separate optimality from uniqueness
Convexity alone does not imply uniqueness. The constant zero function is minimized everywhere, and x squared in two variables is minimized at every point with x=0. Adding y squared yields a unique minimizer at the origin. Strict convexity makes an existing minimizer unique; strong convexity additionally supplies a uniform quadratic curvature lower bound useful for error and convergence analysis. A regularizer can change these properties, but it changes the objective as well. State which property is needed rather than using the word convex as a substitute for every desirable optimization guarantee.
Connect step size to smoothness
Smoothness bounds how quickly the gradient changes. For the quadratic f(x)=a x squared divided by two, with positive a, the gradient Lipschitz constant is a. Gradient descent multiplies x by one minus eta times a. Its magnitude contracts only when eta times a lies strictly between zero and two. Convexity therefore does not allow an arbitrary learning rate. At two, the sign alternates without contraction; above two, magnitude grows. This is an exact calculation for a constant-curvature quadratic, and transferring a step-size claim to other objectives requires their own smoothness assumptions.
Account for feasible directions
Minimizing(x−2) squared subject to x≤1 gives x=1, even though the derivative there is minus two rather than zero. No feasible movement can improve the value at that boundary. Applying the unconstrained stationary condition would miss the solution. Projected gradient methods return an update to the feasible set, which introduces a projection problem of its own. Check feasibility, objective value, and admissible directions separately. More general optimality conditions may require regularity assumptions, so a symbolic condition should always be read together with the constraints and hypotheses that justify it.
Recognize what a counterexample shows
The nonconvex function x to the fourth minus x squared has zero derivative at zero, yet nearby nonzero points have smaller values. Small gradient is therefore not a certificate of global optimality in a general neural-network training problem. As practice, trace the quadratic update for eta times a equal to 0.5,1.5, and 2.5. The first two contract, although one alternates sign; the last diverges. List convexity, existence, uniqueness, smoothness, and constraints separately when explaining a convergence result. That checklist makes the boundary between a theorem and an observed training trajectory explicit.