In many branches of science, including mathematics, mathematical optimization is a branch that is about finding the element that gives an optimal solution to a problem, given some criteria. In the simplest case, this means that a function needs to be minimized or maximized.
The first step is usually to take the derivative of the initial function. From here maxima or minima can be found within a domain by simply finding the critical points. These include domain (x) values where the differentiated function equals 0 or does not exist. Simply finding critical points is not enough to determine a maxima or minima. The final step to determining a maximum, minimum, or neither can be one of two methods.
(1) X values lesser and greater than the critical point must be substituted into the differentiated function to determine if the values change sign. If they go from negative to positive, the critical point is a minimum. If they go from positive to negative, the critical point is a maximum. If neither occur, the point is neither. This is known as the first derivative test.
(2) The derivative of the differentiated function can be taken, creating a second order derivative. The critical point must be subsititued into the second derivative. If the output is positive, the critical point is a minimum. If the output is negative, the critical point is a maximum. This is known as the second derivative test.
The application of optimization span many fields. A simple example is attempting to find smallest possible difference in the distance of two objects in two dimensional space (x and y). In this context, the derivative of the function that gives difference is taken in order to find the minimum. A more complicated example is in Machine Learning, in which the optimization function attempts to find the global minimum of the loss function in order to minimize the difference or loss between the algorithm’s predictions and the actual values. This example is more difficult as Machine learning algorithms often utilize multidimensional data usually in the form of tensors yielding more complicated functions.