fd_sine() itself
20 Constructing derivatives
This chapter shows to use the computer to construct the derivative of any1 function. This is easy because the task of constructing derivatives is well suited to the computer.
We will demonstrate two methods:
Symbolic differentiation, which transforms an algebraic formula for a function into a corresponding algebraic formula for the derivative.
Finite-difference methods that use a “small” \(h\), but not evanescent.
Chapter 23 covers the algorithms used by the computer to construct symbolic derivatives. Why teach you to do with paper and pencil the simpler sorts of problems that the computer does perfectly? One reason is that it helps you to anticipate what the results of the computer calculation will be, providing a mechanism to detect error in the way the computation was set up. Another reason is to enable you to follow textbook or classroom demonstrations of formulas which often come from working out a differentiation problem.
20.1 Why differentiate?
Before showing the easy computer-based methods for constructing the derivative of a function, it is good to provide some motivation: Why is differentiation so frequently in so many fields of study and application?
A primary reason lies in the laws of physics. Newton’s Second Law of Motion reads:
“The change of motion of an object is proportional to the force impressed; and is made in the direction of the straight line in which the force is impressed.”
Newton defined used position \(x(t)\) as the basis for velocity \(v(t) = \partial_t x(t)\). “Change in motion,” which we call “acceleration,” is in turn the derivative \(\partial v(t)\). Derivatives are also central to the expression of more modern forms of physics such as quantum theory and general relativity.
Many relationships encountered in the everyday or technical worlds are more understandable if framed in terms of derivatives. For instance,
- Electrical power is the rate of change with respect to time of electrical energy.
- Birth rate is one component of the rate of change with respect to time of population. (The others are the death rate and the rates immigration and emigration.)
- Interest, as in bank interest or credit card interest, is the rate of change with respect to time of assets.
- Inflation is the rate of change with respect to time of prices.
- Disease incidence is one component of the rate of change with respect to time of disease prevalence. (The other components are death or recovery from disease.)
- Force is the rate of change with respect to position of energy.
- Deficit (as in spending deficits) is the change with respect to time of debt.
Often, we know one member in such function-and-derivative pairs, but to need to calculate the other. Many modeling situations call for putting together different components of change to reveal how some other quantity of interest will change. For example, modeling the financial viability of retirement programs such as the US Social Security involves looking at the changing age structure of the population, the returns on investment, the changing cost of living, and so on. In Block V, we will use derivatives explicitly to construct models of systems, such as an outbreak of disease, with many changing parts.
Derivatives also play an important role in design. They play an important role in the construction and representation of smooth curves, such as a robot’s track or the body of a car. (See Chapter 49.) Control systems that work to stabilize a airplane’s flight or regulate the speed and spacing of cars are based on derivatives. The notion of “stability” itself is defined in terms of derivatives. (See Chapter 45.) Algorithms for optimizing design choices also often make use of derivatives. (See Chapter 50.)
Economics as a field makes considerable use of concepts of calculus—particularly first and second derivatives, the subjects of this Block—although the names used are peculiar to economics, for instance, “elasticity”, “marginal returns” and “diminishing marginal returns.”
Calculus history—The Wealth of Nations
The origins of modern economics, especially the theory of the free market, are attributed to a book published in 1776, The Wealth of Nations. The author, Adam Smith (1723-1790), lays out dozens of relationships between different quantities — wages, labor, stock, interest, prices, profits, and coinage among others. Yet despite the invention of calculus a century before Wealth of Nations, the book uses no calculus.
Consider this characteristic statement in Wealth of Nations:
The market price of every particular commodity is regulated by the proportion between the quantity which is brought to market, and the demand of those who are willing to pay the natural price of the commodity.
Without calculus and the ideas of functions and their derivatives, Smith was not able to think about prices in the modern way where price is shaped by demand and supply. Instead, for Smith, each item has a “natural price”: a fixed quantity that depends on the amount of labor used to produce the item. Nowadays, we understand that productivity changes as new methods of production and new inventions are introduced. But Smith lived near the end of a centuries-long period of static economies. Transportation, agriculture, manufacture, and population size were all much as they had been for the past 500 years or longer. James Watt’s steam engine was introduced only in 1776 and it would be decades before being adapted to the myriad uses of steam power characteristic of the 19th century. The cotton gin (1793), labor-saving agricultural machines such as the McCormick reaper (1831), the assembly line (1901), and the many other innovations of industry all lay in the future when Smith was writing Wealth of Nations.
It took the industrial revolution and nearly a century of intellectual development before economics had to and could embrace the rapid changes in the production process. In this dynamical view, supply and demand are not mere quantities, but functions of which price is the primary input. The tradition in economics is to use the word “curve” instead of “function,” giving us the phrases “supply curve” and “demand curve.” Making the transition from quantity to function, that is, between a single amount and a relationship between amounts, is a core challenge to those learning economics.
20.2 Symbolic differentiation
The R/mosaic function D() takes a formula for a function and produces the derivative. It uses the same sort of tilde expression used by makeFun() or contour_plot() or the other R/mosaic tools. For instance, run the code in Active R chunk 20.1 to see the function created by differentiating \(t \sin(t)\) with respect to \(t\).
If you prefer, you can use makeFun() to define a function, then hand that function to D() for differentiation as in Active R chunk 20.2:
When differentiating a function with two (or more) inputs, the function definition will have the names of the inputs on the right-hand side of the tilde expression, as in Active R chunk 20.3 where the two arguments to f() are named x and y. But notice that in the use of D(), the right-hand side of the tilde expression lists only the “with respect to” variable, in this case, x.
Try it! 20.1 Derivatives of the pattern book functions
Needless to say, D() knows the rules for the derivatives of the pattern-book functions introduced in Section 19.3. For instance,
Try the other eight pattern-book functions. For seven of these, D() produces a symbolic derivative. For the eighth, D() resorts to a “finite-difference” derivative (introduced in Section 20.3). Which is this pattern-book function where D() produces a finite-difference derivative?
20.3 Finite-difference derivatives
Whenever you have a formula amenable to the construction of a symbolic derivative, that is what you should use. Finite-difference derivatives are useful in those situation where you don’t have such a formula. The calculation is simple but has a weakness that points out the advantages of the evanescent-\(h\) approach.
For a function \(f(x)\) and a “small,” non-zero number \(h\), the finite-difference approximates the derivative with this formula:
\[\partial_x f(x) \approx \frac{f(x+h) - f(x-h)}{2h}\ .\] To demonstrate, let’s construct the finite-difference approximation to \(\partial_x \sin(x)\). Since we already know the symbolic derivative—it is \(\partial_x \sin(x) = \cos(x)\)—there is no genuinely practical purpose for this demonstration. Still, it can serve to confirm the symbolic rule.
We will call the finite-difference approximation fd_sine() and use makeFun() to construct it:
Notice that fd_sine() has a parameter, h whose default value is being set to 0.01. Whether 0.01 is “small” or not depends on the context. Operationally, we define “small” to be a value that gives practically the same result even if it is made smaller by a factor of 2 or 10.
As a demonstration that fd_sine() with \(h=0.01\) approximates the genuine \(\partial_x \sin(x)\), we exploit our knowledge that \(\partial_x \sin(x) = \cos(x)\). Figure 20.2 plots out the difference between the \(h=0.01\) approximation and the genuine derivative.
Comparing fd_sine() to the symbolic \(\partial_x \sin(x)\) for two values of \(h\): \(h=0.01\) and \(h=0.001\). :::
A glance at Figure 20.2(a) shows that the finite-difference derivative for \(h=0.01\) (black) is essentially the same as for \(h=0.001\). Thus, \(h=0.01\) is good enough. Figure 20.2(b) highlights this, showing the error between the finite-difference derivative and the (known) symbolic derivative \(\cos(x)\). You will need to look carefully at the vertical axis scales in Figure 20.2 to see what’s happening. For \(h=0.01\), fd_sine() is not exactly the same as cos(), but it is close, always being within $$0.000017. For many purposes, this would be accurate enough. But not for all purposes. We can make the approximation better by using a smaller \(h\). For instance, the \(h=0.001\) version of fd_sine() is one-hundred times more accurate: to within $$0.00000017.
In practical use, one employs the finite-difference method in those cases where one does not already know the exact derivative function. This would be the case, for example, if the function is a sound wave recorded in the form of an MP3 audio file.
In such situations, a practical way to determine what is a small \(h\) is to pick one based on your understanding of the situation. For example, much of what we perceive of sound involves mixtures of sinusoids with periods longer than one-two-thousandth of a second, so you might start with \(h\) of 0.002 seconds. Use this guess about \(h\) to construct a candidate finite-difference approximation. Then, construct another candidate using a smaller h, say, 0.0002 seconds. If the two candidates are a close match to one another, then you have confirmed that your choice of \(h\) is adequate.
It is tempting to think that the approximation gets better and better as h is made even smaller. But that is not necessarily true for computer calculations. The reason is that quantities on the computer have only a limited precision: about 15 digits. To illustrate, let’s calculate a simple quantity, \((\sqrt{3})^2 - 3\). Mathematically, this quantity is exactly zero. However, on the computer, as in Active R chunk 20.4, it is not quite zero:
We can see this loss of precision at work if we make h very small in the finite-difference approximation to \(\partial_x \sin(x)\). In Figure 20.3 we are using h = 0.000000000001. The result is unsatisfactory.
h (\(h=0.0000000000001\), magenta compared to (\(h=0.000000000001\), black) leads to a loss of accuracy in the finite-difference approximation due to machine round-off error.
Evanescent \(h\) doesn’t produce reliable results in the machine arithmetic for finite-difference derivatives, which involves subtracting two almost-identical values!
20.4 Second and higher-order derivatives
Many applications call for differentiating a derivative or even differentiating the derivative of a derivative. In English, such phrases are hard to read. They are much simpler using mathematical notation.
- \(f(x)\) a function
- \(\partial_x f(x)\) the derivative of \(f(x)\)
- \(\partial_x \partial_x f(x)\), the second derivative of \(f(x)\), usually written even more concisely as \(\partial_{xx}f f(x)\).
There are third-order derivatives, fourth-order, and on up, although they are not often used.
To compute a second-order derivative \(\partial_{xx} f(x)\), first differentiate \(f(x)\) to produce \(\partial_x f(x)\). Then, still using the techniques described earlier in this chapter, differentiate \(\partial_x f(x)\).
There is a shortcut for constructing high-order derivatives using D() in a single step. On the right-hand side of the tilde expression, list the with-respect-to name repeatedly. For instance, look at the functions produced in the following active R chunks:
- The second derivative \(\partial_{xx} \sin(x)\):
- The third derivative \(\partial_{xxx} \ln(x)\):
Application area 20.1 Constant acceleration
Physics students learn a formula for the position of an object in free fall dropped from a height \(x_0\) and at an initial velocity \(v_0\): \[ x(t) \equiv -\frac{1}{2} g t^2 + v_0 t + x_0\ .\] The acceleration of the object is the second derivative \(\partial_{tt} x(t)\). Use D() to find the object’s acceleration.
The second derivative of \(x(t)\) with respect to \(t\) is calculated in Active R chunk 20.5. Run that code.
The acceleration does not depend on \(t\); it is the constant \(g\). No wonder \(g\) is called “gravitational acceleration.”
There are functions where the derivative cannot be meaningfully defined. Examples are the absolute-value function or the Heaviside function which we introduced when discussing piecewise functions in Section 9.4. Chapter 22 considers the pathological cases and shows how to spot them at a glance.↩︎