20  Constructing derivatives

This chapter showshow to use the computer to construct the derivative of any function. This is easy because the task of constructing derivatives is well suited to the computer.

There are functions where the derivative cannot be meaningfully defined. Examples are the absolute-value function or the Heaviside function which we introduced when discussing piecewise functions in Chapter Section 9.4. In Chapter Section 22 we will consider the pathological cases and show you how to spot them at a glance.

We will demonstrate two methods:

  1. Symbolic differentiation, which transforms an algebraic formula for a function into a corresponding algebraic formula for the derivative.

  2. Finite-difference methods that use a “small”—but not evanescent—\(h\).

Chapter Section 23 covers the algorithms used by the computer to construct symbolic derivatives. One reason for teaching you to do with paper and pencil the simpler sorts of problems that the computer does perfectly is analogous to why you learned basic arithmetic by hand even though a calculator can perform the task more reliably. Another reason to learn to carry out symbolic differentiation on your own is to enable you to follow textbook or classroom demonstrations of formulas which often come from solve a differentiation problem.

20.1 Why differentiate?

Before showing the easy computer-based methods for constructing the derivative of a function, it is good to provide some motivation: Why is differentiation so frequently in so many fields of study and application?

A primary reason lies in the laws of physics. Newton’s Second Law of Motion reads:

“The change of motion of an object is proportional to the force impressed; and is made in the direction of the straight line in which the force is impressed.”

Newton defined used position \(x(t)\) as the basis for velocity \(v(t) = \partial_t x(t)\). “Change in motion,” which we call “acceleration,” is in turn the derivative \(\partial v(t)\). Derivatives are also central to the expression of more modern forms of physics such as quantum theory and general relativity.

Many relationships encountered in the everyday or technical worlds are more understandable if framed in terms of derivatives. For instance,

  • Electrical power is the rate of change with respect to time of electrical energy.
  • Birth rate is one component of the rate of change with respect to time of population. (The others are the death rate and the rates immigration and emigration.)
  • Interest, as in bank interest or credit card interest, is the rate of change with respect to time of assets.
  • Inflation is the rate of change with respect to time of prices.
  • Disease incidence is one component of the rate of change with respect to time of disease prevalence. (The other components are death or recovery from disease.)
  • Force is the rate of change with respect to position of energy.
  • Deficit (as in spending deficits) is the change with respect to time of debt.

Often, we know one member in such function-and-derivative pairs, but to need to calculate the other. Many modeling situations call for putting together different components of change to reveal how some other quantity of interest will change. For example, modeling the financial viability of retirement programs such as the US Social Security involves looking at the changing age structure of the population, the returns on investment, the changing cost of living, and so on. In Block ?sec-dynamics-part, we will use derivatives explicitly to construct models of systems, such as an outbreak of disease, with many changing parts.

Derivatives also play an important role in design. They play an important role in the construction and representation of smooth curves, such as a robot’s track or the body of a car. (See Chapter Section 48.) Control systems that work to stabilize a airplane’s flight or regulate the speed and spacing of cars are based on derivatives. The notion of “stability” itself is defined in terms of derivatives. (See Chapter Section 44.) Algorithms for optimizing design choices also often make use of derivatives. (See Chapter Section 49.)

Economics as a field makes considerable use of concepts of calculus—particularly first and second derivatives, the subjects of this Block—although the names used are peculiar to economics, for instance, “elasticity”, “marginal returns” and “diminishing marginal returns.”

The origins of modern economics, especially the theory of the free market, are attributed to a book published in 1776, The Wealth of Nations. The author, Adam Smith (1723-1790), lays out dozens of relationships between different quantities — wages, labor, stock, interest, prices, profits, and coinage among others. Yet despite the invention of calculus a century before Wealth of Nations, the book uses no calculus.

Consider this characteristic statement in Wealth of Nations:

The market price of every particular commodity is regulated by the proportion between the quantity which is brought to market, and the demand of those who are willing to pay the natural price of the commodity.

Without calculus and the ideas of functions and their derivatives, Smith was not able to think about prices in a modern way where price is shaped by demand and supply. Instead, for Smith, each item has a “natural price”: a fixed quantity that depends on the amount of labor used to produce the item. Nowadays, we understand that productivity changes as new methods of production and new inventions are introduced. But Smith lived near the end of a centuries-long period of static economies. Transportation, agriculture, manufacture, and population size were all much as they had been for the past 500 years or longer. James Watt’s steam engine was introduced only in 1776 and it would be decades before being adapted to the myriad uses of steam power characteristic of the 19th century. The cotton gin (1793), labor-saving agricultural machines such as the McCormick reaper (1831), the assembly line (1901), and the many other innovations of industry all lay in the future when Smith was writing Wealth of Nations.

It took the industrial revolution and nearly a century of intellectual development before economics had to and could embrace the rapid changes in the production process. In this dynamical view, supply and demand are not mere quantities, but functions of which price is the primary input. The tradition in economics is to use the word “curve” instead of “function,” giving us the phrases “supply curve” and “demand curve.” Making the transition from quantity to function, that is, between a single amount and a relationship between amounts, is a core challenge to those learning economics.

20.2 Symbolic differentiation

The R/mosaic function D() takes a formula for a function and produces the derivative. It uses the same sort of tilde expression used by makeFun() or contour_plot() or the other R/mosaic tools. For instance,

D(t * sin(t) ~ t)
## function (t) 
## sin(t) + t * cos(t)

If you prefer, you can use makeFun() to define a function, then hand that function to D() for differentiation.

myf <- makeFun(sqrt(y * pnorm(1 + x^2, mean=2, sd=3)) ~ x & y)
dx_myf <- D(myf(x, y) ~ x, y=3)
dx_myf
## function (x, y = 3) 
## {
##     .e1 <- 1 + x^2
##     x * y * dnorm(.e1, 2, 3)/sqrt(y * pnorm(.e1, mean = 2, sd = 3))
## }

In the right side of the tilde expression handed off to D() names the with-respect-to input. This is similar to the tilde expressions used in plotting, which name the inputs that form the graphics domain. But it contrasts with the tilde expressions in makeFun(), where the right-hand side specifies the order in which you want the inputs to appear.

Needless to say, D() knows the rules for the derivatives of the pattern-book functions introduced in Section Section 19.3. For instance,

D(sin(t) ~ t)
## function (t) 
## cos(t)
D(log(x) ~ x)
## function (x) 
## 1/x
D(exp(x) ~ x)
## function (x) 
## exp(x)
D(x^2 ~ x)
## function (x) 
## 2 * x

20.3 Finite-difference derivatives

Whenever you have a formula amenable to the construction of a symbolic derivative, that is what you should use. Finite-difference derivatives are useful in those situation where you don’t have such a formula. The calculation is simple but has a weakness that points out the advantages of the evanescent-\(h\) approach.

For a function \(f(x)\) and a “small,” non-zero number \(h\), the finite-difference approximates the derivative with this formula:

\[\partial_x f(x) \approx \frac{f(x+h) - f(x-h)}{2h}\ .\] To demonstrate, let’s construct the finite-difference approximation to \(\partial_x \sin(x)\). Since we already know the symbolic derivative—it is \(\partial_x \sin(x) = \cos(x)\)—there is no genuinely practical purpose for this demonstration. Still, it can serve to confirm the symbolic rule.

We will call the finite-difference approximation fq_sin() and use makeFun() to construct it:

fq_sin <- makeFun((sin(x+h)- sin(x-h))/(2*h) ~ x, h=0.01)

Notice that fq_sin() has a parameter, h whose default value is being set to 0.01. Whether 0.01 is “small” or not depends on the context. Operationally, we define “small” to be a value that gives practically the same result even if it is made smaller by a factor of 2 or 10.

As a demonstration that fq_sin() with \(h=0.01\) approximates the genuine \(\partial_x \sin(x)\), we exploit our knowledge that \(\partial_x \sin(x) = \cos(x)\). Figure 20.1 plots out the difference between the the \(h=0.01\) approximation and the genuine derivative.

slice_plot(fq_sin(x, h=0.01) - cos(x) ~ x, domain(x=-10:10)) %>%
  slice_plot(fq_sin(x, h=0.001) - cos(x) ~ x, color="magenta") %>%
  gf_labs(y="Error from true value.")

You will need to look carefully at the vertical axis scale in Figure 20.1 to see what’s happening. For \(h=0.01\), fq_sin() is not exactly the same as cos(), but it is close, always being within $$0.00017. For many purposes, this would be accurate enough. But not for all purposes. We can make the approximation better by using a smaller \(h\). For instance, the \(h=0.001\) version of fq_sin() is accurate to within $$0.0000017.

In practical use, one employs the finite-difference method in those cases where one does not already know the exact derivative function. This would be the case, for example, if the function is a sound wave recorded in the form of an MP3 audio file.

In such situations, a practical way to determine what is a small \(h\) is to pick one based on your understanding of the situation. For example, much of what we perceive of sound involves mixtures of sinusoids with periods longer than one-two-thousandth of a second, so you might start with \(h\) of 0.002 seconds. Use this guess about \(h\) to construct a candidate finite-difference approximation. Then, construct another candidate using a smaller h, say, 0.0002 seconds. If the two candidates are a close match to one another, then you have confirmed that your choice of \(h\) is adequate.

It is tempting to think that the approximation gets better and better as h is made even smaller. But that is not necessarily true for computer calculations. The reason is that quantities on the computer have only a limited precision: about 15 digits. To illustrate, let’s calculate a simple quantity, \((\sqrt{3})^2 - 3\). Mathematically, this quantity is exactly zero. On the computer, however it is not quite zero:

sqrt(3)^2 - 3
## [1] -4.440892e-16

We can see this loss of precision at work if we make h very small in the finite-difference approximation to \(\partial_x \sin(x)\). In Figure 20.2 we are using h = 0.000000000001. The result is unsatisfactory.

slice_plot(  fq_sin(x, h=0.000000000001) - cos(x) ~ x, 
           domain(x=-10:10)) %>%
  slice_plot(fq_sin(x, h=0.0000000000001) - cos(x) ~ x,
             color="magenta") %>%
  gf_labs(y="Error from true value.")

20.4 Second and higher-order derivatives

Many applications call for differentiating a derivative or even differentiating the derivative of a derivative. In English, such phrases are hard to read. They are much simpler using mathematical notation.

  • \(f(x)\) a function
  • \(\partial_x f(x)\) the derivative of \(f(x)\)
  • \(\partial_x \partial_x f(x)\), the second derivative of \(f(x)\), usually written even more concisely as \(\partial_{xx}f f(x)\).

There are third-order derivatives, fourth-order, and on up, although they are not often used.

To compute a second-order derivative \(\partial_{xx} f(x)\), first differentiate \(f(x)\) to produce \(\partial_x f(x)\). Then, still using the techniques described earlier in this chapter, differentiate \(\partial_x f(x)\).

There is a shortcut for constructing high-order derivatives using D() in a single step. On the right-hand side of the tilde expression, list the with-respect-to name repeatedly. For instance:

  • The second derivative \(\partial_{xx} \sin(x)\):
D(sin(x) ~ x & x)
## function (x) 
## -sin(x)
  • The third derivative \(\partial_{xxx} \ln(x)\):
D(log(x) ~ x & x & x)
## function (x) 
## 2/x^3

Physics students learn a formula for the position of an object in free fall dropped from a height \(x_0\) and at an initial velocity \(v_0\): \[ x(t) \equiv -\frac{1}{2} g t^2 + v_0 t + x_0\ .\] The acceleration of the object is the second derivative \(\partial_{tt} x(t)\). Use D() to find the object’s acceleration.

The second derivative of \(x(t)\) with respect to \(t\) is:

D(0.5*g*t^2 + v0*t + x0 ~ t & t)
## function (t, g, v0, x0) 
## g

The acceleration does not depend on \(t\); it is the constant \(g\). No wonder \(g\) is called “gravitational acceleration.”

20.5 Drill

Part 1 Which pattern-book function is the derivative of the sigmoid function pnorm()? That is, \[{\large \text{pnorm}(x)} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\LARGE ?}\]

Gaussian dnorm(x) Exponential \(e^x\) Sinusoid \(\sin(x)\) Constant \(1\) Reciprocal \(1/x\)

Part 2 Which pattern-book function is the anti-derivative of the reciprocal \(1/x\)? That is, \[{\LARGE ?} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\large \frac{1}{x}}\]

NOTE: Differentiation produces a “child” function from a “parent” function. The child is the derivative of the parent. Putting the relationship the other way, the parent is the anti-derivative of the child. “Derivative” and “anti-derivative” are two ways of looking at the same relationship between a pair of functions. So, if \(f(x)\) is the derivative of \(F(x)\), then \(F(x)\) is the anti-derivative of \(f(x)\).

  1. Gaussian \(\text{dnorm(x)}\)
  2. Logarithm \(\ln(x)\)
  3. Sinusoid \(\sin(x)\)
  4. Constant \(1\)
  5. Reciprocal \(1/x\)

Part 3 Which pattern-book function is the anti-derivative of the gaussian \(\text{dnorm()}\)? That is, \[{\LARGE ?} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\large \text{dnorm}(x)}\]

NOTE: Differentiation produces a “child” function from a “parent” function. The child is the derivative of the parent. Putting the relationship the other way, the parent is the anti-derivative of the child. “Derivative” and “anti-derivative” are two ways of looking at the same relationship between a pair of functions. So, if \(f(x)\) is the derivative of \(F(x)\), then \(F(x)\) is the anti-derivative of \(f(x)\). In other words: \[{\large F(x)} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\Large f(x)}\]

  1. Gaussian \(\text{dnorm(x)}\)
  2. Logarithm \(\ln(x)\)
  3. Sigmoid \(\text{pnorm(x)}\)
  4. Constant \(1\)
  5. Reciprocal \(1/x\)

Part 4 What is the derivative of the power-law function \(x^p\)?i That is, \[{\Large x^p} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\LARGE ?}\]

  1. \((p-1)\, x^{p-1}\)
  2. \((p-1)\, x^p\)
  3. \(p\, x^{p-1}\)
  4. \(\frac{1}{p} x^{p+1}\)
  5. \(p\, x^p\)

Part 5 There are two pattern-book functions whose second derivative is proportional to the function itself. Which are they?

  1. Sinusoid and gaussian
  2. Exponential and sigmoid
  3. Exponential and sinusoid
  4. Exponential and logarithm

Part 6 What is the derivative of \(t^5\) with respect to \(t\)? That is, \[{\Large t^5} \underset{\scriptsize \text{anti-diff}}{{\stackrel{\text{diff}}{\ \ \ \ {\Huge\rightleftharpoons}\ \ \ }}} {\Large ?}\]

\(\frac{1}{4} t^5\)\(4 t^5\)\(5 t^4\)\(\frac{1}{5} t^4\)

Part 7 What is \(\partial_x x^2\)?

\(2/x\)\(2\)\(2 x\)\(2 x^2\)

Part 8 What is \(\partial_t \sin(x)\)

\(-\cos(x)\) 0 \(\cos(x)\)\(-\sin(x)\)

Part 9 Suppose you know only this one fact about \(f(x)\), that \(\partial_{xx}\, f(7.3) = 1.6\). Which of these statements must be true?

  1. \(f(x)\) is concave up at \(x=7.3\), but eventually it will become concave down.
  2. \(f(x)\) is concave up and decreasing at \(x=7.3\)
  3. \(f(x)\) is increasing at \(x=7.3\).
  4. \(f(x)\) is concave up at \(x=7.3\)

Part 10 If \(f(x)\) is discontinuous at \(x=5\), can it possibly be smooth at \(x=6\)?

No Yes

Part 11 If \(g(x)\) is discontinuous at \(x=1\), what will be the value of \(\partial_x g(x)\) at \(x=1\)?

  1. Depends on how big the gap is at the discontinuity.
  2. 0
  3. \(1/x\)
  4. The derivative isn’t defined at a discontinuity.

Part 12 Which of the following is the correct construction for \(\partial_t g(t)\)?

  1. \(\lim_{x \rightarrow 0} \frac{g(t + h) - g(t)}{h}\)
  2. \(\lim_{h \rightarrow 0} \frac{g(t + h) - g(t)}{t}\)
  3. \(\lim_{h \rightarrow 0} \frac{g(t + h) - g(t)}{h}\)
  4. \(\lim_{h \rightarrow 0} \frac{g(t) - g(t+h)}{h}\)

Part 13 Which of these is a reasonable definition of a derivative?

  1. A derivative is the slope of a function.
  2. A derivative is a function whose value tells, for any input, the instantaneous rate of change of the function from which it was derived.
  3. A derivative is a function whose value tells, for any input, the instantaneous change of the function from which it was derived.

Part 14 Which one of these is not the derivative of a pattern-book function?

Sigmoid Zero Reciprocal One

Part 15 Which of the following shapes of functions is not allowed? You are strongly advised to try to draw each shape.

  1. Increasing and concave up.
  2. Decreasing and concave up.
  3. Increasing and concave down.
  4. Decreasing and concave down.
  5. None of them are allowed.
  6. All of them are allowed.