8 Composing functions
This chapter describes tools to build custom functions that represent relationships between inputs and the output quantity. The tools help the modeler compose a function that encompasses their ideas and data about a given relationship.
For instance, in the SIR model introduced in Chapter 1, the number of new cases of the sickness each day, depends on both the number of susceptibles (S) and the number of infectives (I): new_cases(S, I). Depending on our ideas about the mechanism of epidemic infection, we might want new_cases(S, I) to give a large value when both S and I are large, but a small value when either S or I are small. None of our storybook functions take two arguments, so we would have to compose new_cases(S, I) out of more basic functions.
Spotted a problem?
Help us fix it!
Help us fix it!
Another example: Suppose we want to build a function drug(t) to model the amount of drug in the bloodstream of someone taking a pill every eight hours. Soon after taking the pill, the amount in the bloodstream increases sharply. But almost immediately the concentration begins to fall as drug is eliminated by the liver. The overall pattern might look like the function shown in Fig 8. 1.
We don’t have a storybook function that produces the pattern in Fig 8. 1, so we have to build a custom function.
8.1 Where do custom functions come from?
One powerful technique, oddly-named ☞ linear regression ☜, is particularly suited to working with data. Regression, introduced more than a century ago, amounts to specifying the inputs and providing data that record both the inputs and the corresponding output from a laboratory or real-world sample. The computer performs the mathematical work of finding the best function that matches the data, given the modeler’s specification.
Modern generalizations of regression, grouped under the names ☞ machine learning ☜ (sometimes given a grander name: ☞ artificial intelligence ☜), can produce ☞ nonlinear functions ☜. The techniques can be powerful, and once again, the computer does the mathematical work. We will not discuss them in this book, except to note that the functions constructed serve the same modeling purpose as simpler linear regression.
When there is no adequate data, regression techniques do not apply. The modeler then has to construct model functions from first principles. We call this ☞ composing a function ☜ in the same spirit as “composing music” or “composing a poem.” Only a small fraction of those who enjoy music or poetry regularly undertake their own composition. The rest of us sing or strum or read aloud. Likewise, composing a function is usually a task for specialists. The function in Fig 8. 1 is an example, relying on knowledge of the specialized field of pharmacokinetics.
Much, much more common, at least in the sciences, is to look up a formula in a textbook that covers a particular situation. At some point, often obscured by history, such formulas had an original composer. People who can do this typically have a deep knowledge of the field and are, essentially, artists.
Spotted a problem?
Help us fix it!
Help us fix it!
Textbook formulas are akin to photo-realistic drawings or paintings. The realism leads the viewer to imagine an exact process of capturing a real-world relationship. Nevertheless, the formulas are a composition, a human creation. In realistic, day-to-day work, modelers often draw on a small set of mathematical forms, putting them together as intuition directs and revising them as needed, often motivated by poor performance of initial models.
As regards composing functions from first principles, a realistic ambition of this book is to help non-artists decompose (or, “reverse engineer”) the functions they encounter. Typically, it takes months or years of experience with such decomposition, and considerable study, to develop a sense of how to compose. The typical reader of this book may not have enough time, inclination, or an apprenticeship opportunity to learn composition. Decomposition skills are, nevertheless, valuable for understanding, appreciating, and, when called for, critiquing the functions presented in textbooks or composed by other modelers.
8.2 Linear combinations
A ☞ linear combination ☜ of functions brings two or more basic functions together into a single function. We will start with linear combinations of two functions. It would be lovely to have a non-algebraic, intuitive mechanism for illustrating linear combinations of functions, but alas, we have not yet found one. As a substitute, please accept an example based not on functions but on items from the kitchen and pantry.
A famous chocolate-chip cookie recipe has these ingredients:
| Ingredient | unit | coefficient |
|---|---|---|
| flour | cup | 2.25 |
| baking soda | tsp | 1 |
| salt | tsp | 1 |
| butter | cup | 1 |
| granulated sugar | cup | 0.75 |
| brown sugar | cup | 0.75 |
| vanilla extract | tsp | 1 |
| egg | # | 2 |
| chocolate chips | oz | 12 |
The nine ingredients are the recipe’s material inputs. The material output will be a cookie batter, ready to bake. To make the batter, one takes a unit of each ingredient, scales it up or down by a multiplier, adds it to a bowl, mixes, and voilá.
Inspired by chocolate-chip cookies, we turn to linear combinations of functions. For simplicity, we will first consider a combination of two “ingredient” functions, but the technique applies to any number of functions. For the sake of generality, we will name our two functions \(f()\) and \(g()\). They could be any of the storybook functions and, to start, consider them to be functions of a single input.
A linear combination, like chocolate-chip batter, involves scaling each function and adding them together. For example, we might have \[7.3\,f(x) \text{and}\ -0.12\,g(x) .\] The multipliers—here 7.3 and -0.12—are called ☞ coefficients ☜.
In discussing the idea of linear combination, it is conventional to use letters from the start of the alphabet to stand for the coefficients. For example, in high school, one typically encounters the algebraic form of a straight line (e.g., \(ax + b\)) or a quadratic function (e.g., \(ax^2 + b x + c\)). The coefficients are \(a\), \(b\), and \(c\). The functions involved in the combinations are, to use the storybook names, steady() and flat(), and, for the quadratic, the function that squares its input. (Readers may be so used to seeing the simple formulas \(x\) and \(x^2\) that they do not think of them as functions.)
The combination part of “linear combination” comes from adding together the individual function/coefficient pairs. Following this pattern, the generalized version of our linear combination would be
\[a\, f(x) + b\, g(x)\ . \tag{1}\]
A linear combination of functions is itself a function. Rather than writing the bare formula as in Equation 1, we prefer to write a full definition of the function, including a name for it. We will use \(h()\) as that name. So, our linear combination is
\[h(x) \equiv a\, f(x) + b\, g(x)\ . \tag{2}\]
Initially, in the modeling process, the coefficients are typically unknown. The modeler merely specifies what functions are to be included in the combination, that is, giving a specific function (like hill(), or hillside() or even steady() or flat()) to replace the general stand-in functions (\(f()\) and \(g()\) in our example).
After specifying the functions to include in the linear combination, an important and common task is to determine quantitative values for each coefficient.
Spotted a problem?
Help us fix it!
Help us fix it!
The linear-combination pattern supports a clever strategy for creating functions with two or more inputs from simple, one-input functions such as our storybook functions. Consider, by way of illustration, Equation 4, which is simply a linear combination of two single-input functions. The two functions, however, are being applied to different input quantities, x and z. The overall result is a function h(x, z) with two inputs.
\[h(x, z) \equiv a\, f(x) + b\, g(z)\ . \tag{3}\]
Any one-input functions can be used for f(x) and g(z), for instance, the storybook functions. Remarkably, the most common functions used with this technique are the simplest of the storybook functions: flat() and steady(), as in
\[h(x, z) \equiv a\, \text{steady}(x) + b\, \text{steady}(z) + c\ \text{flat}() \tag{4}\]
Recall that steady() and flat() have very simple algebraic definitions:
\(\text{steady}(x) \equiv x\ \ \ \ \text{and}\ \ \ \ \ \text{flat}( ) \equiv 1\)
Flat() is so simple that it does not even require an argument! The simplicity means that we can write Equation 4 in a more compact form:
\[h(x, z) \equiv a\, x + b\, z + c . \tag{5}\]
The conciseness of Equation 5, unfortunately, hides something important: steady() and flat() are a modeler’s choice, usually just an approximation. When this choice is not appropriate, remember that a modeler can use other functions instead of steady().
By way of example, suppose we are building a model of student performance (perf) in elementary school as a function of i) family income, ii) parents’ education (which we will call ed), and iii) the job performance evaluation of the teacher, which we will call skill. There is a different measurement scale for each of these model inputs, in much the same way as butter and chocolate chips are measured differently.
The time-honored, go-to strategy of a person constructing a model perf(income, ed, skill) is to use steady() and flat() in this way:
\[\text{perf(income, ed, skill)} \equiv a\, \text{income} + b\, \text{skill} + c\ \text{ed} + d \tag{6}\]
Anyone familiar with the pattern of family incomes knows that income differs markedly. It is not uncommon for one family to have, say, ten times the income of another. However, it seems unrealistic that the more affluent family gets such a boost in student performance. To create a more realistic model, the modeler can replace the implicit steady(income) in Equation 6. Two good choices would be magnitude(income), which replaces income with its magnitude, or hillside(income).
8.3 A foundry for new functions
In Section 8.2 we presented examples of some functions used in linear combination: flat(), steady(), magnitude(), hillside(). The linear combination strategy is like Lego: build a potentially complicated structure by adding blocks to one another. Almost all children are familiar with the basic Lego bricks, such as 
, 
, 
, and 
. These are analogous to our storybook functions.
Notwithstanding the popularity of the basic Lego bricks, there are some shapes that cannot be constructed by putting the basic ones together. There are many examples, such as 
, 
, and 
. To make such shapes, one needs to visit the Lego foundry, where new blocks are cast. There is an analogous foundry for functions, where it’s possible to build entirely new shapes not seen in the storybook functions.
There is an infinite variety of such special-purpose shapes. This makes the task of choosing one daunting, like looking through the entire Lego catalog; so many possibilities, but which ones to choose? Happily, these special-purpose functions usually appear in only a small variety. And instead of having to set up a furnace to melt down and reshape functions, you can accomplish much by applying just two techniques: ☞ function multiplication ☜ and ☞ pipelining functions ☜
We will illustrate multiplication and pipelining using two generic names for single input functions: \(f()\) and \(g()\). (We haven’t given a name to the argument because there is just a single argument.) Multiplication of these two functions can be done in two different ways:
- A single argument: \(h(x) \equiv f(x) g(x)\). The function created has just a single argument. Multiplication is straightforward.
- Two arguments: \(j(x, z) \equiv f(x) g(z)\).
The newly created functions, \(h(x)\) or \(j(x, z)\) can be used in a linear combination or can participate in yet another multiplication or pipelining.
Spotted a problem?
Help us fix it!
Help us fix it!
For pipelining, the two functions are combined by taking the output of one function and feeding it as the input to the other function (just as a pipeline takes material from an upstream segment and injects it into a downstream segment). Using again our generic function names \(f()\) and \(g()\), there are two distinct forms for the pipeline, depending on which function is used in the interior position and which in the exterior:
- \(f\left(g(x)\right)\) where \(g()\) is the interior function.
- \(g\left(f(x)\right)\) where \(f()\) is the interior function.
Make sure to note the distinction between multiplication and pipelining: \(f(x) g(x)\) is not the same thing as \(f(g(x))\). As mentioned above, function multiplication can produce a function with multiple inputs. But pipelining never can. There is only one slot for an argument, in the parentheses of the interior function.
We have already encountered a use for pipelining: the input-transformation where the interior function is the simple linear combination \(g(x) \equiv a x + b\) and the exterior function was any of the storybook functions. (The output transformation is itself just a linear combination.)
In the Exercises, we will create a whole menagerie of functions by multiplication or pipelining (or both) of the storybook functions. This will give you a sense of how diverse (and sometimes rare and exotic) shapes can be formed. More important for our purposes is to point out a handful of simple forms that appear in a huge variety of modeling contexts. For multiplication, the most common functional forms are built out of our simple friend steady() and look like steady() \(\times\) steady(). There are two varieties:
- steady(x) \(\times\) steady(x), which has just a single input. In algebraic nomenclature, this function is called the square of x and written \(x^2\).
- steady(x) \(\times\) steady(z), which has two different inputs. The algebraic name for this form is hardly ever used, but statistics provides a much more evocative name, justified by how often the form is used in modeling. Steady(x) \(\times\) steady(z) is called the ☞ interaction term ☜, and it is used in just about situation where the effect of x depends on the size of z. We will see it at work in Chapter 11.
8.4 Polynomials
High school students spend a lot of time working with ☞ polynomials ☜. Most famous is the quadratic polynomial, written algebraically as \(a x^2 + b x + c\). Polynomials are always linear combinations of simpler functions. The quadratic polynomial involves flat(), steady(x), and the product steady(x) \(\times\) steady(x). The “poly” in polynomials means “many.” Many of what? The functions that go into the linear combination are called “monomials” and consist of flat(), steady(x), steady(x) \(\times\) steady(x), steady(x) \(\times\) steady(x) \(\times\) steady(x), and so on.
Quadratics and other polynomials played an outsized role in the history of algebra. This is in part a legacy of classical geometric shapes such as squares and cubes. Another historical reason behind the ubiquity of polynomials in mathematics classes is the importance of a mathematical operation called zero-finding, which we will meet in Chapter 14. Zero-finding is one of a small set of basic operations that are used to transform the information present in a model into a different, more directly useful form.
It used to be that zero-finding was hard to apply to functions generally. Polynomials, particularly the quadratic polynomial, lend themselves to standard, formulaic approaches. Many former math students recognize the most famous of these zero-finding formulas, even if they have no idea what it is for. The formula involves the coefficients on the polynomial: \(x_0 = \frac{b \pm \sqrt{b^2 - 4 a c}}{2 a}\). (\(x_0\) stands for the zeros of the function \(a x^2 + b x + c\).)
Nowadays, there are general computational methods that find zero for just about any function, polynomial or not. These have a long history, but one of the most famous methods, still in use today, was invented by Isaac Newton. (The importance of the method is evidence by the very simple name given to it: “Newton’s method.”)
Spotted a problem?
Help us fix it!
Help us fix it!
Because polynomials have been part of the curriculum for hundreds of years, many people use them to build models. (In part, this is because the standard math curriculum fails to cover other methods that may be more suitable in any given circumstance.) For the model builder, they provide a useful tool, serving more or less like “modeling clay” for mathematics. For the reader or the interpreter of a model, the polynomial itself carries little information about specific mechanisms. In Chapter 12 we will look at more general ways of analyzing models to understand what is happening inside.