# 33 Functions as vectors

Starting with Chapter ?sec-vectors, we have been working with the ** dot product**, an operation that combines two vectors to produce a scalar. \[\vec{b}\bullet\vec{a} \equiv
\left[\begin{array}{c}b_1\\b_2\\\vdots\\b_n\end{array}\right] \bullet
\left[\begin{array}{c}a_1\\a_2\\\vdots\\a_n\end{array}\right] \equiv b_1 a_1 + b_2 a_2 + \cdots b_n a_n\] The dot product enables us to use arithmetic to calculate geometric properties of vectors, even in high dimensional spaces that are out of reach of a ruler or protractor. For instance

- length: \(\|\vec{a}\| = \sqrt{\strut\vec{a}\bullet\vec{a}}\)
- included angle: \[\cos(\theta_{ab}) = \left[\vec{a}\bullet\vec{b}\right] / \left[\sqrt{\strut \strut\vec{a}\bullet\vec{a}}\sqrt{\strut \strut\vec{b}\bullet\vec{b}}\right]\]
- projection onto \(\vec{a}\): \[\text{model vector:}\ \ \hat{b} = \left[\vec{b} \bullet \vec{a}\right] \,\vec{a} / {\len{a}^2} = \left[\vec{b} \bullet \vec{a} {\LARGE/} \vec{a} \bullet \vec{a}\right]\ \vec{a}.\]

We used such operations to solve the ** target problem**: finding the best approximation of a vector \(\vec{b}\) as a linear combination of a set of vectors in a matrix \(\mathit{A}\).

As early as Block 1, we constructed functions as a linear combination of other functions, for example: \[g(t) \equiv A + B \sin\left(\frac{2 \pi}{P} t\right)\] where \(A\) is the scalar multiplier for the function \(\text{one}(t) \equiv 1\) and \(B\) the scalar multiplier for the sinusoid of period \(P\).

We will revisit the idea of linear combinations of functions using our new tools of length, included angle, and projection. To do this, we need to have a definition of the dot product suitable for application to functions.

## 33.1 Dot product for functions

Given two functions, \(f(t)\) and \(g(t)\) defined over some domain \(D\), we will compute the dot product of the functions as a sum of the product of the two functions, that is:

\[f(t) \bullet g(t) \equiv \int_{D} f(t)\,g(t)\,dt\ .\]

Suppose that our two functions are \(\text{one}(t) \equiv 1\) and \(\text{identity}(t) \equiv t\) on the domain \(0 \leq t \leq 1\). Find the length of each function and the included angle between them.

Length: \(\|\text{one}(t)\| = \left[\int_0^1 1 \cdot 1\,dt\right]^{1/2} = \left[\ \strut t\left.{\large\strut}\right|_0^1\ \right]^{1/2} = 1\)

Length: \(\|\text{identity}(t)\| = \left[\int_0^1 t \cdot t\,dt\right]^{1/2} = \left[\ \strut \frac{1}{2}t^2\left.{\large\strut}\right|_0^1\ \right]^{1/2} = \frac{1}{\sqrt{2}}\)

Included angle: \[\cos(\theta) = \frac{\text{one}(t) \bullet \text{identity}(t)}{\|\strut\text{one}(t)\| \, \|\text{identity}(t)\|} = \sqrt{2}\ \int_0^1 t\, dt = \sqrt{\strut 2} \left.{\Large\strut}\frac{1}{2} t^2\right|_0^1 = \sqrt{\frac{1}{2}}\] Since \(\cos(\theta) = \sqrt{1/2}\), the angle \(\theta\) is 45 degrees.

Project \(f(t) \equiv t^2\) onto \(g(t) = \text{one}(t)\) over the domain \(-1 \leq t \leq 1\).

The projection of \(f(t)\) onto \(g(t)\) will be

\[\widehat{f(t)} = \frac{f(t) \bullet g(t)}{g(t) \bullet g(t)}\ g(t)\]

\(f(t) \bullet g(t) \equiv \int_{-1}^{1} t^2 dt = \frac{1}{3} \left.{\Large \strut}t^3\right|_{-1}^{1} = \frac{2}{3}\)

\(g(t) \bullet g(t) \equiv \int_{-1}^1 \ dt = 2\)

Thus, \[\widehat{f(t)} = \frac{1}{3} \text{one(t)} = \frac{1}{3}\ .\]

The left panel of Figure 33.1 shows the functions \(f(t) \equiv t^2\) and \(\color{magenta}{\widehat{f(t)} \equiv 1/3}\) on the domain. The center panel shows the residual function, that is \(f(t) - \widehat{f(t)}\). The right panel gives the square of the length of the residual function, which is \(\int_{-1}^1 \left[f(t) - \widehat{f(t)}\right]^{1/2}\, dt\) as indicated by the area shaded in \(\color{blue}{\text{blue}}\).

## 33.2 Sinusoids as vectors

The sinusoid is our fundamental model of periodic phenomena. To get started with using sinusoids as vectors, we will start with a simple setting: a single sinusoid of a specified frequency.

Figure 33.2 shows three sinusoids all with the same frequency, but shifted somewhat in time:

Since we have a dot product for functions, we can treat each of the three sinusoids as a vector. For instance, consider the length of waveforms A and B and the included angle between them.

```
## vector lengths
<- Integrate(waveA(t) * waveA(t) ~ t, bounds(t=0:1)) %>% sqrt()
lengthA
lengthA## [1] 0.7071068
<- Integrate(waveB(t) * waveB(t) ~ t, bounds(t=0:1)) %>% sqrt()
lengthB
lengthB## [1] 0.7071068
<- Integrate(waveC(t) * waveC(t) ~ t, bounds(t=0:1)) %>% sqrt()
lengthC
lengthC## [1] 0.7071068
## dot products
<- Integrate(waveA(t) * waveB(t) ~ t, bounds(t=0:1))
dotAB
dotAB## [1] -3.984443e-18
<- Integrate(waveA(t) * waveC(t) ~ t, bounds(t=0:1))
dotAC
dotAC## [1] -0.1545085
<- Integrate(waveB(t) * waveC(t) ~ t, bounds(t=0:1))
dotBC
dotBC## [1] -0.4755283
```

The cosine of the included angle \(\theta\) between functions A and B is calculated using the the dot product formula: \[\cos(\theta) = \frac{A\bullet B}{\|A\|\, \|B\|}\] or, computationally

```
/ (lengthA * lengthB)
dotAB ## [1] -7.968886e-18
```

Since \(\cos(\theta) = 0\), wave A and B are orthogonal. Admittedly, there is no right angle to be perceived from the graph, but the mathematics of angles gives this result.

The graphical presentation of orthogonality between waveforms A and B is easier to appreciate if we plot out the dot product itself: the integral of waveform A times waveform B. Figure 33.3 shows this integral using colors, blue for positive and orange for negative. The integral is zero, since the positive (blue) areas exactly equal the negative (orange) areas.

In contrast, waveform A is **not** orthogonal to waveform C, and similarly for waveform B. ?fig-AC-BC shows this graphically: the positive and negative areas in the two integrals do not cancel out to zero.

We can project waveform C onto the 2-dimensional subspace spanned by A and B. Since waveforms A and B are orthogonal, This can be done simply by projecting C onto each of A and B one at a time. Here’s a calculation of the scalar multipliers for A and for B and the model vector (that is, the component of C in the A-B subspace):

```
<- dotAC / lengthA^2
A_coef <- dotBC / lengthB^2
B_coef <- makeFun(A_coef*waveA(t) + B_coef*waveB(t) ~ t)
mod_vec # length of mod_vec
Integrate(mod_vec(t)*mod_vec(t) ~ t, bounds(t=0:1)) %>% sqrt()
## [1] 0.7071068
```

You can see that the length of the model vector is the same as the length of the vector being projected. This means that waveform C lies exactly in the subspace spanned by waveforms A and B.

A time-shifted sinusoid of frequency \(\omega\) can *always* be written as a linear combination of \(\sin(2\pi\omega t)\) and \(\cos(2\pi\omega t)\). The coefficients of the linear combination tell us both the amplitude of the time-shifted sinusoid and the time shift.

Consider the function \(g(t) \equiv 17.3 \sin(2*pi*5*(t-0.02)\) on the domain \(0 \leq t \leq 1\) seconds. The amplitude is 17.3. The time shift is 0.02 seconds. Let’s confirm this using the coefficients on the linear combination of sine and cosine of the same frequency.

```
<- makeFun(17.3 * sin(2*pi*5*(t-0.02)) ~ t)
g <- makeFun(sin(2*pi*5*t) ~ t)
sin5 <- makeFun(cos(2*pi*5*t) ~ t)
cos5 <- Integrate(g(t) * sin5(t) ~ t, bounds(t=0:1)) /
A_coef Integrate(sin5(t) * sin5(t) ~ t, bounds(t=0:1))
A_coef## [1] 13.99599
<- Integrate(g(t)*cos5(t) ~ t, bounds(t=0:1)) /
B_coef Integrate(cos5(t) * cos5(t) ~ t, bounds(t=0:1))
B_coef## [1] -10.16868
```

The amplitude of \(g(t)\) is the Pythagorean sum of the two coefficients:

```
sqrt(A_coef^2 + B_coef^2)
## [1] 17.3
```

The time delay involves the ratio of the two coefficients:

```
atan2(B_coef, A_coef) / (2*pi*5)
## [1] -0.02
```

For our purposes here, we will need only the Pythagorean sum and will ignore the time delay.

?fig-cello-seg (top) shows the waveform of a note played on a cello. The note lasts about 1 second. The bottom panel zooms in on the waveform, showing 82 ms (that is, 0.082 s).

The whole note starts with a sharp “attack,” followed by a long period called a “sustain,” and ending with a “decay.” Within the sustain and decay, the waveform is remarkably repetitive, seen best in the bottom panel of the figure.

If you count carefully in the bottom panel, you will see that the waveform completes 9 cycles in the 0.082 s graphical domain. This means that the period is 0.082 / 9 = 0.0091 s. The frequency \(\omega\) is the reciprocal of this: 1/0.0091 = 109.76 Hz. That is, the cello is vibrating about 110 times per second.

In modeling the cello waveform as a linear combination of sinusoids, the frequencies we use ought to respect the period of the cello vibration. Figure 33.4 shows the original waveform as well as the projection of the waveform onto a sinusoid with a frequency of 109.76 Hz. The figure also shows the residual from the projection, which is simply the original waveform minus the projected version.

The sinusoid with \(\omega = 109.76\) is not the only one that will repeat every 0.0091 s. So will a sinusoid with frequency \(2\omega = 219.52\), one with frequency \(3\omega = 329.28\) and so on. These multiples of \(\omega\) are called the ** harmonics** of that frequency. In Figure 33.5 (top) the cello waveform is projected onto \(\omega\) and its first harmonic \(2\omega\). In the middle panel, the projection is made onto \(\omega\) and its first three harmonics. In the bottom panel, the projection is onto \(\omega\) and its first eight harmonics.

As the number of harmonics increases, the approximation gets better and better.

Until now, all the plots of the cello waveform have been made in what’s called the ** time domain**. That is, the horizontal axis of the plots has been

*time*, as seems natural for a function of time.

The decomposition into sinusoids offers another way of describing the cello waveform: the ** frequency domain**. In the frequency domain, we report the amplitude and phase of the projection onto each frequency, plotting that versus frequency. Figure 33.6 shows the waveform in the frequency domain.

From the amplitude graph in Figure 33.6, you can see that only a handful of frequencies account for almost all of the signal. Thus, the frequency domain representation is in many ways much more simple and compact than the time domain representation.

The frequency domain description is an important tool in many fields. As you will see in Block 6, models of many kinds of systems, from the vibrations of buildings during an earthquake, aircraft wings in response to turbulence, and the bounce of a car moving over a rutted road have a very simple form when stated in the frequency domain. Each sinusoid in the input (earthquake shaking, air turbulence, rutted road) gets translated into the same frequency sinusoid in the output (building movement, wing bending, car bound): just the amplitude and phase of the sinusoid is altered.

The construction of the frequency domain description from the waveform is called a ** Fourier Transform**, one of the most important techiques in science.

## 33.3 Exercises

#### Exercise 33.03

Consider these functions/vectors on the domain \(0 \leq t \leq 1\):

- \(s_1(t) \equiv \sin(2\pi t)\)
- \(s_2(t) \equiv \sin(2 \pi 2 t)\) (that is, \(\omega = 2\))
- \(s_3(t) \equiv \sin(2 \pi 3 t)\) (that is, \(\omega = 3\))
- \(c_0(t) \equiv \cos(\pi t)\)
- \(c_1(t) \equiv \cos(2 \pi t)\)
- \(c_2(t) \equiv \cos(2 \pi 2 t)\)

Plot out each of the functions on the domain. How many complete cycles does each function complete as \(t\) goes from 0 to 1?

What is the

*length*of each function?All of the functions are mutually orthogonal except one. Which is the odd one out? (Hint: If the dot product is zero, the vectors are orthgonal.)

#### Exercise 33.05

Here is a square-wave function:

```
<- makeFun(ifelse(abs(t) <= 0.5, 1, 0) ~ t)
sq_wave slice_plot(sq_wave(t) ~ t, bounds(t=-1:1))
```

Find the projection of the square wave onto each of these functions. Use the domain \(-1 \leq t \leq 1\).

- \(c_0 \equiv 1\)
- \(c_1 \equiv \cos(1 \pi t)\)
- \(c_2 \equiv \cos(2 \pi t)\)
- \(c_3 \equiv \cos(3 \pi t)\)
- \(c_4 \equiv \cos(4 \pi t)\)
- \(c_5 \equiv \cos(5 \pi t)\)
- \(c_6 \equiv \cos(6 \pi t)\)
- \(c_7 \equiv \cos(7 \pi t)\)

Hint: To find the scalar multiplier of projecting \(g(t)\) onto \(f(t)\), use

\[\int_{-1}^1 g(t)\, f(t)\, dt {\LARGE /} \int_{-1}^1 f(t)\, f(t)\,dt\] or, in R/mosaic

```
<- Integrate(g(t)*f(t) ~ t, bounds(t=-1:1)) /
A Integrate(f(t)*f(t) ~ t, bounds(t=-1:1))
```

Then the projection of \(g()\) onto \(f()\) is \(A\, f(t)\).

Write down the scalar multiplier on each of the 8 functions above.

If you calculated things correctly, this is the linear combination of the 8 functions that best matches the square wave.