math-prereqs.qmd

# Mathematics

{{< include macros.qmd >}}

These lecture notes use:

- algebra
- precalculus
- univariate calculus
- linear algebra
- vector calculus

Some key results are listed here.

{{< include algebra.qmd >}}

## Derivatives

{{< include calc-derivatives.qmd >}}

## Vector Calculus {#sec-vector-calculus}

(adapted from @fieller2018basics, §7.2)

Let $\vx$ and $\vb$ be vectors of length $p$, 
or in other words, matrices of length $p \times 1$:

$$
\vx = \begin{bmatrix}
x_{1} \\
x_{2} \\
\vdots \\
x_{p}
\end{bmatrix} \\
$$

$$
\vb = \begin{bmatrix}
\beta_{1} \\
\beta_{2} \\
\vdots \\
\beta_{p}
\end{bmatrix}
$$

:::{#def-transpose}
#### Transpose

The transpose of a row vector is the column vector with the same 
sequence of entries:

$$
\vx' \equiv \vx^\top \equiv [x_1, x_2, ..., x_p]
$$

:::

:::{#exm-dot-product}
#### Dot product as matrix multiplication

$$
\begin{aligned}
\vx \cdot \vb
&= \vx \' \vb
\\ &= [x_1, x_2, ..., x_p] 
\begin{bmatrix}
\beta_{1} \\
\beta_{2} \\
\vdots \\
\beta_{p}
\end{bmatrix} 
\\ &= 
x_1\beta_1+x_2\beta_2 +...+x_p \beta_p 
\end{aligned}
$$
:::

:::{#thm-transpose-sum}
#### Transpose of a sum
$$(\vx+\vy)\' = \vx\' + \vy\'$$
:::

---

::: {#def-vector-derivative}
#### Vector derivative of a vector-to-scalar function
If $f(\vb)$ is a function that takes a vector $\vb$ as input and outputs a scalar, 
such as $f(\vb) = x'\vb$, 
then:

$$
\deriv{ \vb} f(\vb) = 
\begin{bmatrix}
\deriv{\beta_1}f(\vb) \\
\deriv{\beta_2}f(\vb) \\
\vdots \\
\deriv{\beta_p}f(\vb)
\end{bmatrix}
$$
:::

---

:::{#thm-deriv-lincom}

#### Derivative of a linear combination

$$
\deriv{\vb} \vx\' \vb = x
$$

:::: notes
This looks a lot like non-vector calculus, except that you have to transpose the coefficient.
::::
:::

---

::: proof

$$ 
\ba
\deriv{ \beta} (x\'\beta) 
&= 
\begin{bmatrix}
\deriv{\beta_1}(x_1\beta_1+x_2\beta_2 +...+x_p \beta_p ) \\
\deriv{\beta_2}(x_1\beta_1+x_2\beta_2 +...+x_p \beta_p ) \\
\vdots \\
\deriv{\beta_p}(x_1\beta_1+x_2\beta_2 +...+x_p \beta_p )
\end{bmatrix} 
\\ &=
\begin{bmatrix}
x_{1} \\
x_{2} \\
\vdots \\
x_{p}
\end{bmatrix}
\\ &= \vx
\ea
$$

:::

---

:::{#def-quadratic-form}
#### Quadratic form

A **quadratic form** is a mathematical expression with the structure

$$\vx\' \matr{S} \vx$$

where $\vx$ is a vector and $\matr{S}$ is a matrix with compatible dimensions for vector-matrix multiplication.

:::

::: notes
Quadratic forms occur frequently in regression models. They are the matrix-vector generalizations of the scalar quadratic form $cx^2 = xcx$.
:::

---

:::{#thm-quadratic-form}

#### Derivative of a quadratic form

If $S$ is a $p\times p$ matrix that is constant with respect to $\beta$, then:

$$
\deriv{\beta} \beta'S\beta = 2S\beta
$$

:::
 
::: notes
This is like taking the derivative of $cx^2$ with respect to $x$ in non-vector calculus.
:::

---

:::{#cor-deriv-normsq}

#### Derivative of a simple quadratic form

$$
\deriv{\vb} \vb'\vb = 2\vb
$$

:::

::: notes
This is like taking the derivative of $x^2$.
:::


## Additional resources

### Calculus 

- @mosaiccalc
- @khuri2003advanced

### Linear Algebra and Vector Calculus

- @fieller2018basics
- @banerjee2014linear
- @searle2017matrix

### Numerical Analysis

- [Hua Zhou](https://hua-zhou.github.io/)'s [lecture notes for "UCLA Biostat 216 - Mathematical Methods for Biostatistics" (2023 Fall)](https://ucla-biostat-216.github.io/2023fall/schedule/schedule.html)