LINEAR MAPS

Alicia Y. Tsai; Authors: Laurent EI Ghaoui; Contributors: Phan Hoang, Tony Tin, Ha To Tam, Nguyen Van Kep, Le Dinh Nam, Juliette Decugis, Nguyen Quoc Cuong.; Giuseppe C. Calafiore

14 LINEAR MAPS

14.1 Definition and Interpretation

Definition

A map $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ is linear (resp. affine) if and only if every one of its components is. The formal definition we saw here for functions applies verbatim to maps.

To an $m \times n$ matrix $A$ , we can associate a linear map $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ , with values $f(x)=A x$ . Conversely, to any linear map, we can uniquely associate a matrix $A$ which satisfies $f(x)=A x$ for every $x$ .

Indeed, if the components of $f, f_i, i=1, \ldots, m$ , are linear, then they can be expressed as $f_i(x)=a_i^T x$ for some $a_i \in \mathbb{R}^n$ . The matrix $A$ is the matrix that has $a_i^T$ as its $i$ -th row:

$f(x)=\left(\begin{array}{c} f_1(x) \\ \vdots \\ f_n(x) \end{array}\right)=\left(\begin{array}{c} a_1^T x \\ \vdots \\ a_n^T x \end{array}\right)=A x, \quad \text { with } A:=\left(\begin{array}{c} a_1^T \\ \vdots \\ a_m^T \end{array}\right) \in \mathbb{R}^{m \times n} .$

Hence, there is a one-to-one correspondence between matrices and linear maps. This is extending what we saw for vectors, which are in one-to-one correspondence with linear functions.

This is summarized as follows.

Representation of affine maps via the matrix-vector product. A function $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ is affine if and only if it can be expressed via a matrix-vector product:

$f(x)=A x+b,$

for some unique pair $(A, b)$ , with $A \in \mathbb{R}^{m \times n}$ and $b \in \mathbb{R}^m$ . The function is linear if and only if $b=0$ .

The result above shows that a matrix can be seen as a (linear) map from the “input” space $\mathbb{R}^n$ to the “output” space $\mathbb{R}^m$ . Both points of view (matrices as simple collections of vectors, or as linear maps) are useful.

Interpretations

Consider an affine map $x \rightarrow y=A x+b$ . An element $A_{i j}$ gives the coefficient of influence of $x_j$ over $y_i$ . In this sense, if $A_{13} \gg A_{14}$ we can say that $x_3$ has much more influence on $y_1$ than $x_4$ . Or, $A_{24}=0$ says that $y_2$ does not depend at all on $x_4$ . Often the constant term $b=f(0)$ is referred to as the “bias” vector.

14.2 First-order approximation of non-linear maps

Since maps are just collections of functions, we can approximate a map with a linear (or affine) map, just as we did with functions here. If $f: \mathbb{R}^n \rightarrow \mathbb{R}^m$ is differentiable, then we can approximate the (vector) values of $f$ near a given point $x_0 \in \mathbb{R}^n$ by an affine map $\tilde{f}$ :

$f(x) \approx \tilde{f}(x):=f\left(x_0\right)+A\left(x-x_0\right),$

where $A_{i j}=\dfrac{\partial f_i}{\partial x_j}\left(x_0\right)$ is the derivative of the $i$ -th component of $f$ with respect to $x_j$ ( $A$ is referred to as the Jacobian matrix of $f$ at $x_0$ ).

See also:

License

Icon for the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License

14.1 Definition and Interpretation

Definition

Interpretations

14.2 First-order approximation of non-linear maps

License

Share This Book