BASICS

Alicia Y. Tsai; Authors: Laurent EI Ghaoui; Contributors: Phan Hoang, Tony Tin, Ha To Tam, Nguyen Van Kep, Le Dinh Nam, Juliette Decugis, Nguyen Quoc Cuong.; Giuseppe C. Calafiore

1 BASICS

1.1. Definitions

Vectors

Assume we are given a collection of $n$ real numbers, $x_1, \cdots, x_n$ . We can represent them as $n$ locations on a line. Alternatively, we can represent the collection as a single point in a $n$ -dimensional space. This is the vector representation of the collection of numbers; each number $x_i$ is called a component or element of the vector.

Vectors can be arranged in a column or a row; we usually write vectors in column format:

$\begin{align*} x = \begin{pmatrix} x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{pmatrix} \end{align*}$

We denote by $\mathbb{R}^n$ denotes the set of real vectors with $n$ components. If $x_i \in \mathbb{R}^n$ denotes a vector, we use subscripts to denote components, so that $x_i$ is the $i$ -th component of $x$ . Sometimes the notation $x(i)$ is used to denote the $i$ -th component.


	A vector can also represent a point in a multi-dimensional space $x_i \in \mathbb{R}^n$ , where each component corresponds to a coordinate of the point.
	Example 1: The vector $x=(2,1)$ in $x_i \in \mathbb{R}^2$ .

See also:

Transpose

If $x$ is a column vector, $x^T$ denotes the corresponding row vector, and vice-versa. Hence, if $x$ is the column vector above:

$\begin{align*} x^T = (x_1 \cdots x_n) \end{align*}$

Sometimes we use the looser, in-line notation $x = (x_1, \cdots, x_n)$ , to denote a row or column vector, the orientation being understood from context.

1.2. Independence

A set of vectors ${x_1, x_2, \cdots, x_m}$ in $\mathbb{R}^n$ is said to be linearly independent if and only if the following condition on a vector $\lambda = (\lambda_1, \lambda_2, \ldots, \lambda_m) \in \mathbb{R}^m$ :

$\begin{align*} \sum\limits_{i=1}^{m} \lambda_i x_i = 0 \end{align*}$

implies $\lambda_i = 0$ for $i = 1, 2, \cdots, m$ . This means that no vector in the set can be expressed as a linear combination of the others.

Example 2: the vectors $x_1 = [1, 2, 3]$ and $x_2 = [3, 6, 9]$ are not linearly independent, since $3x_1 - x_2 = 0$ .

1.3. Subspace, span, affine sets

A subspace of $\mathbb{R}^n$ is a subset that is closed under addition and scalar multiplication. Geometrically, subspaces are ‘‘flat’’ (like a line or plane in 3D) and pass through the origin.

An important result of linear algebra, which we will prove later, says that a subspace ${\bf S}$ can always be represented as the span of a set of vectors $x_i \in \mathbb{R}^n$ , $i = 1, \cdots, m$ , that is, as a set of the form

$\begin{align*} {\bf S} = {\bf span}(x_1, \cdots, x_m):= \left\{\sum\limits_{i=1}^{m} \lambda_i x_i: \lambda \in \mathbb{R}^m\right\} \end{align*}$

An affine set is a translation of a subspace — it is ‘‘flat’’ but does not necessarily pass through $0$ , as a subspace would. (Think for example of a line, or a plane, that does not go through the origin.) So an affine set ${\bf A}$ can always be represented as the translation of the subspace spanned by some vectors:

$\begin{align*} {\bf A} = \left\{x_0 + \sum\limits_{i=1}^{m} \lambda_i x_i: \lambda \in \mathbb{R}^m\right\}, \end{align*}$

for some vectors ${x_0, x_1, x_2, \cdots, x_m}$ where $x_0 \in \mathbb{R}^n$ . In shorthand notation, we write ${\bf A} = x_0 + {\bf S}.$

	Example 3: In $\mathbb{R}^3$ , the span ${\bf S}$ of the two vectors
	$\begin{align} u &= \begin{bmatrix} -1 \\ 2 \\ 0.5 \end{bmatrix}, \quad v = \begin{bmatrix} 1 \\ 3 \\ 0.1 \end{bmatrix} \end{align}$
	is the plane passing through the origin pictured in blue.

When ${\bf S}$ is the span of a single non-zero vector, the set ${\bf A}$ is called a line passing through the point $x_0$ . Thus, lines have the form

$\begin{align*}\{x_0 + tu : t\in \mathbb{R}\}\end{align*}$

where $u$ determines the direction of the line, and $x_0$ is a point through which it passes.

Example 4: A line in $\mathbb{R}^2$ passing through the point $x_0 = (0,1)$ , with direction $u=(0.8944, 0.4472)$ .

1.4. Basis, dimension

Basis

A basis of $\mathbb{R}^n$ is a set of $n$ independent vectors. If the vectors $u_1, \cdots, u_n$ form a basis, we can express any vector as a linear combination of the $u_i$ ‘s:

$\begin{align*} x = \sum\limits_{i=1}^{n} \lambda_i u_i \end{align*}$

for appropriate numbers $\lambda_1, \cdots, \lambda_n$ .

The standard basis (alternatively, natural basis) in $\mathbb{R}^n$ consists of the vectors $e_i$ , where $e_i$ ‘s components are all zero, except the $i$ -th, which is equal to 1. In $\mathbb{R}^3$ , we have

$\begin{align*} e_1 &:= \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix}, & e_2 &:= \begin{pmatrix} 0 \\ 1 \\ 0 \end{pmatrix}, & e_3 &:= \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \end{align*}$

Example 5: The set of three vectors in $\mathbb{R}^3$ :

$\begin{align*} x_1 &:= \begin{pmatrix} 1 \\ 2 \\ 3 \end{pmatrix}, & x_2 &:= \begin{pmatrix} 4 \\ 5 \\ 6 \end{pmatrix}, & x_3 &:= \begin{pmatrix} 3 \\ 3 \\ 3 \end{pmatrix} \end{align*}$

is not independent, since $x_1-x_2+x_3=0$ , and its span has dimension $2$ . Since $x_1, x_2$ are independent (the equation $\lambda_1 x_1 + \lambda_2 x_2 = 0$ has $\lambda = 0$ as the unique solution), a basis for that span is, for example, $\{ x_1, x_2 \}$ . In contrast, the collection $\{ x_1, x_2, x_3 - e_1 \}$ spans the whole space $\mathbb{R}^3$ , and thus forms a basis of that space.

Basis of a subspace

The basis of a given subspace ${\bf S} \in \mathbb{R}^n$ is any independent set of vectors whose span is ${\bf S}$ . If the vectors $u_1, \cdots, u_r$ form a basis of ${\bf S}$ , we can express any vector as a linear combination of the $u_i$ ‘s:

$\begin{align*} x = \sum\limits_{i=1}^{r} \lambda_i u_i \end{align*}$

for appropriate numbers $\lambda_1, \cdots, \lambda_r$ .

The number of vectors in the basis is actually independent of the choice of the basis (for example, in $\mathbb{R}^3$ you need two independent vectors to describe a plane containing the origin). This number is called the dimension of ${\bf S}$ . We can accordingly define the dimension of an affine subspace, as that of the linear subspace of which it is a translation.

Examples:

The dimension of a line is 1 since a line is of the form $x_0 + {\bf span}(x_1)$ for some non-zero vector $x_1$ .
Dimension of an affine subspace.

License

Icon for the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License