Tutorials on Advanced Math and Computer Science Concepts

Spanning Sets and Linear Independence

A linear combination is an expression that includes only the operations of addition and multiplication. In the case of vectors, we utilize vector addition and scalar multiplication. Linear combinations are of interest because they can expose information about vectors, specifically subsets, and subspaces of vectors.

Theorem: If $ \{ \vec{v_1},\vec{v_2},...,\vec{v_k} \} $ is a set of vectors in $\mathbb{R}^n$ and S is the set of all possible linear combinations of these vectors, $S = \{ t_1\vec{v_1} + t_2 \vec{v_2} + ... + t_k \vec{v_k} \} $, then S is a subspace of $\mathbb{R}^n$.

We are saying here that if we take any subset of vectors from $ \mathbb{R}^n $, the set of linear combinations is subspace of $\mathbb{R}^n$. Let's take a look at why this is true.

Proof: To prove that this is true, we need to show that the three properties of subspaces holds for the linear combination we are constructing, S.

Given that these vectors are taken from $\mathbb{R}^n$, we know that their addition and scalar multiplication are closed under $\mathbb{R}^n$, since we know that $\mathbb{R}^n$ is a space itself. From here, we know that using t = 0 would give us the zero vector, so it must be contained in S. Therefore, we can conclude that S is a subspace of $\mathbb{R}^n$

To introduce some new terminology, the subset $\{\vec{v_1},...,\vec{v_k} \}$ is a special subset called a spanning set. It is important because if we have a subset $\{\vec{v_1},...,\vec{v_k} \}$, we can use it construct any other vector in the set it spans using only vector addition and scalar multiplication.

As an example, consider the set $\mathbb{R}^2$. We can construct any vector in $\mathbb{R}^2$ using just two vectors that are in $\mathbb{R}^2$. These two vectors are $[\begin{array}{cc} 0 \\ 1 \end{array}]$ and $\left[\begin{array}{cc} 1 \\ 0 \end{array}\right]$. We call these vectors a basis of $\mathbb{R}^2$, and using just these two vectors, you can build any other vector in $\mathbb{R}^2$. This is particularly useful if we ever want to do translations, scaling, or rotation. They give us a place that we can always start to construct any other vector.

Often, we may also want to use linear combinations to determine if a set of vectors are related to each other or not. The idea of linear independence allows us to find situations like this. If there exists no solution to $t_1\vec{v_1} + t_2\vec{v_2} + ... + t_k\vec{v_k} = 0$, where $t \ne 0$, then we state that $\vec{v_1},\vec{v_2}, ... , \vec{v_k}$ are linearly independent. If there exists a non-zero solution, then we state that the vectors are linearly dependent.

Revisiting our idea of a basis of a set of vectors will help us understand why we care about this idea. If we want to find the basis of a set of vectors, we want to find the smallest one possible. Otherwise, we will have an additional vector that is not useful, since it can be constructed using the others already. Being able to determine if vectors are linearly dependent will allow us to construct basis subsets that are the smallest possible.

Let's take a look at an example to show how we can determine if a set of vectors is linearly independent of each other.

Example: Show that the set $\left[\begin{array}{cc} 7 \\ -14 \\ 6 \end{array}\right], \left[\begin{array}{cc} -10 \\ 15 \\ \frac{15}{14} \end{array}\right], \left[\begin{array}{cc} -1 \\ 0 \\ 3 \end{array}\right]$ is linearly dependent.

To show that this is linearly dependent, we need to show that $t_1 * \left[\begin{array}{cc} 7 \\ -14 \\ 6 \end{array}\right] + t_2 * \left[\begin{array}{cc} -10 \\ 15 \\ \frac{15}{14} \end{array}\right] + t_3 * \left[\begin{array}{cc} -1 \\ 0 \\ 3 \end{array}\right] = \vec{0}$

The easiest way to do this is to construct a system of linear equations, and solve it. Doing this gives us the following system.

$7t_1 - 10t_2 - t_3 = 0$
$-14t_1 + 15t_2 = 0$
$6t_1 + \frac{15}{14}t_2 + 3t_3 = 0$

If we can determine values that satisfies this system, we can conclude that the set of vectors is linearly dependent. You can solve this system however you'd like, I will show one example of a solution.

First, I will rearrange equation 2 for $t_1$

$-14t_1 + 15t_2 = 0$
$t_1 = \frac{15}{14}t_2$

Next, I'll substitute into equation 1 to get a value for $t_3$

$7(\frac{15}{14}t_2) - 10t_2 - t_3 = 0$
$7.5t_2 - 10t_2 -t_3 = 0$
$-2.5t_2 - t_3 = 0$
$-2.5t_2 = t_3$

From here, I can substitute the $t_3$ and $t_1$ values into equation 3 to get a value for $t_1$

$6t_1 + \frac{15}{14}t_2 + 3t_3 = 0$
$6(\frac{15}{14}t_2}) + \frac{15}{14}t_2 + 3(-2.5t_2) = 0$
$\frac{90}{14}t_2 + \frac{15}{14}t_2 - 7.5t_2 = 0$
$\frac{105}{14}t_2 - 7.5t_2 = 0$

This gives us a value of $t_2 = -1$. We can now substitute this value into the other equations to solve the rest of the system. This would give us $t_1 = -\frac{15}{14}$ and $t_3 = 2.5$.

Therefore, there is a non-zero solution for this set of vectors, meaning it is linearly dependent. If you tried solving this problem on your own, you may have gotten different numbers than me. There are actually many correct solutions to this system, so as long as the three vectors add to make 0, you have a correct solution.