3 Vector Spaces

In this chapter, we will mainly be concerned with finite dimensional vector spaces over ℝ or ℂ. Please note that the real and complex numbers have the property that any pair of elements can be added, subtracted or multiplied. Also, division is allowed by a nonzero element. Such sets in mathematics are called field. So, ℝ and ℂ are examples of field. The fields ℝ and ℂ have infinite number of elements. But, in mathematics, we do have fields that have only finitely many elements. For example, consider the set ℤ₅ = {0,1,2,3,4}. In ℤ₅, we respectively, define addition and multiplication, as

Thus, ℤ₅ indeed behaves like a field. So, in this chapter, F will represent a field.

3.1 Vector Spaces: Definition and Examples

Let A ∈ M_m,n(F) and let V denote the solution set of the homogeneous system Ax = 0. Then, by Theorem 2.1.9, V satisfies:

That is, the solution set of a homogeneous linear system satisfies some nice properties. The Euclidean plane, ℝ², and the Euclidean space, ℝ³, also satisfy the above properties. In this chapter, our aim is to understand sets that satisfy such properties. We start with the following definition.

Definition 3.1.1. [Vector Space] A vector space V over F, denoted V(F) or in short V (if the field F is clear from the context), is a non-empty set, satisfying the following conditions:

1.

Vector Addition: To every pair u,v ∈ V there corresponds a unique element u ⊕ v ∈ V (called the addition of vectors) such that

(a): u ⊕ v = v ⊕ u (Commutative law).
(b): (u ⊕ v) ⊕ w = u ⊕ (v ⊕ w) (Associative law).
(c): V has a unique element, denoted 0, called the zero vector that satisfies u⊕0 = u, for every u ∈ V (called the additive identity).
(d): for every u ∈ V there is an element w ∈ V that satisfies u ⊕ w = 0.

2.

Scalar Multiplication: For each u ∈ V and α ∈ F, there corresponds a unique element α⊙ u in V (called the scalar multiplication) such that

(a): α ⋅ (β ⊙ u) = (αβ) ⊙ u for every α,β ∈ F and u ∈ V (⋅ is multiplication in F).
(b): 1 ⊙ u = u for every u ∈ V, where 1 ∈ F.

3.

Distributive Laws: relating vector addition with scalar multiplication
For any α,β ∈ F and u,v ∈ V, the following distributive laws hold: PICT

DRAFT

(a): α ⊙ (u ⊕ v) = (α ⊙ u) ⊕(α ⊙ v).
(b): (α + β) ⊙ u = (α ⊙ u) ⊕(β ⊙ u) (+ is addition in F).

Remark 3.1.2. [Real / Complex Vector Space]

1.: The elements of F are called scalars.
2.: The elements of V are called vectors.
3.: We denote the zero element of F by 0, whereas the zero element of V will be denoted by 0.
4.: Observe that Condition 3.1.1.1d implies that for every u ∈ V, the vector w ∈ V such that u + w = 0 holds, is unique. For if, w₁,w₂ ∈ V with u + w_i = 0, for i = 1,2 then by commutativity of vector addition, we see that $w1 = w1 + 0 = w1 + (u + w2 ) = (w1 + u )+ w2 = 0 + w2 = w2.$ Hence, we represent this unique vector by -u and call it the additive inverse.
5.: If V is a vector space over ℝ then, V is called a real vector space.
6.: If V is a vector space over ℂ then V is called a complex vector space.
7.: In general, a vector space over ℝ or ℂ is called a linear space.

DRAFT

Some interesting consequences of Definition 3.1.1 is stated next. Intuitively, they seem obvious but for better understanding of the given conditions, it is desirable to go through the proof.

Theorem 3.1.3. Let V be a vector space over F. Then,

1.: u ⊕ v = u implies v = 0.
2.: α ⊙ u = 0 if and only if either u = 0 or α = 0.
3.: (-1) ⊙ u = -u, for every u ∈ V.

Proof. Part 1: By Condition 3.1.1.1d, for each u ∈ V there exists -u ∈ V such that -u ⊕ u = 0. Hence, u ⊕ v = u is equivalent to

Now suppose α⊙u = 0. If α = 0 then the proof is over. Therefore, assume that α≠0,α ∈ F. Then, (α)^-1 ∈ F and

Part 3: As 0 = 0 ⋅ u = (1 + (-1))u = u ⊕ (-1) ⋅ u, one has (-1) ⋅ u = -u. _

Example 3.1.4. The readers are advised to justify the statements given below.

1.

Let A ∈ M_m,n(F) with Rank(A) = r ≤ n. Then, using Theorem 2.2.40, the solution set of the homogeneous system Ax = 0 is a vector space over F.

2.

Consider ℝ with the usual addition and multiplication. That is, a ⊕ b = a + b and a ⊙ b = a ⋅ b. Then, ℝ forms a real vector space.

3.

Let ℝ² = {(x₁,x₂)^T|x₁,x₂ ∈ ℝ} Then, for x₁,x₂,y₁,y₂ ∈ ℝ and α ∈ ℝ, define

DRAFT

Verify that ℝ² is a real vector space.

4.

Let ℝⁿ = {(a₁,…,a_n)^T|a_i ∈ ℝ,1 ≤ i ≤ n}. For u = (a₁,…,a_n)^T,v = (b₁,…,b_n)^T ∈ V and α ∈ ℝ, define

u ⊕ v = (a1 + b1,...,an + bn)T and α⊙ u = (αa1, ...,αan )T

(called component wise operations). Then, V is a real vector space. The vector space ℝⁿ is called the real vector space of n-tuples.

Recall that the symbol i represents the complex number √---
- 1 .

5.

Consider ℂ = {x + iy|x,y ∈ ℝ}, the set of complex numbers. Let z₁ = x₁ + iy₁ and z₂ = x₂ + iy₂ and define z₁ ⊕ z₂ = (x₁ + x₂) + i(y₁ + y₂). For scalar multiplication,

(a): let α ∈ ℝ and define, α ⊙ z₁ = (αx₁) + i(αy₁). Then, ℂ is a vector space over ℝ (called the real vector space).
(b): let α +iβ ∈ ℂ and define, (α +iβ)⊙(x₁ +iy₁) = (αx₁ -βy₁)+i(αy₁ +βx₁). Then, ℂ forms a vector space over ℂ (called the complex vector space).

6.

Let ℂⁿ = {(z₁,…,z_n)^T|z_i ∈ ℂ,1 ≤ i ≤ n}. For z = (z₁,…,z_n),w = (w₁,…,w_n)^T ∈ ℂⁿ and α ∈ F, define

z+ w = (z1 + w1, ...,zn + wn )T , and α ⊙ z = (αz1,...,αzn )T.

Then, verify that ℂⁿ forms a vector space over ℂ (called the complex vector space) as well as over ℝ (called the real vector space). Unless specified otherwise, ℂⁿ will be considered a complex vector space.

Remark 3.1.5. If F = ℂ then i(1,0) = (i,0) is allowed. Whereas, if F = ℝ then i(1,0) doesn’t make sense as i ⁄∈ ℝ.

7.

Fix m,n ∈ ℕ and let M_m,n(ℂ) = {A_m×n = [a_ij]|a_ij ∈ ℂ}. For A,B ∈ M_m,n(ℂ) and α ∈ ℂ, define (A + αB)_ij = a_ij + αb_ij. Then, M_m,n(ℂ) is a complex vector space. If m = n, the vector space M_m,n(ℂ) is denoted by M_n(ℂ).

8.

Let S be a non-empty set and let ℝ^S = {f|f is a function from S to ℝ}. For f,g ∈ ℝ^S and α ∈ ℝ, define (f + αg)(x) = f(x) + αg(x), for all x ∈ S. Then, ℝ^S is a real vector space. In particular,

(a): for S = ℕ, observe that ℝ^ℕ consists of all real sequences and forms a real vector space.
(b): Let V be the set of all bounded real sequences. Then, V is a real vector space.
(c): Let V be the set of all real sequences that converge to 0. Then, V is a real vector space.
(d): Let S be the set of all real sequences that converge to 1. Then, check that S is not a vector space. Determine the conditions that fail.

9.

Fix a,b ∈ ℝ with a < b and let

([a,b], ℝ) = {f : [a,b] → ℝ|f is continuous}. Then,

([a,b], ℝ) with (f + αg)(x) = f(x) + αg(x), for all x ∈ [a,b], is a real vector space.

10.

Let

(ℝ, ℝ) = {f : ℝ → ℝ|f is continuous}. Then,

(ℝ, ℝ) is a real vector space, where (f + αg)(x) = f(x) + αg(x), for all x ∈ ℝ. PICT

DRAFT

11.

Fix a < b ∈ ℝ and let

²((a,b), ℝ) = {f : (a,b) → ℝ|f′′ is continuous}. Then,

²((a,b), ℝ) with (f + αg)(x) = f(x) + αg(x), for all x ∈ (a,b), is a real vector space.

12.

Fix a < b ∈ ℝ and let

^∞((a,b), ℝ) = {f : (a,b) → ℝ|f is infinitely differentiable}. Then,

^∞((a,b), ℝ) with (f + αg)(x) = f(x) + αg(x), for all x ∈ (a,b) is a real vector space.

13.

Fix a < b ∈ ℝ. Then, V = {f : (a,b) → ℝ|f′′ + f′ + 2f = 0} is a real vector space.

Note that the in the last few examples we can replace ℝ by ℂ to get corresponding complex vector spaces.

14.

Let ℝ[x] = {a₀ + a₁x +

+ a_nxⁿ|a_i ∈ ℝ, for 0 ≤ i ≤ n}. Now, let p(x),q(x) ∈ ℝ[x]. Then, we can choose m such that p(x) = a₀ + a₁x +

+ a_mx^m and q(x) = b₀ + b₁x +

+ b_mx^m, where some of the a_i’s or b_j’s may be zero. Then, we define

p(x)+ q(x ) = (a0 + b0) + (a1 + b1)x+ ⋅⋅⋅+ (am + bm )xm

and αp(x) = (αa₀) + (αa₁)x + ⋅⋅⋅

+ (αa_m)x^m, for α ∈ ℝ. With the operations defined above (called component wise addition and multiplication), it can be easily verified that ℝ[x] forms a real vector space.

15.

Fix n ∈ ℕ and let ℝ[x;n] = {p(x) ∈ ℝ[x]|p(x) has degree ≤ n}. Then, with component wise addition and multiplication, the set ℝ[x;n] forms a real vector space.

16.

Let ℂ[x] = {a₀ + a₁x + ⋅⋅⋅

+ a_nxⁿ|a_i ∈ ℂ, for 0 ≤ i ≤ n}. Then, component wise addition and multiplication, the set ℂ[x] forms a complex vector space. One can also look at ℂ[x;n], the set of complex polynomials of degree less than or equal to n. Then, ℂ[x;n] forms a complex vector space.

17.

Let V = {0}. Then, V is a real as well as a complex vector space.

18.

Let ℝ⁺ = {x ∈ ℝ|x > 0}. Then, PICT

DRAFT

(a): ℝ⁺ is not a vector space under usual operations of addition and scalar multiplication.
(b): ℝ⁺ is a real vector space with 1 as the additive identity if we define $u ⊕ v = u ⋅v and α ⊙ u = u α, for all u, v ∈ ℝ+ and α ∈ ℝ.$

19.

For any α ∈ ℝ and x = (x₁,x₂)^T,y = (y₁,y₂)^T ∈ ℝ², define

x ⊕ y = (x1 + y1 + 1,x2 + y2 - 3)T and α ⊙ x = (αx1 + α - 1,αx2 - 3α+ 3 )T .

Then, ℝ² is a real vector space with (-1,3)^T as the additive identity.

20.

Let V = {A = [a_ij] ∈ M_n(ℂ)|a₁₁ = 0}. Then, V is a complex vector space.

21.

Let V = {A = [a_ij] ∈ M_n(ℂ)|A = A^*}. Then, verify that V is a real vector space but not a complex vector space.

22.

Let V and W be vector spaces over F, with operations (+,∙) and (⊕,⊙), respectively. Let V × W = {(v,w)|v ∈ V,w ∈ W}. Then, V × W forms a vector space over F, if for every (v₁,w₁),(v₂,w₂) ∈ V × W and α ∈ ℝ, we define

DRAFT

v₁ + v₂ and w₁ ⊕ w₂ on the right hand side mean vector addition in V and W, respectively. Similarly, α ∙ v₁ and α ⊙ w₁ correspond to scalar multiplication in V and W, respectively.

23.

Let ℚ be the set of scalars. Then, ℝ is a vector space over ℚ. As e,π,π - √ --
2

⁄∈ ℚ, these real numbers are vectors but not scalars in this space.

24.

Similarly, ℂ is a vector space over ℚ. Since e - π,i + √2--

,i ⁄∈ ℚ, these complex numbers are vectors but not scalars in this space.

25.

Recall the field ℤ₅ = {0,1,2,3,4} given on the first page of this chapter. Then, V = {(a,b)|a,b ∈ ℤ₅} is a vector space over ℤ₅ having 25 vectors.

Note that all our vector spaces, except the last three, are linear spaces.

From now on, we will use ‘u + v’ for ‘u ⊕ v’ and ‘αu or α ⋅ u’ for ‘α ⊙ u’.

Exercise 3.1.6.

1.

Verify that the vectors spaces mentioned in Example 3.1.4 do satisfy all the conditions for vector spaces.

2.

Does the set V given below form a real/complex or both real and complex vector space? Give reasons for your answer.

(a): For x = (x₁,x₂)^T,y = (y₁,y₂)^T ∈ ℝ², define x + y = (x₁ + y₁,x₂ + y₂)^T and αx = (αx₁,0)^T for all α ∈ ℝ.
(b): Let V = .
(c): Let V = . DRAFT
(d): Let V = {(x,y,z)^T|x + y + z = 1}.
(e): Let V = {(x,y)^T ∈ ℝ²|x ⋅ y = 0}.
(f): Let V = {(x,y)^T ∈ ℝ²|x = y²}.
(g): Let V = {α(1,1,1)^T + β(1,1,-1)^T|α,β ∈ ℝ}.
(h): Let V = ℝ with x ⊕ y = x - y and α ⊙ x = -αx, for all x,y ∈ V and α ∈ ℝ.
(i): Let V = ℝ². Define (x₁,y₁)^T⊕(x₂,y₂)^T = (x₁+x₂,0)^T and α⊙(x₁,y₁)^T = (αx₁,0)^T, for α,x₁,x₂,y₁,y₂ ∈ ℝ.

3.1.1 Subspaces

Definition 3.1.7. [Vector Subspace] Let V be a vector space over F. Then, a non-empty subset S of V is called a subspace of V if S is also a vector space with vector addition and scalar multiplication inherited from V.

Example 3.1.8.

1.

If V is a vector space then V and {0} are subspaces, called trivial subspaces.

2.

The real vector space ℝ has no non-trivial subspace. To check this, let V≠{0} be a vector subspace of ℝ. Then, there exists x ∈ ℝ,x≠0 such that x ∈ V. Now, using scalar multiplication, we see that {αx|α ∈ ℝ}⊆ V. As, x≠0, the set {αx|α ∈ ℝ} = ℝ. This in turn implies that V = ℝ. PICT

DRAFT

3.

W = {x ∈ ℝ³|[1,2,-1]x = 0} is a plane in ℝ³ containing 0 (hence a subspace).

4.

W = {x ∈ ℝ³| [ ]
1 1 1

1 - 1 - 1

x = 0} is a line in ℝ³ containing 0 (hence a subspace).

5.

The vector space ℝ[x;n] is a subspace of ℝ[x].

6.

Is V = {xp(x)|p(x) ∈ ℝ[x]} a subspace of ℝ[x]?

7.

Verify that

²(a,b) is a subspace of

(a,b).

8.

Verify that W = {(x,0)^T ∈ ℝ²|x ∈ ℝ} is a subspace of ℝ².

9.

Is the set of sequences converging to 0 a subspace of the set of all bounded sequences?

10.

Let V be the vector space of Example 3.1.4.19. Then,

(a): S = {(x,0)^T|x ∈ ℝ} is not a subspace of V as (x,0)^T⊕(y,0)^T = (x+y+1,-3)^T ⁄∈ S.
(b): Verify that W = {(x,3)^T|x ∈ ℝ} is a subspace of V.

11.

The vector space ℝ⁺ defined in Example 3.1.4.18 is not a subspace of ℝ.

Let V(F) be a vector space and W ⊆ V, W≠∅. We now prove a result which implies that to check W to be a subspace, we need to verify only one condition.

Theorem 3.1.9. Let V(F) be a vector space and W ⊆ V, W≠∅. Then, W is a subspace of V if and only if αu + βv ∈ W whenever α,β ∈ F and u,v ∈ W.

Proof. Let W be a subspace of V and let u,v ∈ W. Then, for every α,β ∈ F, αu,βv ∈ W and hence αu + βv ∈ W.

Now, we assume that αu + βv ∈ W, whenever α,β ∈ F and u,v ∈ W. To show, W is a subspace of V: PICT

DRAFT

Exercise 3.1.10.

1.

Determine all the subspaces of ℝ and ℝ².

2.

Prove that a line in ℝ² is a subspace if and only if it passes through (0,0) ∈ ℝ².

3.

Fix n ∈ ℕ. Then, is W a subspace of M_n(ℝ), where

(a): W = {A ∈ M_n(ℝ)|A is upper triangular}?
(b): W = {A ∈ M_n(ℝ)|A is symmetric}?
(c): W = {A ∈ M_n(ℝ)|A is skew-symmetric}?
(d): W = {A|A is a diagonal matrix}?
(e): W = {A|trace(A) = 0}?
(f): W = {A ∈ M_n(ℝ)|A^T = 2A}? DRAFT
(g): W = {A = [a_ij]|a₁₁ + a₂₁ + a₃₄ = 0}?

4.

Fix n ∈ ℕ. Then, is W = {A = [a_ij]|a₁₁ + a₂₂ = 0} a subspace of the complex vector space M_n(ℂ)? What if M_n(ℂ) is a real vector space?______________________________________

5.

Are all the sets given below subspaces of

([-1,1])?

(a): W = {f ∈ C([-1,1])|f(1∕2) = 0}.
(b): W = {f ∈ C([-1,1])|f(-1∕2) = 0,f(1∕2) = 0}.

6.

Are all the sets given below subspaces of ℝ[x]? Recall that the degree of the zero polynomial is assumed to be -∞.

(a): W = {f(x) ∈ ℝ[x]|deg(f(x)) = 3}.
(b): W = {f(x) ∈ ℝ[x]|deg(f(x)) ≤ 0}.
(c): W = {f(x) ∈ ℝ[x]|f(0) = 0}.

7.

Which of the following are subspaces of ℝⁿ(ℝ)?

(a): {(x₁,x₂,…,x_n)^T|x₁ ≥ 0}.
(b): {(x₁,x₂,…,x_n)^T|x₁ is rational}.
(c): {(x₁,x₂,…,x_n)^T||x₁|≤ 1}.

8.

Among the following, determine the subspaces of the complex vector space ℂⁿ?

(a): {(z₁,z₂,…,z_n)^T|z₁ is real }.
(b): {(z₁,z₂,…,z_n)^T|z₁ + z₂ = z₃}.
(c): {(z₁,z₂,…,z_n)^T|∣z₁∣ = ∣z₂∣}.

DRAFT

9.

Prove that the following sets are not subspaces of M_n(ℝ).

(a): G = {A ∈ M_n(ℝ)|det(A) = 0}.
(b): G = {A ∈ M_n(ℝ)|det(A) = 1}.

3.1.2 Linear Span

Definition 3.1.11. [Linear Combination] Let V be a vector space over F. Then, for any u₁,…,u_n ∈ V and α₁,…,α_n ∈ F, the vector α₁u₁ + ⋅⋅⋅ + α_nu_n = ∑ _i=1ⁿα_iu_i is said to be a linear combination of the vectors u₁,…,u_n.

Example 3.1.12.

1.

(3,4,3) is a linear combination of (1,1,1) and (1,2,1) as (3,4,3) = 2(1,1,1) + (1,2,1).

2.

(3,4,5) is not a linear combination of (1,1,1) and (1,2,1) as the linear system (3,4,5) = a(1,1,1) + b(1,2,1), in the variables a and b has no solution.

3.

Is (4,5,5) a linear combination of e₁^T = (1,0,0), e₂^T = (0,1,0) and e₃^T = (3,3,1)?
Solution: (4,5,5) is a linear combination as (4,5,5) = 4e₁^T + 5e₂^T + 5e₃^T.

4.

Is (4,5,5) a linear combination of (1,0,0), (2,1,0) and (3,3,1)?
Solution: (4,5,5) is a linear combination if the linear system

DRAFT

a(1,0,0) + b(2,1,0 )+ c(3,3,1) = (4,5,5)

(3.1.1)

in the variables a,b,c ∈ ℝ has a solution. Clearly, Equation (3.1.1) has solution a = 9,b = -10 and c = 5.

5.

Is 4 + 5x + 5x² + x³ a linear combination of the polynomials p₁(x) = 1, p₂(x) = 2 + x² and p₃(x) = 3 + 3x + x² + x³?
Solution: The polynomial 4 + 5x + 5x² + x³ is a linear combination if the linear system

ap1(x )+ bp2(x)+ cp3(x) = 4+ 5x + 5x2 + x3

(3.1.2)

in the variables a,b,c ∈ ℝ has a solution. Verify that the system has no solution. Thus, 4 + 5x + 5x² + x³ is not a linear combination of the given set of polynomials.

6.

a linear combination of the vectors I₃, ⌊ 0 1 1⌋
| |
⌈ 1 1 2⌉
1 2 0

and

?
Solution: Verify that ⌊ ⌋
| 1 3 4|
⌈ 3 3 6⌉
4 6 5

= I₃ + 2

. Hence, it is indeed a linear combination of given vectors of M₃(ℝ).

DRAFT

Exercise 3.1.13.

1.

Let x ∈ ℝ³. Prove that x^T is a linear combination of (1,0,0), (2,1,0) and (3,3,1). Is this linear combination unique? That is, does there exist (a,b,c)≠(e,f,g) with x^T = a(1,0,0) + b(2,1,0) + c(3,3,1) = e(1,0,0) + f(2,1,0) + g(3,3,1)?

2.

Find condition(s) on x,y,z ∈ ℝ such that

(a): (x,y,z) is a linear combination of (1,2,3),(-1,1,4) and (3,3,2).
(b): (x,y,z) is a linear combination of (1,2,1),(1,0,-1) and (1,1,0).
(c): (x,y,z) is a linear combination of (1,1,1),(1,1,0) and (1,-1,0).

Definition 3.1.14. [Linear Span] Let V be a vector space over F and S ⊆ V. Then, the linear span of S, denoted LS(S), is defined as

LS (S) = {α u + ⋅⋅⋅+ α u |α ∈ F,u ∈ S, for 1 ≤ i ≤ n}. 1 1 n n i i

That is, LS(S) is the set of all possible linear combinations of finitely many vectors of S. If S is an empty set, we define LS(S) = {0}. PICT

DRAFT

Example 3.1.15. For the set S given below, determine LS(S).

1.

S = {(1,0)^T,(0,1)^T}⊆ ℝ².
Solution: LS(S) = {a(1,0)^T + b(0,1)^T|a,b ∈ ℝ} = {(a,b)^T|a,b ∈ ℝ} = ℝ².

2.

S = {(1,1,1)^T,(2,1,3)^T}. What does LS(S) represent in ℝ³?
Solution: LS(S) = {a(1,1,1)^T + b(2,1,3)^T|a,b ∈ ℝ} = {(a + 2b,a + b,a + 3b)^T|a,b ∈ ℝ}. Note that LS(S) represents a plane passing through the points (0,0,0)^T,(1,1,1)^T and (2,1,3)^T. To get he equation of the plane, we proceed as follows:
Find conditions on x,y and z such that (a + 2b,a + b,a + 3b) = (x,y,z). Or equivalently, find conditions on x,y and z such that a + 2b = x,a + b = y and a + 3b = z has a solution for all a,b ∈ ℝ. The RREF of the augmented matrix equals ⌊1 0 2y - x ⌋
| |
⌈0 1 x - y ⌉
0 0 z + y - 2x

⌊1 0 2y - x ⌋
| |
⌈0 1 x - y ⌉
0 0 z + y - 2x

. Thus, the required condition on x,y and z is given by z + y - 2x = 0. Hence,

LS (S ) = {a(1,1,1)T + b(2,1,3)T |a,b ∈ ℝ} = {(x,y,z)T ∈ ℝ3|2x- y - z = 0}.

3.

S = {(1,2,1)^T,(1,0,-1)^T,(1,1,0)^T}. What does LS(S) represent?
Solution: As above, LS(S) is a plane passing through the given points and (0,0,0)^T. To get the equation of the plane, we need to find condition(s) on x,y,z such that the linear system

DRAFT

(3.1.3)

in the variables a,b,c is always consistent. An application of GJE to Equation (3.1.3) gives ⌊ ⌋
1 0 1 x+y-
| 1 2x3--y |
⌈0 1 2 3 ⌉
0 0 0 x - y + z . Thus, LS(S) = {(x,y,z)^T ∈ ℝ³|x - y + z = 0}.

4.

S = {1 + 2x + 3x²,1 + x + 2x²,1 + 2x + x³}.
Solution: To understand LS(S), we need to find condition(s) on α,β,γ,δ such that the linear system

a(1 + 2x+ 3x2 )+ b(1+ x + 2x2)+ c(1 + 2x + x3) = α + βx + γx2 + δx3

in the variables a,b,c is always consistent. An application of GJE method gives α + β - γ - 3δ = 0 as the required condition. Thus,

LS (S) = {α + βx + γx2 + δx3 ∈ ℝ[x]|α + β - γ - 3δ = 0}.

5.

S =

( ⌊ ⌋ ⌊ ⌋ )
|{ 0 1 1 0 1 2 |}
I,| |,| |
|( 3 ⌈ 1 1 2⌉ ⌈1 0 2⌉ |)
1 2 0 2 2 4

⊆ M₃(ℝ).
Solution: To get the equation, we need to find conditions of a_ij’s such that the system PICT

DRAFT

⌊ ⌋ ⌊ ⌋ α β + γ β + 2γ a11 a12 a13 |⌈β + γ α + β 2β + 2γ|⌉ = |⌈ a21 a22 a23|⌉, β + 2γ 2β + 2γ α + 2γ a31 a32 a33

in the variables α,β,γ is always consistent. Now, verify that the required condition equals

T a22-+-a33 --a13 LS (S) = {A = [aij] ∈ M3(ℝ )|A = A ,a11 = 2 , a22 - a33 + 3a13 a22 - a33 + 3a13} a12 = -------4-------,a23 = -------2------- .

Exercise 3.1.16. For each S, determine the equation of the geometrical object that LS(S) represents?

1.: S = {-1}⊆ ℝ.
2.: S = {π}⊆ ℝ.
3.: S = {(x,y)^T : x,y < 0}⊆ ℝ².
4.: S = {(x,y)^T : either x≠0 or y≠0}⊆ ℝ². DRAFT
5.: S = {(1,0,1)^T,(0,1,0)^T,(2,0,2)^T} ⊆ ℝ³. Give two examples of vectors u,v different from the given set such that LS(S) = LS(u,v).
6.: S = {(x,y,z)^T : x,y,z > 0}⊆ ℝ³.
7.: S = ⊆ M₃(ℝ).
8.: S = {(1,2,3,4)^T,(-1,1,4,5)^T,(3,3,2,3)^T}⊆ ℝ⁴.
9.: S = {1+2x+x²,x,1+x²}⊆ ℂ[x;2]. Give two examples of polynomial p(x),q(x) different from the given set such that LS(S) = LS(p(x),q(x)).
10.: S = {1 + 2x + 3x²,-1 + x + 4x²,3 + 3x + 2x²}⊆ ℂ[x;2].
11.: S = {1,x,x²,…}⊆ ℂ[x].

Definition 3.1.17. [Finite Dimensional Vector Space] Let V be a vector space over F. Then, V is called finite dimensional if there exists S ⊆ V, such that S has finite number of elements and V = LS(S). If such an S does not exist then V is called infinite dimensional.

Example 3.1.18.

1.: {(1,2)^T,(2,1)^T} spans ℝ². Thus, ℝ² is finite dimensional.
2.: {1,1 + x,1 - x + x²,x³,x⁴,x⁵} spans ℂ[x;5]. Thus, ℂ[x;5] is finite dimensional. DRAFT
3.: Fix n ∈ ℕ. Then, ℂ[x;n] is finite dimensional as ℂ[x;n] = LS({1,x,x²,…,xⁿ}).
4.: ℂ[x] is not finite dimensional as the degree of a polynomial can be any large positive integer. Indeed, verify that ℂ[x] = LS({1,x,x²,…,xⁿ,…}).
5.: The vector space ℝ over ℚ is infinite dimensional. An argument to justify it will be given later. The same argument also implies that the vector space ℂ over ℚ is infinite dimensional.

Lemma 3.1.19 (Linear Span is a Subspace). Let V be a vector space over F and S ⊆ V. Then, LS(S) is a subspace of V.

Proof. By definition, 0 ∈ LS(S). So, LS(S) is non-empty. Let u,v ∈ LS(S). To show, au + bv ∈ LS(S) for all a,b ∈ F. As u,v ∈ LS(S), there exist n ∈ ℕ, vectors w_i ∈ S and scalars α_i,β_i ∈ F such that u = α₁w₁ + ⋅⋅⋅

+ α_nw_n and v = β₁w₁ + ⋅⋅⋅

+ β_nw_n. Hence,

Exercise 3.1.20. Let V be a vector space over F and W ⊆ V.

1.: Then, LS(W) = W if and only if W is a subspace of V.
2.: If W is a subspace of V and S ⊆ W then LS(S) is a subspace of W as well.

DRAFT

Theorem 3.1.21. Let V be a vector space over F and S ⊆ V. Then, LS(S) is the smallest subspace of V containing S.

Proof. For every u ∈ S, u = 1 ⋅u ∈ LS(S). Thus, S ⊆ LS(S). Need to show that LS(S) is the smallest subspace of V containing S. So, let W be any subspace of V containing S. Then, by Exercise 3.1.20, LS(S) ⊆ W and hence the result follows. _

Definition 3.1.22. [Sum of two subsets] Let V be a vector space over F.

1.

Let S and T be two subsets of V. Then, the sum of S and T, denoted S + T equals {s + t|s ∈ S,t ∈ T}. For example,

(a): if V = ℝ, S = {0,1,2,3,4,5,6} and T = {5,10,15} then S + T = {5,6,…,21}.
(b): if V = ℝ², S = and T = then S + T = .
(c): if V = ℝ², S = and T = LS then S +T = .

2.

Let P and Q be two subspaces of ℝ². Then, P + Q = ℝ², if

(a): P = {(x,0)^T|x ∈ ℝ} and Q = {(0,x)^T|x ∈ ℝ} as (x,y) = (x,0) + (0,y).
(b): P = {(x,0)^T|x ∈ ℝ} and Q = {(x,x)^T|x ∈ ℝ} as (x,y) = (x - y,0) + (y,y).
(c): P = LS((1,2)^T) and Q = LS((2,1)^T) as (x,y) = (1,2) + (2,1).

PICT PICT DRAFT Lemma 3.1.23. Let P and Q be two subspaces of a vector space V over F. Then, P + Q is a subspace of V. Furthermore, P + Q is the smallest subspace of V containing both P and Q.

Exercise 3.1.24.

1.

Let a ∈ ℝ²,a≠0. Then, show that {x ∈ ℝ²|a^Tx = 0} is a non-trivial subspace of ℝ². Geometrically, what does this set represent in ℝ²?

2.

Find all subspaces of ℝ³.

3.

Let U =

and W =

be subspaces of M₂(ℝ). Determine U ∩ W. Is M₂(ℝ) = U ∪ W? What is U + W?

4.

Let W and U be two subspaces of a vector space V over F.

(a): Prove that W ∩ U is a subspace of V.
(b): Give examples of W and U such that W ∪ U is not a subspace of V.
(c): Determine conditions on W and U such that W ∪ U a subspace of V?
(d): Prove that LS(W ∪ U) = W + U.

______________________________________________

5.

Prove that {(x,y,z)^T ∈ ℝ³|ax + by + cz = d} is a subspace of ℝ³ if and only if d = 0.

6.

Determine all subspaces of the vector space in Example 3.1.4.19.

7.

Let S = {x₁,x₂,x₃,x₄}, where x₁ = (1,0,0)^T,x₂ = (1,1,0)^T,x₃ = (1,2,0)^T and x₄ = (1,1,1)^T. Then, determine all x_i such that LS(S) = LS(S \{x_i}). PICT

DRAFT

8.

Let W = LS((1,0,0)^T,(1,1,0)^T) and U = LS((1,1,1)^T). Prove that W + U = ℝ³ and W ∩ U = {0}. If v ∈ ℝ³, determine w ∈ W and u ∈ U such that v = w + u. Is it necessary that w and u are unique?

9.

Let W = LS((1,-1,0),(1,1,0)) and U = LS((1,1,1),(1,2,1)). Prove that W + U = ℝ³ and W ∩ U≠{0}. Find v ∈ ℝ³ such that v = w + u, for 2 different choices of w ∈ W and u ∈ U. That is, the choice of vectors w and u is not unique.

Let V be a vector space over either ℝ or ℂ. Then, we have learnt the following:

We try to answer these questions in the subsequent sections. Before doing so, we give a short section on fundamental subspaces associated with a matrix.

3.2 Fundamental Subspaces Associated with a Matrix

Definition 3.2.1. [Fundamental Subspaces] Let A ∈ M_m,n(ℂ). Then, we define the four fundamental subspaces associated with A as PICT PICT DRAFT

1.: Col(A) = {Ax|x ∈ ℂⁿ}⊆ ℂ^m, called the Column space. Observe that Col(A) is the linear span of the columns of A.
2.: Row(A) = {x^TA|x ∈ ℂ^m}, called the row space of A. Observe that Row(A) is the linear span of the rows of A.
3.: Null(A) = {x ∈ ℂⁿ|Ax = 0}, called the Null space of A.
4.: Null(A^*) = {x ∈ ℂ^m|A^*x = 0}.

Remark 3.2.2. Let A ∈ M_m,n(ℂ).

1.: Then, Col(A) is a subspace of ℂ^m and Col(A^*) is a subspace of ℂⁿ.
2.: Then, Null(A) is a subspace of ℂⁿ and Null(A^*) is a subspace of ℂ^m.

Example 3.2.3.

1.

Let A =

. Then, verify that

(a): Col(A) = {x = (x₁,x₂,x₃)^T ∈ ℝ³|x₁ + x₂ - x₃ = 0}.
(b): Row(A) = {x = (x₁,x₂,x₃,x₄)^T ∈ ℝ⁴|x₁ - x₂ - 2x₃ = 0,x₁ - 3x₂ - 2x₄ = 0}.
(c): Null(A) = LS({(1,-1,-2,0)^T,(1,-3,0,-2)^T}). DRAFT
(d): Null(A^T) = LS((1,1,-1)^T).

2.

Let A =

. Then,

(a): Col(A) = {(x₁,x₂,x₃) ∈ ℂ³|(2 + i)x₁ - (1 - i)x₂ - x₃ = 0}.
(b): Col(A^*) = {(x₁,x₂,x₃) ∈ ℂ³|ix₁ - x₂ + x₃ = 0}.
(c): Null(A) = LS((i,1,-1)^T).
(d): Null(A^*) = LS((-2 + i,1 + i,1)^T).

Remark 3.2.4. Let A ∈ M_m,n(ℝ). Then, in Example 3.2.3, observe that the direction ratios of normal vectors of Col(A) matches with vector in Null(A^T). Similarly, the direction ratios of normal vectors of Row(A) matches with vectors in Null(A). Are these true in the general setting? Do similar relations hold if A ∈ M_m,n(ℂ)? We will come back to these spaces again and again.

Exercise 3.2.5.

1.: Let A = [BC]. Then, determine the condition under which Col(A) = Col(C).
2.: Let A = . Then, determine Col(A), Row(A), Null(A) and Null(A^T).

3.3 Linear Independence

Definition 3.3.1. [Linear Independence and Dependence] Let S = {u₁,…,u_m} be a non-empty subset of a vector space V over F. Then, S is said to be linearly independent if the linear system

α u + α u + ⋅⋅⋅+ α u = 0, 1 1 2 2 m m

(3.3.1)

in the variables α_i’s, 1 ≤ i ≤ m, has only the trivial solution. If Equation (3.3.1) has a non-trivial solution then S is said to be linearly dependent.

If S has infinitely many vectors then S is said to be linearly independent if for every finite subset T of S, T is linearly independent.

Observe that we are solving a linear system over F. Hence, linear independence and dependence depend on F, the set of scalars.

Example 3.3.2.

1.

Is the set S a linear independent set? Give reasons.

(a)

Let S = {1 + 2x + x²,2 + x + 4x²,3 + 3x + 5x²}⊆ ℝ[x;2].
Solution: Consider the system [ ]
1+ 2x + x2 2 + x+ 4x2 3 + 3x + 5x2

= 0,

DRAFT or equivalently a(1 + 2x + x²) + b(2 + x + 4x²) + c(3 + 3x + 5x²) = 0, in the variables a,b and c. As two polynomials are equal if and only if their coefficients are equal, the above system reduces to the homogeneous system a + 2b + 3c = 0,2a + b + 3c = 0,a+4b+5c = 0. The corresponding coefficient matrix has rank 2 < 3, the number of variables. Hence, the system has a non-trivial solution. Thus, S is a linearly dependent subset of ℝ[x;2].

(b)

S = {1,sin(x),cos(x)} is a linearly independent subset of

([-π,π], ℝ) over ℝ as the system

⌊a⌋ [ ]| | 1 sin(x) cos(x) ⌈b⌉ = 0 ⇔ a⋅1 + b⋅sin(x)+ c ⋅cos(x) = 0, c

(3.3.2)

in the variables a,b and c has only the trivial solution. To verify this, evaluate Equation (3.3.2) at - π-
2 ,0 and π-
2 to get the homogeneous system a-b = 0,a + c = 0,a + b = 0. Clearly, this system has only the trivial solution.

(c)

Let S = {(0,1,1)^T,(1,1,0)^T,(1,0,1)^T}.
Solution: Consider the system [ ]
(0,1,1) (1,1,0) (1,0,1)

= (0,0,0) in the variables a,b and c. As rank of coefficient matrix is 3 = the number of variables, the system has only the trivial solution. Hence, S is a linearly independent subset of ℝ³.

(d)

Consider ℂ as a complex vector space and let S = {1,i}.
Solution: Since ℂ is a complex vector space, i ⋅ 1 + (-1)i = i - i = 0. So, S is a linear dependent subset of the complex vector space ℂ.

(e)

Consider ℂ as a real vector space and let S = {1,i}.
Solution: Consider the linear system a ⋅ 1 + b ⋅ i = 0, in the variables a,b ∈ ℝ. Since a,b ∈ ℝ, equating real and imaginary parts, we get a = b = 0. So, S is a linear independent subset of the real vector space ℂ.

DRAFT

2.

Let A ∈ M_m,n(ℂ). If Rank(A) < m then, the rows of A are linearly dependent.
Solution: As Rank(A) < m, there exists an invertible matrix P such that PA = [ ]
C
0

. Thus, 0^T = (PA)[m,:] = ∑ _i=1^mp_miA[i,:]. As P is invertible, at least one p_mi≠0. Thus, the required result follows.

3.

Let A ∈ M_m,n(ℂ). If Rank(A) < n then, the columns of A are linearly dependent.
Solution: As Rank(A) < n, by Corollary 2.2.33, there exists an invertible matrix Q such that AQ = [ ]
B 0

. Thus, 0 = (AQ)[:,n] = ∑ _i=1ⁿq_inA[:,i]. As Q is invertible, at least one q_in≠0. Thus, the required result follows.

3.3.1 Basic Results on Linear Independence

Proposition 3.3.3. Let V be a vector space over F.

1.: Then, 0, the zero-vector, cannot belong to a linearly independent set.
2.: Then, every subset of a linearly independent set in V is also linearly independent.
3.: Then, a set containing a linearly dependent set of V is also linearly dependent.

Proof. Let 0 ∈ S. Then, 1 ⋅ 0 = 0. That is, a non-trivial linear combination of some vectors in S is 0. Thus, the set S is linearly dependent. _

PICT PICT DRAFT Proposition 3.3.4. Let S be a linearly independent subset of a vector space V over F. If T₁,T₂ are two subsets of S such that T₁ ∩ T₂ = ∅ then, LS(T₁) ∩ LS(T₂) = {0}. That is, if v ∈ LS(T₁) ∩ LS(T₂) then v = 0.

Proof. Let v ∈ LS(T₁) ∩LS(T₂). Then, there exist vectors u₁,…,u_k ∈ T₁, w₁,…,w_ℓ ∈ T₂ and scalars α_i’s and β_j’s (not all zero) such that v = ∑ _i=1^kα_iu_i and v = ∑ _j=1^ℓβ_jw_j . Thus, we see that ∑ _i=1^kα_iu_i + ∑ _j=1^ℓ(-β_j)w_j = 0. As the scalars α_i’s and β_j’s are not all zero, we see that a non-trivial linear combination of some vectors in T₁ ∪ T₂ ⊆ S is 0. This contradicts the assumption that S is a linearly independent subset of V. Hence, each of α’s and β_j’s is zero. That is v = 0. _

Theorem 3.3.5. Let S = {u₁,…,u_k} be a non-empty subset of a vector space V over F. If T ⊆ LS(S) having more than k vectors then, T is a linearly dependent subset in V.

Proof. Let T = {w₁,…,w_m}. As w_i ∈ LS(S), there exist a_ij ∈ F such that

Corollary 3.3.6. Fix n ∈ ℕ. Then, any subset S of ℝⁿ with |S|≥ n+1 is linearly dependent.

Proof. Observe that ℝⁿ = LS({e₁,…,e_n}), where e_i = I_n[:,i], is the i-th column of I_n. Hence, using Theorem 3.3.5, the required result follows. _

Theorem 3.3.7. Let S be a linearly independent subset of a vector space V over F. Then, for any v ∈ V the set S ∪{v} is linearly dependent if and only if v ∈ LS(S).

Proof. Let us assume that S ∪{v} is linearly dependent. Then, there exist v_i’s in S such that the linear system

in the variables α_i’s has a non-trivial solution, say α_i = c_i, for 1 ≤ i ≤ p + 1. We claim that c_p+1≠0.

For, if c_p+1 = 0 then, Equation (3.3.3) has a non-trivial solution corresponds to having a non-trivial solution of the linear system α₁v₁ + ⋅⋅⋅

+ α_pv_p = 0 in the variables α₁,…,α_p. This contradicts Proposition 3.3.3.2 as {v₁,…,v_p}⊆ S, a linearly independent set. Thus, c_p+1≠0 and we get

Now, assume that v ∈ LS(S). Then, there exists v_i ∈ S and c_i ∈ F, not all zero, such that v = ∑ _i=1^pc_iv_i. Thus, the linear system α₁v₁ + ⋅⋅⋅

+ α_pv_p + α_p+1v = 0 in the variables α_i’s has a non-trivial solution [c₁,…,c_p,-1]. Hence, S ∪{v} is linearly dependent. _

We now state a very important corollary of Theorem 3.3.7 without proof. This result can also be used as an alternative definition of linear independence and dependence.

Corollary 3.3.8. Let V be a vector space over F and let S be a subset of V containing a non-zero vector u₁.

1.: If S is linearly dependent then, there exists k such that LS(u₁,…,u_k) = LS(u₁,…,u_k-1).
2.: If S linearly independent then, v ∈ V \ LS(S) if and only if S ∪{v} is also a linearly independent subset of V.
3.: If S is linearly independent then, LS(S) = V if and only if each proper superset of S is linearly dependent.

DRAFT

3.3.2 Application to Matrices

Theorem 3.3.9. Let A ∈ M_m,n(ℂ). Then, the rows of A corresponding to the pivotal rows of RREF(A) are linearly independent. Also, the columns of A corresponding to the pivotal columns of RREF(A) are linearly independent.

Proof. Let RREF(A) = B. Then, the pivotal rows of B are linearly independent due to the pivotal 1’s. Now, let B₁ be the submatrix of B consisting of the pivotal rows of B. Also, let A₁ be the submatrix of A whose rows corresponds to the rows of B₁. As the RREF of a matrix is unique (see Corollary 2.2.18) there exists an invertible matrix Q such that QA₁ = B₁. So, if there exists c≠0 such that c^TA₁ = 0^T then

Let B[:,i₁],…,B[:,i_r] be the pivotal columns of B. Then, they are linearly independent due to pivotal 1’s. As B = RREF(A), there exists an invertible matrix P such that B = PA. Then, the corresponding columns of A satisfy

The next result follows directly from Theorem 3.3.9 and hence the proof is left to readers.

Corollary 3.3.10. The following statements are equivalent for A ∈ M_n(ℂ).

1.: A is invertible.
2.: The columns of A are linearly independent.
3.: The rows of A are linearly independent.

Example 3.3.11. Let A = ⌊ ⌋
1 1 1 0
|| ||
|1 0 - 1 1|
|⌈2 1 0 1|⌉
1 1 1 2 with RREF(A) = B = ⌊ ⌋
1 0 - 1 0
|| ||
| 0 1 2 0|
|⌈ 0 0 0 1|⌉
0 0 0 0 .

1.: Then, B[:,3] = -B[:,1] + 2B[:,2]. Thus, A[:,3] = -A[:,1] + 2A[:,2].
2.: As the 1-st, 2-nd and 4-th columns of B are linearly independent, the set {A[:,1],A[: ,2],A[:,4]} is linearly independent.
3.: Also, note that during the application of GJE, the 3-rd and 4-th rows were interchanged. Hence, the rows A[1,:],A[2,:] and A[4,:] are linearly independent.

3.3.3 Linear Independence and Uniqueness of Linear Combination

We end this section with a result that states that linear combination with respect to linearly independent set is unique.

Lemma 3.3.12. Let S be a linearly independent subset of a vector space V over F. Then, each v ∈ LS(S) is a unique linear combination of vectors from S.

Proof. Suppose there exists v ∈ LS(S) with v ∈ LS(T₁),LS(T₂) with T₁,T₂ ⊆ S. Let T₁ = {v₁,…,v_k} and T₂ = {w₁,…,w_ℓ}, for some v_i’s and w_j’s in S. Define T = T₁ ∪T₂. Then, T is a subset of S. Hence, using Proposition 3.3.3, the set T is linearly independent. Let T = {u₁,…,u_p}. Then, there exist α_i’s and β_j’s in F, not all zero, such that v = α₁u₁ + ⋅⋅⋅

+ α_pu_p as well as v = β₁u₁ + ⋅⋅⋅

+ β_pu_p. Equating the two expressions for v gives

As T is a linearly independent subset of V, the system c₁v₁ + ⋅⋅⋅

+ c_pv_p = 0, in the variables c₁,…,c_p, has only the trivial solution. Thus, in Equation (3.3.4), α_i -β_i = 0, for 1 ≤ i ≤ p. Thus, for 1 ≤ i ≤ p, α_i = β_i and the required result follows. _

Exercise 3.3.13.

1.

Prove that S = {1,i,x,x+x²} is a linearly independent subset of the vector space ℂ[x;2] over ℝ. Whereas, it is linearly dependent subset of the vector space ℂ[x;2] over ℂ. PICT

DRAFT

2.

Suppose V is a vector space over ℝ as well as over ℂ. Then, prove that {u₁,…,u_k} is a linearly independent subset of V over ℂ if and only if {u₁,…,u_k,iu₁,…,iu_k} is a linearly independent subset of V over ℝ.

3.

Is the set {1,x,x²,…} a linearly independent subset of the vector space ℂ[x] over ℂ?

4.

Is the set {

_ij|1 ≤ i ≤ m,1 ≤ j ≤ n} a linearly independent subset of the vector space M_m,n(ℂ) over ℂ (see Definition 1.3.1.1)?

5.

Let W be a subspace of a vector space V over F. For u,v ∈ V \ W, define K = LS(W,u) and M = LS(W,v). Then, prove that v ∈ K if and only if u ∈ M.

6.

Prove that

(a): the rows/columns of A ∈ M_n(ℂ) are linearly independent if and only if det(A)≠0.
(b): the rows/columns of A ∈ M_n(ℂ) span ℂⁿ if and only if A is an invertible matrix.
(c): the rows/columns of a skew-symmetric matrix A of odd order are linearly dependent.

7.

Let V and W be subspaces of ℝⁿ such that V + W = ℝⁿ and V ∩ W = {0}. Prove that each u ∈ ℝⁿ is uniquely expressible as u = v + w, where v ∈ V and w ∈ W.

8.

Let S = {u₁,…,u_n} and T = {w₁,…,w_n} be subsets of a complex vector space V. Also, let ⌊ ⌋
|w1 |
| .. |
⌈ . ⌉
wn

= A

for some matrix A ∈ M_n(ℂ).

(a): If A is invertible then prove that S is a linearly independent if and only if T is linearly independent. Hint: Suppose T is linearly independent and consider the linear system ∑ _i=1ⁿα_iu_i = 0 in the variables α_i’s. Then, $DRAFT " class="math-display" >$ Since T is linearly independent, one has 0^T = [α₁,…,α_n]A^-1. But A is invertible and hence α_i = 0, for all i.
(b): If T is linearly independent then prove that A is invertible. Further, in this case, the set S is necessarily linearly independent. Hint: Suppose A is not invertible. Then, there exists x₀≠0 such that x₀^TA = 0^T. Thus, we have obtained x₀≠0 such that $⌊ w1⌋ ⌊u1⌋ ⌊ u1⌋ T| .| T | .| T| .| T x0|⌈ ..|⌉ = x0A |⌈ ..|⌉ = 0 |⌈ ..|⌉ = 0 , wn un un$ a contradiction to T being a linearly independent set.

9.

Let S = {u₁,…,u_n}⊆ ℂⁿ and T = {Au₁,…,Au_n}, for some matrix A ∈ M_n(C).

(a): If S is linearly dependent then prove that T is linear dependent.
(b): If S is linearly independent then prove that T is linearly independent for every invertible matrix A.
(c): If T is linearly independent then S is linearly independent. Further, in this case, the matrix A is necessarily invertible.

______________________________________________

10.

Consider the Euclidean plane ℝ². Let u₁ = (1,0)^T. Determine condition on u₂ such that {u₁,u₂} is a linearly independent subset of ℝ².

11.

Let S = {(1,1,1,1)^T,(1,-1,1,2)^T,(1,1,-1,1)^T}⊆ ℝ⁴. Does (1,1,2,1)^T ∈ LS(S)? Furthermore, determine conditions on x,y,z and u such that (x,y,z,u)^T ∈ LS(S).

12.

Show that S = {(1,2,3)^T,(-2,1,1)^T,(8,6,10)^T}⊆ ℝ³ is linearly dependent.

13.

Find u,v,w ∈ ℝ⁴ such that {u,v,w} is linearly dependent whereas {u,v},{u,w} and {v,w} are linearly independent.

14.

Let A ∈ M_n(ℝ). Suppose x,y ∈ ℝⁿ \{0} such that Ax = 3x and Ay = 2y. Then, prove that x and y are linearly independent. PICT

DRAFT

15.

Let A =

. Determine x,y,z ∈ ℝ³ \{0} such that Ax = 6x, Ay = 2y and Az = -2z. Use the vectors x,y and z obtained above to prove the following.

(a): A²v = 4v, where v = cy + dz for any c,d ∈ ℝ.
(b): The set {x,y,z} is linearly independent.
(c): Let P = [x,y,z] be a 3 × 3 matrix. Then, P is invertible.
(d): Let D = . Then, AP = PD.

3.4 Basis of a Vector Space

Definition 3.4.1. [Maximality] Let S be a subset of a set T. Then, S is said to be a maximal subset of T having property P if

1.: S has property P and
2.: no proper superset of S in T has property P.

Example 3.4.2. Let T = {2,3,4,7,8,10,12,13,14,15}. Then, a maximal subset of T of consecutive integers is S = {2,3,4}. Other maximal subsets are {7,8},{10} and {12,13,14,15}. Note that {12,13} is not maximal. Why? PICT PICT DRAFT

Definition 3.4.3. [Maximal linearly independent set] Let V be a vector space over F. Then, S is called a maximal linearly independent subset of V if

1.: S is linearly independent and
2.: no proper superset of S in V is linearly independent.

Example 3.4.4.

1.: In ℝ³, the set S = {e₁,e₂} is linearly independent but not maximal as S ∪{(1,1,1)^T} is a linearly independent set containing S.
2.: In ℝ³, S = {(1,0,0)^T,(1,1,0)^T,(1,1,-1)^T} is a maximal linearly independent set as S is linearly independent and any collection of 4 or more vectors from ℝ³ is linearly dependent (see Corollary 3.3.6).
3.: Let S = {v₁,…,v_k}⊆ ℝⁿ. Now, form the matrix A = [v₁,…,v_k] and let B = RREF(A). Then, using Theorem 3.3.9, we see that if B[:,i₁],…,B[:,i_r] are the pivotal columns of B then {v_i₁,…,v_{i_r}} is a maximal linearly independent subset of S.
4.: Is the set {1,x,x²,…} a maximal linearly independent subset of ℂ[x] over ℂ?
5.: Is the set {_ij|1 ≤ i ≤ m,1 ≤ j ≤ n} a maximal linearly independent subset of M_m,n(ℂ) over ℂ?

PICT PICT DRAFT Theorem 3.4.5. Let V be a vector space over F and S a linearly independent set in V. Then, S is maximal linearly independent if and only if LS(S) = V.

Proof. Let v ∈ V. As S is linearly independent, using Corollary 3.3.8.2, the set S ∪{v} is linearly independent if and only if v ∈ V \ LS(S). Thus, the required result follows. _

Let V = LS(S) for some set S with |S| = k. Then, using Theorem 3.3.5, we see that if T ⊆ V is linearly independent then |T|≤ k. Hence, a maximal linearly independent subset of V can have at most k vectors. Thus, we arrive at the following important result.

Theorem 3.4.6. Let V be a vector space over F and let S and T be two finite maximal linearly independent subsets of V. Then, |S| = |T|.

Proof. By Theorem 3.4.5, S and T are maximal linearly independent if and only if LS(S) = V = LS(T). Now, use the previous paragraph to get the required result. _

Let V be a finite dimensional vector space. Then, by Theorem 3.4.6, the number of vectors in any two maximal linearly independent set is the same. We use this number to define the dimension of a vector space. We do so now.

Definition 3.4.7. [Dimension of a finite dimensional vector space] Let V be a finite dimensional vector space over F. Then, the number of vectors in any maximal linearly independent set is called the dimension of V, denoted dim(V). By convention, dim({0}) = 0.

Example 3.4.8.

1.: As {1} is a maximal linearly independent subset of ℝ, dim(ℝ) = 1.
2.: As {e₁,e₂,e₃}⊆ ℝ³ is maximal linearly independent, dim(ℝ³) = 3. DRAFT
3.: As {e₁,…,e_n} is a maximal linearly independent subset in ℝⁿ, dim(ℝⁿ) = n.
4.: As {e₁,…,e_n} is a maximal linearly independent subset in ℂⁿ over ℂ, dim(ℂⁿ) = n.
5.: Using Exercise 3.3.13.2, {e₁,…,e_n,ie₁,…,ie_n} is a maximal linearly independent subset in ℂⁿ over ℝ. Thus, as a real vector space, dim(ℂⁿ) = 2n.
6.: As {_ij|1 ≤ i ≤ m,1 ≤ j ≤ n} is a maximal linearly independent subset of M_m,n(ℂ) over ℂ, dim(M_m,n(ℂ)) = mn.

Definition 3.4.9. Let V be a vector space over F. Then, a maximal linearly independent subset of V is called a basis/Hamel basis of V. The vectors in a basis are called basis vectors. By convention, a basis of {0} is the empty set.

Definition 3.4.10. [Minimal Spanning Set] Let V be a vector space over F. Then, a subset S of V is called minimal spanning if LS(S) = V and no proper subset of S spans V.

Remark 3.4.11 (Standard Basis). The readers should verify the statements given below.

1.: All the maximal linearly independent set given in Example 3.4.8 form the standard basis of the respective vector space.
2.: {1,x,x²,…} is the standard basis of ℝ[x] over ℝ. DRAFT
3.: Fix a positive integer n. Then, {1,x,x²,…,xⁿ} is the standard basis of ℝ[x;n] over ℝ.
4.: Let V = {A ∈ M_n(ℝ)|A = A^T}. Then, V is a vector space over ℝ with standard basis {_ii,_ij + _ji|1 ≤ i < j ≤ n}.
5.: Let V = {A ∈ M_n(ℝ)|A^T = -A}. Then, V is a vector space over ℝ with standard basis {_ij -_ji|1 ≤ i < j ≤ n}.

Example 3.4.12.

1.: Note that {-2} is a basis and a minimal spanning subset in ℝ.
2.: Let u₁,u₂,u₃ ∈ ℝ². Then, {u₁,u₂,u₃} can neither be a basis nor a minimal spanning subset of ℝ².
3.: {(1,1,-1)^T,(1,-1,1)^T,(-1,1,1)^T} is a basis and a minimal spanning subset of ℝ³.
4.: Let V = {(x,y,0)^T|x,y ∈ ℝ}⊆ ℝ³. Then, = {(1,0,0)^T,(1,3,0)^T} is a basis of V.
5.: Let V = {(x,y,z)^T ∈ ℝ³|x + y - z = 0}⊆ ℝ³. As each element (x,y,z)^T ∈ V satisfies x + y - z = 0. Or equivalently z = x + y, we see that $(x,y,z) = (x,y,x + y) = (x,0,x)+ (0,y,y) = x(1,0,1)+ y(0,1,1).$ Hence, {(1,0,1)^T,(0,1,1)^T} forms a basis of V. DRAFT
6.: Let S = {a₁,…,a_n}. Then, ℝ^S is a real vector space (see Example 3.1.4.8). For 1 ≤ i ≤ n, define the functions ${ ei(aj) = 1 if j = i . 0 otherwise$ Then, prove that = {e₁,…,e_n} is a linearly independent subset of ℝ^S over ℝ. Is it a basis of ℝ^S over ℝ? What can you say if S is a countable set?
7.: Let S = ℝⁿ and consider the vector space ℝ^S (see Example 3.1.4.8). For 1 ≤ i ≤ n, define the functions e_i(x) = e_i(x₁,…,x_n) = x_i. Then, verify that {e₁,…,e_n} is a linearly independent subset of ℝ^S over ℝ. Is it a basis of ℝ^S over ℝ?
8.: Let S = {v₁,…,v_k} ⊆ ℝⁿ. Define A = [v₁,…,v_k]. Then, using Example 3.4.4.3, we see that dim(LS(S)) = Rank(A). Further, using Theorem 3.3.9, the columns of A corresponding to the pivotal columns in RREF(A) form a basis of LS(S).
9.: Recall the vector space [a,b], where a < b ∈ ℝ. For each α ∈ [a,b], define $fα(x) = x - α, for all x ∈ [a,b].$ Prove that the set {f_α|α ∈ [a,b]} is linearly dependent.

3.4.1 Main Results associated with Bases

Theorem 3.4.13. Let V be a non-zero vector space over F. Then, the following statements are equivalent.

1.: is a basis (maximal linearly independent subset) of V.
2.: is linearly independent and spans V.
3.: is a minimal spanning set in V.

Proof. 1 ⇒2 By definition, every basis is a maximal linearly independent subset of V. Thus, using Corollary 3.3.8.2, we see that

spans V.

2 ⇒3 Let S be a linearly independent set that spans V. As S is linearly independent, for any x ∈ S, x

. Hence LS (S - {x })

⊊ LS(S) = V.

3 ⇒1 If

is linearly dependent then using Corollary 3.3.8.1,

is not minimal spanning. A contradiction. Hence,

is linearly independent.

We now need to show that

is a maximal linearly independent set. Since LS(

) = V, for any x ∈ V \

, using Corollary 3.3.8.2, the set

∪{x} is linearly dependent. That is, every proper superset of

is linearly dependent. Hence, the required result follows. _

Remark 3.4.14. Let be a basis of a vector space V over F. Then, for each v ∈ V, there exist unique u_i ∈ and unique α_i ∈ F, for 1 ≤ i ≤ n, such that v = ∑ _i=1ⁿα_iu_i.

The next result is generally known as “every linearly independent set can be extended to form a basis of a finite dimensional vector space”.

PICT PICT DRAFT Theorem 3.4.15. Let V be a vector space over F with dim(V) = n. If S is a linearly independent subset of V then there exists a basis T of V such that S ⊆ T.

Proof. If LS(S) = V, done. Else, choose u₁ ∈ V \LS(S). Thus, by Corollary 3.3.8.2, the set S ∪{u₁} is linearly independent. We repeat this process till we get n vectors in T as dim(V) = n. By Theorem 3.4.13, this T is indeed a required basis. _

3.4.2 Constructing a Basis of a Finite Dimensional Vector Space

We end this section with an algorithm which is based on the proof of the previous theorem.

Exercise 3.4.16.

1.

Let

= {u₁,…,u_n} be a basis of a vector space V over F. Then, does the condition ∑ _i=1ⁿα_iu_i = 0 in α_i’s imply that α_i = 0, for 1 ≤ i ≤ n?

2.

Let S = {v₁,…,v_p} be a subset of a vector space V over F. Suppose LS(S) = V but S is not a linearly independent set. Then, does this imply that each v ∈ V is expressible in more than one way as a linear combination of vectors from S? Is it possible to get a subset T of S such that T is a basis of V over F? Give reasons for your answer.

3.

Let V be a vector space of dimension n. Then, PICT

DRAFT

(a): prove that any set consisting of n linearly independent vectors forms a basis of V.
(b): prove that if S is a subset of V having n vectors with LS(S) = V then, S forms a basis of V.

4.

Let {v₁,…,v_n} be a basis of ℂⁿ. Then, prove that the two matrices B = [v₁,…,v_n] and C = ⌊ T⌋
| v1|
|⌈ ...|⌉
T
vn

are invertible.

5.

Let A ∈ M_n(ℂ) be an invertible matrix. Then, prove that the rows/columns of A form a basis of ℂⁿ over ℂ.

6.

Let W₁ and W₂ be two subspaces of a finite dimensional vector space V such that W₁ ⊆ W₂. Then, prove that W₁ = W₂ if and only if dim(W₁) = dim(W₂).

7.

Let W₁ be a non-trivial subspace of a finite dimensional vector space V over F. Then, prove that there exists a subspace W₂ of V such that

W1 ∩ W2 = {0 },W1 + W2 = V and dim (W2 ) = dim (V)- dim (W1 ).

Also, prove that for each v ∈ V there exist unique vectors w₁ ∈ W₁ and w₂ ∈ W₂ with v = w₁ + w₂. The subspace W₂ is called the complementary subspace of W₁ in V.

8.

Let V be a finite dimensional vector space over F. If W₁ and W₂ are two subspaces of V such that W₁ ∩ W₂ = {0} and dim(W₁) + dim(W₂) = dim(V) then prove that W₁ + W₂ = V.

9.

Consider the vector space

([-π,π]) over ℝ. For each n ∈ ℕ, define e_n(x) = sin(nx). Then, prove that S = {e_n|n ∈ ℕ} is linearly independent. [Hint: Need to show that every finite subset of S is linearly independent. So, on the contrary assume that there exists ℓ ∈ℕ and functions e_k₁,…,e_{k_ℓ} such that α₁e_k₁ + ⋅⋅⋅

+ α_ℓe_{k_ℓ} = 0, for some α_t≠0 with 1 ≤ t ≤ ℓ. But, the above system is PICT

DRAFT equivalent to looking at α₁ sin(k₁x) +

+ α_ℓ sin(k_ℓx) = 0 for all x ∈ [-π,π]. Now in the integral

∫ π sin(mx )(α1sin(k1x)+ ⋅⋅⋅+ αℓsin(kℓx)) dx - π

replace m with k_i’s to show that α_i = 0, for all i,1 ≤ i ≤ ℓ. This gives the required contradiction.]

10.

Is the set {1,sin(x),cos(x),sin(2x),cos(2x),sin(3x),cos(3x),…} a linearly subset of the vector space

([-π,π], ℝ) over ℝ?________________________________________________________

11.

Find a basis of ℝ³ containing the vector (1,1,-2)^T.

12.

Find a basis of ℝ³ containing the vector (1,1,-2)^T and (1,2,-1)^T.

13.

Is it possible to find a basis of ℝ⁴ containing the vectors (1,1,1,-2)^T, (1,2,-1,1)^T and (1,-2,7,-11)^T?

14.

Show that

= {(1,0,1)^T,(1,i,0)^T,(1,1,1 - i)^T} is a basis of ℂ³ over ℂ.

15.

Find a basis of ℂ³ over ℝ containing the basis

given in Example 3.4.16.14.

16.

Determine a basis and dimension of W = {(x,y,z,w)^T ∈ ℝ⁴|x + y - z + w = 0}.

17.

Find a basis of V = {(x,y,z,u) ∈ ℝ⁴|x - y - z = 0,x + z - u = 0}.

18.

Let A =

. Find a basis of V = {x ∈ ℝ⁵|Ax = 0}.

19.

Let u^T = (1,1,-2),v^T = (-1,2,3) and w^T = (1,10,1). Find a basis of LS(u,v,w). Determine a geometrical representation of LS(u,v,w).

20.

Is the set W = {p(x) ∈ ℝ[x;4]|p(-1) = p(1) = 0} a subspace of ℝ[x;4]? If yes, find its dimension.

3.5 Application to the subspaces of ℂⁿ

In this section, we will study results that are intrinsic to the understanding of linear algebra from the point of view of matrices, especially the fundamental subspaces (see Definition 3.2.1) associated with matrices. We start with an example.

Example 3.5.1.

1.

Compute the fundamental subspaces for A = ⌊ ⌋
1 1 1 - 2
|⌈1 2 - 1 1 |⌉

1 - 2 7 - 11

.
Solution: Verify the following

(a): Row(A) = {(x,y,z,u)^T ∈ ℂ⁴|3x - 2y = z,5x - 3y + u = 0}.
(b): Col(A) = {(x,y,z)^T ∈ ℂ³|4x - 3y - z = 0}.
(c): Null(A) = {(x,y,z,u)^T ∈ ℂ⁴|x + 3z - 5u = 0,y - 2z + 3u = 0}.
(d): Null(A^T) = {(x,y,z)^T ∈ ℂ³|x + 4z = 0,y - 3z = 0}.

2.

Let A =

⌊ ⌋
1 1 0 1 1 0 - 1
|⌈0 0 1 2 3 0 - 2|⌉

0 0 0 0 0 1 1

. Find a basis and dimension of Null(A).
Solution: Writing the basic vairables x₁,x₃ and x₆ in terms of the free variables x₂,x₄,x₅ and x₇, we get x₁ = x₇ -x₂ -x₄ -x₅, x₃ = 2x₇ - 2x₄ - 3x₅ and x₆ = -x₇. Hence, the solution set has the form

⌊ ⌋ ⌊ ⌋ ⌊ ⌋ ⌊ ⌋ ⌊ ⌋ ⌊ ⌋ x1 x7 - x2 - x4 - x5 - 1 - 1 - 1 1 || x2|| || x2 || || 1 || || 0 || || 0 || || 0 || || || || || || || || || || || || || || x3|| || 2x7 - 2x4 - 3x5 || || 0 || ||- 2|| ||- 3|| || 2 || | x4| = | x4 | = x2| 0 | + x4 | 1 | + x5| 0 | + x7 | 0 |. || x || || x || || 0 || || 0 || || 1 || || 0 || || 5|| || 5 || || || || || || || || || ⌈ x6⌉ ⌈ - x7 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈ 0 ⌉ ⌈- 1⌉ x7 x7 0 0 0 1

(3.5.1)

Now, let u₁^T = [ ]
- 1,1,0,0,0,0,0 , u₂^T = [ ]
- 1,0,- 2,1,0,0,0 , u₃^T = [ ]
- 1,0,- 3,0,1,0,0 and u₄^T = [ ]
1,0,2,0,0,- 1,1 . Then, S = {u₁,u₂,u₃,u₄} is a basis of Null(A). The reasons for S to be a basis are as follows:

(a)

By Equation (3.5.1) Null(A) = LS(S).

(b)

For Linear independence, the homogeneous system c₁u₁ + c₂u₂ + c₃u₃ + c₄u₄ = 0 in the variables c₁,c₂,c₃ and c₄ has only the trivial solution as

i.: u₄ is the only vector with a nonzero entry at the 7-th place (u₄ corresponds to x₇) and hence c₄ = 0.
ii.: u₃ is the only vector with a nonzero entry at the 5-th place (u₃ corresponds to x₅) and hence c₃ = 0.
iii.: Similar arguments hold for the variables c₂ and c₁.

Exercise 3.5.2. Let A = ⌊ ⌋
|1 2 1 3 2 |
|0 2 2 2 4 |
|| ||
⌈2 - 2 4 0 8 ⌉
4 2 5 6 10 and B = ⌊ ⌋
| 2 4 0 6 |
|- 1 0 - 2 5 |
|| ||
⌈- 3 - 5 1 - 4⌉
- 1 - 1 1 2 .

1.: Find RREF(A) and RREF(B). DRAFT
2.: Find invertible matrices P₁ and P₂ such that P₁A = RREF(A) and P₂B = RREF(B).
3.: Find bases for Col(A), Row(A), Col(B) and Row(B).
4.: Find bases of Null(A), Null(A^T), Null(B) and Null(B^T).
5.: Find the dimensions of all the vector subspaces so obtained.

The next result is a re-writing of the results on system of linear equations. We give the proof for the sake of completeness.

Lemma 3.5.3. Let A ∈ M_m×n(ℂ) and let E be an elementary matrix. If

1.

B = EA then

(a): Null(A) = Null(B), Row(A) = Row(B). Thus, the dimensions of the corresponding spaces are equal.
(b): Null(A) = Null(B), Row(A) = Row(B). Thus, the dimensions of the corresponding spaces are equal.

2.

B = AE then

(a): Null(A^*) = Null(B^*), Col(A) = Col(B). Thus, the dimensions of the corresponding spaces are equal.
(b): Null(A^T) = Null(B^T), Col(A) = Col(B). Thus, the dimensions of the corresponding spaces are equal.

Proof. Part 1a: Let x ∈ Null(A). Then, Bx = EAx = E0 = 0. So, Null(A) ⊆ Null(B). Further, if x ∈ Null(B), then Ax = (E^-1E)Ax = E^-1(EA)x = E^-1Bx = E^-10 = 0. Hence, Null(B) ⊆ Null(A). Thus, Null(A) = Null(B). PICT

DRAFT

Let us now prove Row(A) = Row(B). So, let x^T ∈ Row(A). Then, there exists y ∈ ℂ^m such that x^T = y^TA. Thus, x^T = (yTE -1)

EA =

B and hence x^G ∈ Row(B). That is, Row(A) ⊆ Row(B). A similar argument gives Row(B) ⊆ Row(A) and hence the required result follows.

Part 1b: E is invertible implies E is invertible and B = EA. Thus, an argument similar to the previous part gives us the required result.

For Part 2, note that B^* = E^*A^* and E^* is invertible. Hence, an argument similar to the first part gives the required result. _

Let A ∈ M_m×n(ℂ) and let B = RREF(A). Then, as an immediate application of Lemma 3.5.3, we get dim(Row(A)) = Row rank(A). We now prove that dim(Row(A)) = dim(Col(A)).

Theorem 3.5.4. Let A ∈ M_m×n(ℂ). Then, dim(Row(A)) = dim(Col(A)).

Proof. Let dim(Row(A)) = r. Then, there exist i₁,…,i_r such that {A[i₁,:],…,A[i_r,:]} forms a basis of Row(A). Then, B = ⌊ A[i,:]⌋
| 1. |
|⌈ .. |⌉

A[ir,:]

is an r × n matrix and it’s rows are a basis of Row(A). Therefore, there exist α_ij ∈ ℂ,1 ≤ i ≤ m,1 ≤ j ≤ r such that A[t,:] = [α_t1,…,α_tr]B, for 1 ≤ t ≤ m. So, using matrix multiplication

PICT PICT DRAFT Remark 3.5.5. The proof also shows that for every A ∈ M_m×n(ℂ) of rank r there exists matrices B_r×n and C_m×r, each of rank r, such that A = CB.

Let W₁ and W₁ be two subspaces of a vector space V over F. Then, recall that (see Exercise 3.1.24.4d) W₁ + W₂ = {u + v|u ∈ W₁,v ∈ W₂} = LS(W₁ ∪ W₂) is the smallest subspace of V containing both W₁ and W₂. We now state a result similar to a result in Venn diagram that states |A| + |B| = |A ∪ B| + |A ∩ B|, whenever the sets A and B are finite (for a proof, see Appendix 9.4.1).

Theorem 3.5.6. Let V be a finite dimensional vector space over F. If W₁ and W₂ are two subspaces of V then

dim (W ) + dim(W ) = dim (W + W )+ dim(W ∩W ). 1 2 1 2 1 2

(3.5.2)

For better understanding, we give an example for finite subsets of ℝⁿ. The example uses Theorem 3.3.9 to obtain bases of LS(S), for different choices S. The readers are advised to see Example 3.3.9 before proceeding further.

Example 3.5.7. Let V and W be two spaces with V = {(v,w,x,y,z)^T ∈ ℝ⁵|v + x + z = 3y} and W = {(v,w,x,y,z)^T ∈ ℝ⁵|w - x = z,v = y}. Find bases of V and W containing a basis of V ∩ W.
Solution: One can first find a basis of V ∩ W and then heuristically add a few vectors to get bases for V and W, separately. PICT PICT DRAFT

Alternatively, First find bases of V, W and V∩W, say _V,_W and . Now, consider S = ∪_V. This set is linearly dependent. So, obtain a linearly independent subset of S that contains all the elements of . Similarly, do for T = ∪_W.

So, we first find a basis of V ∩ W. Note that (v,w,x,y,z)^T ∈ V ∩ W if v,w,x,y and z satisfy v + x - 3y + z = 0,w - x - z = 0 and v = y. The solution of the system is given by

(v,w, x,y,z)T = (y,2y,x,y,2y - x)T = y(1,2,0,1,2)T + x(0,0,1,0,- 1)T.

Thus, = {(1,2,0,1,2)^T,(0,0,1,0,-1)^T} is a basis of V ∩ W. Similarly, a basis of V is given by = {(-1,0,1,0,0)^T,(0,1,0,0,0)^T,(3,0,0,1,0)^T,(-1,0,0,0,1)^T} and that of W is given by = {(1,0,0,1,0)^T,(0,1,1,0,0)^T,(0,1,0,0,1)^T}. To find the required basis form a matrix whose rows are the vectors in , and (see Equation(3.5.3)) and apply row operations other than E_ij. Then, after a few row operations, we get

⌊ ⌋ ⌊ ⌋ | 1 2 0 1 2 | |1 2 0 1 2 | |-0---0--1--0--- 1| |0--0--1--0----1| ||- 1 0 1 0 0 || ||0 1 0 0 0 || || || || || || 0 1 0 0 0 || ||0 0 0 1 3 || || 3 0 0 1 0 || → ||0 0 0 0 0 || . ||- 1 0 0 0 1 || ||0 0 0 0 0 || |----------------| |---------------| || 1 0 0 1 0 || ||0 1 0 0 1 || |⌈ 0 1 1 0 0 |⌉ |⌈0 0 0 0 0 |⌉ 0 1 0 0 1 0 0 0 0 0

(3.5.3)

Thus, a required basis of V is {(1,2,0,1,2)^T,(0,0,1,0,-1)^T,(0,1,0,0,0)^T,(0,0,0,1,3)^T}. Similarly, a required basis of W is {(1,2,0,1,2)^T,(0,0,1,0,-1)^T,(0,1,0,0,1)^T}.

DRAFT

Exercise 3.5.8.

1.: Give an example to show that if A and B are equivalent then Col(A) need not equal Col(B).
2.: Let V = {(x,y,z,w)^T ∈ ℝ⁴|x + y - z + w = 0,x + y + z + w = 0,x + 2y = 0} and W = {(x,y,z,w)^T ∈ ℝ⁴|x-y -z + w = 0,x + 2y -w = 0} be two subspaces of ℝ⁴. Think of a method to find bases and dimensions of V, W, V ∩ W and V + W.
3.: Let W₁ and W₂ be two subspaces of a vector space V. If dim(W₁) + dim(W₂) > dim(V), then prove that dim(W₁ ∩ W₂) ≥ 1.
4.: Let A ∈ M_m×n(ℂ) with m < n. Prove that the columns of A are linearly dependent.

Theorem 3.5.9 (Rank-Nullity Theorem). Let A ∈ M_m×n(ℂ). Then,

dim (Col (A))+ dim (Null (A)) = n.

(3.5.4)

PICT PICT DRAFT

Proof. Let dim(Null(A)) = r ≤ n and let

= {u₁,…,u_r} be a basis of Null(A). Since

is a linearly independent set in ℝⁿ, extend it to get

= {u₁,…,u_n} as a basis of ℝⁿ. Then,

in the variables α₁,…,α_n-r. Thus, α₁u_r+1 + ⋅⋅⋅

+ α_n-ru_n ∈ Null(A) = LS(

). Therefore, there exist scalars β_i,1 ≤ i ≤ r, such that ∑ _i=1^n-rα_iu_r+i = ∑ _j=1^rβ_ju_j. Or equivalently,

Theorem 3.5.9 is part of what is known as the fundamental theorem of linear algebra (see Theorem 5.2.16). The following are some of the consequences of the rank-nullity theorem. The proofs are left as an exercise for the reader.

Exercise 3.5.10.

1.

Let A ∈ M_m,n(ℂ).

(a): If n > m then the system Ax = 0 has infinitely many solutions,
(b): If n < m then there exists b ∈ ℝ^m \{0} such that Ax = b is inconsistent.

2.

The following statements are equivalent for an m × n matrix A.

(a): Rank (A) = k.
(b): There exist a set of k rows of A that are linearly independent.
(c): There exist a set of k columns of A that are linearly independent.
(d): dim(Col(A)) = k.
(e): There exists a k × k submatrix B of A with det(B)≠0. Further, the determinant of every (k + 1) × (k + 1) submatrix of A is zero. DRAFT
(f): There exists a linearly independent subset {b₁,…,b_k} of ℝ^m such that the system Ax = b_i, for 1 ≤ i ≤ k, is consistent.
(g): dim(Null(A)) = n - k.

3.6 Ordered Bases

Let V be a vector space over ℂ with dim(V) = n, for some positive integer n. Also, let W be a subspace of V with dim(W) = k. Then, a basis of W may not look like a standard basis. Our problem may force us to look for some other basis. In such a case, it is always helpful to fix the vectors in a particular order and then concentrate only on the coefficients of the vectors as was done for the system of linear equations where we didn’t worry about the variables. It may also happen that k is very-very small as compared to n in which case it is better to work with k vectors in place of n vectors.

Definition 3.6.1. [Ordered Basis, Basis Matrix] Let W be a vector space over F with a basis = {u₁,…,u_m}. Then, an ordered basis for W is a basis together with a one-to-one correspondence between and {1,2,…,m}. Since there is an order among the elements of , we write = (u1,...,um ) . The vector B = [u₁,…,u_m] is an element of W^m and is generally called the basis matrix.

Example 3.6.2.

1.: Consider the ordered basis = (e₁,e₂,e₃) of ℝ³. Then, B = .
2.: Consider the ordered basis = (1,x,x²,x³) of ℝ[x;3]. Then, B = . DRAFT
3.: Consider the ordered basis = (₁₂ -₂₁,₁₃ -₃₁,₂₃ -₃₂) of the set of 3 × 3 skew-symmetric matrices. Then, B = .

Thus,

= (u₁,u₂,…,u_m) is different from

= (u₂,u₃,…,u_m,u₁) and both of them are different from

= (u_m,u_m-1,…,u₂,u₁) even though they have the same set of vectors as elements. We now define the notion of coordinates of a vector with respect to an ordered basis.

Definition 3.6.3. [Coordinate Vector] Let B = [v₁,…,v_m] be the basis matrix corresponding to an ordered basis of W. Since is a basis of W, for each v ∈ W, there exist β_i,1 ≤ i ≤ m, such that v = ∑ _i=1^mβ_iv_i = B ⌊ ⌋
β1
|| . ||
⌈ .. ⌉
βm . The vector ⌊ ⌋
β1
|| .||
⌈ ..⌉
βm , denoted [v], is called the coordinate vector of v with respect to . Thus,

⌊ ⌋ v1 T| .| v = B[v]B = [v1,...,vm ][v]B, or equivalently, v = [v]B|⌈ ..|⌉. v m

(3.6.1)

The last expression is generally viewed as a symbolic expression.

Example 3.6.4. PICT PICT DRAFT

1.

Let f(x) = 1 + x + x³ ∈ ℝ[x;3]. If

= (1,x,x²,x³) is an ordered basis of ℝ[x;3] then,

⌊ ⌋ ⌊ ⌋ 1 1 [ ]|| || [ ]|| || [f(x)]B = 1 x x2 x3 |1| = 1 1 0 1 | x |. |⌈0|⌉ |⌈ x2|⌉ 1 x3

2.

Consider the Bessel polynomials

u0(x) = 1,u1(x) = x and u2 (x ) = 3-x2 - 1, for all x ∈ [- 1,1]. 2 2

Then,

= (u₀(x),u₁(x),u₂(x)) forms an ordered basis of ℝ[x;2]. Note that these polynomials satisfy the following conditions:

(a): deg(u_i(x)) = i, for 0 ≤ i ≤ 2;
(b): u_i(1) = 1, for 0 ≤ i ≤ 2; and
(c): ∫ _-1¹u_i(x)u_j(x)dx = 0, whenever 0 ≤ i≠j ≤ 2.

Verify that

DRAFT

3.

We know that

M3 (ℝ) = U + W, where U = {A ∈ M3 (ℝ)|AT = A } and W = {A ∈ M3 (ℝ )|AT = - A }.

If A =

then, A = X + Y , where X = ⌊ ⌋
1 2 3
|⌈ 2 1 2|⌉

3 2 4

and Y =

(a): If = is an ordered basis of M₃(ℝ) then $[ ] [A ]TB = 1 2 3 2 1 3 3 1 4 .$
(b): If = is an ordered basis of U then, [X]^T = .
(c): If = is an ordered basis of W then, [Y ]^T = .

Thus, in general, any matrix A ∈ M_m,n(ℝ) can be mapped to ℝ^mn with respect to the ordered basis = (E ,...,E ,E ,...,E ,...,E ,...,E )
11 1n 21 2n m1 mn and vise-versa. PICT PICT DRAFT

4.

Let V = {(v,w,x,y,z)^T ∈ ℝ⁵|w - x = z,v = y,v + x + z = 3y}. Then, verify that

can be taken as an ordered basis of V. In this case, [(3,6,0,3,1)] = [ ]
3
5

Remark 3.6.5. [Basis representation of v]

1.

Let

be an ordered basis of a vector space V over F of dimension n.

(a): Then, $[αv + w ]B = α[v]B + [w ]B, for all α ∈ F and v, w ∈ V.$
(b): Further, let S = {w₁,…,w_m}⊆ V. Then, observe that S is linearly independent if and only if {[w₁],…,[w_m]} is linearly independent in Fⁿ.

2.

Suppose V = Fⁿ in Definition 3.6.3. Then, B = [v₁,…,v_n] is an n × n invertible matrix (see Exercise 3.4.16.4). Thus, using Equation (3.6.1), we have

DRAFT " class="math-display" >

(3.6.2)

As B is invertible, [v] = B^-1v, for every v ∈ V.

Definition 3.6.6. [Change of Basis Matrix] Let V be a vector space over F with dim(V) = n. Let A = [v₁,…,v_n] and B = [u₁,…,u_n] be basis matrices corresponding to the ordered bases and , respectively, of V. Thus, using Equation (3.6.1), we have

[v1,...,vn ] = [B [v1]B, ...,B [vn]B ] = B [[v1]B,...,[vn ]B ] = B [A ]B,

where [

] =

. Or equivalently, verify the symbolic equality

⌊ ⌋ ⌊ ⌋ | v1| | u1| | ... | = [A]TB| ... |. ⌈ ⌉ ⌈ ⌉ vn un

(3.6.3)

The matrix [] is called the matrix of with respect to the ordered basis or the change of basis matrix from to .

PICT PICT DRAFT Theorem 3.6.7. Let V be a vector space over F with dim(V) = n. Further, let = (v₁,…,v_n) and = (u₁,…,u_n) be two ordered bases of V

1.: Then, the matrix [] is invertible.
2.: Similarly, the matrix [] is invertible.
3.: Moreover, [x] = [][x], for all x ∈ V. Thus, again note that the matrix [] takes coordinate vector of x with respect to to the coordinate vector of x with respect to . Hence, [] was called the change of basis matrix from to .
4.: Similarly, [x] = [][x], for all x ∈ V.
5.: Furthermore, ([])^-1 = [].

Proof. Part 1: Note that using Equation (3.6.3), we have ⌊ ⌋
v1
| . |
|⌈ .. |⌉
v
n

= [

]^T

and hence by Exercise 3.3.13.8, the matrix [

]^T or equivalently [

] is invertible, which proves Part 1. A similar argument gives Part 2.

PICT PICT DRAFT Example 3.6.8.

1.

Let V = ℂⁿ and

= (e₁,…,e_n) be the standard ordered basis. Then

[A] = [[v ] ,...,[v ]] = [v ,...,v ] = A. B 1 B nB 1 n

2.

Let

be an ordered basis of ℝ². Then, by Remark 3.6.5.2, B = [ ]
1 1
1 2

is an invertible matrix. Thus, verify that [ ]
π
e

^-1

3.

Suppose

(1,0,0)^T,(1,1,0)^T,(1,1,1)^T

and

(1,1,1)^T,(1,-1,1)^T,(1,1,0)^T

are two ordered bases of ℝ³. Then, we verify the statements in the previous result.

(a): = . Thus, = ^-1 = .
(b): = ^-1 = = .
(c): Finally check that [] = , [] = and [][] = I₃.

Remark 3.6.9. Let V be a vector space over F with = (v₁,…,v_n) as an ordered basis. Then, by Theorem 3.6.7, [v] is an element of Fⁿ, for each v ∈ V. Therefore, PICT PICT DRAFT

1.: if F = ℝ then, the elements of V correspond to vectors in ℝⁿ.
2.: if F = ℂ then, the elements of V correspond to vectors in ℂⁿ.

Exercise 3.6.10. Let = (1,2,0)^T,(1,3,2)^T,(0,1,3)^T and = (1,2,1)^T,(0,1,2)^T,(1,4,6)^T be two ordered bases of ℝ³.

1.: Find the change of basis matrix P from to .
2.: Find the change of basis matrix Q from to .
3.: Find the change of basis matrix R from the standard basis of ℝ³ to . What do you notice?

Is it true that PQ = I = QP? Give reasons for your answer.

3.7 Summary

In this chapter, we defined vector spaces over F. The set F was either ℝ or ℂ. To define a vector space, we start with a non-empty set V of vectors and F the set of scalars. We also needed to do the following:

If all conditions in Definition 3.1.1 are satisfied then V is a vector space over F. If W was a non-empty subset of a vector space V over F then for W to be a space, we only need to check whether the vector addition and scalar multiplication inherited from that in V hold in W. PICT

DRAFT

We then learnt linear combination of vectors and the linear span of vectors. It was also shown that the linear span of a subset S of a vector space V is the smallest subspace of V containing S. Also, to check whether a given vector v is a linear combination of u₁,…,u_n, we needed to solve the linear system c₁u₁ + ⋅⋅⋅

+ c_nu_n = v in the variables c₁,…,c_n. Or equivalently, the system Ax = b, where in some sense A[:,i] = u_i, 1 ≤ i ≤ n, x^T = [c₁,…,c_n] and b = v. It was also shown that the geometrical representation of the linear span of S = {u₁,…,u_n} is equivalent to finding conditions in the entries of b such that Ax = b was always consistent.

Then, we learnt linear independence and dependence. A set S = {u₁,…,u_n} is linearly independent set in the vector space V over F if the homogeneous system Ax = 0 has only the trivial solution in F. Else S is linearly dependent, where as before the columns of A correspond to the vectors u_i’s.

We then talked about the maximal linearly independent set (coming from the homogeneous system) and the minimal spanning set (coming from the non-homogeneous system) and culminating in the notion of the basis of a finite dimensional vector space V over F. The following important results were proved.