Functional Analysis

Notes while learning about functional analysis.

Metric Spaces: It is defined as the space which has a set of points, say X, and a distance function, d, where the distance function obeys three rules:

• d(x, y) > 0
• d(x, y) = d(y, x)
• d(x, y) <= d(z, x) + d(z, y) (triangle inequality)
• Examples: Rⁿ and the Eucilidean distance, Complex plane and |x-y| norm distance.

ε-ball: Also denoted by B_ε(x), are defined as {y ∈ X | d(x, y) < ε}.

Open sets: If A is a set, and A ⊂ X, it is open if every point inside B_ε(x) belongs to it (for all x ∈ A).

Boundary Points: If any B_ε(x) has points outside of A, no matter how small the ε value, it is the boundary piont of the set A. Here x ∈ X. Than B_ε(x) is said to have points from both A and it’s compliment A^c. All the boundary points of A are denoted as ∂A.

Closed Sets: A set, A, is defined to be closed, if it’s compliment is open.

Closure: The smallest closed set, is simply, A ∪ ∂A. Remember :

• A is open if, A ∩ ∂A = ∅.
• A is closed if, A ∩ ∂A = A.

In the example, where X = (1, 3] ∪ (4, ∞), the set A = (1, 3] is both open AND closed. (Since A^c is also open, by definition, A is closed).

Convergence: A sequence is defined as a set of points arranged in increasing order, such as x₁ < x₂ < x₃ … x_n, also denoted by {x_i}. A sequence is said to be convergent when, in the limit, it approaches a single value, for example, a sequence, {a}^N_i converges to a single value à. Given a metric space (X, d), we can define the ε-ball, B_ε(x̃), for x ∈ X, such that,

• ∀ε > 0, ∃N ∈ ℕ, ∀n >= N : d(x, x̃) < ε

which means that for any convergent sequence, the ε-ball around it contains all the sequence points (as the ε increases, it engulfs more points from the sequence hence for every ε > 0, every point in sequence is covered).

• x̃ = lim_n->∞ x_n

Proposition: A characterstic of a closed set is: any limit of a convergent sequence is inside the closed set itself.

Proof (by contraposition)

• Let the sequence be {a_n} and the limit be ã. Let’s assume that A is not closed.
• It implies that the compliment of A, A^c is not open either.
• If ã ∈ A^c, then there exists at least one such point in B_ε(ã) that overlaps with A, which means, B_ε(ã) ∩ A ≠ ∅.
• Which means that there is a sub-sequence, {a_n}, which lies in A, but it’s limit does not (since ã ∈ A^c).
• Hence lim_n->∞ a_n = ã ∉ A.

Cauchy Sequence: A sequence which is defined by as

• ∀ε > 0, ∃N ∈ ℕ, ∀ n, m >= N : d(x_n, x_m) < ε
• It is a generalization of a convergent sequence, in which we shift our focus from the distance between the limit and all the points minimising to just the distance between the points to be minimising.
• Which mean after a certain index, N, the distance between two points is always less than ε, for any ε greater than zero.

Complete Metric Space : A metric space is said to be complete when all of it’s Cauchy sequences converge. This simply means that for a complete metric space, Cauchy sequnces = Convergent sequences.

• Example: (0, 3), where d := |x - y| is incomplete when with an index of n, m ∈ N, the distance -> 0 as n, m -> ∞ but zero is excluded from this set, hence it’s incomplete
• [0, 3] is complete, since zero is included in this set.
• (0, 3) under the metric, d := {1 if x ≠ y and 0 if x = y} is complete. Take a Cauchy sequence {x_n}_n∈N, where by definition, d (x_n, x_m) < ε. For any ε, the only way this is possible is if x_n = x_m (which makes d = 0), hence there is only one point, the limit after the index n (since all the points after it must be the same), hence this Cauchy sequence is convergent, which makes this metric space complete.

Norm : Let X be a 𝔽-vector space. (Where 𝔽 = {ℝ, ℂ}). The norm, denoted by ||.|| is a map that transforms a vector to [0, ∞). The norm fulfils the three properties of the distance function as well.

• ||x|| = 0, only when x = 0 (positive definite)
• ||λx|| = λ||x|| (absolutely homogeneous)
• ||x+y|| <= ||x|| + ||y|| where x, y ∈ X (triangle inequality) (Here a since you cannot pull out just the addition, and always get an inquality, it is non-linear)

Hence the space defined by (X, ||.||) is called a normed space. A normed space is a special case of a metric space (by taking the end points of a vector, it can be seen as the distance between those two points, hence the distance function of a metric space, d_||.||).

Banach Space : If a given metric space with a possible norm, (X, d_||.||) is also a complete one, then the underlying norm space, (X, ||.||) is called the banach space. All the properties of a metric space apply as well. The norm function is what ties the real-complex vector space and a complete metric space, hence also defining a possible Banach space.

l^P (ℕ,𝔽): It is defined as a collection of all the sequences in 𝔽, which follow the condition: ∑_n=1∞|x|_p < ∞. Here x = {x_n}, which is a sequence in 𝔽. l^P becomes a vector space if the summation condition is ignored. We define a norm over such vector space, ||.||_p : l^P -> [0, ∞). The norm is given by the formula, (∑_n=1∞|x|_p)^1/P, again, x is a sequence.

Assumption: l^P is a 𝔽-vector space, and ||.||_p is a norm on it.
Proof: (l^P, ||.||_p) is a banach space.

Let {x^(k)} be a cauchy sequence in l^P. Since we are considering sequences of sequences,
x⁽¹⁾ = (x⁽¹⁾₁, x⁽¹⁾₂, x⁽¹⁾₃, x⁽¹⁾₄, x⁽¹⁾₅, …)
x⁽²⁾ = (x⁽²⁾₁, x⁽²⁾₂, x⁽²⁾₃, x⁽²⁾₄, x⁽²⁾₅, …)
x⁽³⁾ = (x⁽³⁾₁, x⁽³⁾₂, x⁽³⁾₃, x⁽³⁾₄, x⁽³⁾₅, …)
x⁽⁴⁾ = (x⁽⁴⁾₁, x⁽⁴⁾₂, x⁽⁴⁾₃, x⁽⁴⁾₄, x⁽⁴⁾₅, …)
x⁽⁵⁾ = (x⁽⁵⁾₁, x⁽⁵⁾₂, x⁽⁵⁾₃, x⁽⁵⁾₄, x⁽⁵⁾₅, …)
. = (., ., ., ., ., …)
. = (., ., ., ., ., …)
. = (., ., ., ., ., …)

The goal here is to prove that the sequence on the left-hand side, x^(k), has a limit, which is also in l^P
We start by picking a random column, say the 4th column, call it : x^(k)₄. This particular sequence is, by itself, a banach space (since it is single valued space with a norm over it, ||.||_p).
The normed distance between elements in this sequence will be: |x^(k)₄ - x^(l)₄|^P.
These normed distance would be lesser than the combined such distance of all columns: ∑∞_n=1|x^(k)_n - x^(l)_n| ^P
Hence it follows that: |x^(k)₄ - x^(l)₄|^P < ∑∞_n=1|x^(k)_n - x^(l)_nP = ||x^(k) - x^(l)||^P_p < ε.
But, since the latter is a Cauchy sequence, the former, that is the indivisual columns, are all also a Cauchy sequence. Since these columns are a Cauchy sequence AND belong to a banach space, they have a limit as well. Let’s call the limit, x’^(k).
Hence, each column has a limit. We can collect every columns limit into a sequence: x’ = (x’₁, x’₂, x’₃, x’₄, x’₅, …)
Now, we have to show that our original Cauchy sequence, x^(k) approaches it’s limit, x’ (which means it’s indeed a convergent sequence), or |x^(k) - x’| < ε.
We start by pulling out the infinity through taking the limit, essentially restricting the columns to N.
lim_n->∞ |x^(k)_n - x’_n|^P.
Now, we also have a second infinite sequence, in x’. We pull that out as well:
lim_l->∞ lim_n->∞ |x^(k)_n - x^(l)_n|^P.
Now, the inner normed distance should look familiar: we know that the indivisual columns can be bounded by any arbirtary upper bound, which we can select. Let it be ε’.
Since, |x^(k)_n - x^(l)_n|^P < (ε’)^P.
|x^(k)_n - x^(l)_n| < ε’.
Hence we simply take the limit now: |x^(k) - x’| < ε’, and set ε’ = ε/2.
Hence we have proved that any sequence {x^(k)} converges to a limit, x’ within some arbitary ε. This means that all Cauchy sequences are convergent, hence the (l^P, ||.||_p) is a banach space.

(Every statement about Cauchy sequence and it’s upper-bound being ε, is true after some kth index, where k > 0.)

Inner Product measures the distance, length and the angle between two vectors in 𝔽-vector space. It is denoted by < x, y >. It follows that:

• < x, x > >= 0 and < x, x > is only equal to 0 if and only if x = 0 vector.
• < x, y > = < y, x > if 𝔽 = ℝ AND < x, y > = < y, x >_conjugate if 𝔽 = ℂ
• < x, y₁ + y₂ > = < x, y₁ > + < x, y₂ > and < x, λy > = λ< x, y >, if 𝔽 = ℝ and < x, λy > = λ_conjugate< x, y >, if 𝔽 = ℂ

Hilbert Space. If (X, ||.||_p) is a banach space, than (X, <., .>) is considered to be a Hilbert space, that is a space which has an inner product that measures distance, length and angle while also being complete metric space.

Examples of Hilbert Spaces. Here are some important examples of hilbert spaces:

• ℝⁿ and ℂⁿ where the norm, < x, y > is := ∑ⁿ_i=1 x_i y_i (x-bar for complex case).
• The space l²(N, 𝔽), where the norm, < x, y > is := ∑^∞_i=1 x_i y_i (x-bar for complex case).

A quick check for the l² being a hilbert space, but check the norm conditions

< x, x > := ∑^∞_i=1 xbar_i x_i = ∑^∞_i=1 |x|² which is >= 0.
but < x, x > = 0. Hence |x|² = 0, so x has to be the zero vector

< x, y > := (∑^∞_i=1 ybar_i x_i)^- = ∑^∞_i=1 (ybar_i x_i)^-
which is ∑^∞_i=1 y_i xbar_i = < y, x >.

< x, λy > = ∑^∞_i=1 x_i λy_i = λ ∑^∞_i=1 x_i y_i = λ< x, y >

(Here xbar or ybar means the conjugate)

Cauchy-Schwarz Inequality This inequality simply states that the inner product between two vectors is always less than or equal to the product of the norms of the two vectors. Given below is the proof of the ineqaulity, given a 𝔽-vector space, (X, < ., . >)

• ||x|| = sqrt(< x, x >)
• |< x, y >| <= ||x|| . ||y||

Proof

Let x^ = x / ||x||, which is the normalized form of the vector x
Let y_|| = < x^, y > x^, which is the y vector projected onto x
Let y_ = y - y_||, which is the perpendicualr, adjacent to both y and x

According to the norm conditions
0 <= || y_ || = ||y - y_|||| = ||y - < x^, y > x^||

Squaring everything:
0 <= ||y_||² = ||y - y_||||² = ||y - < x^, y > x^||²
||y - < x^, y > x^||² = < y - < x^, y > x^, y - < x^, y > x^ >

Expanding
< y - < x^, y > x^, y - < x^, y > x^ > = < y - < x^, y > x^, y > - < y - < x^, y > x^, < x^, y > x^ >

Expanding further:
< y - < x^, y > x^, y > - < y - < x^, y > x^, < x^, y > x^ > = < y,y > - < < x^, y > x^, y > - < y, < x^, y > x^ > + < < x^, y > x^, < x^, y > x^ > =

Norms of y and < < x^, y > x^, y > emerge:
||y||² - (< < x^, y > x^, y > + < < x^, y > x^, y >_conj) + ||< < x^, y > x^, y >||²

Conjugates cancel, real part remains:
||y||² - (2 . Re(< < x^, y > x^, y >)) + |< < x^, y > x^, y >|² . ||x^||²

< < x^, y > x^, y > = |< x^, y >|²
||x^||² = 1

Using the square formula:
||y||² - 2 . |< x^, y >|² + |< x^, y >|²
which is = to: ||y||² - |< x^, y >|² >= 0

||y||² >= |< x^, y >|²
||y||² >= |< x/||x||, y >|²
||y||² >= 1||x||² (|< x, y >|²)

Removing the squares
||y||.||x|| >= < x, y >.

Proof for triangle inequality

||x||+||y|| <= ||x+y||

(||x||+||y||)² = < x+y, x+y > = ||x||² + 2.Re(< x, y >) + ||y||²
(||x||+||y||)² <= ||x||² + 2|< x, y >| + ||y||²
(||x||+||y||)² <= ||x||² + 2||x||.||y|| + ||y||² (cauchy-schwarz inequality)
(||x||+||y||)² <= (||x|| + ||y||)² ||x||+||y|| <= ||x+y||