Functional Analysis

Notes while learning about functional analysis.

Metric Spaces: It is defined as the space which has a set of points, say X, and a distance function, d, where the distance function obeys three rules:

• d(x, y) > 0
• d(x, y) = d(y, x)
• d(x, y) <= d(z, x) + d(z, y) (triangle inequality)
• Examples: Rⁿ and the Eucilidean distance, Complex plane and |x-y| norm distance.

ε-ball: Also denoted by B_ε(x), are defined as {y ∈ X | d(x, y) < ε}.

Open sets: If A is a set, and A ⊂ X, it is open if every point inside B_ε(x) belongs to it (for all x ∈ A).

Boundary Points: If any B_ε(x) has points outside of A, no matter how small the ε value, it is the boundary piont of the set A. Here x ∈ X. Than B_ε(x) is said to have points from both A and it’s compliment A^c. All the boundary points of A are denoted as ∂A.

Closed Sets: A set, A, is defined to be closed, if it’s compliment is open.

Closure: The smallest closed set, is simply, A ∪ ∂A. Remember :

• A is open if, A ∩ ∂A = ∅.
• A is closed if, A ∩ ∂A = A.

In the example, where X = (1, 3] ∪ (4, ∞), the set A = (1, 3] is both open AND closed. (Since A^c is also open, by definition, A is closed).

Convergence: A sequence is defined as a set of points arranged in increasing order, such as x₁ < x₂ < x₃ … x_n, also denoted by {x_i}. A sequence is said to be convergent when, in the limit, it approaches a single value, for example, a sequence, {a}^N_i converges to a single value à. Given a metric space (X, d), we can define the ε-ball, B_ε(x̃), for x ∈ X, such that,

• ∀ε > 0, ∃N ∈ ℕ, ∀n >= N : d(x, x̃) < ε

which means that for any convergent sequence, the ε-ball around it contains all the sequence points (as the ε increases, it engulfs more points from the sequence hence for every ε > 0, every point in sequence is covered).

• x̃ = lim_n->∞ x_n

Proposition: A characterstic of a closed set is: any limit of a convergent sequence is inside the closed set itself.

Proof (by contraposition)

• Let the sequence be {a_n} and the limit be ã. Let’s assume that A is not closed.
• It implies that the compliment of A, A^c is not open either.
• If ã ∈ A^c, then there exists at least one such point in B_ε(ã) that overlaps with A, which means, B_ε(ã) ∩ A ≠ ∅.
• Which means that there is a sub-sequence, {a_n}, which lies in A, but it’s limit does not (since ã ∈ A^c).
• Hence lim_n->∞ a_n = ã ∉ A.

Cauchy Sequence: A sequence which is defined by as

• ∀ε > 0, ∃N ∈ ℕ, ∀ n, m >= N : d(x_n, x_m) < ε
• It is a generalization of a convergent sequence, in which we shift our focus from the distance between the limit and all the points minimising to just the distance between the points to be minimising.
• Which mean after a certain index, N, the distance between two points is always less than ε, for any ε greater than zero.

Complete Metric Space : A metric space is said to be complete when all of it’s Cauchy sequences converge. This simply means that for a complete metric space, Cauchy sequnces = Convergent sequences.

• Example: (0, 3), where d := |x - y| is incomplete when with an index of n, m ∈ N, the distance -> 0 as n, m -> ∞ but zero is excluded from this set, hence it’s incomplete
• [0, 3] is complete, since zero is included in this set.
• (0, 3) under the metric, d := {1 if x ≠ y and 0 if x = y} is complete. Take a Cauchy sequence {x_n}_n∈N, where by definition, d (x_n, x_m) < ε. For any ε, the only way this is possible is if x_n = x_m (which makes d = 0), hence there is only one point, the limit after the index n (since all the points after it must be the same), hence this Cauchy sequence is convergent, which makes this metric space complete.

Norm : Let X be a 𝔽-vector space. (Where 𝔽 = {ℝ, ℂ}). The norm, denoted by ||.|| is a map that transforms a vector to [0, ∞). The norm fulfils the three properties of the distance function as well.

• ||x|| = 0, only when x = 0 (positive definite)
• ||λx|| = λ||x|| (absolutely homogeneous)
• ||x+y|| <= ||x|| + ||y|| where x, y ∈ X (triangle inequality) (Here a since you cannot pull out just the addition, and always get an inquality, it is non-linear)

Hence the space defined by (X, ||.||) is called a normed space. A normed space is a special case of a metric space (by taking the end points of a vector, it can be seen as the distance between those two points, hence the distance function of a metric space, d_||.||).

Banach Space : If a given metric space with a possible norm, (X, d_||.||) is also a complete one, then the underlying norm space, (X, ||.||) is called the banach space. All the properties of a metric space apply as well. The norm function is what ties the real-complex vector space and a complete metric space, hence also defining a possible Banach space.

l^P (ℕ,𝔽): It is defined as a collection of all the sequences in 𝔽, which follow the condition: ∑_n=1∞|x|_p < ∞. Here x = {x_n}, which is a sequence in 𝔽. l^P becomes a vector space if the summation condition is ignored. We define a norm over such vector space, ||.||_p : l^P -> [0, ∞). The norm is given by the formula, (∑_n=1∞|x|_p)^1/P, again, x is a sequence.

Assumption: l^P is a 𝔽-vector space, and ||.||_p is a norm on it.
Proof: (l^P, ||.||_p) is a banach space.

Let {x^(k)} be a cauchy sequence in l^P. Since we are considering sequences of sequences,
x⁽¹⁾ = (x⁽¹⁾₁, x⁽¹⁾₂, x⁽¹⁾₃, x⁽¹⁾₄, x⁽¹⁾₅, …)
x⁽²⁾ = (x⁽²⁾₁, x⁽²⁾₂, x⁽²⁾₃, x⁽²⁾₄, x⁽²⁾₅, …)
x⁽³⁾ = (x⁽³⁾₁, x⁽³⁾₂, x⁽³⁾₃, x⁽³⁾₄, x⁽³⁾₅, …)
x⁽⁴⁾ = (x⁽⁴⁾₁, x⁽⁴⁾₂, x⁽⁴⁾₃, x⁽⁴⁾₄, x⁽⁴⁾₅, …)
x⁽⁵⁾ = (x⁽⁵⁾₁, x⁽⁵⁾₂, x⁽⁵⁾₃, x⁽⁵⁾₄, x⁽⁵⁾₅, …)
. = (., ., ., ., ., …)
. = (., ., ., ., ., …)
. = (., ., ., ., ., …)

The goal here is to prove that the sequence on the left-hand side, x^(k), has a limit, which is also in l^P
We start by picking a random column, say the 4th column, call it : x^(k)₄. This particular sequence is, by itself, a banach space (since it is single valued space with a norm over it, ||.||_p).
The normed distance between elements in this sequence will be: |x^(k)₄ - x^(l)₄|^P.
These normed distance would be lesser than the combined such distance of all columns: ∑∞_n=1|x^(k)_n - x^(l)_n| ^P
Hence it follows that: |x^(k)₄ - x^(l)₄|^P < ∑∞_n=1|x^(k)_n - x^(l)_nP = ||x^(k) - x^(l)||^P_p < ε.
But, since the latter is a Cauchy sequence, the former, that is the indivisual columns, are all also a Cauchy sequence. Since these columns are a Cauchy sequence AND belong to a banach space, they have a limit as well. Let’s call the limit, x’^(k).
Hence, each column has a limit. We can collect every columns limit into a sequence: x’ = (x’₁, x’₂, x’₃, x’₄, x’₅, …)
Now, we have to show that our original Cauchy sequence, x^(k) approaches it’s limit, x’ (which means it’s indeed a convergent sequence), or |x^(k) - x’| < ε.
We start by pulling out the infinity through taking the limit, essentially restricting the columns to N.
lim_n->∞ |x^(k)_n - x’_n|^P.
Now, we also have a second infinite sequence, in x’. We pull that out as well:
lim_l->∞ lim_n->∞ |x^(k)_n - x^(l)_n|^P.
Now, the inner normed distance should look familiar: we know that the indivisual columns can be bounded by any arbirtary upper bound, which we can select. Let it be ε’.
Since, |x^(k)_n - x^(l)_n|^P < (ε’)^P.
|x^(k)_n - x^(l)_n| < ε’.
Hence we simply take the limit now: |x^(k) - x’| < ε’, and set ε’ = ε/2.
Hence we have proved that any sequence {x^(k)} converges to a limit, x’ within some arbitary ε. This means that all Cauchy sequences are convergent, hence the (l^P, ||.||_p) is a banach space.

(Every statement about Cauchy sequence and it’s upper-bound being ε, is true after some kth index, where k > 0.)

Inner Product measures the distance, length and the angle between two vectors in 𝔽-vector space. It is denoted by . It follows that:

• < x, x > >= 0 and < x, x > is only equal to 0 if and only if x = 0 vector.
• < x, y > = < y, x > if 𝔽 = ℝ AND < x, y > = < y, x >_conjugate if 𝔽 = ℂ
• < x, y₁ + y₂ > = < x, y₁ > + < x, y₂ > and < x, λy > = λ< x, y >, if 𝔽 = ℝ and < x, λy > = λ_conjugate< x, y >, if 𝔽 = ℂ

Hilbert Space. If (X, ||.||_p) is a banach space, than (X, <., .>) is considered to be a Hilbert space, that is a space which has an inner product that measures distance, length and angle while also being complete metric space.

Examples of Hilbert Spaces. Here are some important examples of hilbert spaces:

• ℝⁿ and ℂⁿ where the norm, < x, y > is := ∑ⁿ_i=1 x_i y_i (x-bar for complex case).
• The space l²(N, 𝔽), where the norm, < x, y > is := ∑^∞_i=1 x_i y_i (x-bar for complex case).

A quick check for the l² being a hilbert space, but check the norm conditions

< x, x > := ∑^∞_i=1 xbar_i x_i = ∑^∞_i=1 |x|² which is >= 0.
but < x, x > = 0. Hence |x|² = 0, so x has to be the zero vector

< x, y > := (∑^∞_i=1 ybar_i x_i)^- = ∑^∞_i=1 (ybar_i x_i)^-
which is ∑^∞_i=1 y_i xbar_i = < y, x >.

< x, λy > = ∑^∞_i=1 x_i λy_i = λ ∑^∞_i=1 x_i y_i = λ< x, y >

(Here xbar or ybar means the conjugate)