Introduction
We have a Bellman equation and first we want to know if there exists a value function that satisfies the equation and second we want to know the properties of such a solution. In order to answer the question we will define a mapping which maps a function to another function, and a fixed point of the mapping is to be a solution. The mapping we discussed is a mapping on the set of functions, which is a bit abstract. So today we will look at the math review.
So first we consider a set,
, For us what it will be relevant to describe a sort of distance between any two points in a set. We will use the concept of a metric.
Metric
A metric is a function
with the properties that it is non-negative,
, symmetric,
, and satisfies the triangle inequality,
,
A common metric is euclidean distance,
, Another is
,
Space
A space, is a set of objects equipped with some general properties and structure
We may be interested in a metric space, a space with a metric such as,
where
is the set of all bounded rational functions, and
is some distance function. Once we have a metric space we can discuss convergence and continuity.
convergence
A sequence,
, converges to
,
, if
s.t.
for
,
Cauchy sequence
A sequence ,
, is called a Cauchy sequence if
for
,
Question: does every Cauchy sequence converge?
Completeness
The metric space,
is complete if every Cauchy sequence converges.
examples of completeness
is complete.
is not complete. Proof: let
, So
os Cauchy, but does not converge to a point in our set
,
is complete. Are all closed sets complete? A closed subspace of a complete space is complete.
is complete.
Contraction Mapping
A mappting
is a contraction mapping on a metric space,
, if
such that
, Sometimes we write
instead of
,
This means that any two points in our set,
, are mapped such that after the mapping the distance between the points shrinks.
examples of contraction mapping
is a contraction mapping on
,
Now we state the contraction mapping theorem.
Contraction mapping theorem
If
is complete and
is a contraction mapping, then
with
,
We will prove this theorem for a general metric space later on. However, we must remember that it is necessary for this proof that the space be complete.
Let us now look at a criteria to verify that a mapping is a contraction mapping.
Contraction Mapping criteria
For
and
, Let
satisfy the following two conditions:
- (M, monotonic condition)
and
, and
, if
then
,
- (D, discout condition)
, for
,
.
Then
is a contraction mapping.