Definitions and Basics
Vector Norms
Definition of a Vector Norm
The most ordinary kind of vectors are those consisting of
ordered n-tuples of real or complex numbers. They may be written
in row
or column
forms. Commas or other seperators of components or coordinates
may or may not be used. When a vector has many elements, notations like
or
are often used.
A most popular notation to indicate a
vector is .
Vectors are usually added component-wise, for
and
,
.
Scalar multiplication is defined by
.
A vector norm is a generalization of ordinary absolute value of a real or complex number.
For and , vectors, and , a scalar, a vector norm is a real value associated with a vector for which the following properties hold.
.
Common Vector Norms
The most commonly used norms are:
.
Any two norms on dimensional vectors of complex numbers are topologically equivalent in the sense that, if and are two different norms, then there exist positive constants and such that .
Inner Product of Vectors
The inner product (or dot product), of two vectors
and
,
is defined by
,
or when and
are complex valued, by
. |
|
It is often indicated by any one of several notations:
or .
Besides the dot product, other inner products are defined to be a rule that sssigns to each pair of vectors , a complex number with the following properties.
for , is real valued and positive
and
An inner product defines a norm by
. |
|
Inequalities Involving Norms
The Cauchy Schwarz and Holder's inequalities are commonly employed.
for
Matrix Norms
Definition of a Matrix Norm
The most ordinary kind of matices are those consisting of rectangular arrays of real or complex numbers. They may be written in element form and be considered as collections of column or row vectors.
Matrices are usually added element-wise, for
Scalar multiplication is defined by
.
The notation means that all elements of are identically zero.
A matrix norm is a generalization of ordinary absolute value of a real or complex number, and can be considered a type of a vector norm.
For and , matrices, and , a scalar, a matrix norm is a real value associated with a matrix for which the following properties hold.
.
Common Matrix Norms
The most commonly used norms are:
.
Like vector norms any two matrix norms on matrices of complex numbers are topologically equivalent in the sense that, if and are two different norms, then there exist positive constants and such that .
The norm on matrices is an example of an induced norm. An induced norm is for a vector norm defined by
.
This could cause the same subscript notation to be used fot two different norms, sometimes.
Positive definite matrices
A matrix is said to be positive definite if for any vector
for some positive constant ,
not depending on .
The property of being positive definite insures the numerical stability of a variety of common numerical techniques used to solve the equation .
Taking then
.
so that
and .
This gives
and
.
Consistency of Norms
A matrix norm and a vector norm are said to be consistent when
.
When is the matrix norm induced by the vector norm then the two norms will be consistent.
When the two norms are not consistent there will still be a positive constant such that
.
Finite Difference Operators
approximation of a first derivative
The defition of the derivative of a function gives the first and simplest finite difference.
.
So the finite difference
can be defined. It is an approximation to when is near .
The finite difference approximation for is said to be of order , if there exists such that
,
when is near .
For practical reasons the order of a finite difference will be described under the assumption that is sufficiently smooth so that it's Taylor's expansion up to some order exists. For example, if
then
so that
,
meaning that the order of the approximation of by is .
The finite difference so far defined is a 2-point operator, since it requires 2 evaluations of . If
then another 2-point operator
can be defined. Since
,
this is of order 2, and is referred to as a centered difference operator. Centered difference operators are usually one order of accuracy higher than un-centered operators for the same number of points.
More generally, for points a finite difference operator
is usually defined by choosing the coefficients so that has as high of an order of accuracy as possible. Considering
.
Then
.
where the are the right hand side of the Vandermonde system
and
.
When the are chosen so that
then
.
so that the operator is of order .
At the end of the section a table for the first several values of , the number of points, will be provided. The discussion will move on to the approximation of the second derivative.
approximation of a second derivative
The definition of the second derivative of a function
.
used together with the finite difference approximation for the first derivative
gives the finite difference
In view of
for the operator just defined
.
If instead, the difference operator
is used
If the other obvious possibility is tried
In view of
,
.
So
is a second order centered finite difference approximation for .
The reasoning applied to the approximation of a first derivative can be used for the second derivative with only a few modifications.
For points a finite difference operator
is usually defined by choosing the coefficients so that has as high of an order of accuracy as possible. Considering
.
Then
.
where the are the right hand side of the Vandermonde system
and
.
When the are chosen so that
then
.
so that the operator is of order .
The effect of centering the points will be covered somewhere below.
approximation of higher derivatives
Although approximations to higher derivatives can be defined recursively from those for derivatives of lower order, the end result is the same finite difference operators. The Vandermonde type system will be used again for this purpose.
.
The number of points needed to approximate by finite differences is at least .
For points a finite difference operator
is usually defined by choosing the coefficients so that approximates to as high of an order of accuracy as possible. Considering
.
Then
.
where the are the right hand side of the Vandermonde system
and
.
When the are chosen so that
then
.
so that the operator is of order .
An alternative analysis is to require that the finite difference operator differentiates powers of exactly, up to the highest power possible.
effect of the placement of points
Usually the are taken to be integer valued, since the points are intended to coincide with those of some division of an interval or 2 or 3 dimensional domain. If these points and hence are chosen with only accuracy in mind, then a higher accuracy of only one order can be achieved.
So start by seeing how high is the accuracy that can be approximated with three points.
Then accuracy of order 4 can not be achieved, because it would require the solution of
which can not be solved since the matrix
is non-singular. The possibility of an being can be ruled out otherwise.
For accuracy of order 3
So the matrix
is singular and
are the roots of some polynomial
.
Two examples are next.
To see what the accuracy that can be approximated to with three points.
Then accuracy of order 3 can not be achieved, because it would require the solution of
which can not be solved since the matrices
and
would both need to be singular.
If the matrix
is singular, then
are the roots of some polynomial
,
implying
meaning that elementary row operations can transform
to
which is non-singular.
Conversely, if
are the roots of some polynomial
, then
can be solved and approximated to an order of 2 accuracy.
See how high is the accuracy that can be approximated with points.
Then accuracy of order can not be achieved, because it would require the solution of
which can not be solved since the matrix
is non-singular. The possibility of an being can be ruled out otherwise, because, for example, if , then the non-singularity of the block
would force .
For accuracy of order
So the matrix
is singular and
are the roots of some polynomial
.
The progression for the second, third, ... derivatives goes as follows.
If
are the roots of some polynomial
then the system
can be solved, and
approximates to an order of accuracy of .
If
are the roots of some polynomial
then the system
can be solved, and
approximates to an order of accuracy of .
Now, the analysis is not quite done yet. Returning to the approximation of . If for the polynomial
it were that , then the system can be solved for one more order of accuracy. So the question arises as to whether polynomials of the form
exist that have distinct real roots. When there is not. So consider .
If has 4 distinct real roots, then
has 3 distinct real roots, which it does not. So the order of approximation can not be improved. This is generally the case.
Returning to the approximation of . If for the polynomial
it were that , then the system can be solved for one more order of accuracy. So the question arises as to whether polynomials of the form
exist that have distinct real roots.
If has distinct real roots, then
has distinct real roots, which it does not. So the order of approximation can not be improved.
For functions of a complex variable using roots of unity, for example, may obtain higher orders of approximation, since complex roots are allowed.
centered difference operators
For points a finite difference operator
is said to be centered when the points are symmetrically placed about .
When is odd .
To find the centered difference operators, consider
.
Then
.
where the are the right hand side of the over-determined system
and
.
When the are chosen so that
then
.
so that the operator is of order .
Since, for the centered case, the system is over-determined, some restriction is needed for the system to have a solution. A solution occurs when the are the roots of a polynomial
with
.
Observing that when is even
and when is odd
.
So when is even has for all odd and when is odd has for all even .
So a centered difference operator will achieve the one order extra of accuracy when the number of points is even and the order of the derivative is odd or when the number of points is odd and the order of the derivative is even.
Shift Operators
Shift of a Function
Trigometric polynomials
Let
|
be trigometric polynomial defined on .
Define the inner product on such trigometric polynomials by
. |
|
In light of the orthogonalities
,
and when ,
inner products can be calculated easily.
. |
|
and for
the inner product is given by
. |
|
Definition of a shift operator
Define the shift operator on by
. |
|
Since
and
,
so that
. |
|
Approximation by trigometric polynomials
Let be a function defined on and periodic with respect to the interval . That is .
The degree trigometric polynomial approximation to is given by
where
and . |
|
approximates in the sense that
is minimized over all trigometric polynomials, of degree or less, by .
In fact
.
The term in the center
,
so that
|
|
linearity property |
|
If and are the degree trigometric polynomial approximations to and , then the degree trigometric polynomial approximation to is given by .
This follows immediately from (3.4.0) since
and .
Fundamental Property
|
Generally if is the degree trigometric polynomial approximation to a function , periodic on , then is the degree trigometric polynomial approximation to .
To see this calculate the trigometric polynomial approximations for .
,
where
and
.
.
Comparing the results with (3.3.1) finishes the observation.
Another detail of use is
which is
. |
|
Error Estimation
A result used in error estimation is
. |
|
When is a sine polynomial
then
|
and
. |
|
Simple Functions
Let
so that
is a partition of .
The function is said to be simple on , if
|
Of particular interest is when the points have equal spacing .
The intent is to make estimates of .
Begin by making an odd extension of to by setting and continue the definition by extending periodically.
Then approximate with a sine polynomial
where
.
When is large enough so that some are divided by then, for
,
and letting ,
,
so that
.
Now, return to the sum
with for .
If and , then , and in this case
and .
So for with ,
. |
|
Next observe that if is the degree sine polynomial approximation to then is the degree trigometric polynomial approximation to . The assumption that is odd and periodic is still in effect.
Finally the intended results follow.
so that
.
Making use of (3.7.1)
.
Being that simple functions can be approximated by trigometric polynomials to arbitrary accuracy,
. |
|
Now, for , and using the definition of the simple function
To find the sum for list tthe values of over the values of on the whole interval .
This gives
.
So the inequality follows
. |
|
Specialized Norms
Energy and Heat Norms
There are a number of less well known, but important norms. These norms are important in the analysis of many physical problems and are used in error estimation for finite difference and finite element methods. Examples are the energy and heat norms.
These norms are usually expressed in an integral form.
When the inequality below holds.
This follows from a completely elementary, but lengthy calculation, as shown in appendix a). When additional assumptions on are made this inequality can be improved somewhat.
See appendix b) for an explanation.
Discrete Heat Norms
In the analysis of finite difference methods for partial differential equations it is useful to have discrete analogs of norms like the energy and heat norms.
In an attempt not to have notation too cumbersome some indices are suppressed when from the discussion it is clear what they refer to. When is a dimensional vector of complex numbers, finite difference operators are defined as discrete approximations or analogs of derivatives.
The most appropriate definition of a discrete energy or heat norm may vary due to differences in the handling of initial or boudary conditions. So for this reason the reader should make appropriate adjustments, when needed, to apply the same reasoning to another problem.
Before the discrete versions of energy or heat norms can be defined, finite differece operations need to be defined and explained. This was done in the section on finite difference operators.
The next discrete version of the preceding inequality has important applications to the estimation of the error when approximating a second derivative with a finite difference operator.
If then
with
and generally the following under-estimate holds.
See appendix c) for a proof.
The inequality can be improved for general by using (3.7.3) with increased by 2.
If then
and
.
appendix a)
When the inequality below holds.
First apply the Cauchy Schwarz inequality.
,
Next observe the integrals on the right are increasing with .
Integrate, make cancellations, and reapply the first inequality.
.
.
.
After integrating again the inequality is improved.
.
.
Now, assume the inequality immediately below holds for some .
After substituting the inequality above into the next
.
the following observations are made.
.
.
.
.
.
Repeating this iteration leads to a sequence
which converges to , the solution of .
So:
and
.
For ,
.
.
To handle the part
,
.
.
.
Reasoning as before this inequality can be strengthend to
.
.
.
Adding the results of the two main calculations
.
appendix b)
When the inequality below holds.
Assume the conditions that can be approximated by a trigometric polynomial
.
such that is also approximated by the polynomials derivative
That is to say, given , there exixts a trigometric sine polynomial , such that
and .
Now,
and
.
So it is easy to see that
.
In this case the inequality is sharp, since it holds with equality for .
Since
,
and
,
the inequality holds for and .
appendix c)
If then
with
and generally the following under-estimate holds.
The cases for 1 and 2 are simple.
and
with equality, when .
To prove the general under-estimate do as follows.
Use and apply the Cauchy Schwartz inequality together with inequality ().
|
|
So for ,
and
.
After inserting the inequality above into (1)
.
So for ,
and after using formula ()
.
Using the right end of the sum is estimated using nearly the same procedure.
So for ,
,
,
and
.
So for ,
,
.
Combining the two results:
.
When is odd, use so that the inequality becomes
.
When is even, use so that the inequality becomes
.
Special Formulas
The rule named L´Hospital's Rule named by a French math teacher who published it without proof, because he said you need to be taken to the hospital if you can't prove it yourself, states.
If and then,
.
The rule named Leibniz's Rule states how to differentiate a product of two functions times.
There are a number of ways to find the formulas for the sums of the powers of the first integers. One of the most convenient is as follows. Begin with the geometric formula.
By convention .
Apply Leibniz's Rule rule to the right hand side of the equivalent identity
to differentiate both sides times.
Upon noticing that only the first two terms of the sum on the right are non-zero
.
Now take the limit as to establish the formula
.
From term by term differentiation.
and
.
More generally
.
and thus
.
A progression of formulas follow by setting .
.
.
and
.
After adjusting the index of summation and increasing by the progression has the form
.
Letting
.
The formula called summation by parts is a discrete analog of integration by parts . It is verified simply by inspection.
One form of the statement of the Fundamental Theorem of Integral Calculus is
For it holds .
Another rule which slightly generalizes the Fundamental Theorem of Integral Calculus says
.
The following simple inequality can be used to bound sums of products.
and also
,
so that
.
Letting and
.
The particular case with gives
.
save scratch
Now, suppose for some
So for
zzzzzzzzzz
So for
xxxxxxxxxxxxxxxx