Selected from presentations by

1
Basis Basics

• Selected from presentations by
• Jim Ramsay, McGill University,
• Hongliang Fei, and Brian Quanz

2
1. Introduction

• Basis In Linear Algebra, a basis is a set of
  vectors satisfying
• Linear combination of the basis can represent
  every vector in a given vector space
• No element of the set can be represented as a
  linear combination of the others.

3

• In Function Space, Basis is degenerated to a set
  of basis functions
• Each function in the function space can be
  represented as a linear combination of the basis
  functions.
• Example Quadratic Polynomial bases 1,t,t2

4
What are basis functions?

• We need flexible method for constructing a
  function f(t) that can track local curvature.
• We pick a system of K basis functions fk(t), and
  call this the basis for f(t).
• We express f(t) as a weighted sum of these basis
  functions
• f(t) a1f1(t) a2f2(t) aKfK(t)
• The coefficients a1, , aK determine the
  shape of the function.

5
What do we want from basis functions?

• Fast computation of individual basis functions.
• Flexible can exhibit the required curvature
  where needed, but also be nearly linear when
  appropriate.
• Fast computation of coefficients ak possible if
  matrices of values are diagonal, banded or
  sparse.
• Differentiable as required We make lots of use
  of derivatives in functional data analysis.
• Constrained as required, such as periodicity,
  positivity, monotonicity, asymptotes and etc.

6
What are some commonly used basis functions?

• Powers 1, t, t2, and so on. They are the basis
  functions for polynomials. These are not very
  flexible, and are used only for simple problems.
• Fourier series 1, sin(?t), cos(?t), sin(2?t),
  cos(2?t), and so on for a fixed known frequency
  ?. These are used for periodic functions.
• Spline functions These have now more or less
  replaced polynomials for non-periodic problems.
  More explanation follows.

7
What is Basis Expansion?

• Given data X and transformation
• Then we model
• as a linear basis expansion in X, where
• is a basis function.

8
Why Basis Expansion?

• In regression problems, f(X) will typically
  nonlinear in X
• Linear model is convenient and easy to interpret
• When sample size is very small but attribute size
  is very large, linear model is all what we can do
  to avoid over fitting.

9
2. Piecewise Polynomials and Splines

• Spline
• In Mathematics, a spline is a special function
  defined piecewise by polynomials
• In Computer Science, the term spline more
  frequently refers to a piecewise polynomial
  (parametric) curve.
• Simple construction, ease and accuracy of
  evaluation, capacity to approximate complex
  shapes through curve fitting and interactive
  curve design.

10

• Assume four knots spline (two boundary knots and
  two interior knots), also X is one dimensional.
• Piecewise constant basis
• Piecewise Linear Basis

11
(No Transcript)
12
Piecewise Cubic Polynomial

• Basis functions
• Six functions corresponding to a six-dimensional
  linear space.

13
Piecewise Cubic Polynomial
14
Spline Interpolation Method

• Slides taken from the lecture by
• Authors Autar Kaw, Jai Paul

15
What is Interpolation ?

Given (x0,y0), (x1,y1), (xn,yn), find the
value of y at a value of x that is not given.
16
Interpolants

• Polynomials are the most common choice of
  interpolants because they are easy to
• Evaluate
• Differentiate, and
• Integrate.

17
Why Splines ?
18
Why Splines ?
Figure Higher order polynomial interpolation is
a bad idea
19
Linear Interpolation
20
Linear Interpolation (contd)
21
Example

• The upward velocity of a rocket is given as a
  function of time in Table 1. Find the velocity at
  t16 seconds using linear splines.

Table Velocity as a function of time
Figure. Velocity vs. time data for the rocket
example
22
Linear Interpolation




23
Quadratic Interpolation
24
Quadratic Interpolation (contd)
25
Quadratic Splines (contd)
26
Quadratic Splines (contd)
27
Quadratic Splines (contd)
28
Quadratic Spline Example

• The upward velocity of a rocket is given as a
  function of time. Using quadratic splines
• Find the velocity at t16 seconds
• Find the acceleration at t16 seconds
• Find the distance covered between t11 and t16
  seconds

Table Velocity as a function of time
Figure. Velocity vs. time data for the rocket
example
29
Solution






Let us set up the equations
30
Each Spline Goes Through Two Consecutive Data
Points
31
Each Spline Goes Through Two Consecutive Data
Points
32
Derivatives are Continuous at Interior Data Points
33
Derivatives are continuous at Interior Data Points
At t10
At t15
At t20
At t22.5
34
Last Equation
35
Final Set of Equations
36
Coefficients of Spline
37
Quadratic Spline InterpolationPart 2 of
2http//numericalmethods.eng.usf.edu
38
Final Solution
39
Velocity at a Particular Point

• a) Velocity at t16

40
Quadratic Spline Graph
ta2b
41
Quadratic Spline Graph
ta0.5b
42
Natural Cubic Spline Interpolation
SPLINE OF DEGREE k 3

• The domain of S is an interval a,b.
• S, S, S are all continuous functions on a,b.
• There are points ti (the knots of S) such that a
  t0 lt t1 lt .. tn b and such that S is a
  polynomial of degree at most k on each
  subinterval ti, ti1.

ti are knots
43
Natural Cubic Spline Interpolation

• Si(x) is a cubic polynomial that will be used on
  the subinterval xi, xi1 .

44
Natural Cubic Spline Interpolation

• Si(x) aix3 bix2 cix di
• 4 Coefficients with n subintervals 4n equations
• There are 4n-2 conditions
• Interpolation conditions
• Continuity conditions
• Natural Conditions
• S(x0) 0
• S(xn) 0