A Look-Back Technique of Restart for the GMRES(m) Method

1
A Look-Back Technique of Restart for the
GMRES(m) Method
Akira IMAKURA Tomohiro SOGABE
Shao-Liang ZHANG Nagoya University,
Japan. Aichi Prefectural University, Japan.
2
Introduction

• Main subject -- Nonsymmetric linear systems of
  the form

• Krylov subspace methods -- Symmetric linear
  systems
• Based on Lanczos algorithm
• CG, MINRES,
• -- Nonsymmetric linear systems
• Based on two-sided Lanczos algorithm
• Bi-CG, Bi-CGSTAB, GPBi-CG
• Based on Arnoldi algorithm
• GCR, GMRES,

restart ? GMRES(m) method
3
Introduction

• GMRES(m) method Y. Saad and M. H.
  Schultz1986 -- Algorithm (focus on restart)

Algorithm GMRES(m)
-- Update the initial guess
l number of restart cycle
4
Introduction

• GMRES(m) method Y. Saad and M. H.
  Schultz1986 -- Residual polynomials

l number of restart
residual polynomial
Motivation

5
Introduction

• GMRES(m) method with readjustment -- Numerical
  result compared with GMRES(m) method
• parameters

? GMRES(m)
? GMRES(m) with readjustment
CAVITY05 XENON1
Readjustment leads to be better convergence
Question
How does we readjust the function s.t. the root is moved?
6
Introduction

• Look-Back GMRES(m) method -- Algorithm (focus
  on update the initial guess)

Algorithm Look-Back GMRES(m)
Run m iterations of GMRES Input , Output

We can simply achieve the readjustment
7
Introduction

• Look-Back GMRES(m) method -- Extension of the
  GMRES(m) method

? Analyze based on error equations
-- A Look-Back technique of restart
? Analyze based on residual polynomials
8
Extension of the GMRES(m) method-- Analysis
based on error equation --
9
Extension of the GMRES(m) method
Algorithm GMRES(m)
?
Run m iterations of GMRES Input , Output

10
Extension of the GMRES(m) method

• Analysis based on error equations --
  Introduction of error eq. and iterative
  refinement scheme

Definition error equation
Let and be the exact solution and the numerical solution respectively. Then the error vector can be computed by solving the so-called error equation, i.e., where is residual vector corresponding to .
Definition iterative refinement scheme
The technique based on solving error equations recursively to achieve the higher accuracy of the numerical solution is called the iterative refinement scheme.
11
Extension of the GMRES(m) method
Algorithm GMRES(m)
Algorithm Extension of GMRES(m)
Run m iterations of GMRES Input , Output

Run m iterations of GMRES Input , Output

Mathematically equivalent
Mathematically equivalent
Natural extension
12
A Look-Back technique of restart-- Analysis
based on residual polynomials --
13
A Look-Back technique of restart

• Difference between GMRES(m) and its Extension

Extension of GMRES(m) method

residual polynomial
GMRES(m) method

14
A Look-Back technique of restart

• Difference between GMRES(m) and its Extension

Extension of GMRES(m) method

residual polynomial
Look-Back GMRES(m) method

Set by some technique
Look-Back technique
GMRES(m) method

15
A Look-Back technique of restart

• A Look-Back technique

Extension of GMRES(m) method

Look-Back GMRES(m) method

Set by Look-Back technique
Look-Back strategy

16
A Look-Back technique of restart

• A Look-Back technique

Extension of GMRES(m) method

Look-Back GMRES(m) method

Motivation

1.0
0.0
It is expected that readjustment leads to be high
convergence
17
A Look-Back technique of restart

• A Look-Back technique

Extension of GMRES(m) method

Look-Back GMRES(m) method

18
A Look-Back technique of restart

• Proposal of Look-Back GMRES(m) method --
  Algorithm (focus on update the initial guess)

Algorithm Look-Back GMRES(m)
Run m iterations of GMRES Input , Output

-- extra costs for Look-Back technique 1
matrix-vector multiplication per 1 restart
19
Numerical experiments-- Comparison with GMRES(m)
--
20
Numerical experiments

• Test problems obtained from UF Sparse Matrix
  Collection --

CAVITY05, CAVITY16, CHIPCOOL0,
MEMPLUS, NS3DA, RAJAT03,
RDB5000, XENON1, XENON2.

• Compared methods (without preconditioner)
• -- GMRES(m) method
• -- Look-Back GMRES(m) method

(m 30, 100)

• Parameters
• -- right-hand-side
• -- initial guess
• -- stopping criterion

• Experimental conditions
• -- AMD Phenom II X4 940 (3.0GHz)
• -- Standard Fortran 77 using double precision.

21
Numerical experiments
? GMRES(m)
? Look-Back GMRES(m)

• Numerical results for m 30

CAVITY05 CAVITY16 CHIPCOOL0
MEMPLUS NS3DA RAJAT03
RDB5000 XENON1 XENON2
22
Numerical experiments
? GMRES(m)
? Look-Back GMRES(m)

• Numerical results for m 30

CAVITY05 CAVITY16 CHIPCOOL0
MEMPLUS NS3DA RAJAT03
RDB5000 XENON1 XENON2
Time sec. Time sec.
Total 1 Restart


2.94 E00
1.25 E00
Time sec. Time sec.
Total 1 Restart
Time sec. Time sec.
Total 1 Restart


1.86 E-02
1.87 E-02


7.30 E-02
7.33 E-02



3.15 E01



7.83 E01


4.43 E-03
4.57 E-03
Time sec. Time sec.
Total 1 Restart
Time sec. Time sec.
Total 1 Restart
Time sec. Time sec.
Total 1 Restart


5.16 E-02
5.13 E-02


2.30 E-01
2.35 E-01


1.93 E-01
1.93 E-01


1.14 E01
3.93 E00


1.78E 01
1.84 E01



9.16 E00
Time sec. Time sec.
Total 1 Restart
Time sec. Time sec.
Total 1 Restart
Time sec. Time sec.
Total 1 Restart


1.29 E-02
1.29 E-02


2.44 E-01
2.51 E-01


8.36 E-01
8.39 E-01


4.00 E-01
4.42 E-01


9.47 E01
1.57 E01


4.38 E02
6.64 E01
23
Numerical experiments
? GMRES(m)
? Look-Back GMRES(m)

• Numerical results for m 100

CAVITY05 CAVITY16 CHIPCOOL0
MEMPLUS NS3DA RAJAT03
RDB5000 XENON1 XENON2
24
Numerical experiments
? GMRES(m)
? Look-Back GMRES(m)

• Numerical results for m 100

CAVITY05 CAVITY16 CHIPCOOL0
MEMPLUS NS3DA RAJAT03
RDB5000 XENON1 XENON2
Time sec. Time sec.
Total 1 Restart
1.23 E-01
2.38 E01 1.24 E-01
Time sec. Time sec.
Total 1 Restart
5.47 E-01
1.28 E02 5.52 E-01
Time sec. Time sec.
Total 1 Restart
2.65 E00 3.09 E-02
1.15 E00 3.09 E-02
Time sec. Time sec.
Total 1 Restart
1.35 E01 4.49 E-01
7.78 E00 4.50 E-01
Time sec. Time sec.
Total 1 Restart
2.11 E01 1.07 E00
2.11 E01 1.08 E00
Time sec. Time sec.
Total 1 Restart
1.71 E-01
8.56 E00 1.71 E-01
Time sec. Time sec.
Total 1 Restart
4.18 E-01 1.10 E-01
3.93 E-01 1.10 E-01
Time sec. Time sec.
Total 1 Restart
6.41 E01 1.65 E00
3.11 E01 1.63 E00
Time sec. Time sec.
Total 1 Restart
2.70 E02 5.49 E00
1.23 E02 5.43 E00
25
Numerical experiments

• Test problems -- From discretization of partial
  differential equation of the form
• over the unit square
  .

26
Numerical experiments

• Compared methods (with ILU(0) preconditioner)
• -- GMRES(m) method
• -- Look-Back GMRES(m) method

(m 10)

• Parameters
• -- initial guess
• -- stopping criterion

• Experimental conditions
• -- AMD Phenom II X4 940 (3.0GHz)
• -- Standard Fortran 77 using double precision.

27
Numerical experiments

• Numerical results

? GMRES(m)
? Look-Back GMRES(m)
a 0 a 0.5
a 1.0 a 2.0
28
Conclusion and Future works
29
Conclusion and Future works

• Conclusion -- In this talk, from analysis based
  on residual polynomials, we proposed the
  Look-Back GMRES(m) method.
• -- From our numerical experiments, we learned
  that the Look- Back GMRES(m) method shows a
  good convergence than the GMRES(m) method in
  many cases.
• -- Therefore the Look-Back GMRES(m) method will
  be an efficient variant of the GMRES(m)
  method.

• Future works -- Analyze details of the
  Look-Back technique.
• -- Compare with other techniques for the
  GMRES(m) method.

30
Thank you for your kind attention
31
Numerical experiments