Multigrid for Nonlinear Problems

1
Multigrid for Nonlinear Problems
FAS, Newton-MG, Multilevel Nonlinear Method

• Ferien-Akademie 2005, Sarntal, Christoph Scheit

2
Outline

• Motivation
• Basic Idea of Multigrid
• Classical MG-Approaches for nonlinear problems
• Newton-Multigrid
• FAS
• Properties of both approaches
• Multilevel Nonlinear Method
• Conclusions
• Bibliography

3
Motivation

• To solve (linear) systems of equations arising
  from the discretization of nowadays engineering
  science problems fast and robust solvers are
  needed
• A lot of problems arising from engineering
  science contain nonlinearities
• Example nonlinear diffusion equation

4
Basic Ideas of Multigrid

• A lot of relaxations schemes smooth the error.
  Consider Jacobi-relaxation

5
Basic Ideas of Multigrid

• From eigenvalue analysis (Local Mode analysis) or
  numerical experiments one can see that only high
  frequency errors are damped (smoothed)
  efficiently by most relaxation schemes
• The basic idea is now, to smooth the error on a
  grid, on which the error looks high frequent. On
  a twice as coarse grid, the same mode appears
  with the double frequency relative to the number
  of grid points. Therefore high-frequency errors
  can be smoothed efficiently on the coarser grid.

6
Basic Ideas of Multigrid

• The residual equation
• Using the residual equation, it is possible to
  compute the error and update the approximate
  solution
• Bringing it all together, one can use coarser
  grids to efficiently compute a correction for the
  actual solution

7
Basic Ideas of Multigrid

• To use a sequence of Grids, transfer operators
  are needed
• Restriction to transport the residual to a
  coarser grid
• Interpolation (or prolongation) to transport the
  correction back to the finer grid
• Using this ideas, one can construct schemes like
  the well known V-Cycle

8
Basic Algorithm

• MGM_Basic(x,f,l,?1,?2,µ)
• if (l 1)
• return x exact_sol(x,f)
• end
• x_h preSmoothing(x,f,?1)
• r_h f-Ax
• r_H Restriction(r_h)
• for (i 1 i lt µ i)
• x_H 0
• e_H MGM_Basic(x_H,r_H,l-1,?1,?2,µ)
• end
• e_h Prolongate(e_H)
• x_h x_h e_h
• x_h postSmoothing(x_h,f,?2)
• return x_h
• end

9
Nonlinear Problems

• For an equation like our
  resulting operator N for the discretized equation
  is itself depending on the solution u
• How to modify the algorithm for nonlinear
  problems?
• Important, since for nonlinear problems the
  Residual Equation does not hold any more
  (instead, nonlinear Residual Equation / defect
  equation)

10
Nonlinear Approaches for Multigrid

• Newton-Multigrid
• -gt global linearization
• FAS (Full Approximation Scheme/Storage)
• -gt local linearization
• MNM (Multilevel Nonlinear Method)
• -gt combine local and global method

11
Newtons Method

• First consider Newtons Method for scalar
  Problems
• If f(xs) is a solution, then

12
Newtons Method
13
Newtons Method

• How to use Newtons Method for nonlinear equation
  systems? Analog to the scalar case
• If F(vs)F(u) is a solution
• Where s is the error of the current
  approximation, we get

14
Newtons Method

• What is grad(F(u))?
• -gt the Jacobian of F(u)

15
Newtons Method

• A concrete example for J(v)
• Partial derivation yields

16
Newton-Multigrid

• Back to Newtons Method for Multigrid
• Using the nonlinear Residual Equation and the
  truncated taylor serious yields
• The last equation is the linearized equation
  system and has to be computed instead of the
  original one using multigrid methods.

17
Newton-Multigrid - Algorithm

• v init_sol()
• r f-N(u)
• while (r lt tol)
• compute J(v)
• e 0
• for i 0 i lt numV-Cycles i
• e MGM_Basic(e,r,l,?1,?2,µ)
• end
• v v e
• r f-N(u)
• end

18
FAS

• Newtons-Multigrid doesnt use Multigrid ideas to
  solve the nonlinear equation system, but uses a
  global linearization and an outer iteration with
  the basic Multigrid Method embeded as a solver
  for the linearized equation system
• Different from the idea of Newtons Method for
  Multigrid, FAS treats directly the nonlinear
  equation system, using a nonlinear smoother for
  local linearization such as Gauss-Newton
  relaxation

19
FAS

• Back to the nonlinear Residual Equation (defect
  correction equation)
• We can formulate this equation on the coarse grid
  by
• Where is the injection operator (instead of
  full weighted restriction)

20
FAS - Algorithm (only V-Cycle)

• FAS(x,f,l,?1,?2)
• if(l1)
• return x exact_sol(x,f)
• end
• x_h preSmoothing(x,f,?1)
• f_H restriction(f A_h x_h)
• A_h injection(x_h)
• x_H injection(x_h) // initial guess for
  coarse grid
• FAS(x_H,f_H,l, ?1,?2)
• x_h x_h prolongation(x_H injection(x_h))
• x_h postSmoothing(x_h,f,?2)
• return x_h
• end

21
FAS nonlinear relaxation

• Instead of a global linearization FAS uses a
  nonlinear smoother, which is simply obtained by
  Newtons Method (for scalar problem)
• Consider again the nonlinear equation
• Discretized we obtain
• Using Newtons Method yields the following
  iteration scheme

22
FAS implementation hints

• Start first with a linear problem then the
  FAS-Algorithm must yield the same result as the
  standard MG-Algorithm (except roundoff errors)
• For the nonlinear problem considered here, a
  standard Gauss-Seidel relaxation works also. In
  general one has to use a nonlinear smoother like
  presented above
• Since FAS does not approximate the error, but
  directly improves the current solution on the
  different grid levels, dont forget to inject
  also the boundary condition (for the error in the
  standard MG this was not necessary, since for
  Dirichlet b.c. the b.c. for the error is always
  zero)

23
Properties of classical approaches

• Newton
• Fast convergence, often only a few newton steps
• For each newton step, the linearized equation
  must be solved accurately
• A good initial guess is needed to ensure
  convergence (small attraction basin)
• (slow) backtracking to find a good initial guess

• FAS
• No global linearization is needed
• Convergence even for poor initial guess, if a
  good approximation for the nonlinear operator is
  available (large attraction basin)
• Converges slower to the solution than Newton-MG

24
Multilevel Nonlinear Method (MNM)

• While Newtons-MG converges fast, we need a good
  initial guess
• While FAS converges not so fast, it converges
  even for a poor initial guess if we have a good
  approximation for the nonlinear operator
• Idea Combine the properties of both algorithms,
  such that the resulting Method converges fast and
  even for a poor initial guess -gt MNM
• Use a robust approximation for the dominating
  operator -gt MNM, Galerkin Coarsening

25
MNM

• Once again back to the nonlinear Residual
  equation
• Now we want to split this equation into a large
  linear part and a small nonlinear part. The
  linear part corresponds to Newton-MG while the
  nonlinear part corresponds to FAS. The nonlinear
  part should be small, because in this case it
  would not be so bad, if the approximation of the
  nonlinear part is not so good (which was required
  by FAS)

26
MNM

• To obtain this splitting, we add to the left hand
  side of the nonlinear Residual Equation J(v)e e
  u-v
• Rearranging the terms yields a linear and a
  nonlinear term
• Obviously, the linear part is O(e), but what
  about the nonlinear part?

27
MNM

• Consider a Taylor serious
• Hence the nonlinear part is O(e²) and therefor we
  obtain a splitting with a large linear but a
  relatively small nonlinear part

28
MNM

• Back to the complete equation
• There are two methods to bring the operators to
  the coarser grid
• Rediscretization
• Galerkin Coarsening
• We will use rediscretization only for the
  nonlinear part (though rediscretization might
  yield a bad approximation in case of a PDE with
  jumping coefficients, the influence for MNM is
  only O(e²)). Denote rediscretized operators by a
  head Â

29
MNM

• Now we obtain an iteration by defining
• Substituting this into the original equation and
  rearranging yields the defect equation for MNM

30
MNM

• As one can see, the following operators must be
  defined on the several levels
• While the first two will be simply rediscretized
  on each level, the third one is obtained by
  Galerkin coarsening
• To bring the current approximation to the next
  coarser level, we will use injection (as for FAS)

31
MNM - Algorithm

• MNM(u,N,L,f,l,?1,?2)
• if(l 0)
• solve NuLuf
• return u
• end
• Relax ?1 times equation
• NuLuf
• Compute residual
• rf-(NuLu)
• Construct linearized operator
• K LJ(u)
• Initialize coarse grid solution
• u_H injection(u)
• Galerkin Coarsening for linearized operator
• K_H galerkinCoarsening(K)

32
MNM Algorithm(II)

• Compute L for coarse grid
• L_H K_H J_H(u_H)
• Compute RHS for coarse grid
• f_H restriction(r) N_H u_H L_H u_H
• recursive call
• MNM(u_H,N_H,L_H,f_H,l-1,?1,?2)
• add correction
• u u prolongation(u_H - injection(u))
• postsmoothing
• NuLuf
• End
• Where L
• the linear correction to N

33
MNM Concrete Example

• Consider the equation
• For the approximated operators we obtain
• Here B is a scaling to ensure compatibility with
  the linearized coarse grid operator due to
  Galerkin coarsening

34
MNM - Adaptive

• Idea Use parameters to controll how much of
  FAS and Newton should be used
• Consider the complete coarse grid operator
• Two points of view
• The first term is the main term, second and third
  term are a nonlinear correction
• The second term is the main term, while the first
  and the third term are a linear correction

35
MNM - Adaptive

• Now we can use a weighting of the operators
• a1, b0 Newtons Mehtod
• a0, b1FAS
• a1b1, MNM

36
MNM Results

• 2-D diffusion problem
• where

37
MNM - Results
38
MNM Implementation hints

• Due to the Galerkin coarsening we have the
  restriction operator acting on the left hand side
  as well as on the right hand side hence it
  cancels out. But the prolongation operator is
  only on the right hand side, therefor we have to
  introduce a compatible scaling also for the
  rediscretized operators.
• Since N L is just an approximation of the fine
  grid operator (nonlinear), a nonlinear relaxation
  method is needed, such as for FAS (e.g.
  Gauss-Newton)

39
Conclusions(I)

• FAS and Newton-MG have both advantages and
  disadvantages
• MNM combines the good properties of both methods,
  but introduces difficulties due to scaling of the
  coarse grid approximations for the operators
• MNM yields usually fastest convergence factor of
  all three approaches
• Sometimes MNM does not converge, than
  backtracking can be used, but yields poor
  convergence

40
Conclusions(II)

• Adaptive MNM can be used instead of MNM with
  backtracking, yielding a quite good convergence
  factor
• The computational cost per V-Cycle for MNM is
  more expensive than for FAS or Newtons method,
  but less than the sum of both
• MNM is still a research topic
• MNM is more complicated to implement

41
Bibliography

• I. Yavneh and G Dardyk, A Multilevel Nonlinear
  Method, Haifa, 2005
• W. L. Briggs, V. E. Henson, and S F. McCormick, A
  Multigrid Tutorial, SIAM, Philadelphia, second
  ed., 2000
• V. E. Henson, Multigrid for nonlinear problems
  an overview, Center for Applied Scientific
  Computing Lawrence Livermore National Laboratory,
  2003