Optimisation and the NAG Toolbox for MATLAB

1
Optimisation and the NAG Toolbox for MATLAB

• University of Warwick
• Mick Pont, John Holden - NAG

2
Agenda

• NAG licences at Warwick  
• NAG Toolbox for MATLAB
• Optimization and other topics

3
Licences at the University of Warwick

• Site Licence for eight implementations
• Site contact patrick.ryan_at_warwick.ac.uk
• NAGs Serial Libraries (Fortran 77, 90, C DMC)
• REVIEW OF LICENCES UNDERWAY PROBABLE CHANGES
• Windows (Fortran and C DLLs)
• 32-bit Linux (Fortran-g77 and C-gcc)
• 64-bit Linux (Fortran and C)
• NAG Toolbox for MATLAB
• Licence upgrade in progress
• Maple-NAG Connector
• collaborative licence opportunities contact NAG
  direct

4
What does the University licence cover?

• Unlimited use for the licensed implementations
• As long as for academic or research purposes
• Installation may be on any university, staff or
  student machine
• Full access to NAG Support support_at_nag.co.uk
• Need to use an _at_xxxx.ac.uk e-mail address
• Our software
• Includes online documentation - also
  www.nag.co.uk
• Supplied with extensive example programs
• data and results

5
Technical Agenda

• The NAG Engine
• Wrapper functionality
• Algorithmic contents
• Navigating around the NAG toolbox in MATLAB
• NAG Optimization Chapters
• Random numbers

6
The NAG Engine
NAG software is based on NAG Engine technology
NAG C Library
NAG Fortran Library
NAG Engine (algorithmic repository)
User-callable library routines are thin wrappers
NAG Toolbox for MATLAB
NAG SMP Library
Other NAG Software
7
NAG Libraries Ease of Integration

• C (various)
• C / .NET
• Visual Basic
• Java
• Borland Delphi
• Python
• and more

• Excel
• MATLAB
• Maple
• LabVIEW
• R and S-Plus
• SAS
• Simfit
• and more

8
NAG Toolbox for MATLAB

• Built as MATLAB mex files
• Auto-generated from XML documentation
• Contains essentially all NAG functionality
• not a subset
• Currently runs under Windows (32-bit) or Linux
  (32/64-bit)
• Installed under the usual MATLAB toolbox
  directory
• Makes use of a DLL or shared version of the NAG
  Library

ltstart MATLAB heregt
9
NAG Library Contents

• Root Finding
• Summation of Series
• Quadrature
• Ordinary Differential Equations
• Partial Differential Equations
• Numerical Differentiation
• Integral Equations
• Mesh Generation
• Interpolation
• Curve and Surface Fitting
• Optimization
• Approximations of Special Functions

• Dense Linear Algebra
• Sparse Linear Algebra
• Correlation and Regression Analysis
• Multivariate Analysis of Variance
• Random Number Generators
• Univariate Estimation
• Nonparametric Statistics  
• Smoothing in Statistics  
• Contingency Table Analysis  
• Survival Analysis 
• Time Series Analysis
• Operations Research

10
NAG naming scheme

• Based on ACM modified SHARE classification
• John Bolstad - A proposed classification scheme
  for computer program libraries. ACM SIGNUM
  Newsletter, November 1975
• References Nottingham (England) Algorithms
  Group, First Annual Report, June, 1974.
• Routines separated into chapters
• Each routine has a 5-letter base name for ease of
  classification, e.g. (MATLAB/Fortran / C)
• c06eb / C06EBF / c06ebc
• nag_fft_hermitian (long name for readability)
• Each routine is independently documented, with
  complete example program

11
Chapter e04 Minimization / Maximization
Problem minimize F(x1, x2, , xn)
possibly subject to constraints The function F(x)
is called the objective function. We wish to
determine x, the n-vector of variables. May have

• No constraints
• Bound constraints li lt xi lt ui
• Linear or nonlinear constraints l lt G(x) lt u

12
Unconstrained optimization
13
Linearly constrained optimization
14
Nonlinear constraints
15
Chapter e04
Problems categorized according to properties of
objective function

• nonlinear
• sum of squares of nonlinear functions
• quadratic
• sum of squares of nonlinear functions
• linear

Example nonlinear objective and
constraints Minimize f(x,y) (1 - x)2 100(y
- x2)2 subject to x2 y2 lt 2 -2 lt x lt
2
16
E04WD

• Sequential quadratic programming (SQP) algorithm
• obtains search directions from a sequence of QP
  subproblems.
• designed for problems with many variables and
  constraints
• P. Gill (San Diego), W. Murray (Stanford) and M.
  Saunders (Stanford)

17
Chapter e04
It is important to choose a method appropriate to
your problem type, for efficiency and the best
chance of success. NAG documentation is
comprehensive for advice see the Chapter
Introduction for e04 www.nag.co.uk/numeric/FL
/manual/pdf/E04/e04_intro.pdf
www.nag.co.uk/numeric/CL/nagdoc_cl08/pdf/E04/e04_i
ntro.pdf
ltrun rosenbrock_sd_demo, rosenbrock_sqp _demo,
rosenbrock_lsq _demo heregt
18
Some routines available in Chapter e04

• e04ab minimize a function of one variable
• e04dg minimization using conjugate gradients
• e04mf linear programming
• e04nc linear least-squares
• e04nf quadratic programming
• e04nq LP or QP (for sparse problems)
• e04un nonlinear least-squares
• e04vh general sparse constrained nonlinear
• e04wd general nonlinear all-purpose
• etc.

19
New optimization coming at next Mark
Currently many optimization routines in NAG, but
these have all been for local optimization. No
guarantee about which minimum (or maximum) is
returned.
20
Local optimization
21
Global requirements
Users often ask for global optimization
methods. In next releases of NAG Libraries we
will have software based on 'multilevel
coordinate search' (MCS) method - Huyer and
Neumaier http//www.mat.univie.ac.at/neum/ms/
mcs.pdf Search space is recursively split into
sub-boxes, looking for child boxes where gain in
objective is expected. Boxes swept through in
turn, perhaps being split, until a box with
maximum level exists. Then a local search is
performed. Already in NAG Engine - new Chapter
e05 Beta available now on request
22
New NAG Chapter E05

• Main routine named E05JB
• Plus initialization and option setting routines
• Currently handles only bound constraints

Minimize f(x1, x2 xn) Subject to li lt xi lt
ui
ltrun e05jb_demo heregt
23
Chapter g05 Random Numbers
Random numbers used to model real-life
processes. Humans are bad at choosing them.

• e.g. faking a random sequence of coin tosses is
  very difficult HTHHTHTTHTHHTTTHTTHHTTHTHHTHTH
• Most people would choose a sequence easily proved
  not to be random
• (Do you play the lottery?)

Therefore good algorithms are required.
24
Chapter g05 Random Numbers
Random numbers from non-uniform distributions

• Generated from uniform numbers
• Transformation methods
• Rejection methods
• Table search methods

All the likely suspects are there

• Normal (Gaussian) distribution
• Students t distribution
• Beta and exponential
• Chi-squared and binomial
• Geometric, poisson, F, Gamma,

25
Some routines available in Chapter g05

• g05ca pseudo-random from uniform distribution
• g05cb / g05cc initialize generator to repeatable
  or non-repeatable sequence respectively
• g05db exponential distribution
• g05dd normal distribution
• g05fe beta distribution
• g05ff gamma distribution
• g05ya quasi-random numbers
• Other distributions, plus GARCH, copula, etc.

26
Pseudo- versus Quasi-Random Numbers
Pseudo-random numbers

• generated systematically
• e.g. multiplicative congruential ni a ni-1 mod
  m
• properties close to true random numbers
• (assuming that a and m are chosen wisely)
• negligible correlation between consecutive numbers

Quasi-random numbers

• not statistically independent
• give more even distribution in space (looks more
  random)
• useful for Monte Carlo integration

ltrun random_demo heregt
27
Other NAG software

• Wed welcome contact on our other software
• Maple-NAG Connector
• NAG's Excel related products
• Fortran Builder (NAGs Windows Fortran compiler)
• New library functionality
• NAGs High Performance libraries
• ..

28
NAG Toolbox for MATLAB - summary

• Runs under Windows and Linux
• Currently on beta test
• Downloadable from NAG web site
• Can be used with MATLAB compiler
• Release version coming soon

Questions?
sales_at_nag.co.uk or support_at_nag.co.uk
www.nag.co.uk/about/careers.asp