Meta Optimization - PowerPoint PPT Presentation

1 / 28
About This Presentation
Title:

Meta Optimization

Description:

Improving Compiler Heuristics with Machine Learning ... exec. pr. ops. haz. Path. 11. 13. 9. 10. 13 ~0. 0.25. A-C-E-G. 0.02. 0.25. A-C-E-F-G. 1.44. 1.0. A-C-F-G ... – PowerPoint PPT presentation

Number of Views:23
Avg rating:3.0/5.0
Slides: 29
Provided by: mste5
Category:

less

Transcript and Presenter's Notes

Title: Meta Optimization


1
Meta Optimization
  • Improving Compiler Heuristics with Machine
    Learning

Mark Stephenson, Una-May OReilly, Martin Martin,
and Saman Amarasinghe MIT Computer Architecture
Group
2
Motivation
  • Compiler writers are faced with many challenges
  • Many compiler problems are NP-hard
  • Modern architectures are inextricably complex
  • Simple models cant capture architecture
    intricacies
  • Micro-architectures change quickly

3
Motivation
  • Heuristics alleviate complexity woes
  • Find good approximate solutions for a large class
    of applications
  • Find solutions quickly
  • Unfortunately
  • They require a lot of trial-and-error tweaking to
    achieve suitable performance

4
Priority Functions
  • A heuristics Achilles heel
  • A single priority or cost function often dictates
    the efficacy of a heuristic
  • Priority functions rank the options available to
    a compiler heuristic
  • Graph coloring register allocation (selecting
    nodes to spill)
  • List scheduling (identifying instructions in
    worklist to schedule first)
  • Hyperblock formation (selecting paths to include)

5
Machine Learning
  • We propose using machine learning techniques to
    automatically search the priority function space
  • Search space is feasible
  • Make use of spare computer cycles

6
Case Study I Hyperblock Formation
  • Find predicatable regions of control flow
  • Enumerate paths of control in region
  • Exponential, but in practice its okay
  • Prioritize paths based on several characteristics
  • The priority function we want to optimize
  • Add paths to hyperblock in priority order

7
Case Study I IMPACTs Function
8
Hyperblock Formation
  • What are the important characteristic of a
    hyperblock formation priority function?
  • IMPACT uses four characteristics
  • Extract all the characteristics you can think of
    and have a machine learning algorithm find the
    priority function

9
Hyperblock Formation
x1 Maximum ops over paths x2 Dependence height
x3 Number of paths x4 Number of operations
x5 Does path have subroutine calls? x6 Number of branches
x7 Does path have unsafe calls? x8 Path execution ratio
x9 Does path have pointer derefs? x10 Average ops executed in path
x11 Issue width of processor x12 Average predictability of branches in path
xN Predictability product of branches in path
10
Genetic Programming
  • GPs representation is a directly executable
    expression
  • Basically a lisp expression (or an AST)
  • In our case, GP variables are interesting
    characteristics of the program

11
Genetic Programming
  • Searching algorithm analogous to natural
    selection
  • Maintain a population of expressions
  • Selection
  • The fittest expressions in the population are
    more likely to reproduce
  • Sexual reproduction
  • Crossing over subexpressions of two expressions
  • Mutation

12
Genetic Programming
Create initial population (initial solutions)
  • Most expressions in initial population are
    randomly generated
  • It also seeded with the compiler writers best
    guesses

Evaluation
Generation of variants (mutation and crossover)
Selection
Generations lt Limit?
END
13
Genetic Programming
  • Each expression is evaluated by compiling and
    running benchmark(s)
  • Fitness is the relative speedup over the baseline
    on benchmark(s)

Create initial population (initial solutions)
Evaluation
Generation of variants (mutation and crossover)
Selection
Generations lt Limit?
END
14
Genetic Programming
  • Just as with Natural Selection, the fittest
    individuals are more likely to survive and
    reproduce.

Create initial population (initial solutions)
Evaluation
Generation of variants (mutation and crossover)
Selection
Generations lt Limit?
END
15
Genetic Programming
Create initial population (initial solutions)
Evaluation
Generation of variants (mutation and crossover)
Selection
Generations lt Limit?
END
16
Genetic Programming
  • Use crossover and mutation to generate new
    expressions

Create initial population (initial solutions)
Evaluation
Generation of variants (mutation and crossover)
Selection
Generations lt Limit?
END
17
Hyperblock ResultsCompiler Specialization
3.5
Train data set
Alternate data set
3
(add (sub (cmul (gt (cmul b0 0.8982 d17)d7))
(cmul b0 0.6183 d28)))
2.5
(add (div d20 d5) (tern b2 d0 d9))
2
Speedup
1.5
1.54
1.23
1
0.5
0
toast
Average
huff_enc
huff_dec
mpeg2dec
rawcaudio
rawdaudio
g721encode
g721decode
129.compress
18
Hyperblock ResultsA General Purpose Priority
Function
19
Cross ValidationTesting General Purpose
Applicability
20
Case Study II Register AllocationA General
Purpose Priority Function
21
Register Allocation ResultsCross Validation
22
Conclusion
  • Machine learning techniques can identify
    effective priority functions
  • Proof of concept by evolving two well known
    priority functions
  • Human cycles v. computer cycles

23
GP Hyperblock SolutionsGeneral Purpose
  • (add
  • (sub (mul exec_ratio_mean 0.8720) 0.9400)
  • (mul 0.4762
  • (cmul (not has_pointer_deref)
  • (mul 0.6727 num_paths)
  • (mul 1.1609
  • (add (sub
  • (mul (div num_ops dependence_height)
    10.8240)
  • exec_ratio)
  • (sub (mul (cmul has_unsafe_jsr
    predict_product_mean 0.9838)
  • (sub 1.1039 num_ops_max))
  • (sub (mul dependence_height_mean
    num_branches_max) num_paths)))))))

Intron that doesnt affect solution
24
GP Hyperblock SolutionsGeneral Purpose
  • (add
  • (sub (mul exec_ratio_mean 0.8720) 0.9400)
  • (mul 0.4762
  • (cmul (not has_pointer_deref)
  • (mul 0.6727 num_paths)
  • (mul 1.1609
  • (add (sub
  • (mul (div num_ops dependence_height)
    10.8240)
  • exec_ratio)
  • (sub (mul (cmul has_unsafe_jsr
    predict_product_mean 0.9838)
  • (sub 1.1039 num_ops_max))
  • (sub (mul dependence_height_mean
    num_branches_max) num_paths)))))))

Favor paths that dont have pointer dereferences
25
GP Hyperblock SolutionsGeneral Purpose
  • (add
  • (sub (mul exec_ratio_mean 0.8720) 0.9400)
  • (mul 0.4762
  • (cmul (not has_pointer_deref)
  • (mul 0.6727 num_paths)
  • (mul 1.1609
  • (add (sub
  • (mul (div num_ops dependence_height)
    10.8240)
  • exec_ratio)
  • (sub (mul (cmul has_unsafe_jsr
    predict_product_mean 0.9838)
  • (sub 1.1039 num_ops_max))
  • (sub (mul dependence_height_mean
    num_branches_max) num_paths)))))))

26
GP Hyperblock SolutionsGeneral Purpose
  • (add
  • (sub (mul exec_ratio_mean 0.8720) 0.9400)
  • (mul 0.4762
  • (cmul (not has_pointer_deref)
  • (mul 0.6727 num_paths)
  • (mul 1.1609
  • (add (sub
  • (mul (div num_ops dependence_height)
    10.8240)
  • exec_ratio)
  • (sub (mul (cmul has_unsafe_jsr
    predict_product_mean 0.9838)
  • (sub 1.1039 num_ops_max))
  • (sub (mul dependence_height_mean
    num_branches_max) num_paths)))))))

If a path calls a subroutine that may have side
effects, penalize it
27
Case Study I IMPACTs Algorithm
A
4k
24k
Path exec haz ops dep pr
A-B-D-F-G 0 1.0 13 4 0
A-B-F-G 0.14 1.0 10 4 0.21
A-C-F-G 0.79 1.0 9 2 1.44
A-C-E-F-G 0.07 0.25 13 5 0.02
A-C-E-G 0 0.25 11 3 0
B
C
4k
22k
2k
10
E
D
2k
25
10
F
28k
G
28k
28
Case Study I IMPACTs Algorithm
A
4k
24k
Path exec haz ops dep pr
A-B-D-F-G 0 1.0 13 4 0
A-B-F-G 0.14 1.0 10 4 0.21
A-C-F-G 0.79 1.0 9 2 1.44
A-C-E-F-G 0.07 0.25 13 5 0.02
A-C-E-G 0 0.25 11 3 0
B
C
4k
22k
2k
10
E
D
2k
25
10
F
28k
G
28k
Write a Comment
User Comments (0)
About PowerShow.com