GATree Genetically Evolved Decision Trees

1
GATree Genetically Evolved Decision Trees

• Papagelis Athanasios - Kalles
  DimitriosComputer Technology Institute

2
Introduction

• We use GAs to evolve simple and accurate binary
  decision trees
• Simple genetic operators over tree structures
• Experiments with UCI datasets
• very good size
• competitive accuracy results

3
Why it should work ?

• GAs are not
• Hill climbers
• Blind on complex search spaces
• Exhaustive searchers
• Extremely expensive
• They are
• Beam searchers
• They balance between time needed and space
  searched

4
The question

• Are there datasets where hill-climbing techniques
  are really inadequate ?
• e.g unnecessary big misguiding output
• Yes there are
• Conditionally dependent attributes
• e.g XOR
• Irrelevant attributes
• Many solutions that use GAs as a preprocessor so
  as to select adequate attributes
• Direct genetic search can be proven more
  efficient for those datasets

5
The proposed solution

• Select the desired decision tree characteristics
  (e.g small size)
• Create an appropriate fitness function
• Adopt a decision tree representation with
  appropriate genetic operators
• Evolve for as long as you wish!

6
Genetic operators
7
Payoff function

• Balance between accuracy and size

• set x depending on the desired output
  characteristics.
• Small Trees ? ? x near one
• Emphasis on accuracy ? ? x grows big

8
Results
9
Future work

• Minimize evolution time
• Improved node statistics
• Choose the output class using a majority vote
  over the produced tree forest
• Dynamic tuning of initial parameters
• Experiments with synthetic datasets
• Specific characteristics