Project 1: Classification Using Neural Networks

1
Project 1 Classification Using Neural Networks
Artificial Intelligence

• 2009. 03. 23
• Kim, Kwonill
• kikim_at_bi.snu.ac.kr Biointelligence laboratory

2
Contents

• Project outline
• Description on the data set
• Description on tools for ANN
• Guide to Writing Reports
• Style
• Mandatory contents
• Optional contents
• Submission guide / Marking scheme
• Demo

3
Outline

• Goal
• Understand MLP machine learning deeper
• Practice researching and technical writing
• Handwritten digits problem (classification)
• To predict the class labels (digits) of
  handwritten digit data set
• Using Multi Layer Perceptron (MLP)
• Estimating several statistics on the dataset
• Data set
• Variation of the Handwritten digit data set
• http//archive.ics.uci.edu/ml/datasets/Pen-BasedR
  ecognitionofHandwrittenDigits

4
Handwritten Digit Data Set (1/2)

• Original Data Set Description
• Digit database of 11,000 samples from every 44
  writers
• http//archive.ics.uci.edu/ml/datasets/Pen-BasedR
  ecognitionofHandwrittenDigits
• 16 attributes
• (xt, yt), t 1, , 8
• 0 100
• Label (Class)
• 0, 1, 2, , 9

5
Handwritten Digit Data Set (2/2)

• Constitution
• Preprocessed data (.arff, .csv)
• Total data (pendigits_total_set, 1099)
  training data (pendigits_training, 749) test
  data (pendigits_test, 350)
• Data description (pendigits.names)
• For WEKA (.arff)

6
Tools for Experiments with ANN

• Source codes - Choose anything!!
• Free software ? Weka (recommended)
• MATLAB tool box (Toolboxes ? Neural Network)
• ANN libraries (C, C, JAVA, )
• Web sites
• http//www.cs.waikato.ac.nz/ml/weka/
• http//www.faqs.org/faqs/ai-faq/neural-nets/part5/
  

7
Reports Style

• English only!!
• Scientific journal-style
• How to Write A Paper in Scientific Journal Style
  and Format
• http//abacus.bates.edu/ganderso/biology/resource
  s/writing/HTWsections.html

 Experimental process  Section of Paper
What did I do in a nutshell?  Abstract
 What is the problem? Introduction
 How did I solve the problem?  Materials and Methods
 What did I find out?  Results
 What does it mean?  Discussion
 Who helped me out?  Acknowledgments (optional)
 Whose work did I refer to?  Literature Cited
 Extra Information Appendices (optional)
8
Report Contents Mandatory (1/2)

• System description
• Used software and running environments
• Result graphs and tables
• Analysis discussion (Very Important!!)

9
Report Contents Mandatory (2/2)

• Basic experiments
• Changing of epochs (Draw learning curve)
• Various of Hidden Units

Hidden Units Train Train Train Test Test Test
Hidden Units Average ? Std. Dev. Best Worst Average ? Std. Dev. Best Worst
Setting 1 accuracy
Setting 2
Setting 3
??? ??? ??? ??? ??? ??? ???
10
Report Contents Optional

• Various experimental settings
• Normalization
• Learning rates
• Structure of MLP
• Feature selection
• Activation functions
• Learning algorithm
• Evaluation techniques
• ROC curve
• k-fold Crossvalidation

11
Submission Guide

• Due date Apr. 15th (Wed.) 1500
• Submit both hardcopy and email
• Hardcopy submission to the office (301-419 )
• E-mail submission to jakramate_at_bi.snu.ac.kr
• Subject AI Project1 Report Student number,
  Name
• Length report should be summarized within 12
  pages.
• If you build a program by yourself, submit the
  source code with comments
• We are NOT interested in the accuracy and your
  programming skill, but your creativity and
  research ability.
• If your major is not a C.S, team project with a
  C.S major student is possible (Use the class
  board to find your partner and notice the
  information of your team to the 1st project
  TA(jakramate_at_bi.snu.ac.kr) by Mar. 27th)

12
Marking Scheme

• 40 points for experiment analysis
• Extra 4 points for additional expriments
• 20 points for report
• 6 points for overall organization
• Late work
• - 10 per one day
• Maximum 7 days
• The Maximum Score is Changed

13
References

• Materials about Weka
• Weka GUI guide (PPT)
• Explorer guide (PDF)
• Experimenter guide (PDF)

14
WEKA Demo
15
Matlab
16
QnA

• MLP is the simplest form of contemporary neural
  networks. (you can see other forms in the ANN
  section of Wikipedia http//en.wikipedia.org/wiki
  /Artificial_neural_network)
• Neural network is sometimes called as ANN
  (artificial neural network) to stress the
  difference with the original neural network in
  the brain or central nervous system.
• Learning in neural networks consists in the
  optimization of weights by gradient descent
  process. To get the global optimum, we need to
  try not just several configurations of
  parameters, but also various random starting
  points.
• When you use weka, you need to try several
  randomSeed for this reason