Timna: A Framework for Automatically Combining Aspect Mining Analyses

1
Timna A Framework for Automatically Combining
Aspect Mining Analyses

• David Shepherd1
• Jeffrey Palm2
• Lori Pollock1
• Mark Chu-Carroll3

1University of Delaware 2Northeastern
University 3IBM
2
Introduction - What is AOP?

• Aspect Oriented Programming

Each rectangle represents a source file
figures from aspectj.com
Red lines represent source code lines
implementing concept A
Language Support for Crosscutting Concerns (CCCs)
3
Introduction - A Closer Look at AOP
figures from aspectj.com

• Benefits of Refactoring into AOP
• Increased "ilities"
• readability
• maintainability
• extensibility
• Crosscutting concerns (CCCs) are explicit

Responsibility of AOP Community
4
Working Assumptions

• AOP can provide benefits in modularity for some
  problems
• Crosscutting Concerns are dangerous awareness is
  essential

5
Targeted Problem Mining

• Legacy Applications
• refactor into AOP

Aspect Mining is the process of finding these
candidates
figures generated by the AJDT
6
Targeted Problem Mining
Applications written with AOP, still problems...

• 3 different programmers implement 1 concept

7
State of the Art

• Researchers currently create a single analysis to
  perform aspect mining
• Examples
• Fan-in Analysis Marin et al, WCRE 05
• Code Clone Analysis Shepherd et al, SERP 05
• Dynamic Analysis Breu et al, ASE 04
• ...
• Little work done on combining analysis

8
State of the Art

• Fan-In

Good Candidate for Refactoring
    public void credit(float amount)        
AccessController.checkPermission(            
new BankingPermission("accountOperation"))
        _balance _balance amount      
    public void debit(float amount) throws
InsufficientBalanceException        
AccessController.checkPermission(            
new BankingPermission("accountOperation"))    
    if (_balance lt amount)             throw
new InsufficientBalanceException("Insufficient
total balance")         else            
_balance _balance - amount              
Marin et al, WCRE 04
checkPermission
Bad Candidate for Refactoring
    public void checkOut(SpecialList items)
SpecialIterator it items.iterator()
while(it.hasNext()) checkOut(it.next())
          public void markItems(SpecialList
items) SpecialIterator it
items.iterator() while(it.hasNext())
((Item)it.next()).mark()      
next
9
State of the Art
Conventional Transaction Management
public void debit(float amount) throws
InsufficientBalanceException UserTransaction
ut ... try ut.begin() ... business
logic ... ut.commit() catch (Exception ex)
ut.rollback() // rethrow after logging and
wrapping

• Clone Detection

Shepherd et al, SERP 05
public void credit(float amount)
UserTransaction ut ... try
ut.begin() ... business logic
... ut.commit() catch (Exception ex)
ut.rollback() // rethrow after logging and
wrapping
10
Remaining Challenges

• Combining Analyses
• if (code clone fan-in high), more likely to be
  a refactoring candidate
• Running a large number of analyses
• Methods with void return types
• Getters and setters
• ...

Humans do this during manual mining
Our framework (Timna) combines analyses to make a
decision
We invented several new analyses, use 11 total
analyses
11
Key Insight

• Currently, humans are the best miners.
• What is their process?
• Manual Inspection
• Learn to identify candidates in specific system
• Generalize to other systems
• Apply in other systems

12
Automated Approach

• Create Training Data
• Manual Tag Known Program
• Automatically Run Individual Mining Analyses
• Learn
• Output set of rules to classify
• boolean or categories
• Classify Unknown programs
• Output refactoring candidates

13
Approach - Learning

• Create Training Data
• Learn
• Apply

Create Training Data
Classification Table
Known Program
Manual Tagging
Mining Analyses
Augmented Classification Table
Learn Rules
Machine Learning
Classification Rules
14
Approach - Learning

• Create Training Data
• Learn
• Apply

Create Training Data

• If( Fan-in gt 5 and is-Void true ), then
  (refactor)
• If ( true ), then (don't refactor)

Classification Table
Program
Manual Tagging
Final Result Only output to Classifying Phase
Mining Analyses
Augmented Classification Table
Learn Rules
Machine Learning
Classification Rules
15
Approach - Classifying
Classify Unknown Program
Unknown Program
Mining Analyses
Augmented Classification Table
Completed Classification Table
Classification Table
Classifier
Classification Rules
16
Evaluation Questions

• Does combining analyses increase precision and
  recall?
• Are generated rules effective on other programs?
• Does categorical tagging increase performance?
• What is the (time) overhead?
• Can rules help direct research and evaluate new
  analyses?

Tagged in two different ways Boolean either
refactor or don't Categorical either don't
refactor or a reason (category) why to refactor
17
Experimental Setup

• Subject Programs
• Training Program (JHotDraw, 11K LOC)
• Testing Program (PetStore, 9K LOC)
• Steps
• Train
• Test on Training Program
• Test on Testing Program
• Metrics
• Precision and Recall
• Time

18
Experimental Results
Timna Cat
Timna Bool
Timna Cat
Timna Bool
Precision (number of good candidates returned)
/ (number of candidates returned)
Recall (number of candidates returned) /
(number of actual good candidates)
Program not tagged, so can't calculate recall
19
Experimental Results

• Does combining analyses increase precision and
  recall?

Why is Fan-In performing poorly? Single analyses
work well for specific cases, but fail to find
all aspects.
Timna Cat
Timna Bool
Timna Cat
Timna Bool
In this case, combining analyses does increase
precision and recall.
20
Experimental Results
2. Are generated rules effective on other
programs?
Timna Cat
Timna Bool
Timna Cat
Timna Bool
In this case, the rules effectively mine from the
testing program.
21
Experimental Results
3. Does categorical tagging increase performance?
Timna Cat
Timna Bool
Timna Cat
Timna Bool
In this case, the categorical tagging and the
boolean tagging perform similarly.
22
Experimental Results
4. What is the (time) overhead?
Analyze
Learn
6.24s
5m28s
Timna Cat
1.88s
5m28s
Timna Bool
2m04s
--
Timna Cat
2m04s
Timna Bool
--
Only done once
Can be done incrementally, at each compile/edit,
or overnight
Can be done incrementally, at each compile/edit,
or overnight
From these results, we believe Timna could be
integrated into an IDE without degrading response
time.
23
Experimental Results

• 5. Can rules help direct research and evaluate
  new analyses?
• WARE 05 elaborates on use in evaluating new
  analyses
• if analyses does not appear in rules, it is
  providing no new information
• Human readable rules can help define style

24
Contributions

• Technique to combine mining analyses to
  automatically identify refactoring candidates
• Demonstrated how to apply machine learning to
  learn good AOP style from canonical examples
• generate human readable rules
• Invented several (7) novel mining analyses during
  our initial use of Timna
• Experimentally shown evidence that combining
  analyses can improve performance

25
Possible Application
Provide hints, shaded by level of confidence
Implicit Rule Always create UndoActivity before
"executing" undoable
Possible Rule Clear Selection before changing
drawing
Of course! Whenever I change something in the
drawing, I should check to see if I damaged the
drawing. Moving this concept to an aspect can
eliminate a lot of similar calls from my OOP code.
Implicit Rule Check damage after changing drawing
AOSD-style programming with reduced costs and
burdens on the developer
26
Future Work

• Examine other aspect categorization
• "Sorts" (Marin et al, ICSM 05)
• Extend with additional analyses
• NLP-based analyses
• Apply to more unknown programs