Chapter III Instruction Level Parallelism

1
Chapter IIIInstruction Level Parallelism

• Soner Onder
• Michigan Technological University, Houghton MI

2
Outline

• Instruction level parallelism, concepts.
• Data dependencies
• Control dependencies
• Overcoming data hazards with dynamic scheduling
  (Tomasulos algorithm).
• Overcoming control hazards with dynamic branch
  prediction.
• Basic two bit predictors.
• Corrolating predictors.

3
Forms of parallelism

• Process-level
• How do we exploit it? What are the challenges?
• Examples?
• Thread-level
• How do we exploit it? What are the challenges?
• Examples?
• Loop-level
• What is really loop level parallelism? What
  percentage of a programs time is spent inside
  loops?
• Instruction-level
• What is shaping our future.

Coarse grain
Human intervention?
Fine Grain
4
Loop level parallelism

• Consider the following loop from your book
• for( i1 i lt 1000 i)
• Xi Xi Yi
• What is wrong with it?

• How about this one?
• for( i1 i lt 1000 i)
• Xi Xi-1 Yi
• What is it called?

5
Instruction-level parallelism (ILP)

• Briefly, ability to execute more than one
  instruction simultaneously.
• In order to achieve this goal, we should not have
  dependencies among instructions which are
  executing in parallel
• - H/W terminology Data hazards
• (I.e. RAW WAR WAW)
• - S/W terminology Data dependencies
• Name true dependencies

6
ILP Challenges

• How many instructions can we execute in parallel?
• Basic block What is between two branch
  instructions
• Typical MIPS programs have 15-25 branch
  instruction
• One every 4-7 instructions is a branch.
• How many of those are likely to be data dependent
  on each other?
• We need the means to exploit parallelism across
  basic blocks. What stops us from doing so?

7
Dependences and hazards

• Dependences are a property of programs.
• If two instructions are data dependent they
  cannot execute simultaneously.
• Whether a dependence results in a hazard and
  whether that hazard actually causes a stall are
  properties of the pipeline organization.
• Data dependences may occur through registers or
  memory.

8
Dependences and hazards

• The presence of the dependence indicates the
  potential for a hazard, but the actual hazard and
  the length of any stall is a property of the
  pipeline. A data dependence
• Indicates that there is a possibility of a
  hazard.
• Determines the order in which results must be
  calculated, and
• Sets an upper bound on the amount of parallelism
  that can be exploited.

9
Dependencies
Name dependencies
Output dependence
Anti-dependence
Data
True dependence
Control
Do you remember the hazards they may lead to?
10
Name dependences

• Output dependence
• When instruction I and j write the same register
  or memory location. The ordering must be
  preserved to leave the correct value in the
  register
• add r7,r4,r3
• div r7,r2,r8
• Antidependence
• When instruction j writes a register or memory
  location that instruction i reads
• i add r6,r5,r4
• j sub r5,r8,r11

11
Data dependences

• Data dependence, true dependence, and true data
  dependence are terms used to mean the same thing
  
• An instruction j is data dependent on
  instruction i if either of the following holds
• instruction i produces a result that may be used
  by instruction j, or
• instruction j is data dependent on instruction k,
  and instruction k is data dependent on
  instruction i.
• Chains of dependent instructions.

12
Data Dependences through registers/memory

• Dependences through registers are easy
• lw r10,10(r11)
• add r12,r10,r8
• just compare register names.
• Dependences through memory are harder
• sw r10,4 (r2)
• lw r6,0(r4)
• is r24 r40 ? If so they are dependent, if
  not, they are not.

13
Control dependences

• An instruction j is control dependent on i if the
  execution of j is controlled by instruction i.
• I If a lt b
• j aa1 j is control dependent on I.
• 1. An instruction that is control dependent on a
  branch cannot be moved before the branch so that
  its execution is no longer controlled by the
  branch.
• 2. An instruction that is not control dependent
  on a branch cannot be moved after the branch so
  that its execution is controlled by the branch.

14
Preserving the exception behavior

• A simple pipeline preserves control dependences
  since it executes programs in program order.
• DADDU r2,r3,r4
• beqz r2,l1
• lw r1,0(r2)
• l1
• Can we move lw before the branch? (Dont worry,
  it is OK to violate control dependences as long
  as we can preserve the program semantics)

15
Preserving the exception behavior

• Corollary
• Any changes in the ordering of instructions
  should not change how exceptions are raised in a
  program.

16
Preserving the data flow

• Consider the following example
• daddu r1,r2,r3
• beqz r4,L
• dsubu r1,r5,r6
• L
• or r7,r1,r8
• What can you say about the value of r1 used by
  the or instruction?

17
Preserving the data flow

• Corollary
• Preserving data dependences alone is not
  sufficient when changing program order. We must
  preserve the data flow.
• These two principles together allow us to execute
  instructions in a different order and still
  maintain the program semantics.
• This is the foundation upon which ILP processors
  are built.

18
Dynamic scheduling

• Consider the example
• div.d f0,f2,f4
• add.d f10,f0,f8
• sub.d f12,f8,f14
• Where are the data dependences?

19
Dynamic scheduling

• Consider the example
• div.d f0,f2,f4
• add.d f10,f0,f8
• sub.d f12,f8,f14

sub.d
add.d
div.d
What happens on a machine with an in-order
pipeline?
20
Dynamic scheduling

• Consider the example
• div.d f0,f2,f4
• add.d f10,f0,f8
• sub.d f12,f8,f14
• What happens on a machine with an out-of-order
  pipeline?

add.d
div.d
sub.d
21
Dynamic scheduling
Program order
Completed
add.d
div.d
div.d
add.d
sub.d
sub.d
executing
22
Dynamic scheduling
Program order
Completed
add.d
div.d
sub.d
div.d
add.d
sub.d
executing
23
Dynamic scheduling
Program order
Completed
add.d
sub.d
div.d
div.d
add.d
sub.d
executing
24
Dynamic scheduling
Program order
Completed
sub.d
div.d
div.d
add.d
add.d
sub.d
executing
Note that The program now executed faster But
instructions completed in a different order.
25
Imprecise exceptions

• An exception is imprecise if the processor state
  when an exception is raised does not look exactly
  as if the instructions were executed sequentially
  in strict program order
• The pipeline may have already completed
  instructions that are later in program order than
  the instruction causing the exception (we may
  have div.d raising an exception but have sub.d
  completed already).
• The pipeline may have not yet completed some
  instructions that are earlier in program order
  than the instruction causing the exception.
• We will look into how to handle imprecise
  exceptions later.