Hazards in a Pipeline

1
Hazards in a Pipeline

• What is a pipeline hazard?
• A situation that prevents the Normal Flow of
  Execution Sequence In an Instruction Pipeline
  i.e. which prevents an instruction from executing
  during its designated clock cycles.
• There are 3 classes of hazards
• Structural Hazards.
• Data Hazards.
• Control Hazards.

2
Structural Hazards

• Arise from resource conflicts among instructions
  executing concurrently
• Same resource is required by two (or more)
  concurrently executing instructions at the same
  time.
• The Resources under consideration happen to be
  ALL Hardware Resources of the following types
• Memory .
• Various CPU Registers.
• ALU.

3
Common methods for eliminating structural hazards

• Duplicate resources
• Pipeline the resource
• Reorder the instructions
• It may be too expensive to eliminate a structural
  hazard, in which case the pipeline should stall
• No new instruction is issued until the hazard has
  been resolved

4
Possible Structural Hazard in a 5 Stage Pipeline
Example - 1
IF
ID
EXE
MEM
WB
IF
ID
EXE
MEM
WB
IF
ID
EXE
MEM
WB
IF
ID
EXE
MEM
WB
Instruction Fetch IF Operand Read MEM
trying to ACCESS same Memory
5
Resolving IF vs MEM Hazard

• Spilt the MEMORY into two parts namely
  Instruction Memory IM and Data Memory DM
  at the ISA Level.
• Implemented by split Level 1 L1 Cache at the
  Micro Architecture Level.
• Requires separate dedicated set of System
  Buses as well as separate Address Data
  Registers within the CPU. Havard Architecture
  Model ?!

6
The Principle Adopted

• Providing Additional Resources.
• Increased Hardware Cost.
• Benefit Avoiding MOST frequent conflicts.
• Problem
• a) Is there a Possibility of such a conflict
  at the Lower Levels of Memory Hierarchy ?
• b) How to resolve that anyway ?

7
Resources involved in a 5 Stage Pipeline
Reg
Mem
DM
Reg
Reg
Mem
DM
Reg
Instruction 1
Reg
Mem
DM
Reg
Instruction 2
Reg
Mem
DM
Reg
Instruction 3
Reg
Mem
DM
Reg
Instruction 4
Time
Any Possibility of a Structural Hazard from
this Point Onwards ?
8
Pipeline Stages vs. Resources in Use

• Stage 1.Instruction Fetch (IF) Instruction
  Pointer , Instruction Memory, Instruction Memory
  Address Register, Instruction Register WRITE,
  Update Instruction Pointer.
• Stage 2. Instruction Decode (ID) , Instruction
  Register READ, Control Unit, CPU Registers READ,
  ALU Operand Registers WRITE.
• Stage 3. Instruction Execute (EX) Control
  Unit, ALU , ALU Result Register WRITE.
• Stage 4. Data Memory Access (MEM) Control
  Unit, Data Memory Address Register, Data Memory,
  Memory Buffer Register.
• Stage 5. Write Back (WB) Control Unit, CPU
  Register, Data Memory Address Register, Data
  Memory, Memory Buffer Register .

9
Possible Structural Conflict Ways to Resolve
it

• Instruction I LOAD rj, M1 Requires
  Memory Access in Stage 4 and Register Access in
  Stage 5 (WB) Stage
• Instruction I1 STORE M2 , rk will
  require Memory Access First , may overlap with
  the Memory Access of Previous Instruction.
• Possible Ways to Resolve
• 1) How can Hardware Resolve it ?
• 2) How can Compiler Resolve it?
• 3) How can Operating System Resolve it?

10
Hardware Techniques to Deal with Hazards

• Simple solution
• Stall pipeline.
• Pipeline stall
• Lets some instruction(s) in pipeline proceed,
  others are made to wait for data, resource, etc.

11
Resolving through STALL
Reg
Mem
DM
Reg
LOAD
Reg
Mem
DM
Reg
Instruction 1
No Instruction COMPLETED
Reg
Mem
DM
Reg
Instruction 2
Stall / NOP Bubbles
Bubble
Bubble
Bubble
Bubble
Bubble
Reg
Mem
DM
Reg
Instruction 3
A Pipeline can be stalled by inserting a
bubble or NOP
Time
12
A Likely Pipeline Stall For the given Example
Clock Number

• STORE introduces a STALL that is propagated
  through the Succeeding Pipeline Stages.
• Thus, no instruction completes on clock cycle 9.

13
Structural Hazard Example

• Occurs when two or more instructions need the
  same resource

• A pipeline stalled for a structural hazard when
  both instruction and data shares the same memory

14
Performance with Stalls

• Stalls degrade performance of a pipeline
• Result in deviation from 1 instruction
  executing/clock cycle.
• Lets examine by how much stalls can impact CPI

15
Stalls and Performance

• CPI pipelined
• Ideal CPI Pipeline stall cycles per
  instruction
• 1 Pipeline stall cycles per instruction
• Ignoring overhead and assuming stages are
  balanced (?!)

16
Speedup Due to Pipelining
17
Alternate Speedup Expression
18
An Example of Performance Impact of Structural
Hazard

• Assume
• Pipelined processor.
• Data references constitute 40 of an instruction
  mix.
• Ideal CPI of the pipelined machine is 1.
• Consider two cases
• Unified data and instruction cache vs. separate
  data and instruction cache.
• What is the impact on performance?

19
An Example

• Avg. Inst. Time CPI x Clock Cycle Time
• (i) For Separate cache Avg. Instr. Time111
• (ii) For Unified cache case
• (1 0.4 x 1) x (Clock cycle timeideal)
• 1.4 x Clock cycle timeideal1.4
• Speedup 1/1.4
• 0.7
• 30 degradation in performance

20
Program Dependences Can Cause Hazards!

• Hazards can be caused by dependences within a
  program.
• There are three main types of dependences in a
  program
• Data dependence.
• Name dependence.
• Control dependence.

21
Data Dependences

• An instruction j is data dependent on instruction
  i, iff either of
• 1) Direct Instruction i produces a result that
  is used by instruction j.
• 2) Transitive
• Instruction j is data dependent on instruction
  k and Instruction k is data dependent on
  instruction i.

r3 ? r1 op r2 r5 ? r3 op r4
r3 ? r1 op r2 r4 ? r3 op r2 r5 ? r6 op
r4
22
Types of Data Dependences

• Two types of data dependences
• True Data Dependence. Will Cause DATA Hazards
  The Output generated by one Instruction is being
  used by the Next Instruction as Input Read After
  Write (RAW).
• Name Dependence
• Two instructions use the same register or memory
  location (called a name).
• There is no true flow of data between the two
  instructions.
• Example ABC APQ
• Two types of name dependences
• i. Anti-dependence. Write After Read
  (WAR)
• ii. Output dependence. Write After
  Write (WAW)

23
Classification of Data Hazards

• Let I be an earlier instruction, J a later one
• RAW (read after write)
• J tries to read a value before I writes into it
• WAW (write after write)
• I and J write to the same place, but in the wrong
  order
• Only occurs if more than one pipeline stage can
  write (in-order)
• WAR (write after read)
• J writes a new value to a location before I has
  read the old one
• Only occurs if writes can happen before reads in
  pipeline (in-order)

24
Detecting Data Dependences

• A data value may flow between instructions
• (i) through registers .
• (ii) through memory locations.
• When data flow is through a register
• Detection is rather straight forward.
• When data flow is through a memory location
• Detection is difficult.
• Two addresses may refer to the same memory
  location but look different.
• 100(R4) and 20(R6)

25
Data Hazards

• Data hazards are caused by data dependences
• But, mere existence of data dependence in a
  program may not result in a data Hazard.

26
Data Hazard

• Arises out of change of relative timing of
  instructions by overlapping their execution
• Example
• add r1, r2, r3
• sub r4, r1, r5
• and r6, r1, r7
• or r8, r1, r9
• xor r10, r1, r11

27
Data Hazards

• Occur when an instruction under execution depends
  on
• Data from an instruction ahead in pipeline.
• Example
• Dependent instruction uses old data
• Results in wrong computations

ABC DAE
IF
ID
EXE
MEM
WB
ABC
IF
ID
EXE
MEM
WB
DAE
28
Data Hazard Example
29
Read after Write (RAW) Hazards

• RAW hazard
• instruction j tries to read its operand before
  instruction i writes it.
• j would incorrectly receive an old or incorrect
  value.
• Example

i ADD R1, R2, R3 j SUB R4, R1, R6
Instruction j is a read instruction issued after i
Instruction i is a write instruction issued
before j
30
RAW Dependency More Examples

• Example program (a)
• i1 load r1, addr
• i2 add r2, r1,r1
• Program (b)
• i1 mul r1, r4, r5
• i2 add r2, r1, r1
• Both cases, i2 does not get operand until i1 has
  completed writing the result
• In (a) this is due to load-use dependency
• In (b) this is due to define-use dependency

31
Output Dependence Write after Write (WAW)
Hazards

• Output dependence occurs between two instructions
  i and j, iff
• The two instructions write to the same memory
  location.
• Ordering of the instructions must be preserved to
  ensure
• Finally written value corresponds to j.
• Example
• ADD R3 , R1, R2 R3 ? R1 PLUS R2
• SUB R5 , R3, R4 R5 ? R3 MINUS R4
• ADD R3, R6 , R7 R3? R6 PLUS R7

32
Write After Write (WAW) Hazards

• WAW hazard
• instruction j tries to write an operand before
  instruction i writes it.
• Writes are performed in wrong order.
• Example

(How can this happen???)
i DIV F1, F2, F3 j SUB F1, F4, F6
Instruction j is a write instruction issued after
i
Instruction i is a write instruction issued
before j
33
Write After Read (WAR) Hazards

• WAR hazard
• instruction j tries to write an operand before
  instruction i reads it.
• Instruction i instead of getting old value,
  receives newer, undesired value
• Example

i DIV F7, F1, F3 j SUB F1, F4, F6
Instruction j is a write instruction issued after
i
Instruction i is a read instruction issued before
j
34
WAR Hazards

• Called an anti-dependence by compiler writers.
• This results from reuse of the name (register)
  F1.
• Cant happen in MIPS 5 stage pipeline because
• All instructions take 5 stages, and
• Reads are always in stage 2, and
• Writes are always in stage 5.

35
WAR and WAW Dependency More Examples

• Example program (a)
• i1 mul r1, r2, r3
• i2 add r2, r4, r5
• Example program (b)
• i1 mul r1, r2, r3
• i2 add r1, r4, r5
• Both cases have dependence between i1 and i2
• in (a) r2 must be read before it is written into
• in (b) r1 must be written by i2 after it has been
  written into by i1

36
Is There Really a Dependence/Hazard ?

• Exercise
• i1 mul r1, r2, r3
• i2 add r2, r4, r5

37
Inter-Instruction Dependences

• ? Data dependence
• r3 ? r1 op r2 Read-after-Write
• r5 ? r3 op r4 (RAW)
• ? Anti-dependence
• r3 ? r1 op r2 Write-after-Read
• r1 ? r4 op r5 (WAR)
• ? Output dependence
• r3 ? r1 op r2 Write-after-Write
• r5 ? r3 op r4 (WAW)
• r3 ? r6 op r7
• Control dependence
• To be Taken up Later

False Dependency
38
Data Dependencies Summary
Data dependencies in straight-line code
RAW Read After Write dependency
WAR Write After Read dependency
WAW Write After Write dependency
( Flow dependency )
( Anti dependency )
( Output dependency )
Load-Use dependency
Define-Use dependency
True dependency
False dependency
Cannot be overcome
Can be eliminated by register renaming
39
Dependences and Hazards
RAW
True
WAW
Data
Name
Output
WAR
Anti
Control
Control
Structural
------
40
Techniques to Reduce Data Hazards

• Forwarding and bypassing
• Basic compiler pipeline scheduling
• Basic dynamic scheduling
• Dynamic scheduling with renaming
• Hardware speculation

41
Hazard Resolution

• Name dependences
• Once identified, can be easily eliminated through
  simple compiler renaming techniques.
• Memory-related dependences are difficult to
  identify
• Hardware techniques (scoreboarding and dynamic
  instruction scheduling) are being used.
• True data dependences
• More difficult to handle.
• Can not be eliminated can only be overcome!
• Many techniques have evolved over the years. (?!)

42
Out-of-order Pipelining
IF



ID



ALU



INT
Fadd1
Fmult1
LD/ST
EX
Fadd2
Fmult2
Fmult3
WB



43
A Solution to WAR and WAW Hazards

• Rename Registers
• i1 mul r1, r2, r3
• i2 add r6, r4, r5
• Register renaming can get rid of most false
  dependencies
• Compiler can do register renaming in the register
  allocation process (i.e., the process that
  assigns registers to variables).

44
Use of Compiler Techniques to Tackle Data hazards

• A compiler can help eliminate some stalls caused
  by data hazards
• Example an instruction that uses result of a
  LOADs destination register should not
  immediately follow the LOAD instruction.
• The technique is called
• compiler-based pipeline instruction scheduling

45
Resolving Data Hazards by the Hardware

• What causes them?
• Pipelining changes the order of read/write
  accesses to operands.
• Order differs from that of an unpipelined
  machine.
• Example
• ADD R1, R2, R3
• SUB R4, R1, R5

For MIPS, ADD writes the register in WB but SUB
needs it in ID. This is a data hazard
46
Illustration of a Data Hazard
ADD R1, R2, R3
SUB R4, R1, R5
Reg
Mem
DM
AND R6, R1, R7
Reg
Mem
OR R8, R1, R9
Reg
Mem
XOR R10, R1, R11
Time
ADD instruction causes a hazard in next 3
instructions because register not written until
after those 3 read it.
47
Data Forwarding - 1

• Simplest solution to data hazard
• Forwarding
• Result of the ADD instruction not really needed
• - until after ADD actually produces it.
• Can we move the result from EX/MEM register to
  the beginning of ALU (where SUB needs it)?
• Yes!

48
Data Forwarding - 2

• Generally speaking
• Forwarding occurs when a result is passed
  directly to the functional unit that requires it.
• Result goes from output of one pipeline stage to
  input of another stage bypassing the NORMAL Flow.

49
When Can We Forward?
ADD R1, R2, R3
SUB gets info. from EX/MEM pipe register AND
gets info. from MEM/WB pipe register OR gets
info. by forwarding from register file
SUB R4, R1, R5
Reg
Mem
DM
AND R6, R1, R7
Reg
Mem
OR R8, R1, R9
Reg
Mem
XOR R10, R1, R11
If line goes forward you can do forwarding. If
its drawn backward, its physically impossible
(why)?.
Time
50
Implementing Forwarding in HW

• To support data forwarding, additional hardware
  is required multiplexers to allow data to be
  transferred back and control logic for the
  multiplexers

51
Avoidance of Data Hazards using Forwarding
52
Data Hazards and Pipeline Stall

• Do all kinds of data hazards translate into
  pipeline stalls?
• NO, whether or not a data hazard results in a
  stall depends on the pipeline structure
• For the simple five-stage RISC pipeline
• Only RAW hazards result in a pipeline stall
• Instruction reading a register needs to wait
  until it is written
• WAR and WAW hazards cannot occur because
• All instructions take 5 stages
• Reads happen in the 2nd stage (ID)
• Writes happen in the 5th stage (WB)
• No way for a write from a subsequent instruction
  to interfere with the read (or write) of a prior
  instruction

53
How to Implement Hazard Control Logic?

• In a pipeline,
• All data hazards can be checked during ID phase
  of pipeline. ?!
• If a data hazard is detected, next instruction
  should be stalled. ?!
• Whether forwarding is needed can also be
  determined at this stage, control signals set.
• If any Structural Hazard is detected,
• Control unit of pipeline must stall pipeline and
  prevent instructions in IF, ID from advancing.

54
An Unavoidable Stall