Funded%20by%20NSF%20Grant%20ITR%200313203

1
Adaptive Thermoregulation for Applications on
Reconfigurable Devices
Phillip Jones Applied Research
Laboratory Washington University Saint Louis,
Missouri, USA http//www.arl.wustl.edu/arl/phjone
s Iowa State University Seminar April 2008

• Funded by NSF Grant ITR 0313203

2
What are FPGAs?

• FPGA Field Programmable Gate Array
• Sea of general purpose logic gates

3
What are FPGAs?

• FPGA Field Programmable Gate Array
• Sea of general purpose logic gates

CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
4
What are FPGAs?

• FPGA Field Programmable Gate Array
• Sea of general purpose logic gates

CLB
CLB
CLB
CLB
CLB
CLB
CLB
CLB
5
FPGA Usage Models

• Experimental ISA
• Experimental Micro
• Architectures

• Run-time adaptation
• Run-time Customization

CPU Specialized HW - Sparc-V8 Leon
Partial Reconfiguration
System on Chip (SoC)
Fast Prototyping
Full Reconfiguration
Parallel Applications

• Image Processing
• Computational
• Biology

• Remote Update
• Fault Tolerance

6
Some FPGA Details
CLB
CLB
CLB
CLB
7
Some FPGA Details
CLB
CLB
CLB
Z
A
LUT
B
C
D
8
Some FPGA Details
CLB
CLB
CLB
Z
A
LUT
B
C
D
ABCD Z
0000 0001 1110 1111
0 0 0 1
A
Z
AND
B
4 input Look Up Table
C
D
9
Some FPGA Details
CLB
CLB
CLB
Z
A
LUT
B
C
D
ABCD Z
0000 0001 1110 1111
0 1 1 1
A
Z
OR
B
4 input Look Up Table
C
D
10
Some FPGA Details
CLB
CLB
CLB
Z
A
LUT
B
C
D
ABCD Z
B
X000 X001 X110 X111
0 1 1 1
Z
21 Mux
C
4 input Look Up Table
D
11
Some FPGA Details
CLB
CLB
CLB
Z
A
LUT
B
C
D
12
Some FPGA Details
CLB
CLB
Programmable Interconnection Point
PIP
CLB
Z
A
LUT
DFF
B
C
D
13
Some FPGA Details
CLB
CLB
Programmable Interconnection Point
PIP
CLB
Z
A
LUT
DFF
B
C
D
14
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Temperature-Safe Real-time Systems
• Future Directions

15
Why Thermal Management?
16
Why Thermal Management?
Location?
17
Why Thermal Management?
Mobile?
Hot
Cold
Regulated
18
Why Thermal Management?
Reconfigurability
FPGA
19
Why Thermal Management?
Exceptional Events
20
Why Thermal Management?
Exceptional Events
21
Local Experience

• Thermally aggressive application
• Disruption of air flow

22
Damaged Board (bottom view)

• Thermally aggressive application
• Disruption of air flow

23
Damaged Board (side view)

• Thermally aggressive application
• Disruption of air flow

24
Response to catastrophic thermal events
25
Solutions

• Over provision
• Large heat sinks and fans
• Restrict performance
• Limiting operating frequency
• Limit amount chip utilization
• Use thermal feedback
• Dynamic operating frequency
• Adaptive Computation
• Shutdown device

My approach
26
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Temperature-Safe Real-time Systems
• Future Directions

27
Measuring Temperature
FPGA
28
Measuring Temperature
FPGA
A/D
60 C
29
Background Measuring Temperature
S. Lopez-Buedo, J. Garrido, and E. Boemo,
. Thermal testing on reconfigurable computers,.
IEEE Design and Test of Computers, vol. 17, pp.
84.91, 2000.
FPGA
Temperature
.0
1.
.1
0.
.0
1.
Period
30
Background Measuring Temperature
FPGA
Temperature
.0
1.
.1
0.
1.
1.
.0
1.
0.
Period
31
Background Measuring Temperature
FPGA
Temperature
.0
1.
.1
0.
1.
1.
.0
1.
0.
Period
32
Background Measuring Temperature
S. Lopez-Buedo, J. Garrido, and E. Boemo,
. Thermal testing on reconfigurable computers,.
IEEE Design and Test of Computers, vol. 17, pp.
84.91, 2000.
FPGA
Temperature
1.
.1
.0
Period
Voltage
33
Background Measuring Temperature
FPGA
Temperature
1.
.1
.0
Period
Voltage
34
Background Measuring Temperature
FPGA
Adaptive Thermoregulation for Applications on
Reconfigurable Devices,by Phillip H. Jones,
James Moscola, Young H. Cho, and John W.
LockwoodField Programmable Logic and
Applications (FPL07), Amsterdam, Netherlands
Temperature
Period
35
Background Measuring Temperature
FPGA
Mode 1
Core 2
Core 1
Temperature
Core 4
Core 3
Period
Frequency High
36
Background Measuring Temperature
FPGA
Mode 2
Mode 1
Core 2
Core 1
Temperature
Core 4
Core 3
Period
Frequency High
37
Background Measuring Temperature
FPGA
Mode 3
Mode 2
Mode 1
Core 2
Core 1
70C
Temperature
40C
Core 4
Core 3
Period
8,300
8,000
Frequency Low
Frequency High
38
Background Measuring Temperature
FPGA
Mode 3
Mode 2
Mode 1
Pause
Sample Controller
Core 2
Core 1
Temperature
Core 4
Core 3
Period
Frequency High
39
Background Measuring Temperature
FPGA
Mode 3
Mode 2
Mode 1
Pause
Time out
Counter
Core 1
Core 2
Temperature
Core 3
Core 4
Period
Frequency High
40
Background Measuring Temperature
FPGA
Mode 3
Mode 2
Mode 1
Pause
Time out
Counter
5
4
3
2
1
0
0
1
2
3
5
Core 1
Core 2
Temperature
Core 3
Core 4
Period
Frequency High
Frequency Low
41
Background Measuring Temperature
FPGA
Mode 3
Mode 2
Mode 1
Pause
Time out
Counter
5
3
4
3
2
1
0
0
1
2
3
5
Core 1
Core 2
Temperature
Core 3
Core 4
Period
Frequency High
Frequency Low
42
Background Measuring Temperature
FPGA
Pause
Time out
Counter
5
3
4
3
2
1
0
0
1
2
3
5
Core 1
Core 2
Temperature
Core 3
Core 4
Period
Frequency High
43
Temperature Benchmark Circuits

• Desired Properties
• Scalable
• Work over a wide range of frequencies
• Can easily increase or decrease circuit size
• Simple to analyze
• Regular structure
• Distributes evenly over chip
• Help reduce thermal gradients that may cause
  damage to the chip
• May serve as standard
• Further experimentation
• Repeatability of results

A Thermal Management and Profiling Method for
Reconfigurable Hardware Applications,by Phillip
H. Jones, John W. Lockwood, and Young H.
ChoField Programmable Logic and Applications
(FPL06), Madrid, Spain,
44
Temperature Benchmark Circuits
Core Block (CB) Array of 48 LUTs and 48 DFF
45
Temperature Benchmark Circuits
RLOC Row, Col 0 , 0 7 , 5
Core Block (CB) Array of 48 LUTs and 48 DFF Each
LUT configured to be a 4-input AND gate
46
Temperature Benchmark Circuits
RLOC Row, Col 0 , 0 7 , 5
Core Block (CB) Array of 48 LUTs and 48 DFF Each
LUT configured to be a 4-input AND gate
Thermal workload unit
Computation Row
01
Input Gen
CB 1
CB 0
CB 16
CB 17
00
1
0
1
0
8
8
(1 LUT, 1 DFF)
Array of 18 core blocks (864 LUTs, 864 DFFs)
47
Example Circuit Layout(Configuration 1x, 9 LUTs
and DFFs)
Thermal Workload Unit
48
Example Circuit Layout(Configuration 4x, 36
LUTs and DFFs)
49
Observed Temperature vs. Frequency
T P
P FCV2
Steady-State Temperatures
Cfg4x
Cfg10x
Cfg2x
Cfg1x
50
Observed Temperature vs. Active Area
Max rated Tj 85 C
T P
P FCV2
Steady-State Temperatures
200 MHz
100 MHz
50 MHz
25 MHz
10 MHz
51
Projecting Thermal Trajectories
Tj_ss Power ?jA TA ?jA is the FPGA Thermal
resistance (ºC/W)
Exponential specific equation Temperature(t)
½(-41e(-t/20) 71) ½(-41e(-t/180) 71)
52
Projecting Thermal Trajectories
Estimate Steady State Temperature
Exploit this phase for performance
Tj_ss Power ?jA TA ?jA is the FPGA Thermal
resistance (ºC/W)
Use measured power at t0
Exponential specific equation Temperature(t)
½(-41e(-t/20) 71) ½(-41e(-t/180) 71)
53
Thermal Shutdown
54
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Temperature-Safe Real-time Systems
• Future Directions

55
Image Correlation Application
56
Image Correlation Application

• Heats FPGA a lot! (gt 85 C)

Virtex-4 100FX Resource Utilization
57
Infrastructure
Thermoregulation Controller
Sample Controller
Pause
65 C
Application
Mode
Adaptive Thermoregulation for Applications on
Reconfigurable Devices,by Phillip H. Jones,
James Moscola, Young H. Cho, and John W.
LockwoodField Programmable Logic and
Applications (FPL07), Amsterdam, Netherlands
58
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Image Buffer
Mode
Image Processor Core 3
Mask 2
Mask 1
Score Out
59
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Image Buffer
Mode
200
8
MHz
Score Out
60
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Frequency
Quality
Image Buffer
200
8
MHz
Image Processor Core 3
Mask 2
Mask 1
Low Priority Features
High Priority Features
Score Out
61
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Frequency
Quality
Image Buffer
200
8
180
150
100
MHz
Image Processor Core 3
Mask 2
Mask 1
Low Priority Features
High Priority Features
Score Out
62
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Frequency
Quality
Image Buffer
100
8
75
50
MHz
MHz
Image Processor Core 3
Mask 2
Mask 1
Low Priority Features
High Priority Features
Score Out
63
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Frequency
Quality
Image Buffer
8
50
7
6
5
4
MHz
Image Processor Core 3
Image Processor Core 4
Mask 2
Mask 2
Mask 2
Mask 1
Mask 1
Mask 2
Low Priority Features
High Priority Features
Score Out
64
Application Specific Adaptation
Temperature
Thermoregulation Controller
Sample Controller
Pause
65 C
Frequency
Quality
Image Buffer
8
7
6
5
4
100
75
50
200
180
150
MHz
MHz
Image Processor Core 3
Image Processor Core 4
Mask 2
Mask 2
Mask 1
Mask 1
Low Priority Features
High Priority Features
Score Out
65
Thermally Adaptive Frequency
An Adaptive Frequency Control Method Using
Thermal Feedback for Reconfigurable Hardware
Applications,by Phillip H. Jones, Young H. Cho,
and John W. LockwoodField Programmable
Technology (FPT06), Bangkok, Thailand
Junction Temperature, Tj (C)
Time (s)
66
Thermally Adaptive Frequency
Junction Temperature, Tj (C)
Time (s)
67
Thermally Adaptive Frequency
Junction Temperature, Tj (C)
S. Wang (Reactive Speed Control, ECRTS06)
Time (s)
68
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Temperature-Safe Real-time Systems
• Future Directions

69
Platform Overview
70
Thermal Budget Efficiency
50 MHz
65 MHz
106 MHz
184 MHz
200 MHz
Fixed
Adaptive
70
Adaptive
Thermal Budget (65 C)
65
4 Features 50 MHz
6 50
8 65
8 106
8 184
4 50
60

Fixed
8 200
55
50
Junction Temperature (C)
45
40
35
30
40 C
35 C
30 C
25 C
25 C
25 C
0 Fans
1 Fan
2 Fans
0 Fans
0 Fans
0 Fans
Thermal Condition
71
Conclusions

• Motivated the need for thermal management
• Measuring temperature
• Application dependent voltage variations effects.
• Temperature benchmark circuits
• Examined application specific adaptation for
  improving performance in dynamic thermal
  environments

72
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Temperature-Safe Real-time Systems
• Future Directions

73
Thermally Constrained Systems
Space Craft
Sun
Earth
74
Thermally Constrained Systems
75
Temperature-Safe Real-time Systems

• Task scheduling is a concern in many embedded
  systems
• Goal Satisfy thermal constraints without
  violating real-time constraints

76
How to manage temperature?

• Static frequency scaling

• Sleep while idle

T1
T2
T3
Time
77
How to manage temperature?

• Static frequency scaling

• Sleep while idle

Too hot?
Deadlines could be missed
T1
T2
T3
Time
78
How to manage temperature?

• Static frequency scaling

• Sleep while idle

Deadlines could be missed
T1
T2
T3
Idle
Idle
Idle
Time
Generalization Idle task insertion
79
Idle Task Insertion More Powerful
Task for schedule at F_max (100 MHz)
Deadline (s)
Cost (s)
Period (s)
Utilization ()
10.0
10.0
30
33.33
120
30.0
120
25.00
480
30.0
480
6.25
960
20.0
960
2.08
66.66
a. No idle task inserted
80
Sleep when idle is insufficient
81
Idle-task inserted
82
Idle-Task Insertion
83
Related Research
Power Management Thermal Management
EDF, Dynamic Frequency Scaling Yao (FOCS95) EDF, Minimize Temperature Bansal (FOCS04)
Worst Case Execution Time Shin (DAC99) RMS, Reactive Frequency, CIA Wang (RTSS06, ECRTS06)
84
Outline

• Why Thermal Management?
• Measuring Temperature
• Thermally Driven Adaptation
• Experimental Results
• Conclusions
• Temperature-Safe Real-time Systems
• Future Directions

85
Research Fronts

• Near term
• Exploration of adaptation techniques
• Advanced FPGA reconfiguration capabilities
• Other frequency adaptation techniques
• Integration of temperature into real-time systems
• Longer term
• Cyber physical systems (NSF initiative)

86
Questions/Comments?

• Near term
• Exploration of adaptation techniques
• Advanced FPGA reconfiguration capabilities
• Other frequency adaptation techniques
• Integration of temperature into real-time systems
• Longer term
• Cyber physical systems (NSF initiative)

87
Temperature per Processing Core
Temperature vs. Number of Processing Core
70
2.21x
y 60.1
S1
65
2.24x
y 57.1
S2
60
2.23x
y 52.1
S3
55
2.07x
Junction Temperature (C)
y 44.2
50
S4
45
1.43x
y 37.5
S5
40
1.22x
y 34.0
S6
35
2
1
3
4
Number of Processing Cores
88
Temperature Sample Mode
89
Ring Oscillator Thermometer Characteristics
Thermometer size Ring oscillator size Oscillation period Incrementer Cycle Period Temperature resolution
100 LUTs 48 LUTs (47 NOT 1 OR) 40 ns .16 ms (40ns 4096) .1ºC/ count Or .1ºC/ 20ns
90
(No Transcript)
91
Application implementation statistics
Virtex-4 100FX Resource Utilization
92
Application implementation statistics
93
Scenario Descriptions
94
High Level Architecture
95
Periodic Temperature Sampling
96
Ring Oscillator Based Thermometer
97
ASIC, GPP, FPGA Comparison

• Cost
• Performance
• Power
• Flexibility

98
Frequency Multiplexing Circuit
Frequency Control
clk
Clk Multiplier (DLLs)
clk
to global clock tree
21 MUX
4xclk
BUFG
99
Thermally Adaptive Frequency
Junction Temperature, Tj (C)
Time (s)
100
Thermally Adaptive Frequency
Junction Temperature, Tj (C)
Time (s)
101
Thermally Adaptive Frequency
Junction Temperature, Tj (C)
Time (s)
102
Worst Case Thermal Condition
Thermally Safe Frequency 50 MHz
103
Worst Case Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency
104
Worst Case Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency 48.5 MHz
105
Typical Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency 48.5 MHz
106
Typical Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency 95 MHz
Adaptive Frequency 48.5 MHz
107
Best Case Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency 95 MHz
108
Best Case Thermal Condition
30/120MHz
Thermally Safe Frequency 50 MHz
Adaptive Frequency 119 MHz
Adaptive Frequency 95 MHz
2.4x Factor Performance Increase