Lab Board

1
Lab Board

• A description of the board you will do
  measurements on

2
Lab Board Description

• Code name is Kings Canyon
• Referred to hereafter as KC
• You will do measurements on this board in the lab
• Background for KC
• Low-end server board
• Inexpensive board for customers who want servers
• Server
• Computer which serves data to others
• Supports more memory, more processing power, and
  more I/O (i.e., network bandwidth, hard drives)
  than desktop computers
• More reliable (crashes less and recovers better)
  than desktop computers

3
Server in a network
4
Inside the Server
5
Circuit Board Block Diagram
CPU 0
CPU 1
Front Side Bus (FSB)
CC
SS
P64H2 HI to PCI bridge
SS
SS
CC
SS
Memory
MCH
4 PCI-X busses
SS
Hub Interface
Memory
SS
Two (2) DDR (Double Data Rate) Memory Busses
CC
ICH
Video
CC
P64H2 HI to PCI bridge
CC
Low speed 33MHz PCI bus
Details to Follow .
SS Source Synchronous
CC- Common Clock
6
Components on KC

• the chips

7
CPU

• CPUs Central Processing Units
• The brains of the computer that run your
  program
• Runs the Operating System (e.g. windows) and all
  the software
• Speed of CPU has large effect on speed of system
• Currently measured in GHz (e.g., 3.0 GHz)
• Consumer pcs typically have 1 cpu inside the box
  called UP
• KC has 2 cpus this is called DP, i.e. dual
  processor
• Some boards have 4 cpus, they are called MP,
  i.e. multi-processor
• KC uses Intel Xeon CPUs
• Currently at 3GHz
• Similar to Pentium 4
• Bigger cache (superfast memory inside CPU)
• Can operate with other CPUs on same bus

The Silicon die (2) is under this metal heat
spreader (1). The die is significantly smaller
than the heat spreader. A Fansink mounts to
the heat spreader to keep die cool.
8
MCH

• MCH Memory Controller Hub
• Helps CPU communicate with rest of system
• The gatekeeper to the outside world from
  perspective of CPU
• Talks to memory, CPU, and I/O
• I/O means busses and chips and boards which
  ultimately connect to things like hard drives,
  networks, keyboards, monitor, etc..
• Maintains memory
• tells it when to refresh, etc.. (dynamic memory
  has to be refreshed or it forgets the data it
  holds)
• Detects errors in what is stored in memory (ECC)
  and periodically corrects these errors
• KC uses Intel E7501 MCH

Package decoupling capacitors. Try to hold
voltage rails (vcc) steady. Closer to the die
the better.
Actual Die Passive (no fan) Heatsink mounts on
top.

• http//www.intel.com/design/chipsets/e7501/index
  .htm?iidipp_srvr_proc_xeone7501

9
ICH

• ICH Input/Output Controller Hub
• I/O device that deals with critical but slow
  speed devices
• Video controller (basic 2-d video on servers, not
  meant for 3-d graphics)
• Talks to Keyboard, mouse via Super IO chip
• BIOS (contains software which computer runs when
  its first powered on. BIOS gets computer up and
  running and then passes control to the OS)
• IDE Hard drive (slow hard drive that OS boots off
  of. The big capacity hard drives do not connect
  to ICH)
• Helps with initial bootup of system
• Talks to mch over a slow version of the Hub
  Interface bus
• KC uses Intel 82801CA ICH

Still uses wire-bond technology. Doesnt require
heatsink.
10
Memory Modules

• Memory High speed storage place for code and
  data used by CPU
• Much faster than hard drive storage, but not as
  massive in quantity
• 2-16 Gigabytes vs. 100s of Gigabytes
• Slower than CPU cache (small amount of memory
  internal to CPU)
• Memory dies are inside wire-bond packages called
  DRAMs
• Dynamic Random Access Memory
• Many DRAMs solder to a memory board called the
  DIMM
• Dual In-line Memory Module
• Anywhere from 9 to 36 DRAMs on one memory board
• 2 types of memory boards Unbuffered and
  Registered. Unbuffered is used in desktop. KC
  (and most servers) only uses Registered DIMMs.
• Memory board plugs into KC board via a connector.
• KC can support up to 8 DIMMs
• Many different DIMM manufacturers
• Micron, Samsung, Elpida,
• Infineon,

DRAM 2 DIMMs
http//www.micron.com/products/modules/ddrsdram/in
dex.html
11
PCI-bridge

• P64H2 PCIX 64-bit to Hub Interface 2 bridge
• Gateway to High Speed I/O devices such as hard
  drive arrays adapters or ethernet adapters
• Talks to MCH on one side
• Via high speed variey of Hub Interface Bus
• Talks to 2 separate pci-busses on other side
• Several connectors connect to each pci bus
• User can slide a card into each of these
  connectors
• Typically Hard Disk Controller Cards
• Each Provides several SCSI or SATA connection to
  hard drive
• Can end up with quite a few hard drives and large
  storage capacity
• Or Network Adapater Cards
• Provide several 100Mb/s ethernet connections,
  which go to a switch or router
• Servers tend to read data from the hard drive
  into memory, process the data with the CPUs, and
  then send it out to the requesting computer via
  the network (e.g., web server)
• P64H2 is the code name for the Intel 82870P2

P64H2 die, may require passive heatsink
12
Identifying parts on the board

• CPUs
• there are two of them and they have really huge
  fansinks.
• NOTE If the fansink doesnt run, and the system
  is on, the heatsink will get VERY hot (burn skin)
  and system will eventually shut itself off.
• Memory
• memory consists of 8 wide connectors all in a
  row.
• At least 2 DIMMs must be plugged in (populated)
  for system to boot up. There are rules regarding
  what kind of dimms to use and which order to plug
  them into the connectors.
• MCH
• Its the chip between the cpus and the memory
• Probably has a passive heatsink (a small heatsink
  w/ no fan)
• ICH P64H2s
• Hard to find, but it looks like the chip pictured
  in the above slide
• There is one ICH and 2 P64H2s (second one not
  always populated, i.e. sometimes its missing)
• PCI Slots
• Larger connectors on opposite side of board from
  DIMM connectors.
• One slot is from ICH and is slow speed legacy PCI
  bus (33MHz, 32-bit)
• The rest are high speed (66-133MHz, 64-bit i.e.
  64 data bits wide)

13
Interconnects on KC

• the wires that connect the chips together

14
Front Side Bus (FSB)

• Function CPUs talk to each other and to the MCH.
  This is how the CPUs get data and code from
  memory, and communicate with rest of the system.
• Technology Source synchronous
• 533MBit/second/wire (266MHz signals)
• This means each data wire on this bus can send
  533 million bits of data in one second
• 64 data bits wide
• So the whole bus can transfer 64533 million bits
  of data per second
• Topology Multi-drop bus. CPU1 has to listen to
  when CPU0 acceses memory because CPU1 may have
  that memory data in its cache
• Cache Coherency Each cpu has a small superfast
  memory inside it called the cpu cache. The
  cache holds recently accessed memory data. If
  one cpu holds the data for a memory address that
  another cpu needs, it has to let it know not to
  get that data from main memory.
• Termination Parallel termination at both ends of
  bus (MCH and CPU0)

15
FSB Topology
16
FSB Signal Groups

• Data Signal Group
• HD630 64 bits of data
• DINV30 - dynamic inversion
• HDSTBP30 - P strobes
• HDSTBN30 - N strobes
• Example, HDSTBP0 HDSTBN0 are used to
  strobe in HD0-16 DINV0
• E.g., two strobe signals are used to clock in a
  set of data bits, more on this later
• Address Signal Group
• Switches at ½ the rate of the FSB Data Signals
• HAB353 - Address Bits. E.g., what memory
  address is being requested
• HADSTB10 - Strobes for address signal group
• HADSTB0 strobes A163, HADSTB1 strobes
  the rest
• Common Clock Signals
• There a whole lot of common clock signals
• HCLKINP, HCLKINN Common clock for all signals
  in the Host Clock Domain, i.e. all common clock
  signals, and outer loop timings for strobes (also
  called BCLK)

17
FSB Timings 4X Source Synchronous
Timings not published for 533 MHz FSB, 400 MHz
timing diagrams shown
18
FSB Timings 2X Source Synchronous
Timings not published for 533 MHz FSB, 400 MHz
timing diagrams shown
19
DDR (Double Data Rate) Memory Bus

• Function MCH reads and writes data from memory
  over this high speed bus.
• Signaling Technology Source Synchronous
• 266Mbit/sec/wire (133Mhz signals)
• A DDR bus is 64 data bits wide
• KCs MCH uses 2 DDR busses in parallel, so it
  likes having a 128-bit wide bus
• Bandwidth is 128266 Million bits per second
• TopologyMulti-Drop Bus.
• Each of the two separate instances of this bus on
  this system go to four (4) DIMM slots.
• Each DIMM can have up to 2GBytes of memory, for a
  max total of 16GB.
• Topology and Timings
• The subject of the next class

20
Memory speed and size within a system
When we study DDR signaling and timings, it will
be clear the overhead and latency that main
memory exhibits, and thus the need for local
L1-L3 CPU cache.
Hard Drives
21
HI (Hub Interface) Bus

• Function Intel proprietary bus to provide
  flexible high speed data transfer from MCH to
  various downstream components
• Signaling Technology Source Synchronous
• HI1 From MCH to ICH 266Mbit/sec/wire (133Mhz
  signals)
• 8 data bits wide, bandwidth 8266 Million bits
  per second
• HI2 From MCH to P64H2s 533Mbit/sec/wire
  (266Mhz signals)
• 16 data bits wide, bandwidth 16533 Million
  bits per second
• Topology Point to Point.
• Termination There are parallel terminations at
  both ends of the bus

22
Hub Interface Topology
23
Hub Interface Signal Groups

• Data Bits
• HI150 Data Signals
• PSTRB_10, PSTRB_10 Strobe Signals
• PSTRB_0 PSTRB_0 strobe in HI07
• PSTRB_1 PSTRB_1 strobe in HI158
• Other
• HI1816 Command Signals
• Common Clock signals
• GCLKIN
• 66MHz common clock for HI1816 and outer loops
  of the PSTRBs
• Timings not published for Hub Interface

24
PCI and PCI-X

• Function Industry standard bus for plug-in
  peripherals
• In servers, quite often plug-in hard drive
  adapters or network adapters are connected to the
  pci bus
• Technology Common Clock
• PCI-X From P64H2 133Mbit/sec/wire, clocks are
  133MHz signals, data is 66MHz
• 64 data bits wide, bandwidth 64133 Million
  bits per second
• PCI From ICH 33Mbit/sec/sire, clocks are
  33MHz, data is 16MHz
• 32 data bits wide, bandwidth 3233 Million bits
  per second
• This speed and width are common in current
  desktop pc pci slots
• Topology Multi-Drop Bus.
• The bus consists of devices soldered down on the
  board and slots for plug-in devices. They all
  dangle off the PCI bus.
• Termination
• Series terminated inside the transmitting agent.
  
• This signaling technolgy depends on a doubling of
  the waveform at the end of the bus (due to
  open-circuit) in order to function.

25
PCIX-133 2-slot Topology
x
x
x
26
PCI-X Signal Groups

• Data and Address use same signals
• AD630
• Common Clocked
• Control Signals
• DEVSEL, TRDY, a bunch more
• Common Clocked
• Asynchronous Signals
• REQ, GNT Arbitration signals
• Interrupt Signals
• Clocks
• PCLK60 goes to every device that hangs off
  the pci bus including the bridge itself

27
PCI-X Data to Common Clock Timings
ns
ns
28
PCI-X Common Clock
29
KC Clock Distribution
30
Magic Decoder Ring

• Names for Signals given so far are from the
  device sheets
• KC Board uses its own names
• Boards have to name signals clearly and uniquely
  across the whole board
• Devices only have to name then uniquely across
  the device
• Decoder Ring
• FSB HD630 ? FSB_HD63_N
• Rule is append FSB_ in front and replace with
  _N in the device signal name
• HI HI150 ? P64H2_1_HI150
• Rule is append P64H2_1_ before the device signal
  name
• Note that the 1 in P64H2_1_ may be a 1 or a
  2 because KC has two P64H2 devices
• PCI-X AD630 ? P64H2_1_PB_AD630
• Rule is append P64H2_1_PB to beginning of device
  signal name
• Note that the 1 in P64H2_1_PD.. Can be a 2 or
  3 depending on which pci bus the signal
  belongs too
• DDR Given Later