Computer Memory Organization

1
Computer Memory Organization

• Elephants dont forget
• but do computers?

2
Memory Design

• GOAL to best supply programmers with (almost)
  infinitely extensible memory that is as fast as
  possible to use
• The general design is a hierarchy
• The most expensive, fastest memory is in close
  proximity to the processor
• Memory contents are duplicated and expanded in
  banks lower in the hierarchy
• Successive memory banks increase in storage
  capacity and access latency, but decrease in cost
  and operational speed

3
Performance Considerations

• Performance measurements between any two levels
  of hierarchy
• Hit rate ratio of times a desired element is in
  that level of memory to the number of total
  accesses to memory (1-miss rate)
• Hit time time to access the upper level of
  memory and deliver the item to the requesting
  device
• Miss penalty time to bring the item into upper
  level and deliver the item

4
Memory and Design

• Average memory access time is equal to (hit time
  miss rate miss penalty)
• Computer designers favor a block size with the
  lowest average access time rather than lowest
  miss rate
• Memory hierarchies also complicate the design of
  the CPU
• Variable access time prevents memory from being
  directly on the datapath

5
The Memory Hierarchy

• Cache
• Smallest and closest to processor
• Many different organization strategies
• May be multiple levels of cache
• Main memory
• The motherboard RAM you know and love
• Virtual memory
• Switch out memory to disk
• Essential for shared, multiprocess systems

6
Caches

• Where can you put a block in a cache?
• direct mapped a function maps a block to a
  unique location in the cache
• set associative a function maps a block to a
  unique group of locations in the cache, but block
  can go anywhere in the cache
• fully associative put the block anywhere
• How do you find things in a cache?
• For direct map, might invert map function
• Otherwise, must check each block (in parallel)

7
More Caches

• What block is replaced on a miss?
• Random selection promotes uniform allocation,
  easy to implement
• Least recently used takes advantage of temporal
  locality (recently used blocks are more likely to
  be used again)
• What happens on a write?
• Write through - also writes to lower-level block
• Write back - doesnt write lower-level block
  until the corresponding cache block is replaced
• dirty bit on a cache block indicates update is
  necessary

8
Main Memory

• We try to make main memory contain the entire
  state of the currently running system process
  otherwise, we swap
• Penalty for swapping with virtual memory very
  high (magnetic, mechanical storage)
• We dont worry about sync between main memory and
  virtual memory because we do not replicate main
  memory in virtual memory (very costly)

9
Improving Main Memory

• Make memory bus wider (more at one time)
• Also must widen cache
• Must multiplex before entering the datapath
  (convert many signals into one)
• May be wasteful (especially on writes)
• Interleave memory banks (parallelize reading)
• Successive words rotate through memory banks
  (word 1 bank 1, word 2 bank 2, , word 5
  bank 1, word 6 bank 2, )
• Must still multiplex into the datapath
• Allows parallel writes (if one bank not
  overloaded)

10
Looking Ahead

• Next we will talk hardware performance and
  analysis
• What the heck are MIPS? MFLOPS?
• Whats the big deal about clock cycle times?
• Why is I/O (especially virtual memory) so slow?
• Lab 1 will let you implement some integer
  arithmetic algorithms, address memory, and decode
  simple instructions
• Homework 4 has some thought questions on memory
  organization and CPU design