Architecting Testbenches

1
Architecting Testbenches
2
Architecting Testbenches

• Reusable Verification Components
• Verilog Implementation
• VHDL Implementation
• Autonomous Generation and Monitoring
• Input and Output Paths
• Verifying Configurable Designs

3
Reusable Verification Components

• Want to maximize amount of verification code
  reused across testbenches
• This minimizes the development effort
• Testbench is made up of 2 components
• Reusable test harness
• Testcase specific code

4
Procedural Interfaces for reusable components

• For the components to be reusable, must define
  procedural interfaces that are independent of the
  detailed implementation
• This way as the interfaces change, the testcases
  do not.
• Provide flexibility through thin layers.
• This gives the ability to have fine control or
  greater abstraction (byte access vs.. packet
  access)
• Do not try to implement all functionality at one
  level
• Preserve the procedural interfaces

5
Development Process for reusable components

• Introduce flexibility as needed.
• This way you dont spend time on unnecessary
  functions
• Use the verification plan to determine what is
  functions are necessary
• When you need more complex functions, build upon
  already verified functions.

6
VHDL Implementation

• Want to refine the monolithic view into a
  client/server BFM, access and utility functions,
  and testcases.
• Goal is to obtain a flexible implementation
  strategy promoting reusability of verification
  components
• To change a testcase, just change the control
  block.

7
VHDL Implementation Packaging Bus Functional
Procedures

• BFMs can be located in a package to be re-used.
• Driven/monitored signals are passed as
  signal-class arguments
• BFM procedures are cumbersome to use.
• All signals involved in the transaction must be
  passed to the procedure
• Duplication across testbenches, each testbench
• Declares all interface signals
• Instantiates the DUV
• Properly connects everything

8
VHDL Implementation Creating a test harness

• Instead add some abstraction, use a test
  harness.
• In contains declarations and functionality common
  to all testbenches
• Reduces amount of duplicated information
• Determine common elements
• Declaration of the component
• Declaration of the interface signals
• Instantiation of the DUV
• Mapping of Interface signals to the ports on the
  design
• Mapping of interface signals to the signal class
  arguments of the bus-functional procedures

9
VHDL Implementation Creating a test harness
(cont)

• Use an intermediate level of hierarchy for test
  harness encapsulation
• Have the test harness own the procedures
• All the signals associated with the DUV are
  local.
• Testcase control process instructs the local
  process
• Use control signals to communicate to/from the
  testcase to the test harness

10
VHDL Implementation Creating a test harness
(cont)

• Control signals
• Server signals signals used in the local (test
  harness) process
• Client signals signals used in the testcase
• These signals have to be visible to both the
  client and server (these are located in different
  architectures)
• Pass as ports in the test harness
• Global signals in a package

11
VHDL Implementation Creating a test harness
(cont)

• Global signals eliminate need for each testbench
  architecture to declare them
• Use transaction to synchronize operations
• Wont miss two consecutive operations (compared
  to using event)
• Control signals utilize records
• Minimize maintenance if signals change
• Minimizes impact on client/server processes

12
VHDL Implementation Abstracting the
Client/Server Protocol

• Client must properly operate the control signals
• Must wait for the server process to finish the
  transaction
• Use transactions, not event based waits
• Defining the communication protocol on signals
  between the client/server dont seem to
  accomplish much.
• Now instead of having to deal with a physical
  interface, you have an arbitrary interface
• Encapsulate this for ease of maintenance
• Removes the client having to know the details of
  the protocol
• Change the protocol without having to change
  testcase

13
VHDL Implementation Abstracting the
Client/Server Protocol (cont)

• Encapsulate by placing server procedures into a
  package
• Client processes are now free from knowing the
  details of the protocol between the client and
  server
• To perform an operation, use the appropriate
  procedure
• Pass the control signals in the procedure
• Use the package to get access to the global
  signals and procedures
• Simply need 2 signals (records)
• One used in communications to the server process
• One used in communications from the server process

14
VHDL Implementation Abstracting the
Client/Server Protocol (cont)

• What has been gained signals are still passed
  from the bus-functional access procedures
• Testcase need not declare all the interface
  signals to the DUV
• Testcase need not instantiate and connect the
  design
• Reduction of duplication
• Regardless of number of signals involved in
  physical interface, only 2 need to be passed
• Testcase is completely removed from the physical
  interface of the DUV
• Pins can be removed or added, polarities changed,
  etc no effect on existing testcases

15
VHDL Implementation Managing Control Signals

• Use two control signals (simplest method)
• One to send control and synchronization
  information to the server
• One to send control and synchronization
  information to the client
• Use one control signal requires additional
  development effort
• Must have a resolution function for this method
• Include a mechanism for differentiating value
  driven from client and server
• Proper interlocking for parallel procedure calls
• May have dozens of server processes
• Each has its own packages/procedures
• Use qualified names to prevent identifier
  collision

16
VHDL Implementation Multiple Server Instances

• Many designs interface to multiple instances
• If using one BFM instantiated multiple times,
  need a way to differentiate which instance to use
  to perform an operation
• Use an array of control signals
• One pair for each server
• Can use a generate statement to instantiate the
  BFMs

17
VHDL Implementation Utility Packages

• Utility routines can provide additional level of
  abstraction
• Composed of procedures
• Encapsulated in separate packages using
  lower-level access packages
• Example
• Use the read/write access functions to perform
  packet read/write access functions

18
Autonomous Generation and Monitoring

• You now have a properly packaged BFM
• These can become active entities
• Removes tedious task of generating
  background/random data
• Removes task of performing detailed response
  checking
• Can include variety of tasks
• Safety checks
• Data generation
• Data collection
• Etc
• Constant attention from testcase not required
• Instead, testcase just configures BFM

19
Autonomous Stimulus

• Protocols may require more data than is relevant
  for the testcase
• ATM protocol when not active cell, must send
  dummy cell
• A procedure (send_cell) would inject the cell,
  but what about the other cycles.
• Dont want the testcase to worry about it
• Add Autonomous stimulus to BFM
• Now send_cell would inject the cell at the
  appropriate time (after current dummy cell is
  sent) and then return back to the testcase

20
Autonomous Stimulus (Cont)

• For Autonomous stimulus BFMs, 2 type of
  procedures
• Blocking procedure does not return control back
  to testcase until BFM is done with function
• Non-blocking procedure returns control back to
  testcase immediately
• For this, actions must be queued.
• Used to control constraints on the fly

21
Random Stimulus

• Generated data can be random
• BFM can contain algorithms to generate random
  data, in a random sequence, at random intervals
• May want controls (procedures) to BFM to change
  random constraints
• Given enough information, the Autonomous BFM can
  assist in verifying the output of the DUV
• Example of packet router header and filler info
  could be randomly generated, but expect is a
  function of this. BFM has knowledge, testcase
  doesnt. Use BFM to generate the expect.
• Procedures used to start/stop the BFM and fill in
  the routing table.

22
Injecting Errors

• How do you ensure that your monitors are working?
• Could also have BFM to be configured to be enable
  to generate error conditions to ensure they are
  detected.
• Example parity generation
• Example CRC generation

23
Autonomous Monitoring

• We have differentiated between monitor and
  generator based on who is in control of timing
  (who initiates)
• Monitors must be continuously listening
• If not, outputs can be missed
• If procedures are used to activate monitoring,
  potential for missed activities
• Must have 0-delays for back to back calls.

24
Autonomous Monitoring (Cont)

• Role of testbench is to detect as many errors as
  possible!
• Even errors from within the testbench (Usage
  errors)
• If unexpected output operation is not caught,
  then possible for bad design to go unnoticed
• Solution Autonomous Monitors
• Continuously calling monitors is cumbersome Use
  autonomous one
• Continuously monitors outputs all the time
• Use queues if needed
• Testcase can still use procedure, BFM just
  dequeues from its queues
• Can have error generated if there are still
  items in queue at end of test

25
Autonomous Error Detection

• Go one step further and place all checking in
  Autonomous Monitor
• Used with environments where data contains
  description of expected response
• Could also have stimulus BFM give the monitor the
  expected response
• Procedures could pass configuration options

26
Input and Output Paths

• Each testcase must provide different stimulus and
  expect different responses
• Create these differences by configuring test
  harness in various ways, and providing different
  data sequences
• To this point we have assumed that data was hard
  coded in the testcase

27
Programmable Testbenches

• Most cases, stimulus data and expected response
  are specified in verification plan and hard coded
  in testcase
• Use BFM, in test harness, to apply/receive data
  using procedures from testcase.
• Alternative is to have programmable environment
• Have stimulus/expect read from a file
• Do not have to recompile for new testcases
• Common method where you have external C program
  generating data and expect (reference model)

28
Programmable Testbenches (Cont)

• VHDL
• Can read any text file, but very primitive
• Verilog
• Only read binary/hex values
• May want to massage data before placing into
  these files

29
Configurable Files

• Packaged BFMs offer the possibility of making
  testbench difficult to understand or manage
• To minimize possibility, use self-contained
  BFMs, controlled by procedures.
• Only exception may be seed for randomization
• Functionality of a testcase originates from
  top-level testcase control description.
  Non-default configuration settings originate from
  that point.
• Avoid external configuration files
• Creates more complicated management task
• Task grows exponentially with of files

30
Configurable Files (Cont)

• If you must use files, then
• Do NOT hardcode filenames inside BFM or
  procedure, instead place the name as a parameter
  to procedure
• Not obvious where data is coming from
• Used in procedure, then top-level testcase shows
  what file
• Talk more about output file management this later!

31
Concurrent Simulation

• Other problem with hard coding filenames,
  concurrent simulation
• Causes collisions between multiple simulations
• Try to make filenames unique

32
Compile Time Configuration

• Avoid using Compile time configurations for BFM.
• Different configuration requires compiling the
  source code with a different header file
• Makes managing a testcase more complicated
  extra files to maintain
• May make it impossible to run concurrent compiled
  simulations
• Verilog always compiles before simulation, may
  make it tempting to use.
• Minimize of files. Use a single compile-time
  configuration

33
Verifying Configurable Designs

• If different configurations must be verified,
  architect you environment for this
• Two type of configurable designs
• Soft Can be changed by a testcase.
• Randomization constraints
• This type is implicitly covered by the
  verification process
• Hard Cannot be changed once simulation starts.
• Clock frequency
• Testbench must be properly designed to support
  this in a reproducible fashion

34
Configurable Testbenches

• Configure testbench to match DUV
• If a design can be made configurable, so can a
  testbench
• Use generics/parameters to achieve this
• This allows configuration details to be
  propagated down the hierarchy from the top-level
• Using this approach, almost anything can be
  configured
• Can be used in signal definitions, generate
  statements, etc
• If a component is configurable via generics or
  parameters, make this part of its interface

35
Top Level Generics and Parameters

• Top-level modules and entities can have generics
  and parameters
• Each configurable component has these
• May want control from top level
• Top level does not have pins or ports, but it can
  have generics/parameters
• Can control configurations based on these values
• Some simulators allow setting these from command
  line
• Not portable among simulators
• Can wrap the top level with another layer of
  hierarchy
• Verilog can use defparam statement
• VHDL can use configurations to do this

36
Randomization
37
Random Environments

• Random used to describe many environments
• Some teams call testcase generators random
  they have randomness in the generation process,
  these are one type
• Two types
• Pre-test randomization
• During Test Randomization
• Both utilize autonomous stimulus and autonomous
  checking

38
Pre-Test Randomization

• This is a technique used to randomize the
  controls to the BFMs
• Introduce another layer of hierarchy to the
  verification environment
• Common testcase
• Example DUV has 4 major functions, could do
  pre-test randomness to randomly pick which of the
  4 major functions.
• Input to common testcase could be how many to
  test.

39
During-Test Randomization

• This utilizing the methods of autonomous stimulus
  and monitoring.
• For a given interface, done entirely from within
  a BFM
• Controlled via procedures from the testcase
  and/or common testcase.

40
Random Drivers/Checkers

• The most robust random environments use
  autonomous stimulus and autonomous monitoring
• Autonomous stimulus
• Allows more flexibility and more control
• Allows the capability to stress the DUV
• Autonomous monitoring will flag interim errors.
  Testcase can be stopped as soon as a random input
  sequence causes an error.
• Overall quality is determined by verification
  engineer.
• Scenarios missing from verification plan
• Checks not implemented correctly or missing

41
Random Drivers/Checkers

• Costs of optimal random environment
• Code intensive
• Need experienced engineer to oversee effort to
  ensure quality
• Benefits of optimal random environment
• More stress on the logic than any other
  environment, can include real hardware
• Find nearly all the bugs
• Most devious
• Easy ones

42
Random Drivers/Checkers

• Too much randomness can prevent from uncovering
  problems.
• Want controlled and constrained randomness
• May want un-randomizing controls to be built in
• Hangs due to looping
• Low activity scenarios
• Directed tests
• Micro-modes may be put in to the drivers
• Allows user to specify specific scenarios
• Allows the narrowing of scope for testing

43
Randomization Seeds

• Testcase seed is used to seed the pseudo-number
  generator.
• The pseudo-number generator is then used for all
  decision making in the BFMs.
• This is chosen at the beginning of the test.
• Usually passed in as a parameter or generic
• Caution Watch for seed synchronization across
  drivers, especially ones instantiated multiple
  times.