Skip to content

Using the PeekPokeTester

Jump to bottom
Chick Markley edited this page Feb 5, 2018 · 8 revisions

How to build a test using the PeekPokeTester

This will be a brief walk-through of the PeekPokeTester. We will look at the src/test/scala/examples/GCDSpec.scala example.

A couple of imports

import chisel3._
import chisel3.util._
import chisel3.iotesters._
import org.scalatest.{Matchers, FlatSpec}

The device under test.

For the purposes of this discussion we are focused on the IO ports of the circuit or device under test (DUT).

object RealGCD2 {
  val num_width = 16
}

class RealGCD2Input extends Bundle {
  val a = Bits(width = RealGCD2.num_width)
  val b = Bits(width = RealGCD2.num_width)
}

class RealGCD2 extends Module {
  val io  = new Bundle {
    val in  = Decoupled(new RealGCD2Input()).flip()
    val out = Valid(UInt(width = RealGCD2.num_width))
  }
...

The Test Harness

The test harness API allows 4 interactions with the DUT

  1. To set the DUT'S inputs: poke
  2. To look at the DUT'S outputs: peek
  3. To test one of the DUT's outputs: expect
  4. To advance the clock of the DUT: step

The tester is constructed by subclassing PeekPokeTester

class GCDPeekPokeTester(c: RealGCD2) extends PeekPokeTester(c)  {
  for {
    i <- 1 to 10
    j <- 1 to 10
  } {
    val (gcd_value, cycles) = GCDCalculator.computeGcdResultsAndCycles(i, j)

    poke(c.io.in.bits.a, i)
    poke(c.io.in.bits.b, j)
    poke(c.io.in.valid, 1)

    var count = 0
    while(peek(c.io.out.valid) == BigInt(0) && count < 20) {
      step(1)
      count += 1
    }
    if(count > 30) {
      println(s"Waited $count cycles on gcd inputs $i, $j, giving up")
      System.exit(0)
    }
    expect(c.io.out.bits, gcd_value)
    step(1)
  }
}

The GCD test pushes two numbers into the inputs and then checks that the returned GCD value is correct. There is some complexity here in the code that waits for the c.io.out.valid to become high a sign that the GCD computation is complete.

Wrapping the tester in a scalatest.

The test (a subclass of a PeekPokeTester) is now ready to run. The simplest way is to embed the invocation of the test in a scala test.

class GCDSpec extends FlatSpec with Matchers {
  behavior of "GCDSpec"

  it should "compute gcd excellently" in {
    chisel3.iotesters.Driver(() => new RealGCD2) { c =>
      new GCDPeekPokeTester(c)
    } should be(true)
  }
}

Running the test

The test can now be executed via sbt.

sbt
> test-only examples.GCDSpec

What you will see

The test-only command will report with much text the result of your actions. The main types of output are

  1. Success. Everything worked, there is a lot's of output showing the progress of the test and a summary of the result
  2. Problem in the design. There is some error in the design of your circuit that caused either Chisel or Firrtl to error.
  3. Problem in the execution of the circuit. The circuit was successfully compiled into a simulatable state, but there is some problem with the execution. Typically this involves some error in the circuit logic, but can also result from an incorrect test with peek or expect.

If there is Problem in the Design

The full scope of Chisel/Firrtl error reporting is a broad area that we are working hard to provide more materials for. As a simple illustration here. I have introduced a bad connection io.in := 7777.U at line 40 of GCDSpec.scala. The error message in this case looks like

[info] - should compute gcd excellently *** FAILED ***
[info]   chisel3.internal.ChiselException: cannot connect chisel3.util.DecoupledIO@10 and chisel3.core.UInt@2f
[info]   at chisel3.internal.throwException$.apply(Error.scala:13)
[info]   at chisel3.core.Data.badConnect(Data.scala:50)
[info]   at chisel3.core.Bundle.$less$greater(Aggregate.scala:288)
[info]   at chisel3.core.Bundle.$colon$eq(Aggregate.scala:292)
[info]   at examples.RealGCD2.<init>(GCDSpec.scala:40)
[info]   at examples.GCDSpec$$anonfun$5$$anonfun$apply$mcV$sp$3.apply(GCDSpec.scala:86)
[info]   at examples.GCDSpec$$anonfun$5$$anonfun$apply$mcV$sp$3.apply(GCDSpec.scala:86)
[info]   at chisel3.core.Module$.do_apply(Module.scala:30)
[info]   at chisel3.Driver$$anonfun$elaborate$1.apply(Driver.scala:117)
[info]   at chisel3.Driver$$anonfun$elaborate$1.apply(Driver.scala:117)

A good place to start when confronted with an error message like this is to look at the earliest line in your code listed in the stack trace. In this case

[info]   at examples.RealGCD2.<init>(GCDSpec.scala:40)

Find and fix the error and try again.

If there is a problem with execution

Here I introduced and error at line 49 of GCDSpec.scala



The output in this case is quite long but it ends with


STEP 779 -> 780
  POKE io_in_bits_a <- 0xa
  POKE io_in_bits_b <- 0xa
  POKE io_in_valid <- 0x1
ti 780  x 0 y 1  in_ready 1  in_valid 1  out 1 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 780 -> 781
ti 781  x 10 y 10  in_ready 0  in_valid 1  out 11 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 781 -> 782
ti 782  x 10 y 0  in_ready 0  in_valid 1  out 11 out_valid 1==============
  PEEK io_out_valid -> 0x1
  EXPECT io_out_bits -> 0xb == 0xaFAIL
STEP 782 -> 783
RAN 783 CYCLES FAILED FIRST AT CYCLE 2
[info] GCDSpec:
[info] GCDSpec
[info] - should compute gcd excellently *** FAILED ***
[info]   false was not true (GCDSpec.scala:86)
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 1 second, 193 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 0, failed 1, canceled 0, ignored 0, pending 0
[info] *** 1 TEST FAILED ***
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] 	examples.GCDSpec
[error] (test:testOnly) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 4 s, completed Sep 20, 2016 8



Note in particular EXPECT io_out_bits -> 0xb == 0xaFAIL and the


[info] *** 1 TEST FAILED ***
[error] Failed: Total 1, Failed 1, Errors 0, Passed 0
[error] Failed tests:
[error] 	examples.GCDSpec
[error] (test:testOnly) sbt.TestsFailedException: Tests unsuccessful



There are a number of approaches to finding and fixing these types of errors. Going over the circuit. Printf's can be introduced into the circuit.  It is even possible to run the firrtl interpreter backend in a scala debugger.


Things are still not working, more information needed


There are quite a few options options available when using the PeekPokeTester.
A different invocation method is required, when invoking the PeekPokeTester use the Driver.execute method like this.


    chisel3.iotesters.Driver.execute(Array("--fint-write-vcd"), () => new RealGCD2) { c =>
      new GCDPeekPokeTester(c)
    } should be(true)


Note the Array("--fint-write-vcd") argument to the execute function, it allows you to specify that a VCD file should be produced.

That file will generally be found in a subdirectory of you where you are starting with "test_run_dir", the full path will use the name of your DUT plus a random string and many other arguments.  For example when testing this the relative path to my VCD file was test_run_dir/examples.GCDSpec1996890495/RealGCD2.vcd. (There are options that can control where the file goes)


To see the full gamut of available options change "--fint-write-vcd" to "--help".
There are a number of good examples of uses in the
/src/test/scala/examples/GCDSpec.scala
Where it shows how to make the interpreter run verbosely, how to use verilator instead among other things.


Everything worked


The first thing you see will be the firrtl version of the circuit.


circuit RealGCD2 :
  module RealGCD2 :
    input clk : Clock
    input reset : UInt<1>
    output io_in_ready : UInt<1>
    input io_in_valid : UInt<1>
    input io_in_bits_a : UInt<16>
    input io_in_bits_b : UInt<16>
    output io_out_valid : UInt<1>
    output io_out_bits : UInt<16>

    reg x : UInt<16>, clk with :
      reset => (UInt<1>("h0"), x)
    reg y : UInt<16>, clk with :
      reset => (UInt<1>("h0"), y)
    reg p : UInt<1>, clk with :
      reset => (reset, UInt<1>("h0"))
    reg ti : UInt<16>, clk with :
      reset => (reset, UInt<16>("h0"))
    node T_31 = add(ti, UInt<1>("h1")) @[GCDSpec.scala 30:12]
    node T_32 = tail(T_31, 1) @[GCDSpec.scala 30:12]
    node T_34 = eq(p, UInt<1>("h0")) @[GCDSpec.scala 32:18]
    node T_36 = eq(p, UInt<1>("h0")) @[GCDSpec.scala 34:24]
    node T_37 = and(io_in_valid, T_36) @[GCDSpec.scala 34:21]
    node GEN_0 = mux(T_37, io_in_bits_a, x) @[GCDSpec.scala 34:28]
    node GEN_1 = mux(T_37, io_in_bits_b, y) @[GCDSpec.scala 34:28]
    node GEN_2 = mux(T_37, UInt<1>("h1"), p) @[GCDSpec.scala 34:28]
    node T_39 = gt(x, y) @[GCDSpec.scala 41:13]
    node GEN_3 = mux(T_39, y, GEN_0) @[GCDSpec.scala 41:19]
    node GEN_4 = mux(T_39, x, GEN_1) @[GCDSpec.scala 41:19]
    node T_41 = eq(T_39, UInt<1>("h0")) @[GCDSpec.scala 41:19]
    node T_42 = sub(y, x) @[GCDSpec.scala 42:30]
    node T_43 = tail(T_42, 1) @[GCDSpec.scala 42:30]
    node GEN_5 = mux(T_41, T_43, GEN_4) @[GCDSpec.scala 42:21]
    node GEN_6 = mux(p, GEN_3, GEN_0) @[GCDSpec.scala 40:12]
    node GEN_7 = mux(p, GEN_5, GEN_1) @[GCDSpec.scala 40:12]
    node T_45 = eq(reset, UInt<1>("h0")) @[GCDSpec.scala 45:9]
    node T_47 = eq(y, UInt<1>("h0")) @[GCDSpec.scala 50:21]
    node T_48 = and(T_47, p) @[GCDSpec.scala 50:33]
    node GEN_8 = mux(io_out_valid, UInt<1>("h0"), GEN_2) @[GCDSpec.scala 51:23]
    io_in_ready <= T_34
    io_out_valid <= T_48
    io_out_bits <= x
    x <= GEN_6
    y <= GEN_7
    p <= GEN_8
    ti <= T_32
    printf(clk, T_45, "ti %d  x %d y %d  in_ready %d  in_valid %d  out %d out_valid %d==============\n", ti, x, y, io_in_ready, io_in_valid, io_out_bits, io_out_valid) @[GCDSpec.scala 45:9]



Followed by a lot of output of the execution


SEED 1474359604763
  POKE io_in_bits_a <- 0x1
  POKE io_in_bits_b <- 0x1
  POKE io_in_valid <- 0x1
ti 0  x 0 y 0  in_ready 1  in_valid 1  out 0 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 0 -> 1
ti 1  x 1 y 1  in_ready 0  in_valid 1  out 1 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 1 -> 2
ti 2  x 1 y 0  in_ready 0  in_valid 1  out 1 out_valid 1==============
  PEEK io_out_valid -> 0x1
  EXPECT io_out_bits -> 0x1 == 0x1PASS
STEP 2 -> 3
  POKE io_in_bits_a <- 0x1
  POKE io_in_bits_b <- 0x2
  POKE io_in_valid <- 0x1
ti 3  x 0 y 1  in_ready 1  in_valid 1  out 0 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 3 -> 4
ti 4  x 1 y 2  in_ready 0  in_valid 1  out 1 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 4 -> 5
ti 5  x 1 y 1  in_ready 0  in_valid 1  out 1 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 5 -> 6
ti 6  x 1 y 0  in_ready 0  in_valid 1  out 1 out_valid 1==============
  PEEK io_out_valid -> 0x1
  EXPECT io_out_bits -> 0x1 == 0x1PASS
STEP 6 -> 7
  POKE io_in_bits_a <- 0x1
  POKE io_in_bits_b <- 0x3
  POKE io_in_valid <- 0x1
ti 7  x 0 y 1  in_ready 1  in_valid 1  out 0 out_valid 0==============
  PEEK io_out_valid -> 0x0
STEP 7 -> 8
...



Followed by the summary


  EXPECT io_out_bits -> 0xa == 0xaPASS
STEP 782 -> 783
RAN 783 CYCLES PASSED
[info] GCDSpec:
[info] GCDSpec
[info] - should compute gcd excellently
[info] ScalaCheck
[info] Passed: Total 0, Failed 0, Errors 0, Passed 0
[info] ScalaTest
[info] Run completed in 1 second, 135 milliseconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: 6 s, completed Sep 20, 2016 8:20:05 AM



Success! How sweet it is


Too much output


In the examples above the amount of output for this particular test was too large for the scrollback buffer in my terminal so I did the following.


sbt 'test-only examples.GCDSpec' > out
tail -100 out  # to see the result
head -100 out  # to see if the circuit was included
less -r out    # to move around through the output to figure out what was going on.





              


          



  
          

            


            
Clone this wiki locally