Simulator Independent Coverage for RTL Hardware Languages

Kevin Laeufera, Vighnesh Iyera, David Biancolinb, Jonathan Bachrachc, Borivoje Nikolića, Koushik Sena
aUniversity of California, Berkeley, bSiFive, cJITX
ASPLOS 2023,

Coverage: Who tests the testers?

Problem 1: Most open-source simulators lack coverage

Problem 2: Verilog coverage is an inadequate replacement for native Chisel coverage

In this example, the translation to structural Verilog replaces a branch with a conditional assignment. Therefore, 100% line coverage on the generated Verilog does not necessarily imply complete line coverage of the Chisel source.

Our Solution: Simulator Independent Coverage

We show how a single new cover statement allows us to move all coverage instrumentation into the FIRRTL compiler. Adding support for the cover statement to a simulator is fairly simple.

Line Coverage

The line coverage pass instruments every when statement in the FIRRTL circuit. The mapping from lines to branches is used to generate the coverage report from the counts reported by the simulator.

Finite State Machine Coverage

Finite state machine (FSM) coverage assumes that the state register uses a ChiselEnum. We first analyze all possible next states by simplifying the state update expression for each possible current state. We then add cover statements for all states and possible transitions.

Verilator: cover Statement Support

Verilator: Coverage Instrumentation Overhead

Coverage instrumentation overhead on Verilator v4.034. For TLRAM, the measured overhead of our FIRRTL line coverage is close to zero.

The cover Statement

Toggle Coverage

The toggle coverage pass adds a register and a xor gate. It avoids redundant instrumentation for signals that always have the same value.

FireSim: cover Statement Support on FPGA

We generate saturating counters and a scan chain for all cover statements for FPGA-accelerated simulation with FireSim.

FireSim: Utilization and \(f_{max}\)

Extensive Simulator Support

More Features

  • Applicable to other hardware languages.
  • Feedback-Directed Fuzz Testing
  • Coverage merging
  • Formal cover trace generation
Paper PDF, Code & Slides