Test Behavior, Not Implementation

This is also stated as “test interface, not implementation”. Test only the behavioral aspects that are presented via the interface of the CUT, not the details of the underlying implementation.

Continuing the example above of a FIFO data structure, that implies specific behavior that will be apparent in the interface as you work it out. For instance, you know that adding items to the structure will change its state from empty to non-empty, retrieving them should remove them in the order in which they were added, and retrieving all of them will change it from non-empty to empty. You might also be able to query the number of items it contains.

This interface will tend to be stable long-term once you’ve settled on it, but the implementation could change. You might track internal state differently, or you might switch from a linked-list implementation to a circular buffer implementation. Even if you do all this correctly, it would cause failures in tests that tested the old implementation.

If the interface doesn’t allow a way to check for evidence of behavior, you can use mocks to verify that the CUT called underlying dependencies as expected. For example, testing code that sends data out a UART may only return a success or failure status. But it will need to interact with hardware, so using a mock version of the HAL would allow you to check what was actually sent to the HAL interface.

If you find that you have to fix tests when you make changes to the implementation, that’s a signal that you’re testing implementation. The only time you should have to fix tests is when you change the interface, and that should be limited in scope to the specific tests that use the part of the interface that has changed.

Test One Thing At A Time

Each test case should test only one specific behavior. Each one should be very focused on establishing initial conditions, triggering the CUT, and checking the results.

This makes each case standalone, without order dependencies, so that behaviors can be tested in isolation at fine granularity, and test failures will point to very specific parts of the CUT.

Test cases should not perform additional tests. In particular, avoid “run-on” tests, or the “one test that does it all”. This can produce a cascade of failures that are difficult to analyze for root cause.

Make Your Test Fast

Each test case should be very fast, and the entire suite should complete in seconds, no more than a minute at most. This enables a fast edit-build-run cycle for TDD.

Use fakes to achieve this, because one of the main consumers of run time is slow or long-running operations performed by dependencies. For example, database or file operations, network communications, and physical device interactions. Timed delays are another; make these instantaneous by faking the passage of time.

Follow The BDD Pattern

Behavior-Driven Development (BDD) is a way of structuring TDD test cases that embodies all the guidelines above.

BDD is a form of “specification by example,” using a simple pattern: given a set of initial conditions, when an event occurs, then there should be expected outcomes. Another way of stating this is given/should/when.

For example, given an empty FIFO, when an item is added to it, then it should be non-empty. Alternatively, given an empty FIFO, it should be non-empty when an item is added to it.

You can name test cases directly according to this pattern, so that the names read as a set of specifications for what should happen. For example, “GivenEmptyFIFO_WhenItemAdded_ThenShouldBeNonEmpty” or “EmptyFIFO_ShouldBeNonEmpty_WhenItemAdded”.

Then write the test case code to follow the pattern, setting up the initial conditions as the given, calling the CUT as the when, and asserting expectations as the then. For example:

// Given
IntFifo_t fifo;
IntFifo_init(&fifo);

// When
bool was_added = IntFifo_add(&fifo, 1);

// Then
EXPECT_TRUE(was_added);
EXPECT_FALSE(IntFifo_is_empty(&fifo));

There may be multiple steps to setting up the initial conditions, and multiple expectations asserted to verify the results, but the event being tested, the when, should be a single call to the CUT.

The tests should form an interlocking set, so that once you’ve tested a particular behavior, you don’t have to verify it in each test case. For example, the above assumes that an initialized FIFO is empty, so there should be a case for that, “FIFO_ShouldBeEmpty_WhenInitialized”.

When one of these fine-grained tests fails, the intended behavior and specific failing CUT are clear.