Introduction
Behavior-Driven Development (BDD) is a software development methodology that focuses on defining the behavior of a system in plain, human-readable language.
While commonly associated with end user business requirements and web development, it can be applied to embedded systems as well. This includes not only behaviors visible to end users, but also internal behaviors between modules, where one module is the “user” of another module’s behaviors.
BDD can be used across the entire range of embedded system project environments, from exploratory development that starts on the back of a napkin to fully-specified regulated industries like medical, automotive, and aerospace products.
What Is BDD?
BDD is a layer of discipline on top of Test-Driven Development (TDD). It was created by Dan North (see his post Introducing BDD), who found that developers learning TDD would frequently experience confusion and misunderstanding, often leading to poor results applying the methodology.
In the worst case, this would result in projects abandoning TDD, losing all the benefits of an extremely effective technique.
A major source of confusion and misunderstanding is due to the word “test”. That carries a number of connotations, most often that it’s simply a test methodology, and that it should be done by testers, not developers.
While it’s true that TDD uses tests, it’s better thought of as a design and development methodology, driven by developer tests that exercise the code in isolation and verify the results. It allows developers to check their work as they go. The net result is a unit test suite and known-good functioning code, with test results to prove it.
There are also ways to get TDD wrong, primarily by focusing on the internals of the code being developed.
Adding the BDD practices avoids these issues.
BDD replaces the word “test” with the word “behavior” to change the focus from tests and internal implementation, to behavior, i.e. what the code should accomplish regardless of the specific internal implementation. Required behaviors may be known up front or they may emerge as development proceeds.
BDD still uses tests just like TDD, but with some constraints to maintain the proper focus. The tests verify that the code meets the required behavior.
What Is TDD?
TDD is an iterative methodology used during software development to simultaneously build a unit test suite and the functional code. It’s performed entirely by the developer, not by a separate tester.
The developer performs the following cycle (CUT is the Code Under Test):
- Write a single new unit test to exercise the CUT before writing the actual code that will pass the test. Thus the test is logically guaranteed to fail initially.
- Run the current test suite to verify that the new test does indeed fail.
- Write the code that will pass the test.
- Run the current test suite to verify that the code changes didn’t break anything unexpectedly, and the new code passes the new test. Revise as necessary until this is true.
- With a test now showing that the code runs as required, refactor as needed to improve the code, while keeping all the tests passing.
This is known as the Red, Green, Refactor cycle for the colors of a failing test, a passing test, and the refactoring step. The developer repeats the cycle until the code is complete.
While it seems like writing tests and code at the same time would slow development, it actually makes very fast progress as the developer iterates the cycle. In the end, the code is known to be good and doesn’t need time-consuming debugging. This reduction or even total elimination of debugging is a significant net speed-up to the overall project.
TDD tests are run off-target, meaning they are run on the development host or build server, not on the target embedded system. That means they can only test the aspects of the code that are platform-independent (where “platform” is the combination of target hardware, target RTOS or other runtime environment, and target third-party software).
However, this tends to be the bulk of a project’s codebase. That encourages an architecture that segregates the platform-independent components from the platform-dependent components. The firmware crosses that boundary by using thin abstraction layers.
The platform dependencies are simulated via “test doubles” that stand in for the real dependencies during tests.
What Does BDD Add?
These are the BDD constraints, the extra layer of discipline it adds to TDD:
- Test scenarios that are readable and understandable by non-developers.
- It exercises the CUT only through its public interface without any knowledge of the internal implementation.
- It uses a well-defined 3-part test structure; Given-When-Then written in a natural language.
- It focuses each test on just one specific aspect of behavior.
Up-Front vs. Emergent Requirements
Many embedded systems, particularly in regulated industries, start from up-front requirements. Less formal environments may rely more on emergent requirements. BDD works in all cases to prove that the code meets the final resultant set of functional requirements.
Where requirements are known up front, they can be translated to tests following the BDD constraints. This can help facilitate requirements traceability for auditing and review purposes.
Where requirements emerge as the project progresses, the BDD test names can be compiled into a list of requirements.
By using sentences for test names, the list of test names becomes a requirements document that is understandable to a wider range of technical and non-technical stakeholders. That improves communication to ensure both business and technical considerations are addressed. All stakeholders are able to review and contribute behavioral requirements.
What Are The Benefits of BDD?
There are a number of benefits to BDD, some hinted at above:
- The resulting code is known to be good, proven to execute correctly by the tests.
- The resulting test suite serves as a safety net for future development.
- The test names form a human-readable specification.
- The test suite acts as executable documentation of how to use the code, automatically updated when the code and tests are updated.
- When a test fails unexpectedly, it points directly to the problem.
- Overall development is faster, because fewer bugs escape the developer, resulting in much less debugging and bug fixing.
- Test coverage is high, often 100% for platform-independent code.
- The codebase is lean, avoiding bloat, because it only contains code that is required to pass the tests, and the tests are only added for the behaviors that are required.
- Firmware can be implemented before hardware is available, using test doubles.
- When failures occur in off-target testing, it’s much easier to investigate them than it is when things fail running on the embedded target.
In this context, test coverage becomes a quality signal (as opposed to a goal metric). If it drops, that’s an indication someone has added code that isn’t covered by a test and may not be a required behavior. That can be checked against the test name list to see if a requirement (and test) needs to be added, or if the code needs to be removed. These offer cross-checking opportunities.
What Are The Limitations Of BDD?
BDD isn’t a magic solution to all problems. Like any tool, it has its limitations. The key is to remember that it’s only one part of a comprehensive development strategy. It’s one of a set of tools.
Any testing takes time to write and maintain. The goal with BDD/TDD is to take small amounts of time doing it in order to save large amounts of time later by avoiding prolonged debugging: detecting, chasing down, isolating, and fixing bugs. It’s a small investment with a big ROI. That’s what James Grenning calls the “Physics of Test Driven Development” vs. the “Physics of Debug Later Programming” (see his article on this).
BDD is an example of the 80/20 rule: it provides the methodology for implementing and verifying 80% of the firmware, and other methods must be used to perform the final 20% (actual projects may see a different specific ratio).
Specifically, the following areas need additional testing and verification methods, because they have to be verified on the specific target platform:
- Concurrency
- RTOS interaction
- Hardware interaction
- Performance
These are all critically important, and are often mistakenly cited as reasons not to use TDD-based methods such as BDD. However, BDD is still an excellent foundation, providing known good functional components up to the final platform-dependent interaction points.
This is where the test doubles stand in for the final level of target platform interaction, allowing BDD to cover as much of the code path as possible up to the actual concurrency, RTOS, and hardware function call or on-chip register access. Using thin adapter and abstraction layers allows swapping out between the test doubles and the real things. See James Grenning’s article Progress Before Hardware for an overview of this, and his 3-part series Unit testing RTOS dependent code – RTOS Test-Double for an example of one way to do this with RTOS concurrency.
These areas must then be tested further via other methods using on-target testing. Because BDD provides known good functionality, when something fails in this testing, the investigation can focus much more directly on the specifics of the interactions, reducing the problem space.
For example, if concurrent firmware components work properly in a single-threaded BDD context but experience a failure during on-target testing, the problem is most likely related to the concurrency control, such as a mutex not used where it should have been.
Performance is a completely separate domain, because it depends on the entire system, from the target MCU, the on-chip and off-chip peripherals, and PCB design, to all the firmware components built with BDD and integrated from vendors and third parties. Performance and resource utilization need to be measured under real operational conditions on the real system. This is an entirely different testing regime from what BDD covers.
Similarly, there will be other system attributes that can only be verified on the total system with different testing methods, such as security. However, BDD plays an active role in developing the firmware that will be tested for these attributes, since they need to be built in from the ground up. To hear more about this topic, listen for the author of this post, Dojo Five alumni Steve Branam, on The Agile Embedded Podcast.
Do Embedded Firmware Right with Dojo Five on Your Team
Are you looking to modernize your embedded firmware development and support processes? You can book a call with us to get the conversation started.
