Effective Requirements is Pillar 6 of the Dojo Five Modern Embedded development practices. Project goals and requirements clearly defined, documented, and prioritized with traceability through verification and validation.
The set of requirements at the start of a project fall somewhere in the continuum of “nothing known up front” to “everything known up front.” Most projects start out as a hybrid on this scale, a combination of parts with little or nothing known, parts with something but not everything known, and parts with everything known.
Anything in the range of 10-90% known is realistic. That’s the practical range of “exploratory” to “mostly defined” projects.
The extreme ends of the scale are unrealistic. The “nothing” end is unrealistic because even a vague, ill-defined idea will have some basic set of target features and capabilities. The “everything” end is unrealistic because no matter how well-defined things are already, there will always be unknowns (the “known unknowns” and the “unknown unknowns”); that’s the nature of developing a new product.
Modern teams therefore take a hybrid approach, acknowledging this. They work in an environment where learning and iteration are the primary goals. As requirements are identified (including changes to already defined requirements), they’re documented, communicated to all stakeholders, and prioritized. Good requirements are user-facing to keep focused on what the end user gets out of the project.
Each iteration starts with the set of requirements that have been captured so far. As work proceeds and the conversation continues throughout the iteration, new requirements and changes may emerge, to be incorporated into the set for the next iteration. Understanding that change is inevitable as everyone learns more is key.
What causes change?
Change is a consequence of many things:
- Learning that additional features are needed to create a complete product
- Learning that additional details are required to work on the features already known
- Learning that an initial approach doesn’t work (or the customer doesn’t like it), and a different approach is required
- Changes to the hardware because of supply constraints or specification changes
- Changes to the business environment
- Assumptions that don’t pan out
The recent pandemic is an extreme example of unknown unknowns, causing chaos in supply chains and finished product supply and demand. It caused both technical and business changes that continue to ripple through the market.
Avoid All or None
Trying to build a system with no requirements is problematic because the team has no guidance, increasing the possibility that they’ll focus on developing the wrong things. That leads to frequent rework, or produces a system that doesn’t meet the business level goals. It wastes precious time building the wrong things.
Trying to build a system with all the requirements defined up front is problematic because it’s too rigid and prescriptive. It doesn’t provide the flexibility to adapt to the inevitable changes that will come up. As a result, it may still fail to meet the business level goals. It also invests a large amount of time before anyone starts getting feedback. This is one of the reasons most companies have moved away from the waterfall model.
In an interview with Dave Farley, host of The Engineering Room on YouTube, Kent Beck relates an anecdote where a skeptical workshop attendee asked, “Wouldn’t it be easier to just do it right the first time?” His response was “Yes, it would be, but that’s just not an option. Because you don’t know everything about what it is or how to do it, and the target changes, and the team changes, etc. Change happens.”
Efficiency Vs. Effectiveness
We often optimize for efficiency, when we really need to focus on effectiveness. Efficiency in a management process often means minimizing cost. But effectiveness means optimizing for a higher level success criteria to accomplish the best result.
For software development, efficiency often results in missing a higher level goal. It would be more efficient to get all requirements up front, because it’s predictable, it’s the quickest way to put something together. That’s what the attendee in Beck’s workshop was referring to.
But that comes at the cost of ignoring all the things we learn as we proceed with the development of the project. It ignores all the changes that may happen, whether driven by the customer’s understanding, the developer’s understanding, or external market conditions. That’s what Beck was referring to.
That’s why we talk about being effective with the pillars. It means getting the total job done. Efficiency is still important, but it’s a second order optimization. First be effective, then be efficient. An ineffective project that is done efficiently doesn’t produce the results we need.
Verification & Validation
Having effective requirements means that the product can be verified and validated.
- Verification: the product complies with requirements, regulations, and specifications.
- Validation: the product meets the needs of the customer and other stakeholders, and is acceptable to them.
Validation answers the question, “Are you building the right thing?” and verification answers the question, “Are you building it right?” It’s possible to fail on either axis: building the wrong thing, that works flawlessly; or building the right thing, that works terribly.
Effective requirements ensure that the product is valid, meets the customer’s needs, and provides a reference against which it can be verified. That happens over the entire course of the project as things change.
Using Behavior-Driven Development
Performing verification and validation is a mix of manual and automated activity. Automated tests can be written in a way that expresses requirements in a human-readable form that everyone understands, including non-technical stakeholders, and then can be executed to verify that the product meets them.
This is known as Behavior-Driven Development (BDD), extending the concept of Test-Driven Development (TDD). The tests express the behavior that the product needs to provide, capturing the requirements. This is useful for both acceptance tests and unit tests.
BDD uses the given, when, then format to specify preconditions for initial state, a command to execute (the thing being tested), and expected postconditions. As Kevlin Henney points out in his presentation Structure and Interpretation of Test Cases, this is known as a Hoare triple. The given initial state for a group of tests is implemented as a test fixture, and each of the tests for that given state use that fixture.
Tests can be named in a form that is a human-readable specification, and then written to carry out that specification. This allows non-technical stakeholders to work together with technical stakeholders to describe what the code will do, and then specify that in an executable form. It still gets technical, but this is where human language and programming language coincide.
For example, in the test fixture Given_Uninitialized_Connection for a communications service, the test name When_Opened_Then_Sends_Connection_Request clearly states what should happen. The test code then verifies that’s what the code under test does, using its public interface and the techniques of off-target testing for embedded systems.
Similarly, there would be test fixtures and tests for the various states, such as Given_Open_Connection, Given_Failed_Connection, etc. As the system requirements are understood and evolve, the tests evolve.
The tests themselves become the requirements specification. Any implementation that passes the tests, that provides the desired behavior, is acceptable. If the implementation changes, the tests continue to act as the criteria for automatically verifying it. If the requirements change, the tests need to be changed accordingly, which drives implementation changes.
This also provides traceability. The code in the implementation is traceable back to requirements because of the tests it passes.
Two things are crucial to the success of BDD:
- The product is architected to be testable.
- The tests focus on its behavior as defined by its interfaces, not on its internal implementation.
The other pillars of Modern Embedded development help you to achieve this success.
Start with Prototyping
Phillip Johnston of Embedded Artistry agrees on verification and BDD. He also says prototyping is an invaluable tool for pinning down requirements and mitigating the risk of significant change later in the process. It can be used to:
- Determine whether or not a requirement is even feasible
- Down-select from multiple interesting paths
- Clarify some of the unknowns
- Give customers a chance to give feedback early to refine/modify requirements.
He says contemporary tools and techniques enable embedded teams to do extensive prototyping on their development machines. But most embedded teams just prototype the hardware, with little thought given to the software system until after hardware is designed, missing out on a truly valuable practice.
It can be challenging to get people to put effort into traceability. Philip Koopman, Associate Professor at Carnegie Mellon University and expert in autonomous vehicle safety and dependable embedded systems, says, “In my experience, the only way to get someone to pay attention is if there is a smoking gun failure that just hit them recently that could have obviously been avoided with better traceability. They have to be ready to listen.”
In conclusion, “Effective Requirements” stands as a pivotal pillar within modern embedded development practices. This pillar underscores the imperative of meticulously defining, documenting, and prioritizing project objectives while maintaining traceability through verification and validation processes. The blog acknowledges the dynamic nature of initial knowledge and advocates a flexible approach to accommodate change. Shifting the focus from efficiency to effectiveness, the concept of Behavior-Driven Development (BDD) is introduced as a means to automate testing and foster collaboration. Architectural prototyping’s value in refining requirements and minimizing future revisions is highlighted, along with the critical need for traceability to avert preventable failures. This holistic approach ensures that projects align with goals and stakeholders, ultimately leading to successful and well-executed embedded systems development.
recommended additional reading
The free e-book Shape Up: Stop Running in Circles and Ship Work that Matters from Basecamp offers an interesting way to break up work to deal with unknowns, working in 6-week iterative high-level epics they call “bets” and “scopes”.
Need assistance with your project and want to ensure you’re utilizing effective requirements? We adhere to an agile process focused on rapid, repeatable, and reliable system development. In addition to building a great project and establishing effective requirements, we’ll ensure you’re set up for long-term success. You have projects–we can help. Learn about Dojo Five Services.