> Mr Wright's Rules
Article

Why is Nobody Talking About Business Rule Coverage?

How business rule coverage is a game-changer for software development

By Greg Wright · 1 June 2022 · 5 min read · Originally published on Medium (opens in a new tab)

Where to you keep your business rules? Over the years I’ve seen them everywhere: front-end, back-end, domain objects, business-rule layer, database access layer, in stored procedures in the database — everywhere.

What if we could have one source of truth for business rules and for that source of truth to drive our tests?

We would only have one place to look for business rules. We would not have to trawl through the code at each layer trying to work out what is actually going on.

One source of truth for business rulesRules scattered across the front-end, back-end, domain objects and database drift out of sync. Instead, a single set of business rules drives the tests, and the tests verify the application.Rules scattered across the layersFront-endBack-endDomainDatabaseThe same rule, copied and drifting out of syncOne source of truth driving the testsBusiness rulesdriveTestsverifyApplication
Scattered rules versus one source of truth that drives the tests.

If we had 100% business rule test coverage, and a green build, we would have huge confidence in the functional correctness of our application. We would be able to refactor or even rewrite our application in a new language and have huge confidence that it was still correct.

If a business rule changed a BA would update the rule and corresponding tests on a branch, causing a test failure on that branch (because the test has changed but the application has not). A developer could then take over to make the build green again.

What happens when a business rule changesThe BA updates the rule and tests on a branch, which turns the build red. A developer then implements the change to make the build green again.The BA updates the rule and its testson a branchThe build goes redthe tests changed, the application hasn'tA developer takes overand implements the changeThe build goes greena very good release candidate
How a changed business rule flows through the team.

Focusing on business rules at an early stage of User Story development leads to stories where testing has been considered early in the development process. These tests both elucidate the User Stories and lead to better estimation.

The business rules can be tested via Feature Specs using Behaviour Driven Development (BDD). Those Feature Specs can be drawn out as the User Story is developed. Then the availability of those specs can be part of the “Definition of Ready” and feed directly into the development process.

Practical Learnings

We implemented Business Rule-Based Testing using precisely this process, following a fairly simple recipe:

  • All tests must be capable of running individually and independently. This allows the tests to be run in parallel. To do this we tagged each test with a unique ID and business rule IDs of the business rules under test. The test runner could take a tag (or set of tags) as input and run only that one test or a set of tests covering a particular business rule.
  • Put the business rules into the feature specs as comments. Each Business Rule can have one or more tests, but the tests should be tagged with the business rule’s number.
  • The tests need to be robust. Flakey tests (often due to poor handling of eventual consistency) cause problems.
  • Keep an eye on long-running tests. Log the time each test takes to run, so you know which your long running tests are and work to split them up or optimise them. (If you could do massively parallel testing, and every test started at the same time, then the length of time your test suite took to run would be the length of your longest test.) Since we are looking for fast feedback we need the tests to be efficient.

We developed our User Stories in a number of simple stages:

  • In the first stage BA and tester would work together on the Business Rules and Feature Specs in Jira, as well as any other supporting information. Feature specs can be developed in Jira or on a git branch such as ‘feature/Jira-1234'
  • ‘3-Amigos in’ : The Tester, BA and Developer discuss the requirements
  • Developers would then refine and plan development based on the testing requirements in the Jira tickets. We would then do technical analysis and estimate the work needed to write the implementation of the tests and application code
  • Developers write both the test and application code, ensuring we have a green build
  • ‘3-Amigos out’: The developer demonstrates the new functionality to the tester and BA for signoff
The stages a User Story goes throughFive stages: BA and tester write rules and feature specs, 3-Amigos in, developers refine and estimate, developers write the test and application code until the build is green, then 3-Amigos out for sign-off.1. BA and tester write rules and feature specsin Jira or on a feature branch2. 3-Amigos inTester, BA and Developer discuss the requirements3. Developers refine, analyse and estimatebased on the testing requirements4. Developers write the test and the codeuntil the build is green5. 3-Amigos outDeveloper demonstrates to tester and BA for sign-off
The stages a User Story goes through.

We found that the tests were both great integration tests and acceptance tests. After a while we had to build a system to run the tests in parallel in order to give fast feedback to developers. Focusing on parallelisation sounds expensive, and its certainly not cheap, but it is much cheaper than the cost of manual testers. In practice our manual tester was roughly part time, focusing mainly on developing User Stories and manual acceptance checking. A human can apply common sense in ways that an automated test never could. That one tester was supporting a team of 14 developers. The parallel test solution, though expensive, cost much less than the cost of a manual tester’s salary, and was much more precise and efficient.

Do Business Rule Tests Make Unit Tests redundant?

Absolutely not. Unit tests are still extremely important and it is important that they are proper unit tests (not coupled to a database or external service, not coupled to the actual implementation and that they test behaviour) and that they are super-quick to supply the developers with fast feedback. One annoyance is developers waiting for the whole test suite to run when running the unit test suite could provide fast evidence of a problem.

Business rule tests are part of your test-driven strategy but are not TDD as such. The tests are larger and less iterative than TDD tests. They do however have the advantage that they describe the desired behaviour of the system, are never coupled to implementation.

Do Business Rule Tests take a long time to run?

They can do — but you shouldn’t let that get out of control. You are aiming for fast feedback. You need to keep an eye on test times, deal with long running tests and run the tests in parallel.

What other advantages did you find?

The process made BAs, testers and developers work closely together. We were much more aware of User Stories under development and put much more thought into the testing requirements. We got better at estimation and we knew that whenever we had a green build we had a very good release candidate. We did continuous delivery into a production-like environment and did some manual verification (applying common-sense) before releasing.