Experimenting: Touch Point Coverage

Areas where tests prevent refactoring pin-point how you can improve your code quality. Useful for teams that rely mostly on E2E tests and want an easy way toward unit testing.

Nov 05, 2024

This is an exploratory concept that I’m sharing with you today for the first time. Over the past few years this concept has remained a blind spot with most teams that I work with via Leadership coaching, Continuous Delivery and Test-driven development trainings.
I’m looking for feedback of any kind. Is this helpful? Did you find similar concepts? What worked for you, are the ideas clear and actionable?

How many times were you told to Test behaviors, not implementation details? Of course, nods all around. Until you realise no one can agree on what that means.

This idea keeps popping up in my conversations with engineering teams. Improving your test design requires rules that are pedantic but simple to follow at the same time.

The closest I came up with so far in terms of naming—and I’m terrible with naming these things, sorry!"—is Touch Point Coverage.

Touch Point Coverage

The number of public interfaces touched in a test suite per line of coverage.

There are some caveats of course when measuring too naïvely, I’ll get to those in a moment. TPC is less of a metric and more of an intuition about how tightly the testing surface area couples with the underlying class or function structures. Lower percentage is better, unless it is zero.

If it’s 1 to 1 (100%), the TPC is probably too high.
If it’s in the low 5-20%, then you probably have proxy classes or use cases masking the core behavior.

Okay, that’s the basics. I want your feedback so hit me!

Caveats

As always it’s easy to poke holes into too simplistic rules, so here’s a few examples with context how this could be used and where it doesn’t make sense. What did you find? Did I miss anything obvious?

What about End to End tests?

E2E tests would have a very low percentage of “public interface”, however they are too wide. End-to-end testing’s actual deterministic coverage is likely zero. It is smoking out possible issues, but because E2E tests can have multiple reasons for failing, they do not provide adequate coverage for the unit under test.

You still would want tests in addition to E2E and smoke tests that are faster and more deterministic, focused on the failure of only 1 behavior at a time, through a shared interface.

What about testless workflows?

Event Modeling comes to mind. Itsup front design produces public interfaces and desired behaviors which focus on provability and correctness. The TPC of such a system would be close to 100% when tested against each component’s behavior specs (Given-when-then) even when not done in a TDD fashion or with zero abstractions.

Given the effort required to repeatedly do smalll-design-up-front diligently and keep the models up to date would be worth it. For example, Adam Dymitruk’s business is focused on event modeling and event sourcing as their core process and are famously not doing any TDD or technical shenanigans, relying purely on the correctness of the model.

I think that’s okay but the small-design-up-front with a detail event model is a huge factor in this Along with the experience of keeping the data flow behavior focused. I’d need to talk to Adam to get his insights on this.

EDIT: Perhaps in an event model the “public interface” can be focused on the views and projections rather than actual coding structures (classes, functions) to maintain the efficacy of the metric.

Conclusion

“Test behaviors, not implementation” is hardly ever actionable advice
Playing with a metric that is easy to follow, so that you can improve your design
There will be edge cases where the metric doesn’t make sense
Let’s keep testing and improve our design
Continuous Delivery studies show that well-designed tests and small batch cycles enable tech teams to spend 44% more time on meaningful pursuits (rather than bugfixing or meetings). That’s what we’re all about at Crafting Tech Teams.

Thoughts?