The Many Ways of Disagreeing What Testing Really Is
A few thoughts on testing diligence
A controversial view on testing that I posted on LinkedIn today that sparked a few interesting conversations:
The only purpose of manual testing is to figure out how to automate it in the future.
Or do you think you will only ever deploy this once?
There were two general camps of responses: those arguing about what automation and manual means - and those that argue whether testing is QA or about development. I find that distinction interesting, let’s dive in!
☕️ Software is like Coffee
Testing is akin to tasting while cooking or brewing. You may follow the recipe to the letter but there are thousands of little things plus your overestimated lack of understanding that can cause quick unexpected shifts in your dish.
I’m a part-owner of a local specialty coffee business, and was surprised to learn at the beginning that micro-shifts in atmospheric pressure and half a degree temperature shifts can greatly impact (ruin) an espresso shot.
Well, to be exact — that’s what integration testing is. Eliminating all local interference, we test whether the current environment makes our expectations shift. But let’s not get ahead of ourselves. What are expectations?
Expectations vs. Assessments
When you combine Event Modeling diagrams, Extreme Programming stories and ubiquitous languages (Domain-Driven Design) a very consistent pattern emerges.
Everybody keeps asking questions like:
How will the user know it works?
How will we know it works?
What does work mean?
What do you mean by that?
Why do they want it?
Why not use something cheaper / more practical?
What questions will we ask to determine its value?
Edit: Reader Bogdan Veliscu suggested adding the following to the list:
How can we ensure that our solution remains adaptable and maintainable as the business requirements evolve over time?
I’m excited to receive feedback, glad to see your responses!
You can use these sets of questions to gather business requirements, slice user stories down to acceptance criteria level or even extract important business terminology and artefacts from a subject-matter expert.
It’s worth writing down. Or if you prefer, bookmark this post on Substack.
Ultimately, answering these questions via tests gives us confidence in our work.
Confidence that it will work once delivered.
Expectations require an understanding of the functional working state
Assessments are about observing our own emotional state and responses
The former is for function, the latter form.
When to automate
There is a common advice to do it manually first, then automate. I automate on step zero. I recommend you do too. When possible. If not, I follow the advice above, but only to learn how to automate it.
When you have access to Docker images and are about to install new packages, I don’t do it manually first on my host machine and then Dockerise the second time. I use Docker from the beginning.
I follow the same pattern when using Test-driven Development. The testing-first drives my mind to ask the questions above. Not to have full code coverage. But to understand the problem scope separated from the solution-scope.
Then I dip my toe in the solution space. Make it work. This is usually very quick, which is surprising. Now that the functionality works and is test-covered I’m confident I’ll make the deadline. So I refactor it for better maintainability. And —should it be necessary— removal.
After all, the feature may work functionally, but fail to provide value to the customer. I want such features to be pruned easily and without much effort.
QA does not create, merely assures what’s already there
I firmly believe understanding the system behavior, how we test and write acceptance criteria is a joint effort of engineering, product and business analysis. A QA process separate from the prior trio as part of a deployment process has shown to cause more harm than good.
Testing, unit testing, automation and creating tools for assessments and support research (triage) all belong into the domain of the engineering team that creates the software.
At its core, software is a liability and all contact with it should be minimized of human manual labor. The business wants the capability the software provides, not the lines of code and tests that make up its structure.
The Properties of High Quality Tests
They run fast
They tell you where the problem is when they fail
They are easy to remove when they become obsolete
They are written in the language of the user, using terminology natural to the product
They are testing expectations of behavior, not the current implementation
I won’t go into detail right now, on the more controversial aspects on the above list. That will make a great weekend essay instead.
How do you test? What does your team automate?