I spoke with Dave Farley on Test-Driven Development and DORA Metrics

“I think that software engineering is one of the most important things that we do in our society. It matters a lot. We ought to be better at it in our industry. And I think [this] is how we get better at it.“
—Dave Farley on Continuous Delivery

As you can see on the still frame, we had a blast! I’ve finally been able to sift through the master recording. I captured the highlights from our conversation below where we discussed the following topics:

• The Engineering Mindset

• Alternatives to DORA for Continuous Delivery

• A Finished Feature is Ready to Go

• TDD Improves on These Metrics

• Unanswered Questions

The Engineering Mindset

The challenge in Continuous Delivery adoption is going from an “inspection” to an “experimentation” mindset. The DORA data claims that if you adopt continuous delivery, you’ll have 44% more capacity to devote to building new features. Slam dunk!

But the shift carries friction, pulling everyone out of their comfort zone: engineers, managers, product designers, testers, the business stakeholders, data scientists.

Measuring stuff is hard. The big ticket problem is the mindset to start measuring meaningfully across the entire value stream, to get past the resistance of changing processes which seem fashionable (ie. git flow, feature branching) while focusing on the engineering improvement rather than being dogmatic about it.

Alternatives to DORA for Continuous Delivery

DORA Metrics seem to be enough for most modern software engineering as a generic measure to stability and throughput. The fundamental areas of improving measures of success are:

• The quality of the software we produce

• The rate at which we can produce software of that quality

They provide actionable feedback on building the software the right way. But don’t put all your eggs in one basket. What DORA metrics lack in leading/trailing metrics is whether you are building the right software. That’s what product and support feedback is for.

But those three things will generally cover most your team’s needs which is a huge step up from the bleak confusion of how to build software products in the past few decades.

A Finished Feature is Ready to Go

Release every day or sooner, when possible. Social meetings to deal with the “team stuff” every 2 weeks.

This is the original sprint/kanban flow with continuous delivery. Notice how the human cycle is separate from the delivery cycle. I notice in my practice coaching engineering teams that they are stuck with these two processes glued together, forcing them to “slow down” delivery to their meeting regiment.

Such a setup is rigid and is indeed the main anti-thesis to high performing teams who separate these two following the mantra: as many releases as possible, as few meetings as possible.

For leaders especially, a Genba approach of being “on the floor” with the team as part of the team is a strong predictor of being able to influence and implement change in an engineering team.

TDD Improves on These Metrics

Stability is the measure that most strongly corresponds to quality indicators with regards to a team that does TDD compared to one that isn’t. DORA Stability measures the quality of the changes delivered and the team’s ability to repair failures. Practically a combination of change fail percentage and recovery time given a certain critical subsystem (benign greenfield projects are not worth measuring for this).