Don't Optimise what Shouldn't be There
The purpose of planning and deeper understanding
Today’s Crafting Tech Teams issue covers a topic many like to avoid: thinking. Specifically the kind of thinking that doesn’t involve doing—analysis and planning. As you may already be used to our main pillar, I will approach this issue from a strategic and tactical vantage point.
Today’s topics:
Tunnel vision: the importance of zooming out your focus. How to zoom out? When to zoom out?
Partially done work is the root cause of all delays. Learn to split up work so it can be finished fast. This includes integration and testing for software.
Defer commitment and remove processes first. Try the process a few times and learn from it. Do not optimise it before you have taken lessons from a process. This can lead to: making bad processes more expensive or analysis paralysis.
Optimise what’s left. Optimise last or not at all. This should be the last step. Only optimise after you have clear feedback or metrics that give you a clear picture on what performance you are aiming for.
💬 Join September’s Book Club: Implementing Lean Software Development. I host a monthly book club on tech leadership and craftsmanship topics. We combine speed reading with an accountability system. Learn how to read practically and apply in new knowledge in a month. It’s freefor now.
Tunnel vision: the importance of zooming out your focus
Two aspects of Tunnel Vision can hamper a team, regardless of their good intentions:
Strategy / Blinders: Being stuck on a problem for so long they can’t see the forest for the trees
Tactical / Solution attachment: Sunk cost fallacy kicks in and everyone is attached to “this is how things are”
Jocko Willink talks about this refocusing of efforts as “Step back and detach” in a stress or heated situation. No matter if in combat, a public space, a meeting or a phone call, an email.
The Google Software Engineering handbook notoriously mentions Blinders in the context of changing a team’s management. Often the problem evolves and changes while you’re working on it. Your team easily gets stuck in the deficiencies of your processes and follow a plan they set out 2 weeks ago without adjustment. This is wasteful, as they may be building a solution to perfection for a problem that doesn’t exist anymore.
Or the client moved on.
Or changed their mind on your competence.
The second aspect is trickier: inherit a codebase you don’t know and your gut reaction will tell you “I’m sure this works and is necessary, otherwise it wouldn’t be here.” The investment and hardships of the current and prior teams creates an attachment to a fake sense of value that went into building it.
The solutions to both are simple, yet difficult to emotionally come to terms with:
Cut features
Trim the roadmap
Deprecate features
Stop shipping unused functionality
Partially done work is the root cause of all delays
Unfinished work happens due to two reasons:
The deadline wasn’t serious—the feature-work should be dead after the line!
The feature is only deployable at the end of its process lifecycle
We touched on this already on the aspect of slicing and feature flags with vertical slice architecture. Linking it below to jog your memory.
A high performing team is capable of delivering continuously. This is possible by balancing two competing forces hourly or daily:
Feature flagging so there is continuous proof of progress and value towards the deadline. For this to be smooth, requires investment in Point (2).
Improving—or bypassing— the parts of the deployment process that are cumbersome. If it’s painful, do it more often. However, this impedes Point (1).
You may wonder why old branches with unfinished features are such a problem. After all, they are already there and ready—why don’t we just finish it? Such level of understanding is a common problem faced by more junior engineers and even non-technical managers or founders.
Looking at releases holistically, you may divide into several decreasing layers of “reality”:
On production, used by real users—this is the Strongest level of “finished”
On production, but feature flagged, being tested by a subset of users or internal personnel only with test accounts and test data
On “staging”, exposed to a fake environment and lacking major integration points
On “staging” on a non-main branch in a simulated, automated Q&A environment with known, deterministic fixture-scenarios
Isolated in a pipeline on non-main branch, running subset of isolated, non-I/O tests only
Running the non-main branch locally on an engineer’s machine, running automated tests in a deterministic environment
Running the non-main branch locally, engineer is testing manually—this is the Weakest level of “finished”
As you can see, the “stuck on a branch” is Level 5. There are actually many combinatorially increasing possibilities of things going wrong as it gets closer to Level 1. The older the branch, the more these unaccounted for, untested problems accumulate. This inevitably causes a huge surprise for any engineer trying to merge this to mainline and deploy it without issue.
If your process is such that this is allowed to happen in the first place, it is very unlikely you have 5-6 states for a story between Level 5 and 1. Ready for Deploy, testing or QA are the ones most often suggested by Scrum/Agile JIRA modules and these stages notoriously hide a lot of problems that result in delays if not accounted.
However, you would be mistaken in thinking this process is healthy and should be expanded upon. The solution for this issue is much simpler: do not allow branches to age. Merge them fast and early and do not hold on to old ones.
Delete old unmerged branches. If someone could wait 2-3 months for it, regardless of effort invested, it obviously wasn’t an important high-market demand feature. If it cannot be safely merged, then it naturally calls for rework. Most often this additional investment is an endless distraction. Simply delete such branches and open PRs.
Defer commitment and remove processes first
What I love about Event Modeling is that it allows for a quick exploration of the solution and problem space. You can quickly asses which verticals are UI-facing, which ones touch sensitive data and which ones no one cares about or has gaps.
Any slice outside of the space of valuable or productivity enabling should be left alone and deferred to a future step. Decide when the best moment is to make the decision and move on.
For a non-trivial feature or project this manifests as a simple workflow:
Explore by making a plan, slice the plan and prioritise
Decide on what to work on next. This is usually 5-10% of the overall plan.
Throw the other 90% of the plan away.
The working and replanning is what creates knowledge. Knowledge is what drives decisions. Do not optimise a process centred around second-guessing, worrying and anxiety. Eliminate these doubts instead by lowering the cost of taking the next step.
“An early design cannot fully anticipate the complexity encountered during implementation, nor can it take into account the ongoing feedback that comes from actually building the software. Worse, early detailed designs are not amenable to feedback from stakeholders and customers.“ —Implementing Lean Software Development, Mary and Tom Poppendieck
Optimise what’s left.
Only when you have adopted a healthy process with features that have shown successful iterations via deployment and user feedback should you attempt to automate and optimise anything.
Most teams obsess about automating so fast that they do this entire process in reverse—creating a tangled mess of processes and review hand-off stages for activities that shouldn’t even exist. To name a few:
Multi-stage Pull Request Blocking
Needless sign-off for deployment
Running automated QA or CI/CD pipelines manually
Automating email response systems rather than creating visibility and self-correcting UIs to the end-users
Optimising meetings that shouldn’t exist
Optimising review flows for documents that are being written by the wrong people
Optimising feedback and review loops that can be resolved with a decision or email
Indulging too much in analysis and estimations rather than looking at past evidences of progress and projecting
To summarise:
Do it manually
Check if it makes sense
Try to remove it
If it stays and it’s valuable, automate/optimise it
The only reason for you to do tasks and activities manually is to figure out how and whether they should be automated. The software you are building is not a commodity, but the processes and tools with which you build them are. Involve your team and simplify your tools. Simplify your process. Then make your tools more complex to handle non-negotiable, valuable and cumbersome workloads.
And exactly in that order!
💬 Join September’s Book Club: Implementing Lean Software Development
We are about to kick off a book challenge in September. Topics: Apply lean improvements in your software teams, learn speed reading, have fun. DM me LEAN if you’d like to join.
In other news, we have a live stream tomorrow on Event Modeling with Clean Architecture. If you have any questions, reply to this email or drop by tomorrow.