4 Common But Insidious Patterns That Derail Software Projects
With experience, you can recognize the less obvious patterns which lead to costly disasters.
Welcome to the Scarlet Ink newsletter. I'm Dave Anderson, an ex-Amazon Tech Director and GM. Each week I write a newsletter article on tech industry careers, and specific leadership advice.
Free members can read some amount of each article, while paid members can read the full article. For some, part of the article is plenty! But if you'd like to read more, I'd love you to consider becoming a paid member! This note was originally written in 2021. I re-wrote it to match my current writing length/style.
Random side note. People often ask me how I use AI in my writing. My general answer is that I don’t find AI to be all that good at writing. I mean, it’s pretty good at writing basic code, but I really don’t like its writing style when it comes to paragraphs and words.
What I do find it great for though is finding words. I’ll know a word exists, but I can’t find it in my head. So I’ll explain the word to AI, and it’ll almost always come up with the right word. Such as today’s headline. I knew there was a word which was a combination of “subtle” and “not well known” and “bad”, and I asked AI, and it said insidious, which is precisely the word I was looking for.
So how do I use AI? It’s a glorified thesaurus. Anyway, moving on to our topic!
We had a junior software engineer join one of my teams, and not a week later, they sent this email to the team early in the morning.
Hey all!
I was taking a look at our sprint board today, and I see a number of in-progress tasks which haven't had any updates in over a week. I hope you don't all mind, but I've taken the liberty of moving every task without an update into a temporary parking lot status. I scheduled a meeting this afternoon to review those parking lot tasks. If we don’t know the current status of tasks, I think it will be hard to know if we’re on track to complete this sprint.
Ignoring our janky process at the time (something else we fixed later), what does that message tell you about that new hire?
The other (relatively junior) software engineers on the team shrugged and essentially said, “That was nice of them.”
But I was immediately convinced that this was an excellent hire. What signals did I see in that small action?
I recognized the courage and ownership it took to take action on tasks which weren’t theirs. Particularly as someone new on the team, it takes a lot to step in.
I saw bias for action which had them take action so quickly. Not just to complain (which is easy), but actually do something about it.
There was empathy combined with high standards. The engineer pointed out (subtly) that their co-workers weren’t properly managing their tasks, but phrased it around addressing the issue, instead of complaining.
What would lead me to be impacted more than the other employees? I think it’s pattern recognition. I recognized behaviors in this employee which I’d seen in other exceptional new hires in the past. As soon as I saw this email, I thought, “oh, got a good hire here!”
Some patterns indicate that you’re seeing something good. The signs of a future excellent employee in how they communicate with their team. A project which has closed out all ambiguous and risky tasks early. A cleanly designed feature which is likely to scale well.
Other times you’ll see patterns which suggest future problems. Some of these patterns are obvious. Perhaps it’s an employee who insults their co-workers when providing feedback. Or a project which is late on every milestone. In these cases, almost everyone can see the issue.
In other cases, patterns are subtle. Other people may think things are fine, but the experienced employee knows that things are not at all ok! We’re like, “Oh no, dudes and dudettes! We’re in trouble!”
These are the types of patterns you learn with experience. I’m going to go over four of those tricky and subtle and insidious patterns.
1. The belief that a big bang launch will make things easier.
It might be a critical piece of infrastructure that needs to change. Or a core technology component which needs to be replaced. Perhaps your entire UI needs a redesign.
Regardless, engineering teams have two choices.
A big bang launch — Where everything is written to the side of the production system, and then “bang!”, it’s released. If we were talking about cars, it’s like you’re driving around your Corolla, while you build a Prius in your garage. When it’s done, you sell the Corolla, and drive the Prius.
An incremental launch — Where bits and pieces are re-written one at a time, and they’re released into production as they’re complete. Again with the car analogy, it’s like driving your Corolla, but you’re replacing old parts while at red lights.
In my car analogy, it sounds insane to do an incremental launch. That’s just so complex!
Engineering teams often feel the same way. They’ll suggest that the change would be easier to make if it was all written and deployed at once, rather than piecemeal. But it’s not just the engineering team. The marketing team is excited about the big drop because they believe a large launch will be more dramatic for customers.
I'll jump to the punch line — it is (almost) never a good idea to do a big bang launch. With almost no exceptions, every single time the big launch path was chosen, it was later flagged as a mistake.
Why big launches are a bad idea:
You learn from deploying code to customers.
When you write code in isolation, you miss learning. With a big bang launch, you’re writing significant amounts of code without taking advantage of what you learned with your previous code. With a big bang launch, every single line of code or every design choice could be wrong. Keeping with our car analogy, it’s entirely possible that this new Prius thing you’re building won’t run at all.
Mistakes are more impactful.
When you’re doing incremental updates, that single page or feature changed has limited potential impact. When you update an entire product, it has a much larger potential impact. Any mistake is compounded by the size of the launch. If you were changing a tire on a car at a red light, the worst-case scenario is that you have one flat tire when the light turns green. If you built an entirely new car in your garage, the engine could explode when you turn it on.
Value is held back.
If you believe there is value in what you're delivering, you're holding that value back from customers for the length of the project. If you’re incrementally building your work, you receive value from your spent effort by launching whatever you’ve built once you’ve built it. If you’re doing a big bang launch, no one benefits until the entire project is done. In the car analogy, your new tires immediately make your Corolla more functional. If you build a Prius from scratch, you keep driving your old Corolla with bald tires for as long as it takes you to build that new car.
Delays are larger.
Delays tend to scale with the work involved. If your 1-week project is late, it might slip by 2-3 days. If your 1-year project is delayed, you could slip by 6+ months. Larger projects mean more variability in delivery times. You might think that you’ll have to drive your trashy Corolla for 1 year, but you might be driving that clunker for 2+ years as you try to get that Prius working.
Inevitably, the combination of the above means that almost all big bang projects go wrong in some way. For all the work put into the project, you get zero value for longer than expected. You create some mistakes which are high impact, and when it’s all done, you learn things that you should have done differently anyway.
In the end, everyone says, “I know it sounded easier to do a big bang launch, but there was no way to predict that an incremental launch would have been a better idea.” But I’m telling you that it’s absolutely predictable, and you should really find that way.
You avoid this pain by splitting projects into multiple parts. However, there’s a reason that people think a big bang launch will be easier. There are almost always business or technical barriers in the way. Avoiding or breaking through these barriers, even with expensive heroics, is preferable to a future disaster.
If I had to choose between a 12-person year effort big-bang launch, and a 20-person year effort incremental launch, I would choose the 20-person year incremental effort every time.
Why? Because I guarantee that the 12-person year effort will balloon into a disaster of 25-person year effort project with escalations and delays.
That incremental 20-person year effort launch may end up being 25-person years as well, but it will be smoother, calmer, higher quality, and more predictable. It will also have delivered value to customers every step of the way.