Technical Debt is Known and Created On Purpose
Technical debt suffers, like many other software development metaphors, from multiple, sometimes conflicting, interpretations. From being present the moment any code is written, to associations with “quick and dirty” decisions that will result in future refactoring being required, you’d be forgiven for thinking that technical debt is a “genium malignum”, akin to Descartes’ evil genius/demon for software developers that’s always present to slip a design into the mire of hard-to-maintain legacy code.
In all the exploration of technical debt’s debits, two characteristics to its credit are often common but rarely highlighted, they are:
- Technical debt is known.
- Technical debt is created on purpose.
When you are deciding between two solutions and opt for the faster, less clean approach, you are doing so with intent. Technical debt is not a surprise. You know you’ve taken a specific path. You’ve consciously and, hopefully, collectively decided to accrue some design debt, and everyone is comfortable with that fact.
Far from being an evil demon, technical debt is a powerful metaphor to guide you to collaborate on “good enough for now” solutions, while ideally not losing touch with those decisions that might lead to problems further down the line.
Technical Debt is far from being an evil demon looking to cause you problems, you’re a willing collaborator! However, not so with Technical Debt’s twin, Dark Debt…
Dark Debt is all about Surprise!
Dark debt, in stark contrast to Technical Debt, is always a surprise. That’s why it is dark; you don’t know you’re creating it and, even more importantly, you can’t know you’re creating it. Dark Debt is a natural occurrence of a sufficiently complex system, and it doesn’t matter how hard you think in advance, like a Pokemon that hasn’t read its marketing materials with Dark Debt you absolutely won’t catch it all as “Dark Debt is not recognisable at the time of creation”.
The effects of Dark Debt are complex system failures; anomalies that just were not predicted when a system was placed into production. I often encourage people to think of “incidents” as “surprises” for this exact reason, they are surprising! All incidents look anticipatable and avoidable in retrospect. At the time of the incident though, when Dark Debt is making itself known, things are surprising, confusing, and even brutally embarrassing. Far from obvious and avoidable.
Surfacing Dark Debt
Technical debt has the advantage of being known, so you can choose when you pay it back. You can choose to do that in advance or at the least-responsible-moment when your design is about to go bankrupt in the face of inevitable change, but it is a choice.
Dark debt doesn’t give you that choice. To deal with dark debt organisations are proactively investing in practices and tools to help them explore, experiment, and even experience it.
Dark Debt requires you need to push and prod at your design in production so that you can uncover your dark debt before it chooses to rear its ugly head in an expensive outage on its own terms and timescale, and this is one of the major reasons that Chaos Engineering and Learning From Incidents are getting so much attention across the industry.
Chaos engineering is a proactive practice that embraces the job of surfacing dark debt by running controlled chaos engineering experiments. Through deliberately injecting turbulent conditions, such as failures, into a system you can throw a light on your dark debt ahead of time. Through chaos engineering you can explore and invest in being better prepared for dark debt on your own terms and timescales.
The Learning From Incidents (LFI) work, originally headed up by Nora Jones (CEO of Jeli), and now augmented by her team’s recent work, “Howie: The Post-Incident Guide”, emphasizes how we can best learn from when dark debt rears its head in chaos experiments and actual surprise incidents.
The combination of LFI and chaos engineering are leading the charge on surfacing and improving a system’s resilience to dark debt. While you can never be sure all dark debt has been overcome, by starting to explore these practices you can at least get ahead of your dark debt and even be better prepared for when, not if, it makes an appearance!
