Technical Debt In Software – The Fine Line Between No Architecture and Over Architecture

As I am writing this, I am sitting on my condo’s terrace in downtown Toronto, baking in the sun while I take in the noise of the city and cars and people down below at the street level.  There is a helicopter that seems to be circling the downtown core for quite a while now.  It’s freaking hot today and I could probably use some water.  Be right back.

At the time of writing...

(1 minute later…)

Ok – water has been acquired….. I also grabbed a Kilkenny and poured it into my Kilkenny glass (Kilkenny requires the right glass to be enjoyed properly)

It’s been a while since I’ve posted, and career wise a lot of big and exciting changes have been made.  I made the move to independent consulting, and I am enjoying it big time!  Now my efforts are shifting from helping one organization develop bleeding edge scalable systems to helping many organizations with development, architecture, minimizing technical debt, and team building.

Along the lines of the type of work I’m focusing on, I want to write a bit about bad architecture, over architecture, and technical debt.

Ok, so I’m going to talk about the benefits of a “value added approach” to software.  I’ve seen a lot of systems in my day – ranging from poorly architected apps that deliver high amounts of business value to over architected systems that, instead of delivering their expected business value, became a total utter failure for a multitude of reasons… and everything between.

To the business, a successful system is typically one that delivers on it’s promises on value to the business.  The negative impact of technical debt is not always seen by the business teams and is sometimes seen, albeit indirectly, as “necessary”.  So, in these scenarios – why is it necessary?  is it job security for the dev team? Lack of training?  Lack of standards?  There could be a plethora of reasons, but in the end the technical debt introduced by these systems is high and could cost the organization millions of dollars.

High business value systems and mission critical systems can suffer from an endless amount of technical debt due to lack of design standards and architecture.  This technical debt is not always apparent to the senior business teams. The business loves the system however, but they sure don’t understand why it takes such a long time to add new features or track down bugs.  The business leaders at the top see the system as great too, but the fact that it’s overly fragile, requires daily maintenance and an overly large team just to support it seems somewhat necessary – plus it’s “just the way it is”, right?

The real deal here is that technical debt, and overly large dev support teams are just not necessary at all.  The right people, training, technical skills, system architecture, and business leaders have the potential to create the right systems and find that fine line between a proper development architecture and business value.

Yet, there is another extreme…..Over architecting and big ego’s….

People have ego’s… Fact of life .. When architecting a system, I’ve seen many software architects with an ego.  Their system is great, using all the right design patterns.. Look at how cool my undo system is with the command pattern implementation I came up with.  Watch how I can add one entry to the configuration file and all the sudden the entire behavior and business logic of the app can be changed… Our customers can now create their own custom modules using my handy dependency injection techniques and augment the app with their own fancy things they want to do…..  Pretty sweet eh?

I agree, ok it’s pretty sweet (and fun to work on) and I’ve seen and created my fair share of ‘coolness’ in my systems.  There are always business cases for these types of things and having the right technical team and abilities to implement them properly is key.  The problem is over architecting when they aren’t needed or for the dreaded reason ‘just in case in the future’.  The fact is, this can sometimes delay shipping the product and complicate the development.  There is typically a big disconnect between the development team and the business value in these scenarios.  You end up having a technical team who is more focused on architecting than on providing business value.  Focus on what’s needed, and if it is, and there is a case for having it… Build away and have fun!

Both of these scenarios above can lead to long term technical debt.  The trick in software is building a team who can leave their ego’s out of it, share knowledge of the system and the business value proposition, and focus on what’s important for the long term success of the project while minimizing technical debt.  Doing this has the potential to create phenomenal systems that are well architected, bugs become easy to track down, the system can grow organically the way it needs to, the size of the development team can remain minimal, and the business is happy knowing that maintenance costs are reduced and new features can be added to the system quickly.  There is no more fragility – the system becomes clear and small changes are less likely to have unexpected bad consequences.

There is a fine line between both of these scenarios and it requires practice and discipline to build the right team who can achieve and continually create well architected solutions that maximize business value and eliminate technical debt.  It can be done and can be done very successfully.  To truly master this as a software/business team, it needs to be instilled as part of the culture of the team.  The right leaders and people are very important in truly creating world class software solutions.