This is part one of a two part series. The original article Refactoring: 4 Key Principles was originally featured on the Agile Advice blog by Mishkin Berteig.
Every software system that we build is inside a dynamic environment. The organization(s) using the software are all in a state of constant change. The people using the software are also constantly changing. Due to this constant change, every software system needs to be adapted to the environment in which it is used. Most of the time, businesses think of this constant change in terms of new features and enhancements – the scope of functionality that a system can handle. Less commonly, businesses think of this change in terms of the obvious external qualities and attributes of the system such as performance or security. But almost never does an organization, from a business perspective, think of the invisible qualities of the software system such as simplicity and technical excellence.
What happens when the business does not recognize those invisible qualities? I’m sure almost every software developer reading this can answer this question easily: the system becomes “crufty”, hard to maintain, bug-prone, costly to change, maze-like, complex. Some people refer to this as legacy code or technical debt.
The longer this state is allowed to continue, the more it costs to add new features – the stuff that the business really cares about. It is pretty easy to see how this works – for someone who has a technical background. But for those without a technical background it can be hard to understand. Here is a little analogy to help out.
Imagine that you set up a system for giving allowance to your kids. In this system, every week your kids have to fill out a simple form that has their name, the amount that they are requesting, and their signature. After a few weeks of doing this, you realize that it would be helpful to have the date on the form. You do this so that you can enter their allowance payments in your personal bookkeeping records. Then you decide that you need to add a spot for you to counter-sign so that the paper becomes a legal record of the allowance payment. Then your kids want extra allowance for a special outing. So you add some things on the form to allow them to make these special requests. Your accountant tells you that some portions of your kids allowance might be good to track for tax purposes. So, the form gets expanded to have fields for the several different possible uses that are beneficial to your taxes. Your form is getting quite complex by this point. Your kids start making other requests like to be paid by cheque or direct-deposit instead of in cash or to be paid advances against future allowances. Every new situation adds complexity to the form. The form expands over multiple pages. Filling out the form weekly starts to take significant time for each child and for you to review them. You realize that in numerous places on the form it would be more efficient to ask for information in a different way, but you’re not sure if it will have tax implications, so you decide not to make the changes… yet. You decide you need your own checklist to make sure that the forms are being filled out correctly. A new tax law means that you could claim some refunds if you have some additional information… and it can be applied retroactively, so you ask your kids to help transcribe all the old versions of the form into the latest version. It takes three days, and there is lots of guess-work. Your allowance tracking forms have become a bureaucratic nightmare.
The forms and their handling is what software developers have to deal with on a daily basis – and the business usually doesn’t give time to do that simplification step. The difference is that in software development there are tools, techniques and skills that allow your developers to maintain a system so that it doesn’t get into that nightmare state.
For a more in-depth description of this process of systems gradually becoming more and more difficult to improve, please see these two excellent articles by Kane Mar:
Technical Debt and Design Death
Technical Debt and Design Death: Part II
Ultimately, a software system can become so crufty that it costs more to add features than the business benefit of adding those features. If the business has the capacity, it is usually at this point that the business makes a hard decision: let’s re-write the system from scratch.
I used the word “decision” in that last sentence. What are the other options in that decision? Ignoring the problem might be okay for a while longer: if the company is still getting benefit from the operation of the system, then this can go on for quite a while. Throwing more bodies at the system can seem to help for a bit, but there are rapidly diminishing returns on that approach (see The Mythical Man-Month for details). At some point, however, another threshold is reached: the cost of maintaining the operation of the system grows to the point where it is more expensive than the operational value of the system. Again, the business can make a hard decision, but it is in a worse place to do so: to replace the system (either by re-writing or buying a packaged solution), but without the operating margin to fund the replacement.
In his articles, Kane Mar describes this like so:
It’s pretty clear that a company in this situation has some difficult decisions ahead. There may be some temporary solution that would allow [a company] to use the existing system while building a new product, [A company] may decide to borrow money to fund the rewrite, or [a company] may want to consider returning any remaining value to their shareholders.
In other words, refactor or die.
About the Author
Mishkin Berteig is the President and Co-Founder of Berteig Consulting Inc. He has been training, coaching and consulting for organizations adopting Agile methods since 2001 and is committed to helping individuals, teams and organizations apply Agile methods. Mishkin is a Certified Scrum Trainer, and qualified to deliver OpenAgile and Agile Project Management training. He has developed and delivered Agile training both in public and in-house seminars for over 3000 people in Canada and abroad. Courses have been as short as three hour intro-style and as long as five day boot-camp-style, and audiences have ranged from junior team members to senior executives. He has also assisted organizations of all sizes to make the transformation from traditional methods to Agile/Scrum methods (Extreme Programming, Scrum, Lean, OpenAgile). Assistance includes Agile Engineering Practices, Agile teamwork, Agile project and product management, Agile management and executive management.
