In the Scrum Master and Product Owner classes that we teach, this topic comes up frequently: how does the business account for refactoring? How do we “govern” it? How do we make good decisions about refactoring?
There are a few principles that are important in helping to answer these questions. All of these principles assume that we are talking about refactoring in an Agile team using a framework like Scrum, OpenAgile, or Kanban.
Refactoring Principle One: Keep It Small
Refactoring is safest and cheapest when it is done in many small increments rather than in large batches. The worst extreme is the complete system re-write refactoring. The best refactoring activities take seconds or minutes to execute. Small refactorings create a constant modest “overhead” in the work of the team. This overhead then becomes a natural part of the pace of the team.
Not all refactoring moves can be kept so small. For example, upgrading a component or module from a third party might show that your system has many dependencies on that module. In this case, efforts should be made to allow your system to use both the old and the new versions of the component simultaneously. This allows your system to be partially refactored. In other words, to break a large refactoring into many small refactorings. This, in turn, may force you to refactor your system to be more modular in its dependencies.
Another common problem with keeping refactorings small is the re-write problem. Your own system may have a major component that needs to be re-written. Again, finding creative technical means to allow for incremental refactoring of the component is crucial. This can often mean having temporary structures in your system to allow for the old and new parts to work harmoniously. One system that I was working on had to have two separate database platforms with some shared data in order to enable this “bi-modal” operation.
Refactoring Principle Two: Business Catalysts
When is the earliest that a refactoring should be done? Not whenever the technical team wants to do it. Instead, the technical team needs to use business requests as catalysts for refactoring. If the business needs a new feature, then refactoring should only be done on those parts of the system that are required to enable that feature. In other words, don’t refactor the whole user interface, just refactor the parts that relate to the specific business request.
Again, there can be exceptions to doing this… but only in the sense that some refactorings might be delayed until a later date. This is tricky: we want to make sure that we are not accumulating technical debt or creating legacy code. So, instead, we need to allow the technical team to refactor when they detect duplication. Duplication of code, data or structure in the system. A business request might impact a particular part of the system and the team sees how it might be necessary to refactor a large swath of the system as a result. But, the cost of doing so is not yet justified: the single request is not enough of a catalyst, and the team can also choose a simple temporary solution. Later, the business makes another request that also implies the same large refactoring. Now is the time to seriously consider it. It is now a question of duplication of another simple temporary solution. The business may not be happy with the extra expense of the large refactoring so the principle of keeping it small still applies. However, the technical team must also be willing to push back to the business under the right circumstances.
Refactoring Principle Three: Team Cohesion
Teamwork in Agile requires high levels of communication and collaboration. In refactoring work, teamwork applies just as much as in any other activity. Here, it is critical that all members of the team have a unified understanding of the principles and purpose of refactoring. But that is just the first level of team cohesion around refactoring.
The next level of team cohesion comes in the tools, techniques and practices that a team uses in refactoring. Examples include the unit testing frameworks, the mocking frameworks, the automation provided by development tools, continuous integration, and perhaps most importantly, the team working agreements about standard objectives of refactoring. This last idea is best expressed by the concept of refactoring to patterns.
The highest level of team cohesion in refactoring comes from collective code ownership and trust. Usually, this is built from practices such as pair programming or mob programming. These practices create deep levels of shared understanding among team members. This shared understanding leads to self-organizing behaviour in which team members make independent decisions that they know the other team members will support. It also impacts research and learning processes so that teams can do experiments and try alternatives quickly. All of which leads to the ability to do refactoring, large and small, quickly and without fear.
Refactoring Principle Four: Transparency
In many ways, this is the simplest refactoring principle: the team needs to be completely open and honest with all stakeholders about the cost of refactoring. This can be difficult at first. Another analogy helps to see the value of this. A surgeon does not hide the fact that care is put into creating a clean operating environment: washing hands, sterilizing instruments, wearing face masks and hair covers, restricted spaces, etc. In fact, all of those things contribute to the cost of surgery. A surgeon is a professional who has solid reasons for doing all those things and is open about the need for them. Likewise, software professionals need to be open about the costs of refactoring. This comes back to the main point of the first part of this article: hidden and deferred costs will still need to be paid… but with interest. Software professionals are up-front about the costs because doing so both minimizes the costs and gives stakeholders important information to make decisions.
The challenge for business stakeholders is to accept the costs. Respecting the team and trusting their decisions can sometimes be very hard. Teams sometimes make mistakes too, which complicates trust-building. The business stakeholders (for example, the Product Owner), must allow the team freedom to do refactoring. Ideally, it is continuous, small, and low-level. But once in a while, a team will have to do a large refactoring. How do you know if the cost is legitimate? Unfortunately, as a non-technical stakeholder, you can’t know with certainty. However, there are a few factors that can help you understand the cost and its legitimacy, namely, the principles that are described here.
If the refactoring is small, it is more likely to be legitimate.
If the refactoring is in response to a business catalyst, it is more likely to be legitimate.
If the refactoring is reflective of team cohesion, it is more likely to be legitimate.
And, of course, if the refactoring is made transparent, it is more likely to be legitimate.
About the Author
Mishkin Berteig is the President and Co-Founder of Berteig Consulting Inc. He has been training, coaching and consulting for organizations adopting Agile methods since 2001 and is committed to helping individuals, teams and organizations apply Agile methods. Mishkin is a Certified Scrum Trainer, and qualified to deliver OpenAgile and Agile Project Management training. He has developed and delivered Agile training both in public and in-house seminars for over 3000 people in Canada and abroad. Courses have been as short as three hour intro-style and as long as five day boot-camp-style, and audiences have ranged from junior team members to senior executives. He has also assisted organizations of all sizes to make the transformation from traditional methods to Agile/Scrum methods (Extreme Programming, Scrum, Lean, OpenAgile). Assistance includes Agile Engineering Practices, Agile teamwork, Agile project and product management, Agile management and executive management.