Wednesday, October 24, 2018

How to address technical debt in an evolving IT landscape?


Can you wipe out technical debt from your enterprise landscape? Can you always remain one step ahead? These are tough questions that require a calibrated answer. Before answering, we need to understand what is meant by technical debt, why it occurs and ways to manage them.

The first realization should be that technical debt is bound to happen no matter your toll gates and quality check.  The second is an organized program should be initiated, at periodic intervals, to estimate the size and manage its growth. The third is that not all technical debt is bad.
Like a financial debt, if not managed or contained, it would grow to such an extent that it affects the time to market, sink the morale of employees and cease to make the IT shop attractive. Technical debt accrues interest in the sense that it can generate new debts and make it progressively difficult to manoeuvre the development. Tech debt will always be there.

What is TECHNICAL DEBT?

In simple terms, it is the marginal (incremental) work required to complete the software development in order to address the drawbacks. It doesn’t apply only to those projects that are in development stage. It follows even when it is in BAU (Business As Usual). Technical debt holds the organization back from introducing new functionality quickly. We can get an estimate of the debt as the time or money required for refactoring. This refactoring could result in change in design, cleaning up of bad code or porting to a new technology.

What causes technical debt?

The experts cite 4 major reasons for introduction of technical debt.

Cause #1 - Poor Conception: It is the rush or speed in delivery that causes poorly designed software.

Cause #2 – Poor Scheduling: Underestimating the time to develop a product or complete a project often is the culprit that introduces technical debt.

Cause #3 - Bad and inconsistent development practices: Various developers working across different modules tend to introduce their own practices that affect the design and possibly rebuild the logic independently.

Cause #4 - Outdated Technology: As technology evolves, software standards become higher every day. With each improvement, new technical debt can arise.



Types of TECHNICAL DEBT

Type #1: Intentional debt – The software engineers almost always know the right way to code something and the quicker to way to do it. In many cases, the quick way also turns out to be the right way and in others, it may not be. To quickly deliver a project or a product, the functionality will be achieved but not in the best possible manner.

Type #2: Design tech debt – Do we spend time thinking ahead? Or Do we future-proof our design with quicker delivery? As systems become richer with more functionality, the developers may find implementing a new feature very difficult. It may be easier to refactor the original design if it was constructed in a suitable manner. If not, what do you do? Be ready to bite the bullet and get into significant refactoring.

Type #3: Obsolete debt – As many people work on a system, it tends to evolve in a rotten manner. Some symptoms like copy-paste and cargo-cult programming can be easily seen to identify this type. This is directly incurred by the developers. This is one debt that we should avoid in a consistent manner.

Another way to look at this is using the following quadrant that classifies using two dimensions – deliberate or accidental and manner of introduction of such debt.




How to manage this?

Type #1 – Record the debt as backlog at the time it is incurred. This can be revisited later.

Type #2 – Whenever the system is in steady state BAU, allocate some time to look at this type of debt and see if any refactoring has to be done.

Type #3 – Continuous refactoring is the solution to address this. Very experienced and strong teams tend to take time to understand the design of the system before they work on it. When they work, the improve the design incrementally and clean up the bad code en route.

Measures / Signs of technical debt:

Source Code Formatting: It is a common measure. Insisting on the right tool as well as template during and before the SDLC can reduce this type.

Low Test Coverage: It is a measure of code quality. A very low level of the test coverage reduces the certainty of the accuracy of the software's behaviour and makes it difficult to solve problems when they occur.

Lack of Modularity: This generally results from a poor code design. Some code sometimes serves different business logic. The more codes developers write, the more lack of modularity can bottleneck. It is harder to manage software that has logic all over the codes and have parts of codes handling several logic.

Code Complexity: Complexity can be measured in several different ways, but it measures the dependence and path length to perform an operation. A long path leads to complex code.

Lack of the Documentation: Documentation is part of software development best practices. The software is often driven to evolve. It is important that the written code is always understood at all times by everyone who may be involved in the development process.

One approach is to perform a static analysis of code using tools that support the analysis. The following is a list of the most popular tools used: Coverity, SonarQube, Checkstyle, Closure Compiler.

There are two ways of measuring technical debt. The first one is to get a ratio of technical debt according to code volume, and the second one is to use directly the estimates given by the tools (like SonarQube), along with the list of technical debts and their references to the code, SonarQube gives an estimate in days or hours needed to fix this debt. For the ratio approach, we can use the initial estimates or even better, the overall time needed to develop the software so far and extrapolate the value according to the technical debt ratio. The time needed for the development is very accurate so measuring technical debt from ratio can give an accurate estimate of the work needed to fix the issues.

How to Reduce or Eliminate Technical Debt

Being agile is the best way of managing technical debt and reducing it when it appears. The sooner we address the issue, the less interest we'll have to pay over time. To address technical debt, software development teams can use the following approach.

The Quickest to Solve: Fixing debts that take little time to fix is an excellent way to eliminate technical debt gradually. Debts like code formatting can be solved in a little time, making up a template and apply these templates to all the codes that have been developed so far, then integrate these templates in the tools used by the developers.

Priority: It is also important to address issues by priority. All issues that can lead to more significant issues should be addressed quickly and should be prioritized to avoid accumulation.

Technology Update: When outdated technology leads to technical debt, it is important to update the software to the newest versions of the frameworks, application servers, databases etc. It is even important to include every stable evolution of a framework used for instance to always have the latest update and to bring small change without breaking the software.

Refactoring: Reviewing the software architecture and refactoring codes often can be useful when we don't want to end up with duplicate code or codes that lack modularity.


The quadrant, shown, can help to categorize the technical debt to identify which ones to fix first. There are many such approaches. Coming to agile processes, how to estimate / eliminate technical debt? It should be entered in the product backlog as a user story and should be prioritized like any user story. The prioritization should take into account the impact of not managing technical debt identified at the beginning of a new iteration. When deciding which stories to include in the next development iteration, we should analyze whether postponing technical debt correction for the next iteration is more or less advantageous in the long term. As a rule of thumb, we should address every critical issue as soon as it is identified.

Closing remarks

We have to learn to live with certain amount of technical debt. A good IT shop will always have a handle on the measure in terms of either the time or money needed to fix the accumulated debt to a manageable level. Understanding of different types of debt and the rate at which the organization accumulates will help in introducing specific measures to eliminate or minimize such debt.

References:




No comments: