5. DISCUSSION

5.1 Contributions and implications

Process Debt is a phenomenon that has not been studied as much as Technical Debt. Through a 3-step investigation, we have collected qualitative data from five companies and quantitative data from four of them for validation purposes.
The main contributions are:
We discuss these points in the following sections.

5.1.1 A framework to reason about Process Debt

We propose an overall PD conceptual framework (Figure 2) where the key elements and their relationships are visible. We also give detailed taxonomies of types of PD, occurrence patterns, causes, effects and mitigation strategies.
Our framework can support reasoning about PD for practitioners who would like to systematically manage PD and need a comprehensive reference point. First and foremost, TD research has shown how the first important step to managing debt is to build awareness around it and to start a discussion in the organizations.
From the theoretical point of view, our framework is not complete, as the taxonomies can be extended with additional evidence collected from new and specific case studies. However, our results show that, when extending our initial framework with new, in-depth evidence from a longitudinal case study with a different context from the initial four companies, the framework was substantially validated with some little additions. This gives confidence that the framework represents a robust first step towards building a comprehensive theory of PD and is already usable as a key theoretical reference point.
Our findings show that the PD lens can enrich the knowledge related to Software Process Improvement with a new perspective on how to prioritize SPI work. Much of the existing literature is dedicated to assessing SPI or to improving specific processes: however, little is found about how to prioritize if an SPI intervention is actually more important than another one or even if improving a process is more important than developing features and fixing bugs. The debt perspective and our characterization using entities such as principal and interest borrowed from the financial domain, can help to reason about the prioritization of SPI to avoid or repay an existing PD that is particularly disruptive, while avoiding SPI work and spending resources that would reduce PD with limited negative effects.

5.1.2 Process Debt state of practice

Our investigation shows that PD is an existing phenomenon that can cause a huge amount of negative effects, among which mentioned the limitation of applying modern practices to support vital business approaches such as continuous delivery. PD is also recognized as generating Technical Debt.
Although a quantification of the (negative) effects of PD is not provided in this study, according to the interviewees the magnitude of its impact is comparable to the TD one. Further studies and surveys should be conducted on the PD phenomenon to assess its actual impact. Our taxonomies can help create a solid instrument to investigate such a phenomenon.
Throughout our report, we have mentioned several concrete instances mentioned by the practitioners supporting our taxonomies. Besides underpinning our framework, such instances constitute a concrete list of examples of PD that can occur in organizations. Practitioners can also use our taxonomy and even such concrete list of instances to identify and differentiate across PD types in their context, to recognize their causes and effects even before PD becomes disruptive and be able to reason about prioritizing PDs’ avoidance and removal.
A key point revealed by observing PD in the longitudinal study, is that removing PD often introduces new PD. This is different from TD, where often repaying TD implies that the new solution is without better (excluding a few corner cases). This means that evaluating if repaying PD is worth it, a more complex assessment is needed, and more than one “repayment” might be included in the equation to reach a substantial improvement. More research is needed to understand such evaluation and assessment. This is in line with Argyris and Schön [21] and the use of double-loop learning, where additional evaluation is done after a solution is applied.
The most valuable part of our reported PD state of practice is perhaps related to the reported mitigation strategies used by the practitioners to manage PD, which can also be applied in other contexts (if not precisely, at least can give inspiration for new practices). In addition, besides what is used today, we show what mitigation strategies are in practitioners’ plans for better managing PD in the future. This can give key targets for companies developing tools for SPI and to researchers to investigate novel approaches along the directions mentioned by our informants.

5.1.3 Validity, reliability and limitations

Our study relies on interviews, observations and quantitative data collection with a limited set of organizations. This means that our results cannot be considered general and suffer from external validity threats [25]. For example, additional organizations with different domains and contexts should be interviewed and surveyed to reach a complete conceptual framework. Some of our results would hold in different settings (e.g., the types of PD), but, for example, some of the causes and consequences might not apply or be different in such contexts. We, therefore, acknowledge that further research is required. However, we argue that our exploratory effort is already valuable, in that we included different companies with different contexts and different stakeholders, ranging from process designers to software developers, to team leaders and practitioners in other organizations, such as hardware designers. To mitigate the threats to external validity, we have also applied two additional validation steps by running a workshop and collecting additional qualitative and quantitative data, and comparing our results to the in-depth findings from a longitudinal case study in a new and different company E. We argue that it would not be possible to cover the phenomenon in its entirety with just one study, but we call for the software engineering community to contribute to studying PD with additional investigations.
As for reliability, although it’s often the case that qualitative studies rely heavily on the researchers’ interpretation, we have used several mitigation strategies to limit the bias introduced by the authors. First, we made sure that in the majority, at least two researchers were present at the interviews, and two researchers analyzed the interviews in parallel and then discussed and agreed on the resulting coding. At least two researchers have discussed the findings and definitions across studies to ensure that the data from the longitudinal study would be correctly interpreted and mapped to the framework obtained from the first phase.
One limitation is the lack of input for the workshop run in phase 3 from the practitioners in the longitudinal case study related to phase 2, but only related to the first four companies investigated in the first phase. However, similar insights were collected and analyzed in the longitudinal case study with a different method, the only difference is the data collection format.