2. BACKGROUND

2.1 Process Debt and Technical Debt

Technical Debt (TD) research has increased steadily in the last ten years, showing great interest both from academia as well as the industry [9]. However, the TD metaphor is mainly restricted to enclose only technical issues, such as those related to code, tests, architecture, documentation, etc. [11] Other issues, such as social debt [12], were excluded from the set of TD issues and considered as different kinds of debt. In addition, issues concerning builds, infrastructures, etc., which were previously considered TD (as in the systematic mapping [9]), were also removed from the TD scope. The main reasons were not related to the applicability of the debt metaphor but rather to the substantial difference in managing issues related to technical artifacts with respect to non-technical aspects. For example, tools, methods, and strategies to manage social debt, where the relationships among humans are involved, can differ significantly from static code analyzers used to assess code. In summary, while TD has become well defined and has a community to study it (e.g. the International Conference on Technical Debt), several issues, also deserving the “debt” status, seem to have been left behind. At the same time, new kinds of debts emerge continuously in practice as important issues to be addressed [13], as testimony that the debt metaphor can help to reason about other aspects than the strict technical ones.
Our study used the only existing definition, given in [10]: PD is “a sub-optimal activity or process that might have short-term benefits but generates a negative impact in the medium-long term.” However, such a definition is based on TD’s definition. Therefore, we also aim to develop a more nuanced and comprehensive definition based on empirical evidence in this study.
Same as for TD [14], we assume that the PD metaphor is composed of similar components, namely the debt, the interest, and the principal. A debt item consists of the divergence, in practice, from an optimal process; the interest constitutes several negative effects generated by the occurrence of such debt; the principal corresponds to the cost of changing the process to avoid or repay the debt.
Although a process is, most of the time, the result of a tradeoff across several stakeholders’ needs, some tradeoffs might be better or worse than others. In this case, we do not recognize the debt as a specific sub-optimality (which can still be acceptable in the best possible tradeoff), but by assuming that a better tradeoff (and therefore a better process) could be achieved to solve the same problem. However, a sub-optimal process is not considered PD if it does not have a clear interest to be paid, as it has been postulated for the TD metaphor.

2.2 Process and Workflow

A process is a structured set of activities and decisions to do a certain job. “A software development process, also known as a software development lifecycle, is a structure imposed on the development of a software product. A software process is represented as a set of work phases that is applied to design and build a software product. There is no ideal software process that suits all different types of software developing companies, and many organizations have developed or tailored their own approach to software development.” [15].
In practice, there might be several sub-processes that are interconnected. The process can emerge or be designed. There can be one or more process designers (those who are in charge of designing and managing the process) and one or more process executors (those who perform one or more steps in the process). In some cases, these roles might coincide. In agile software development, software process initiatives are often built into retrospectives [17].
The difference between workflow and process is that a workflow is a series of repeatable activities that one needs to carry out to finish a task. A process, on the other hand, is a set of repeatable activities that need to be carried out to accomplish some organizational goals [16]. In practice, the workflow represents the everyday sequence of activities from an individual point of view. In contrast, a process represents an overall sequence of activities carried out by specific roles to reach an overall goal. Different processes (software related or not) might affect different aspects of software development: for example, a process can be defined to implement a testing strategy or to produce documents for certification of the software with respect to safety, security, or other requirements, etc. Other processes, for example in mechatronic products, might be used to manage other parts of the product but can still affect software development.

2.3 SPI and process debt

In software development, there is a long tradition of regularly improving the processes [26] Software Process Improvement (SPI) is about making the software processes better and is mainly a human activity [18]. SPI has been found to be positively associated with business success in small to medium-sized companies [19]. However, process problems are frequently identified but rarely solved [20], and then, therefore, only the potential for improvement exists.
One important driving force for SPI initiatives is that the team members learn how to improve their activities. SPI can be seen as an organizational change mechanism, which requires group learning [27] since collaboration is essential in the software development process. . Agile software development supports group learning through frequent feedback sessions like stand-up meetings, demos and retrospectives involving the whole team.
In their learning theory, Argyris and Schön [21] distinguish between two types of learning in the organization, which they call ”single loop learning” and ”double loop learning”: a single loop learning is a change in practice in the event of a problem, in order to avoid that same problem for the future. Single-loop learning involves changing processes when a problem occurs in order to prevent the same problem in the future. Examples include managers monitoring development costs, sales, or customer satisfaction to ensure that organizational activities remain within predetermined limits to keep the organization ”on track.” In single-loop learning, actions are modified slightly to attain the desired results if their consequences are not met. There is a feedback loop between observed effects and modifications or refinements that influence these effects. On the other hand, double-loop learning is when time is taken to understand the factors that influence the effects, and the nature of this influence is called the governing values [21]. It involves using the problems being experienced to understand their root causes and then take actions to address these causes. One example is what happens when a software error is corrected. Correcting the error itself can be seen as single-loop learning, but if something is done with whatever caused the error to be introduced, that is considered double-loop learning. The changes based on this type of understanding will be more thorough. To sum up: Single-loop learning is about asking, “are we doing things right?” while double-loop learning is about asking, “are we doing the right things?”.
If organizations focus only on single-loop learning, the underlying cause of PD will not be solved, and the sub-optimal process will continue to exist. Therefore, we argue that to reduce PD, companies need to conduct double-loop learning. That is, the team or the organization need to “learn how to learn”, also known as deutero-learning [21]. Teams need both to conduct single-loop learning (questioning if they are doing the process right) and double-loop learning (questioning if they are doing the right process). In addition, they need to make the right decisions when replacing a suboptimal process with an optimal one in order to reduce PD.
Although there exists a large body of knowledge related to software process improvement and change management, our goal for this study is to understand what PD is and how it is accumulated and managed, which has not been studied before. The results will provide companies with new knowledge on how to conduct double-loop learning and better understand their PD. To do so, and to maintain an exploratory approach, we decided to conduct our investigation without using previous knowledge related to software process improvement to formulate our questions, but rather to let the lens of PD direct our interviews, in a fully exploratory fashion.

2.4 Extension to our previous work

This study was partly and originally published at the APSEC 2020 [24]. The delta of this manuscript over the prior published study is based on an additional empirical extension of the previous study, where this manuscript includes an in-depth value-added analysis of the results derived from the first study to provide a more detailed and comprehensive understanding and perception of the concept of PD.
This manuscript has been extended to include six additional research questions (RQ2.3, RQ2.5, RQ3, RQ3.1, RQ3.2 and RQ4), where the results of these questions are derived from a totally new set of independent data collection.
This extended manuscript includes the results from a follow-up cross-company interview with company A-D, which reports a validation of the results from the previous study and additional insights coming from a close-ended questionnaire on the extent to which the causes, consequences and mitigation strategies are the most crucial for the interviewed companies (new RQ2.3, RQ2.5, RQ3, RQ3.1, RQ3.2). Further, the current study includes data collected during a six-year-long case study focusing primarily on software process improvements on an additional company (Company E), which was primarily used to validate and complement the results using a new method and data source as well as for adding a new RQ related to the time dimension of PD (RQ4).