U-QASAR Methodology WebBook

Section 3: Driving Process Improvement with Measurements ( Section 2.3 )

Checklist

Focus needs to be on trends
Common problems have to be recognized
The root causes of the problems are detected
Action proposals are developed
The necessary changes are implemented

Description

This section introduces how to use the measurement program to control the daily software development. The goal is to provide useful knowledge on the quality of software system development process and/or product.

Part 3.1 : Process improvement analysis ( Section 2.3.1 )

Flyers

Using ARCA root cause analysis method to improve software development practice

Guidelines

Guideline	Description
Postmortem analysis	Postmortem analysis is a lightweight retrospective method that has been commonly used to determine what went well in the software project, what could be improved in the next project how to make it better. It takes positive and negative software development experiences as an input and provides knowledge on how to make it better the next time as an output. Usually, post-mortems are conducted in a group and they can also utilize techniques like Root cause analysis.
Root cause analysis	Root cause analysis is a structured investigation of a problem, which aims to state what changes are needed in the future projects. Root cause analysis takes a problem as an input and provides a set of its perceived causeand-effect relationships as an output. It creates a causal model, which can be used to explain how the problems are related to one another, ultimately helping to determine process improvement targets. Additionally, the causal model can be used to consolidate and extend the prediction models and project experience questionnaires.

Guideline

Description

Postmortem analysis

Postmortem analysis is a lightweight retrospective method that has been commonly used to determine

what went well in the software project,
what could be improved in the next project
how to make it better.

It takes positive and negative software development experiences as an input and provides knowledge on how to make it better the next time as an output. Usually, post-mortems are conducted in a group and they can also utilize techniques like Root cause analysis.

Root cause analysis

Root cause analysis is a structured investigation of a problem, which aims to state what changes are needed in the future projects. Root cause analysis takes a problem as an input and provides a set of its perceived causeand-effect relationships as an output. It creates a causal model, which can be used to explain how the problems are related to one another, ultimately helping to determine process improvement targets. Additionally, the causal model can be used to consolidate and extend the prediction models and project experience questionnaires.

Process improvement represents the proactive QA approach, which is used to increase the likelihood of success in the upcoming projects. In practice, the goal is to optimize the process and product quality in order to avoid the most typical software defects and development failures. Similarly than the reactive QA approach, this goal divides into automated analyses and expert judgements. The automated analyses help to express the most typical types of software defects and development failures. Instead, the expert judgement helps to explain why software defects are done and what are the main problems causing the development failures. The topic of process improvement is closely related to organizational knowledge covering the subtopics of individual knowledge, team knowledge, organization knowledge, and company knowledge.

Process improvement is about controlling reoccurring problems in the product or process. That requires the detection of the reoccurring problems. Otherwise, if not detected, problems are not known, which means that the software organization is not fully capable to learn from their failures. There are data analysis techniques which can be automated. Pareto analysis can be conducted after the project in order to determine the types of the most common defects (e.g. usability defects). Respectively, in terms of process quality, similar approach can be used to reveal the most common types of failures (e.g. lack of condition coverage). The results from Pareto analyses could also be combined with Artificial intelligence, e.g., genetic algorithms, which can be used to teach new “problem patterns” for the prediction models. Such problem patterns, i.e., the problems that occur commonly and simultaneously, can be further validated with the expert judgement increasing the reliability of the prediction models.

Determining the problems of software product and process does not always require the use of automated analyses. Expert judgement analyses including Postmortem analysis, Root cause analysis and Maturity models analysis have also been shown to be valuable and lightweight for such a need. Questionnaires can also be used in order to learn from the project stakeholder whether they considered the project as a success or a failure, and what product and process

Using ARCA root cause analysis method to improve software development practice

Step 1: Target problem detection

The target problem detection step uses the outcome of analysing the measurement results as an input. The analyses of measures help to determine the process improvement opportunities, i.e., target problems that need to be controlled. For example, the analyses could reveal that the technical debt has increased or that the test coverage is insufficiently low. It is important to conclude what happens and where. Timelining and the use of focus groups have been presented as efficient and evidencebased approaches for detecting the target problems. It is also possible to use checklists and questionnaires in order to detect target problems.

Step 2: Root cause detection

The root cause detection step uses the detected target problems as an input. In this step, the sources of target problems are detected and considered. The root cause detection step uses both anonymous and public data collection approaches in order to conclude what causes the target problems. There are various tools to enable anonymous data collection, e.g., ARCAtool and Google Docs. There are also various tools to organize the collected root cause data into a causeeffect diagram.

Step 3: Corrective action innovation

The corrective action innovation step uses the outcome of root cause detection as an input. In this step, the process improvement ideas are developed and evaluated. It includes two work phases: root cause selection and corrective action workshop. Root cause selection defines the problems to be controlled. You should use expert opinions while defining the problems. Keep focusing on controllable root causes detected in the Root cause detection step. Corrective action workshop is a meeting wherein process improvement ideas are developed, evaluated, and analysed. The meeting is prepared by the process improvement facilitator, who selects the key stakeholders to join the meeting.

Step 4: Implementing the change

In order to make improvements, the best process improvement ideas have to be implemented and monitored. Therefore, the results from the target problem detection, root cause detection, and corrective action innovation need to be documented first, and then, communicated over the company members. It is also possible to refine the process improvement ideas by reflecting the analysed target problems and their underlying causes on the capability maturity models and other best practices on software engineering education.