Diagrams or structural lists in software project retrospectives – An experimental comparison

https://doi.org/10.1016/j.jss.2015.01.020Get rights and content
Under a Creative Commons license
open access

Highlights

  • Root cause analysis is a recommended practice in retrospectives.

  • We compare the use of cause–effect diagram to the use of structural lists in root cause analysis.

  • Cause–effect diagram improves the root cause analysis.

  • Cause–effect diagram is preferred by participants (75%).

Abstract

Root cause analysis (RCA) is a recommended practice in retrospectives and cause–effect diagram (CED) is a commonly recommended technique for RCA. Our objective is to evaluate whether CED improves the outcome and perceived utility of RCA. We conducted a controlled experiment with 11 student software project teams by using a single factor paired design resulting in a total of 22 experimental units. Two visualization techniques of underlying causes were compared: CED and a structural list of causes. We used the output of RCA, questionnaires, and group interviews to compare the two techniques. In our results, CED increased the total number of detected causes. CED also increased the links between causes, thus, suggesting more structured analysis of problems. Furthermore, the participants perceived that CED improved organizing and outlining the detected causes. The implication of our results is that using CED in the RCA of retrospectives is recommended, yet, not mandatory as the groups also performed well with the structural list. In addition to increased number of detected causes, CED is visually more attractive and preferred by retrospective participants, even though it is somewhat harder to read and requires specific software tools.

Keywords

Root cause analysis
Retrospective
Post mortem analysis
Cause–effect diagram
Controlled experiment

Cited by (0)

Timo O.A. Lehtinen received M.Sc. in software engineering from Aalto University, Espoo, Finland, in 2010. He is a researcher at Aalto University and his Ph.D. work focuses on root cause analysis, which he has studied both in the software industry and education contexts. He has years of software project management experiences in industry.

Mika V. Mäntylä is a professor at Aalto University. His current research interests include software defects, human cognition, and software evolution. He has a D.Sc. degree in software engineering from Aalto University.

Juha Itkonen is a postdoctoral researcher at Aalto University. His research focuses on experience-based and exploratory software testing and human issues in software engineering, including quality assurance in agile contexts. He received his D.Sc. degree in software engineering in 2012 from Aalto University.

Jari Vanhanen received D.Sc. in software engineering from Aalto University, Espoo, Finland, in 2011. He is a university lecturer in software engineering with Aalto University, and has been the responsible teacher for the capstone software development project course at Aalto University since 2001.