Detecting Inconsistencies in Software Architecture Documentation Using Traceability Link Recovery

Terminology Update (V2): This paper uses the original terminology “Missing Model Elements (MME)” and “Unmentioned Model Elements (UME)”. As of ARDoCo V2, these have been renamed to Text Entity Absent from Model (TEAM) and Model Entity Absent from Text (MEAT) respectively for improved clarity.

Abstract

Documenting software architecture is important for a system’s success. Software architecture documentation (SAD) makes information about the system available and eases comprehensibility. There are different forms of SADs like natural language texts and formal models with different benefits and different purposes. However, there can be inconsistent information in different SADs for the same system. Inconsistent documentation then can cause flaws in development and maintenance. To tackle this, we present an approach for inconsistency detection in natural language SAD and formal architecture models. We make use of traceability link recovery (TLR) and extend an existing approach. We utilize the results from TLR to detect unmentioned (i.e., model elements without natural language documentation) and missing model elements (i.e., described but not modeled elements). In our evaluation, we measure how the adaptations on TLR affected its performance. Moreover, we evaluate the inconsistency detection. We use a benchmark with multiple open source projects and compare the results with existing and baseline approaches. For TLR, we achieve an excellent F1-score of 0.81, significantly outperforming the other approaches by at least 0.24. Our approach also achieves excellent results (accuracy: 0.93) for detecting unmentioned model elements and good results for detecting missing model elements (accuracy: 0.75). These results also significantly outperform competing baselines. Although we see room for improvements, the results show that detecting inconsistencies using TLR is promising.

@inproceedings{keim_detecting_2023,
  title        = {{Detecting} {Inconsistencies} in {Software} {Architecture} {Documentation} {Using} {Traceability} {Link} {Recovery}},
  author       = {Keim, Jan and Corallo, Sophie and Fuch{\ss}, Dominik and Koziolek, Anne},
  booktitle    = {2023 IEEE 20th International Conference on Software Architecture (ICSA)},
  doi          = {10.1109/icsa56044.2023.00021},
  month        = {3},
  pages        = {141--152},
  publisher    = {IEEE},
  year         = {2023}
}