Enabling Architecture Traceability by LLM-based Architecture Component Name Extraction

Abstract

Traceability Link Recovery (TLR) is an enabler for various software engineering tasks. One important task is the recovery of trace links between Software Architecture Documentation (SAD) and source code. Here, the main challenge is the semantic gap between the two artifact types. Recent research has shown that this semantic gap can be bridged by using Software Architecture Models (SAMs) as intermediates. However, the creation of SAMs is a manual and time-consuming task. This paper investigates the use of Large Language Models (LLMs) to extract component names as simple SAMs for TLR based on SAD and source code. By doing so, we aim to bridge the semantic gap between SAD and source code without the need for manual SAM creation. We compare our approach to the state-of-the-art TLR approaches TransArC and ArDoCode. TransArC is the currently best-performing approach for TLR between SAD and source code, but it requires SAMs as an additional artifact. Our evaluation shows that our approach performs comparable to TransArC (weighted average F1 with GPT-4o: 0.86 vs. TransArC’s 0.87), while only needing the SAD and source code. Moreover, our approach significantly outperforms the best baseline that does not need SAMs (weighted average F1 with GPT-4o: 0.86 vs. ArDoCode’s 0.62). In summary, our approach shows that LLMs can be used to make TLR between SAD and source code more applicable by extracting component names and omitting the need for manually created SAMs.

@inproceedings{fuchss_enabling_2025,
  title        = {{Enabling} {Architecture} {Traceability} by {LLM}-based {Architecture} {Component} {Name} {Extraction}},
  author       = {Fuch{\ss}, Dominik and Liu, Haoyu and Hey, Tobias and Keim, Jan and Koziolek, Anne},
  booktitle    = {2025 IEEE 22nd International Conference on Software Architecture (ICSA)},
  doi          = {10.1109/icsa65012.2025.00011},
  eventdate    = {2025-03-31/2025-04-04},
  eventtitle   = {22nd IEEE International Conference on Software Architecture},
  eventtitleaddon = {ICSA 2025},
  language     = {english},
  month        = {3},
  pages        = {1--12},
  publisher    = {IEEE},
  venue        = {Odense, Denmark},
  year         = {2025},
  keywords     = {Large Language Models, Model Extraction, Software Architecture, Traceability Link Recovery}
}