"Augmenting general-purpose large-language models with domain-specific " by Shenghua Zhou, Keyan Liu et al.
 

Augmenting general-purpose large-language models with domain-specific multimodal knowledge graph for question-answering in construction project management

Document Type

Article

Publication Date

5-2025

Department

Department of Civil, Environmental, and Geospatial Engineering

Abstract

Current studies on Question-Answering of Construction Project Management (CPM-QA) face challenges, including the small-scale CPM-related knowledge repositories, the limited effectiveness of QA methods using grammar rules or tiny machine-learning models, and the shortage of testing sets for comparing QA performance. Hence, this research augments general-purpose large-language models (GLMs) with the multimodal CPM knowledge graph (CPM-KG) for CPM-QA. It encompasses (i) building the multimodal CPM-KG covering 36 CPM subfields, (ii) combining CPM-KG and GLMs through three stages, (iii) developing a 2435-question CPM-QA testing set, and (iv) assessing and comparing CPM-QA accuracies for eight pairs of original and CPM-KG-augmented GLMs. The results demonstrate that CPM-KG-augmented GLMs’ CPM-QA accuracy rate is 30.0 % superior to original GLMs on average, and top-performing CPM-KG-augmented GLMs (e.g., ERNIE-Bot 4.0) pass CRCEEs. Within 36 CPM subfields, CPM-QA accuracy enhancements resulting from CPM-KG are between 12.2 % and 57.8 %. Furthermore, CPM-KG leads to CPM-QA accuracy enhancements of 19.6 % for single-answer, 48.0 % for multiple-answer, 30.6 % for text-only, and 20.4 % for image-embedded questions. The multimodal CPM-KG also outperforms the text-only single-modal CPM-KG in enhancing CPM-QA performance. This work contributes to unveiling the significance of CPM-specific knowledge in augmenting GLMs, sharing a reusable multimodal CPM-KG-formatted knowledge repository, and delivering a testing set of CPM-QA.

Publication Title

Advanced Engineering Informatics

Link to Full Text

Share

COinS