Document Type
Article
Publication Date
4-9-2023
Department
Department of Computer Science
Abstract
Media convergence works by processing information from different modalities and applying them to different domains. It is difficult for the conventional knowledge graph to utilise multi-media features because the introduction of a large amount of information from other modalities reduces the effectiveness of representation learning and makes knowledge graph inference less effective. To address the issue, an inference method based on Media Convergence and Rule-guided Joint Inference model (MCRJI) has been proposed. The authors not only converge multi-media features of entities but also introduce logic rules to improve the accuracy and interpretability of link prediction. First, a multi-headed self-attention approach is used to obtain the attention of different media features of entities during semantic synthesis. Second, logic rules of different lengths are mined from knowledge graph to learn new entity representations. Finally, knowledge graph inference is performed based on representing entities that converge multi-media features. Numerous experimental results show that MCRJI outperforms other advanced baselines in using multi-media features and knowledge graph inference, demonstrating that MCRJI provides an excellent approach for knowledge graph inference with converged multi-media features.
Publication Title
CAAI Transactions on Intelligence Technology
Recommended Citation
Lin, F.,
Li, D.,
Zhang, W.,
Shi, D.,
Jiao, Y.,
Chen, Q.,
Lin, Y.,
&
Zhu, W.
(2023).
Multi-modal knowledge graph inference via media convergence and logic rule.
CAAI Transactions on Intelligence Technology.
http://doi.org/10.1049/cit2.12217
Retrieved from: https://digitalcommons.mtu.edu/michigantech-p/17088
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.
Version
Publisher's PDF
Publisher's Statement
© 2023 The Authors. Publisher’s version of record: https://doi.org/10.1049/cit2.12217