[1]王海荣,王怡梦,周北京,等.融合多模态信息的知识感知推荐方法[J].郑州大学学报(工学版),2025,46(06):15-22.[doi:10.13705/j.issn.1671-6833.2025.03.010]
　WANG Hairong,WANG Yimeng,ZHOU Beijing,et al.Knowledge-aware Recommendation Method Integrating Multi-modal Information[J].Journal of Zhengzhou University (Engineering Science),2025,46(06):15-22.[doi:10.13705/j.issn.1671-6833.2025.03.010]
点击复制

融合多模态信息的知识感知推荐方法()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
46
期数:
2025年06期
页码:
15-22
栏目:
出版日期:
2025-10-25

文章信息/Info

Title:
Knowledge-aware Recommendation Method Integrating Multi-modal Information
文章编号:
1671-6833(2025)06-0015-08
作者:
王海荣1; 2;  王怡梦1;  周北京1;  易之航1
1.北方民族大学 计算机科学与工程学院,宁夏 银川 750021;2.北方民族大学 图像图形智能处理国家民委重点实验室, 宁夏 银川 750021
Author(s):
WANG Hairong1; 2;  WANG Yimeng1;  ZHOU Beijing1;  YI Zhihang1
1.College of Computer Science and Engineering, North Minzu University, Yinchuan 750021,China; 2.The Key Laboratory of Images & Graphics Intelligent Processing of State Ethnic Affairs Commission, North Minzu University, Yinchuan 750021,China
关键词:
知识图谱;  推荐系统;  多模态信息;  特征融合;  嵌入传播
Keywords:
knowledge graph;  recommendation system;  multi-modal information;  feature fusionl;  embedding propagation
分类号:
TP391TP18TN912
DOI:
10.13705/j.issn.1671-6833.2025.03.010
文献标志码:
A
摘要:
图片、文本等多模态信息具有语义互补性,能够有效增强知识图谱中的实体表示,从而提高推荐的准确率和可解释性。通过分析推荐系统中具有语义相关性的多模态数据特点,提出了一种融合多模态信息的知识感知推荐方法。在知识图谱传播的基础上,整合与图谱中实体语义相关的多模态信息,并将其与对应的实体进行特征融合,用来丰富实体表示,以便探索用户潜在的兴趣偏好。该方法充分考虑了多模态信息间的依赖性和交互性,采用模态间注意力关注各模态的重要信息,获取具有语义关联的多模态嵌入特征;通过门控注意力将实体对应的多模态嵌入特征与实体表示融合,进一步丰富实体的多模态语义信息,从而增强用户和项目的表示。为了验证方法的有效性,在MovieLens-1M和Book-Crossing数据集上进行实验,并与RippletNet、KGAT、CKAN、LKGR、COAT、CKE、KGCN、SKGCR和KGCL这9种方法进行对比分析,实验结果表明:所提方法在AUC和ACC上均优于对比方法;在MovieLens1M和Book-Crossing数据集上,所提方法的AUC分别为0.936 6和0.763 7,与其他模型的平均值相比,增幅为0.027 2和0.029 1;所提方法的ACC分别为0.862 3和0.708 9,与其他模型的平均值相比,增幅为0.028 3和0.030 5。
Abstract:
It is found that multi-modal information such as images and text possesses semantic complementarity, which could effectively enhance the representation of entities in knowledge graphs, thereby improving the accuracy and interpretability of recommendations. A knowledge-aware recommendation method that could integrate multimodal information was proposed by analyzing the characteristics of semantically related multimodal data in recommendation systems. On the basis of knowledge graph propagation, multi-modal information that was semantically related to entities in the graph was integrated, and feature fusion was performed with the corresponding entities to enrich entity representation, aiming to explore users′ potential interest preferences. In this method, the dependency and interactivity between multimodal information was considered, intermodal attention was adopted to focus on important information of each modality, and semantically associated multimodal embedding features were obtained. Through gated attention, the multi-modal embedding features corresponding to entities were fused with entity representations, further enriching the multi-modal semantic information of entities, thereby enhancing the representation of users and items. In order to verify the effectiveness of the method, experiments were conducted on MovieLens-1M and Book-Crossing data sets, and comparative analysis was conducted with 9 methods including RippletNet, KGAT, CKAN, LKGR, COAT, CKE, KGCN, SKGCR and KGCL. The experimental results showed that it was better than the other two indicators in AUC and ACC. On the MovieLens-1M and Book-Crossing datasets, the AUC of the proposed method were 0.936 6 and 0.763 7, respectively, with an increase of 0.027 2 and 0.029 1 compared to the average values of other models. The ACC values of the proposed methods were 0.862 3 and 0.708 9, respectively, with an increase of 0.028 3 and 0.030 5 compared to the average values of other models.

参考文献/References:

[1]GUO H, YANG C Y, ZHOU L Q, et al. A novel knowledge graph recommendation algorithm based on graph convolutional network[J]. Connection Science, 2024, 36 (1): 2327441. 

[2]LIU T Y, SHEN H J, CHANG L, et al. Iterative heterogeneous graph learning for knowledge graph-based recommendation[J]. Scientific Reports, 2023, 13(1): 6987. 
[3]LI D Z, QU H B, WANG J Q. A survey on knowledge graph-based recommender systems[C]∥2023 China Automation Congress (CAC). Piscataway: IEEE, 2023: 2925-2930. 
[4]GAO C, ZHENG Y, LI N, et al. A survey of graph neural networks for recommender systems: challenges, methods, and directions[J]. ACM Transactions on Recommender Systems, 2023, 1(1): 1-51. 
[5]WANG H W, ZHANG F Z, XIE X, et al. DKN[C]∥ Proceedings of the 2018 World Wide Web Conference. New York: ACM, 2018: 1835-1844. 
[6]HU B B, SHI C, ZHAO W X, et al. Leveraging metapath based context for top-N recommendation with a neural co-attention model[C]∥Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2018: 1531-1540. 
[7]WANG F, LI Y S, ZHANG Y J, et al. KLGCN: knowledge graph-aware light graph convolutional network for recommender systems[J]. Expert Systems with Applications, 2022, 195: 116513. 
[8]RUAN S Q, YANG C, LI D S. Knowledge-enhanced personalized hierarchical attention network for sequential recommendation[J]. World Wide Web, 2024, 27(1): 2. 
[9]CHEN F K, YIN G S, DONG Y X, et al. KHGCN: knowledge-enhanced recommendation with hierarchical graph capsule network[J]. Entropy, 2023, 25(4): 697. 
[10] TAO S H, QIU R H, CAO Y, et al. Intent with knowledge-aware multiview contrastive learning for recommendation[J]. Complex & Intelligent Systems, 2024, 10 (1): 1349-1363. 
[11]WANG H W, ZHANG F Z, WANG J L, et al. RippleNet: propagating user preferences on the knowledge graph for recommender systems[C]∥Proceedings of the 27th ACM International Conference on Information and Knowledge Management. New York: ACM, 2018: 417-426. 
[12]WANG X, HE X N, CAO Y X, et al. KGAT: knowledge graph attention network for recommendation[C]∥Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 950-958. 
[13]WANG Z, LIN G Y, TAN H B, et al. CKAN[C]∥Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 219-228. 
[14] CHEN Y K, YANG M L, ZHANG Y X, et al. Modeling scale-free graphs with hyperbolic geometry for knowledgeaware recommendation[C]∥Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. New York: ACM, 2022: 94-102. 
[15]王海荣, 徐玺, 王彤, 等. 多模态命名实体识别方法研究进展[J]. 郑州大学学报(工学版), 2024, 45 (2): 60-71. 
WANG H R, XU X, WANG T, et al. Research progress of multimodal named entity recognition[J]. Journal of Zhengzhou University (Engineering Science), 2024, 45 (2): 60-71. 
[16] BAI H Y, WU L, HOU M, et al. Multimodality invariant learning for multimedia-based new item recommendation [C]∥Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2024: 677-686. 
[17] CUI X H, QU X L, LI D M, et al. MKGCN: multi-modal knowledge graph convolutional network for music recommender systems[J]. Electronics, 2023, 12(12): 2688. 
[18] SUN R, CAO X Z, ZHAO Y, et al. Multi-modal knowledge graphs for recommender systems[C]∥Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: ACM, 2020: 14051414. 
[19] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2024-06-11]. https:∥arxiv. org/ abs/1409.1556v6. 
[20] DEVLIN J, CHANG M W, LEE K, et al. BERT: pretraining of deep bidirectional transformers for language understanding[EB/OL]. (2019-05-24)[2024-0611]. https:∥arxiv.org/abs/1810.04805v2. 
[21] MA T, HUANG L T, LU Q Q, et al. KR-GCN: knowledge-aware reasoning with graph convolution network for explainable recommendation[J]. ACM Transactions on Information Systems, 2023, 41(1): 1-27. 
[22] DAI Q Y, WU X M, FAN L, et al. Personalized knowledge-aware recommendation with collaborative and attentive graph convolutional networks[J]. Pattern Recognition, 2022, 128: 108628. 
[23] ZHANG F Z, YUAN N J, LIAN D F, et al. Collaborative knowledge base embedding for recommender systems [C]∥Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 353-362. 
[24]WANG H W, ZHAO M, XIE X, et al. Knowledge graph convolutional networks for recommender systems[C]∥The World Wide Web Conference. New York: ACM, 2019: 3307-3313. 
[25] LIU X K, YANG B, XU J Y. SKGCR: self-supervision enhanced knowledge-aware graph collaborative recommendation[J]. Applied Intelligence, 2023, 53(17): 19872-19891. 
[26] YANG Y H, HUANG C, XIA L H, et al. Knowledge graph contrastive learning for recommendation[C]∥Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2022: 1434-1443.

相似文献/References:

[1]陈 宏,陈新财,巩晓赟,等.基于知识图谱的风电机诊断系统构建与应用[J].郑州大学学报(工学版),2023,44(06):54.[doi:10.13705/j.issn.1671-6833.2023.06.007]
　CHEN Hong,CHEN Xincai,GONG Xiaobin,et al.Construction and Application of Wind Turbine Diagnosis System Based on Knowledge Graph[J].Journal of Zhengzhou University (Engineering Science),2023,44(06):54.[doi:10.13705/j.issn.1671-6833.2023.06.007]

更新日期/Last Update: 2025-10-21