[1]冯皓楠,何智勇,马良荔.基于图文注意力融合的主题标签推荐[J].郑州大学学报(工学版),2022,43(06):30-35.[doi:10.13705/j.issn.1671-6833.2022.03.001]
　FENG H N,HE Z Y,MA  L L.Multimodal Hashtag Recommendation Based on Image and Text Attention Fusion[J].Journal of Zhengzhou University (Engineering Science),2022,43(06):30-35.[doi:10.13705/j.issn.1671-6833.2022.03.001]
点击复制

基于图文注意力融合的主题标签推荐()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
43卷
期数:
2022年06期
页码:
30-35
栏目:
出版日期:
2022-09-02

文章信息/Info

Title:
Multimodal Hashtag Recommendation Based on Image and Text Attention Fusion
作者:
冯皓楠; 何智勇; 马良荔
中国人民解放军海军工程大学电子工程学院;

Author(s):
FENG H N ; HE Z Y; MA  L L
School of Electronic Engineering, the University of the People’s Liberation Army Naval Engineering University;

关键词:
Keywords:
分类号:
TP301. 6TP391. 1
DOI:
10.13705/j.issn.1671-6833.2022.03.001
文献标志码:
A
摘要:
为了解决社交媒体平台上的信息超载问题,帮助用户快速捕捉所需信息,对基于多模态内容的标签推荐问题进行研究。 针对不同模态间的异质性差异,采用共注意力机制进行跨模态内容的特征建 模与融合;针对多标签分类方法只能推荐出数据集标签空间中标签的不足,采用 Seq2Seq 框架生成新的标签序列,并通过一种聚合策略将分类方法的推荐结果聚合到生成的标签序列中,得到 2 种方法的统一推荐模型。 在大规模数据集上的实验结果表明:多模态方法比单模态方法更具优势,所提出的统一推荐模型的F1 值比仅使用单模态的对比模型高 9. 44 百分点;生成新标签序列的方法也优于传统的分类方法,所提出的标签序列生成模型的 F1 值比对比模型 COA 高 3. 41 百分点;所提出的统一推荐模型 UNIFIED-CO-ATT 的 F1 值比 GEN-CO-ATT 模型高 1. 25 百分点,其效果优于其他对比模型。 所提出的模型综合了分类方法和生成方法的特点,可以使推荐的标签同时具有准确性和新颖性。
Abstract:
In order to solve the information overload problem on social media platforms and help users quickly capture the required information, in this study the problem of hashtag recommendation based on multimodal content was investigated. To address the heterogeneous differences between different modalities, a co-attention mechanism was used to model and fuse features of cross-modal content, and use Seg2Seg framework was used to generate new hashtag sequences to address the deficiency that multi-label classification methods could only recommend hashtags in the hashtag space of the dataset. An aggregation strategy was used to aggregate the rec- ommendation results of classification methods into the generated hashtag sequences to obtain a unified recom- mendation model for both methods. The experimental results on a large-scale dataset showed that, firstly, the multimodal approach was more advantageous than the unimodal approach, and the unified recommendation model proposed in this paper had 9. 44 percentage points improvement in F1 value over the comparison model using unimodal approach, and 3. 41 percentage points improvement over the comparison model using the clas- sification method. Finally, the unified recommendation model UNIFIED-CO-ATT is 1. 25 percentage points higher than GEN-CO-ATT in F1 values. The model proposed in this study could combine the advantages of classification and generation methods and could make the recommended hashtags have the advantages of accu- racy and novelty at the same time.
更新日期/Last Update: 2022-10-03