[1]许胜新,梁弼政,胡　飞,等.基于时间序列成像的多任务学习驱动情感识别[J].郑州大学学报(工学版),2026,47(01):73-80.[doi:10.13705/j.issn.1671-6833.2026.01.005]
　XU Shengxin,LIANG Bizheng,HU Fei,et al.Multi-task Learning-driven Emotion Recognition Based on Time Series Imaging[J].Journal of Zhengzhou University (Engineering Science),2026,47(01):73-80.[doi:10.13705/j.issn.1671-6833.2026.01.005]
点击复制

基于时间序列成像的多任务学习驱动情感识别()
分享到：

《郑州大学学报(工学版)》[ISSN:1671-6833/CN:41-1339/T]

卷:
47
期数:
2026年01期
页码:
73-80
栏目:
出版日期:
2026-01-06

文章信息/Info

Title:
Multi-task Learning-driven Emotion Recognition Based on Time Series Imaging
文章编号:
1671-6833(2026)01-0073-08
作者:
许胜新1;  梁弼政2;  胡　飞3;  徐华兴3
1.中国电子科学研究院 社会安全风险感知与防控大数据应用国家工程研究中心,北京 100041;2.中国空间技术研究院 通信与导航卫星总体部,北京 100094;3.郑州大学 电气与信息工程学院,河南 郑州 450001
Author(s):
XU Shengxin1;  LIANG Bizheng2;  HU Fei3;  XU Huaxing3
1.National Engineering Center for Risk Perception and Prevention (RPP), China Academy of Electronics and Information Technology, Beijing 100041, China; 2. Institute of Telecommunication and Navigation Satellites, China Academy of Space Technology, Beijing 100094, China; 3.School of Electrical and Information Engineering, Zhengzhou University, Zhengzhou 450001, China
关键词:
脑电;  情感识别;  时间序列成像;  多任务;  特征融合
Keywords:
EEG;  emotion recognition;  time series imaging;  multi-task;  feature fusion
分类号:
TP181G202
DOI:
10.13705/j.issn.1671-6833.2026.01.005
文献标志码:
A
摘要:
针对脑电情感识别依赖特征抽取或时频谱图导致的计算复杂度高的问题,提出一种基于时间序列成像(TSI)的多任务学习驱动情感识别方法。通过格拉姆角场、马尔可夫转移场以及模式差分场,实现一维脑电信号到二维图像的直接编码;基于ResNet18分类骨干网络,设计多任务特征融合框架,融合3种TSI编码提取的特征表示。实验结果表明:在DEAP数据集上,所提方法在Valence和Arousal的二分类中的平均分类准确率分别为96.51%和97.22%,在AMIGOS数据集上分别为98.59%和99.64%,扩展到四分类和八分类时,DEAP上的平均分类准确率分别为91.06%和87.43%,AMIGOS上的平均准确率分别为97.41%和89.84,在脑电情感识别中具备良好的鲁棒性。
Abstract:
To overcome the high computational complexity of EEG-based emotion recognition methods based on feature extraction or time-frequency representations, a multi-task learning-driven method for emotion recognition based on time series imaging (TSI) was proposed. EEG signals were directly transformed into two-dimensional images using Gramian angular field, Markov transition field, and motif difference field. Built upon the ResNet18 architecture, a multi-task feature fusion framework was designed to integrate features from the three imaging methods to enhance emotional feature representation. Experimental results showed that with the DEAP dataset, the proposed method achieved average classification accuracies of 96.51% and 97.22% for binary classification of Valence and Arousal, respectively, and with the AMIGOS dataset, the accuracies reached 98.59% and 99.64%. When extended to four-class and eight-class classification tasks, the proposed method achieved average accuracies of 91.06% and 87.43% with DEAP, and 97.41% and 89.84% with AMIGOS, respectively. These results demonstrated the robustness of the proposed method in EEG-based emotion recognition.

参考文献/References:

[1]WANG Y M, ZHANG B, DI L M. Research progress of EEG-based emotion recognition: a survey[J]. ACM Computing Surveys, 2024, 56(11): 1-49.

[2]LIU Y, WEI Y, LI C, et al. Bi-CapsNet: a binary capsule network for EEG-based emotion recognition[J]. IEEE Journal of Biomedical and Health Informatics, 2023, 27(3): 1319-1330.
[3]YANG Y L, WU Q F, FU Y Z, et al. Continuous convolutional neural network with 3D input for EEG-based emotion recognition[J]. Lecture Notes in Computer Science,2018,11307(1):433-443.
[4]SHARMA L D, BHATTACHARYYA A. A computerized approach for automatic human emotion recognition using sliding mode singular spectrum analysis[J]. IEEE Sensors Journal, 2021, 21(23): 26931-26940.
[5]GALVÃO F, ALARCÃO S M, FONSECA M J. Predicting exact valence and arousal values from EEG[J]. Sensors, 2021, 21(10): 3414.
[6]TAO W, LI C, SONG R C, et al. EEG-based emotion recognition via channel-wise attention and self attention[J]. IEEE Transactions on Affective Computing, 2023, 14(1): 382-393.
[7]魏琛, 陈兰岚, 张傲. 基于集成卷积神经网络的脑电情感识别[J]. 华东理工大学学报(自然科学版), 2019, 45(4): 614-622.
WEI C, CHEN L L, ZHANG A. Emotion recognition of EEG based on ensemble convolutional neural networks[J]. Journal of East China University of Science and Technology, 2019, 45(4): 614-622.
[8]SONG T F, ZHENG W M, SONG P, et al. EEG emotion recognition using dynamical graph convolutional neural networks[J]. IEEE Transactions on Affective Computing, 2020, 11(3): 532-541.
[9]YANG L Q, LIU J W. EEG-based emotion recognition using temporal convolutional network[C]∥2019 IEEE 8th Data Driven Control and Learning Systems Conference (DDCLS). Piscataway: IEEE, 2019: 437-442.
[10] SALAMA E S, EL-KHORIBI R A, SHOMAN M E, et al. EEG-based emotion recognition using 3D convolutional neural networks[J]. International Journal of Advanced Computer Science and Applications, 2018, 9(8): 329-337.
[11] RAMZAN M, DAWN S. Fused CNN-LSTM deep learning emotion recognition model using electroencephalography signals[J]. International Journal of Neuroscience, 2023, 133(6): 587-597.
[12] YANG Y L, WU Q F, QIU M, et al. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network[C]∥2018 International Joint Conference on Neural Networks (IJCNN). Piscataway: IEEE, 2018: 1-7.
[13] LU Y F, ZHENG W L, LI B B, et al. Combining eye movements and EEG to enhance emotion recognition[C]∥International Joint Conference on Artificial Intelligence. Buenos Aires: IJCAI, 2015: 1170-1176.
[14] BUSA M A, VAN EMMERIK R E A. Multiscale entropy: a tool for understanding the complexity of postural control[J]. Journal of Sport and Health Science, 2016, 5(1): 44-51.
[15] ZHANG H Y, CISSE M, DAUPHIN Y N, et al. Mixup: beyond empirical risk minimization[EB/OL]. (2018-0427)[2025-03-30]. https:∥arxiv.org/abs/1710.09412.
[16] GAGLIARDI G, ALFEO A L, CATRAMBONE V, et al. Improving emotion recognition systems by exploiting the spatial information of EEG sensors[J]. IEEE Access, 2023, 11: 39544-39554.
[17] CUI F C, WANG R Q, DING W W, et al. A novel DECNN-BiLSTM multi-fusion model for EEG emotion recognition[J]. Mathematics, 2022, 10(4): 582-593.
[18] VAZQUEZ-RODRIGUEZ J, LEFEBVRE G, CUMIN J, et al. Emotion recognition with pre-trained transformers using multimodal signals[C]∥2022 10th International Conference on Affective Computing and Intelligent Interaction (ACII). Piscataway: IEEE, 2022: 1-8.
[19] TOPIC A, RUSSO M, STELLA M, et al. Emotion recognition using a reduced set of EEG channels based on holographic feature maps[J]. Sensors, 2022, 22(9): 3248.
[20] ZHAO Y X, YANG J, LIN J L, et al. A 3D convolutional neural network for emotion recognition based on EEG signals[C]∥2020 International Joint Conference on Neural Networks (IJCNN). Piscataway: IEEE, 2020: 1-6.

更新日期/Last Update: 2026-01-17