李辉, 黄钰杰.基于多头自注意力和并行混合模型的文本情感分析[J].河南理工大学学报（自然科学版）,2021,40(1):125-132.

LI H, HUANG Y J.Text sentiment analysis based on multi self-attention and parallel hybrid model[J].Journal of Henan Polytechnic University(Natural Science) ,2021,40(1):125-132.

基于多头自注意力和并行混合模型的文本情感分析

李辉1, 黄钰杰2

1.河南理工大学 物理与电子信息学院，河南 焦作 454000;2.河南理工大学 电气工程与自动化学院，河南 焦作 454000

摘要:针对以往研究大多使用单一模型进行文本情感分析，导致无法很好地捕捉相关文本的情感特征，从而造成情感分析效果不理想的问题，提出一种基于多头自注意力和并行混合模型的文本情感分析方法。首先，利用Word2 vec模型捕捉单词的语义特征，训练词向量；其次，借助双层多头自注意力机制（double layer multi-head self-attention ， DLMA）学习文本内部的词依赖关系，捕获其内部结构特征；再次，使用并行的双向门限循环神经网络（bi-directional gated recurrent nnit， BiGRU ）获取文本的序列特征；最后，借助改进的并行卷积神经网络（convolutional neural network ，CNN）提取深层次特征信息。将该模型分别在2个数据集上进行实验验证，其准确率分别达到92.71%和91.08%。结果表明，该方法比其他模型具有更好的学习能力。

关键词:多头自注意力;双向门限循环神经网络;卷积神经网络;文本情感分析

doi:10.16186/j.cnki.1673-9787.2019100022

基金项目:河南省基础与前沿技术研究计划资助项目（152300410103 ）

收稿日期:2019/10/10

修回日期:2019/12/26

出版日期:2021/01/15

Text sentiment analysis based on multi self-attention and parallel hybrid model

LI Hui1, HUANG Yujie2

1.School of Physics and Electronic Information， Henan Polytechnic University， Jiaozuo  454000 ， Henan， China;2.School of Electrical Engineering and Automation， Henan Polytechnic University， Jiaozuo  454000 ， Henan， China

Abstract:Most of the past studies used a single model for text sentiment analysis， which led to the inability to capture the emotional features of related texts well， and led to the problem of unsatisfactory sentiment analysis. A model of text sentiment analysis method based on self-attention and parallel hybrid model was proposed. First， the Word2vec model was used to capture the semantic features of words and to train word vectors. Secondly ，the double layer multi-head self-attention （ DLMA） was used to learn the word dependence within the text and to capture its internal structural features. The sequence characteristics of the text were then acquired by using a parallel bi-directional gated recurrent unit （ BiGRU ）.Finally， the deep hierarchical feature information was extracted by the improved convolutional neural network （ CNN ）.The model was validated on two data sets， and the accuracy rate reached 92. 71% and 91.08%.The experimental results showed that the method had better learning performance than other models.

Key words:multi-head self-attention;bi-directional gated recurrent unit;convolutional neural network;text

 基于多头自注意力和并行混合模型的文本情感分析_李辉.pdf