首页期刊简介编委会投稿启事审稿流程读者订阅广告服务联系我们English
引用本文
  •    [点击复制]
  •    [点击复制]
【打印本页】 【下载PDF全文】 查看/发表评论下载PDF阅读器关闭

←前一篇|后一篇→

过刊浏览    高级检索

本文已被:浏览 36次   下载 0 本文二维码信息
码上扫一扫!
基于改进Transformer模型的文本摘要生成方法
王侃,曹开臣,徐畅,潘袁湘,牛新征
0
(中国西南电子技术研究所,成都 610036;电子科技大学 计算机科学与工程学院;,成都 610000;电子科技大学 信息与软件工程学院,成都 610000)
摘要:
传统的文本摘要方法,如基于循环神经网络和Encoder-Decoder框架构建的摘要生成模型等,在生成文本摘要时存在并行能力不足或长期依赖的性能缺陷,以及文本摘要生成的准确率和流畅度的问题。对此,提出了一种动态词嵌入摘要生成方法。该方法基于改进的Transformer模型,在文本预处理阶段引入先验知识,将ELMo(Embeddings from Language Models)动态词向量作为训练文本的词表征,结合此词对应当句的文本句向量拼接生成输入文本矩阵,将文本矩阵输入到Encoder生成固定长度的文本向量表达,然后通过Decoder将此向量表达解码生成目标文本摘要。实验采用Rouge值作为摘要的评测指标,与其他方法进行的对比实验结果表明,所提方法所生成的文本摘要的准确率和流畅度更高。
关键词:  文本摘要  Transformer模型  先验知识  动态词向量  句向量
DOI:
基金项目:
Text abstract generation based on improved Transformer model
WANG Kan,CAO Kaichen,XU Chang,PAN Yuanxiang,NIU Xinzheng
(Southwest China Institute of Electronic Technology,Chengdu 610036,China;School of Computer Science and Engineering,University of Electronic Science and Technology of China,Chengdu 610000,China;School of Information and Software Engineering,University of Electronic Science and Technology of China,Chengdu 610000,China)
Abstract:
The traditional text abstract methods,such as the abstract generation model based on the circular neural network and Encoder-Decoder framework,have the shortcomings of parallel ability,long-term dependent performance defects,as well as the accuracy and fluency problem in the generation of text abstracts.So,a dynamic word embedding abstract generation model(DWEM) is proposed.This method is based on the improved Transformer model.Firstly,prior knowledge is introduced in the text preprocessing stage,and the Embeddings from Language Models(ELMo) dynamic word vector is taken as the word representation of the training text.Secondly,the input text matrix is generated by combining this word with the text sentence vector of the corresponding sentence.The text matrix is then input into the Encoder to generate a fixed-length text vector representation.Finally,this vector is expressed and decoded by a Decoder to generate the target text abstract.In experiment,Rouge value is used as the evaluation index of the abstract,and the DWEM method is compared with other methods.The results show that the proposed method is more accurate and fluent.
Key words:  text abstract  Transformer model  prior knowledge  dynamic word vector  sentence vector
安全联盟站长平台