一种基于LDA模型的关键词抽取方法

来源期刊:中南大学学报(自然科学版)2015年第6期

论文作者:李淼 朱泽德 张健 曾伟辉 曾新华

文章页码:2142 - 2149

关键词:信息抽取;关键词抽取;LDA模型;主题相似性

Key words:information extraction; keyphrase extraction; LDA model; topic similarity

摘    要:为解决现有方法未能综合考察文档主题的全面性、关键词的可读性以及差异性,提出一种基于文档隐含主题的关键词抽取新算法TFITF。算法根据大规模语料产生隐含主题模型计算词汇对主题的TFITF权重并进一步产生词汇对文档的权重,利用共现信息排序和选择相邻词汇形成候选关键短语,再使用相似性排除隐含主题一致的冗余短语。此外,从文档统计信息、词汇链和主题分析3方面来进行关键词抽取的对比测试,实验在1 040篇中文摘要及5 408个关键词构成的测试集上展开。结果表明,算法有效地提高文档关键词抽取的准确率与召回率。

Abstract: Due to the shortage of the comprehensive analysis of the coverage of document topics, the readability and difference of keyphrases, a new algorithm of keyphrase extraction TFITF based on the implicit topic model was put forward. The algorithm adopted the large-scale corpus and producted latent topic model to calculate the TFITF weight of vocabulary on the topic and further generate the weight of vocabulary on the document. And adjacent lexical was ranked and picked out as candidate keyphrases based on co-occurrence information. Then according to the similarity of vocabulary topics, redundant phrases were eliminated. In addition, the comparative experiments of candidate keyphrases were executed by document statistical information, vocabulary chain and topic information. The experimental results, which were carried out on an evaluation dataset including 1 040 Chinese documents and 5 408 standard keyphrases, demonstrate that the method can effectively improve the precision and recall of keyphrase extraction.

相关论文

  • 暂无!

相关知识点

  • 暂无!

有色金属在线官网  |   会议  |   在线投稿  |   购买纸书  |   科技图书馆

中南大学出版社 技术支持 版权声明   电话:0731-88830515 88830516   传真:0731-88710482   Email:administrator@cnnmol.com

互联网出版许可证:(署)网出证(京)字第342号   京ICP备17050991号-6      京公网安备11010802042557号