文本特征和复合统计量的领域术语抽取方法

发布时间：2019-02-09 12:30

【摘要】：中文领域术语的抽取,是文本知识挖掘的重要内容。传统的中文领域术语抽取方法以人工方法为主,显然这种方法费时费力。目前,处于研究阶段的中文领域术语自动化抽取方法主要有:基于字典的方法、基于规则的方法以及基于统计的方法。但由于中文自然语言的复杂性,这些自动化抽取方法都存在一定的局限性,比如对特定领域的用户字典及规则存在更新速度慢、文本特征考虑不足等,从而导致抽取的效果不佳。针对这一问题,提出了一种基于文本特征和复合统计量的中文领域术语抽取方法,该方法在对中文文档中的词语进行粗粒度筛选后,再综合考虑候选术语的词性、长度、边界词语等文本特征,构造出信息熵和TFIDF等统计量,计算其综合权值,并将综合权值大于设定阈值的候选术语抽取出来,作为最终的领域术语。实验结果表明,该方法在测试语料下,获得了较好的正确率、召回率和F值。
[Abstract]:The extraction of terms in Chinese domain is an important part of text knowledge mining. The traditional Chinese domain term extraction method is mainly manual, obviously this method is time-consuming and laborious. At present, the automatic extraction methods of Chinese terminology are mainly based on dictionaries, rules and statistics. However, due to the complexity of Chinese natural language, these automatic extraction methods have some limitations, such as slow updating of user dictionaries and rules in specific fields, insufficient consideration of text features, etc. As a result, the effect of extraction is not good. In order to solve this problem, this paper proposes a method of extracting terms in Chinese domain based on text features and compound statistics. After coarse-grained selection of the words in Chinese documents, the method synthetically considers the part of speech and length of candidate terms. Based on the text features such as boundary words, information entropy and TFIDF statistics are constructed, the synthetic weights are calculated, and the candidate terms whose weights are greater than the threshold are extracted as the final domain terms. The experimental results show that the proposed method has good accuracy, recall and F value under the test data.
【作者单位】：西北工业大学计算机学院;
【基金】：陕西省自然科学基金(2015JM6290)资助
【分类号】：TP391.1

【相似文献】