古汉语编年体的人名实体识别与词性标注

发布时间：2018-06-22 08:29

本文选题：古汉语编年体 + 条件随机场　；参考：《复旦大学》2012年硕士论文

【摘要】：自然语言处理是人工智能中的重要领域,不但能够帮助人们从庞大的语言数据中提取出需要的信息,甚至能够理解语法语义并对其做出合理的应答。随着近十年来现代汉语的语言处理研究逐步进展,研究者们已经逐渐能够从现代汉语中抽提出有效的实体信息并进行简单的语法解析。然而古汉语作为中国的传统语言,同样有着智能处理的需要,古汉语文献中蕴藏着大量的历史资料需要被结构化的整理出来。本文以编年体《明史本纪》为范例,首次制作了详细词性标注的古汉语语料,并分别从基于统计与基于规则的角度尝试对其进行名词实体识别与词性标注实验。首先使用了在现代汉语中效率最高的序列模型条件随机场作为统计模型,并且根据古汉语的词法特征设计了多套标签模板以及图模型对语料分别进行了实体识别与词性标注实验。在实验中大部分词性的判断有着良好的结果,而且词性的标注对实体识别的效率有着很好的帮助。从实验结果中,还发现了未登录人名的判断效率要明显高于其他未登录词,从而暗示着编年体人名周围有着很强的规则。随后使用了基于规则的方法尝试对相同的语料进行人名识别。通过观察,发现了编年体裁中人物与官职的关联,并设计了所有含有官职与姓名的句式规则。通过这些规则实现的算法在测试结果中有着良好的表现,最后分析了漏检与错判的姓名的原因和规则中的不足。词性标注实验结果表明古汉语的信息处理不需要进行分词就可以达到较好的效果,但语料库与专有名词库的完整性对结果有着重要的影响。而人名识别的实验反映出编年体中的人名句式十分规则,能够精确的提取出绝大多数姓名,但对于其他体裁的处理则需要更深入的研究。
[Abstract]:Natural language processing is an important field in artificial intelligence. It can not only help people extract the necessary information from the huge language data, but also understand the syntax and semantics and make a reasonable response to it. With the development of language processing in modern Chinese in the past ten years, researchers have been able to extract effective entity information from modern Chinese and make simple grammar analysis. However, as the traditional language of China, ancient Chinese also has the need of intelligent processing. There are a large number of historical materials in ancient Chinese literature that need to be structured out. Taking chronological style Ming Shi Ben Ji as an example, this paper for the first time makes detailed ancient Chinese corpus of part of speech tagging, and tries to carry on the experiment of noun entity recognition and part of speech tagging from the point of view of statistics and rule. Firstly, the conditional random field, which is the most efficient sequential model in modern Chinese, is used as the statistical model. According to the lexical features of ancient Chinese, several sets of tag templates and graph models are designed to carry out entity recognition and part of speech tagging experiments respectively. In the experiment, most of the judgments of part of speech have good results, and the tagging of part of speech is very helpful to the efficiency of entity recognition. From the experimental results it is also found that the judgment efficiency of unrecorded names is obviously higher than that of other unrecorded words which implies that there are strong rules around chronological names. Then a rule-based approach is used to identify the same corpus. Through observation, we find out the relationship between characters and official posts in chronological genre, and design all sentence rules containing official posts and names. The algorithm realized by these rules has a good performance in the test results. Finally, the reasons of missing and wrong names and the shortcomings of the rules are analyzed. The experimental results of part of speech tagging show that the information processing of ancient Chinese can achieve better results without the need of word segmentation, but the integrity of corpus and proper noun database has an important impact on the results. The experiment of human name recognition shows that the name sentence pattern in chronology is very regular, and can extract most names accurately, but the processing of other genres needs more in-depth research.
【学位授予单位】：复旦大学
【学位级别】：硕士
【学位授予年份】：2012
【分类号】：H109.2

【共引文献】