基于字典的藏文分词系统实现研究

发布时间：2019-03-11 07:47

【摘要】：随着对语言文字信息处理研究工作的不断加深,藏文信息处理技术也逐渐从字信息处理走向了语言信息处理。跟日语、汉语、韩语等语种的信息处理相同,藏文自动分词(Tibetan Automatic Word Segmentation)是藏文信息处理中的一项必不可少的基础性工作,在此基础上才能划分短语、抽取概念以及分析主题,以至自然语言理解,最终实现智能化。对于不同应用环境,藏文自动分词需要采用最合适的算法,本文通过对藏语语料的统计分析和藏语词的分布特点、语法功能的研究,提出了设计开发基于词典库的藏文自动分词系统,力求为藏文输入法研究、藏文电子词典建设、藏文字词频统计、搜索引擎的设计和实现、机器翻译系统的开发、网络信息安全、藏文语料库建设以及藏语语义分析研究奠定基础。
[Abstract]:With the deepening of the research work on language information processing, Tibetan information processing technology has gradually changed from word information processing to language information processing. Like the information processing of Japanese, Chinese, Korean and other languages, Tibetan automatic word segmentation (Tibetan Automatic Word Segmentation) is an essential basic work in Tibetan information processing. On this basis, we can divide phrases, extract concepts and analyze topics. Even natural language understanding, finally to achieve intelligence. For different application environments, Tibetan automatic word segmentation needs to adopt the most suitable algorithm. Through the statistical analysis of Tibetan language data, the distribution characteristics of Tibetan words and the study of grammatical function, This paper puts forward the design and development of Tibetan automatic word segmentation system based on lexicon database, which aims at the research of Tibetan input method, the construction of Tibetan electronic dictionary, the statistics of Tibetan word frequency, the design and implementation of search engine, and the development of machine translation system. The network information security, the construction of Tibetan corpus and the research of Tibetan semantic analysis lay the foundation.
【作者单位】：西藏大学藏文信息技术研究中心;西南交通大学信息科学与技术学院;
【基金】：2011年度国家自然科学基金项目“基于群体智能涌现的藏文网络舆情分析及突发事件预警机制研究”(项目编号:61165013)阶段性成果
【分类号】：TP311.52;TP391.1

【参考文献】